- Windows 10 Fall Creators Update installation and features - Thu, Nov 2 2017
- Install Microsoft SQL Server on Ubuntu Linux - Thu, Jan 5 2017
- Use PowerShell with Google Cloud Platform - Thu, Dec 8 2016
The AWS virtual machine shares two iSCSI Volumes with your on-premises Windows server and the cloud. If you’re proficient with VMWare ESXi, you can easily sync local files with Amazon AWS and sync your on-premises data with the cloud.
Getting started ^
This article will assume that you already have an AWS account and you have VMWare ESXi running with a Windows VM also running on your host. Your Storage Gateway VM will need 7.5 GB of memory, and its two additional disks will require their own datastore. Storage Gateways aren’t exactly cheap at $125/month, but the first 60 days after activation are free. While it may sound like a lot of money, a service such as Iron Mountain offsite storage will cost a similar amount but with less flexibility.
First, we’re going to log into AWS, and then start the Storage Gateway setup. While there are several storage options available (for example, a virtual tape disk), we’ll stick to setting up synched volumes, which are called “Gateway-stored volumes.” Do this by navigating to the Storage & Content Delivery section of AWS, and select Storage Gateway:
The following page will describe the Storage Gateway and the general procedure for setting up the gateway. A wizard will ask you to Setup and Activate a New Gateway.
We’re now presented with three options: Gateway-Cached volumes, Gateway-Virtual Tape Library, and Gateway-Stored Volumes: Schedule off-site backups to Amazon S3 for your on-premises data. Select the third option, and click Continue.
After a warning, we’re presented with three different virtual machines to download. Select the first option, “I want to run the AWS Storage Gateway on VMware ESXi,” and then click Continue.
On the following page, click Download to download the AWS Storage Gateway VM software. The download is a zip file approximately 900 MB in size.
After unzipping the .ova file, be sure to place the file on a computer where the VSphere Client is installed. Log into your VSphere Client with administrative privileges. Similarly, click Continue in the Storage Gateway setup wizard.
Building the virtual machine ^
In vSphere, click on the File drop-down menu, and select Deploy OVF Template. Point the installer to your downloaded and unzipped .ova file. Name the storage gateway.
Select a datastore where you want to store the .ova.
When prompted for Disk Format, choose Thick provisioned format, and then click Finish.
It could take several minutes for the deployment to complete. Once you have completed the deployment, the new VM will appear on your ESXi host.
It’s important that your ESXi host has its clock synchronized with a Network Time Protocol (NTP) server. Additionally, we want to be certain that the new virtual machine uses the host to synchronize its own clock. Right-click on the Storage Gateway VM, and click Edit Settings. In the Options tab, select the VMware Tools setting in the Settings column. In the Advanced section, select Synchronize guest time with host, and click OK.
Provision local disks ^
Here we’ll provision local storage with two new volumes. For more detailed step-by-step instructions, refer to the documentation here. In short, we’ll create two disks inside the Storage Gateway VM settings. Once again, use Thick Provisioning, and ensure that the disks use Paravirtualized Disk Controllers. Keep in mind that you’ll first create the two disks and then enable the Paravirtualized disk controllers. The disks can be anywhere from 1 GB to 1 TB in size. It is recommended that you use a datastore separate from the one in which the virtual machine is located, as demonstrated in the following screenshot.
Once the disks have been created, right-click on the Storage Gateway VM, and click Edit Settings. Select the SCSI controller, and click Change Type.
Activate the Storage Gateway ^
Back in the AWS Setup and Activate Gateway wizard, after configuring your disks, click Continue to continue with activating the gateway.
Power up your Storage Gateway VM, go to the Summary tab, and copy the IP address.
Paste the VM’s IP address into the AWS wizard. It’s OK that the IP address is one that came from your local network—you do not need to enter your external IP here or mess around with a firewall. Then, click Proceed to Activation.
You’ll now be prompted to enter your time zone, provide a Gateway Name, and then Activate My Storage Gateway.
Now that the Storage Gateway has been activated, we want to Create Volumes in the AWS console to begin using the AWS Storage Gateway. Note that if we want to shut down or delete our Storage Gateway, we’ll do that under the Gateway tab in this screen.
Create AWS storage volumes ^
Under the Volumes tab, click Create Volumes, select one of your SCSI disks, name the volume, and click Create Volume.
Next, we will configure the iSCSI settings. Find your unique initiator name in your Windows Server iSCSI Initiator Configuration tab and enter it, enter an initiator secret and a Mutual CHAP secret, and click Save. If you’re unsure how to configure CHAP for iSCSI, consult the documentation here. In the interest of experimentation, you can skip CHAP authentication altogether.
While creating your volumes, select which volume will be used as the upload buffer and which will be the cache volume.
When we’ve completed the creation of our storage volumes, AWS will notify us that our volume is available.
Configure Windows iSCSI Initiator ^
Back on your Windows server, configure your CHAP secret.
When your volumes and CHAP are ready, connect to your volumes from the Discovery tab, then click Discover Portal, enter your IP address of your Storage Gateway, and then click the Advanced button.
Place a checkmark in Enable CHAP login, specify the Name, Target Secret, and place a checkmark in Perform mutual authentication.
Connect to the Storage Gateway volume ^
Next we’ll go into the Targets tab and connect to the IP address of our local Storage Gateway. If all goes well, you should see you’re now connected.
We are now able to go into the iSCSI Initiator Volumes and Devices tab and click the AutoConfigure button, which will populate the volume in the Volume List.
Initialize the iSCSI volume in Windows ^
We can finally head into Windows Computer Management tool, click on Disk Management, initialize our new disk, provide a drive letter, and begin synching data with our AWS volume.
Subscribe to 4sysops newsletter!
Where to go from here? ^
Now that we’ve got our Storage Gateway configured, we can manage our activated gateway, set bandwidth rate limits, optimize performance, take snapshots, and use those snapshots to build new EC2 instances. If your organization is already using AWS, it may prove worthwhile to bridge the storage gap between your on-premises datacenter and AWS S3 with Storage Gateways.
How do you integrate with S3 buckets?
There are 2 types of gateways, Volume and File. The differences between these two can be found in the Q&A FAQ found here and perhaps this may answer your question:
Q: When I look in Amazon S3 why can’t I see my volume data?
Your volumes are stored in Amazon S3 and accessible through AWS Storage Gateway. You cannot directly access them by using Amazon S3 API actions. You can take point-in-time snapshots of gateway volumes that are made available in the form of Amazon EBS snapshots. Use the file interface to work with your data natively in S3.