- Windows 10 Fall Creators Update installation and features - Thu, Nov 2 2017
- Install Microsoft SQL Server on Ubuntu Linux - Thu, Jan 5 2017
- Use PowerShell with Google Cloud Platform - Thu, Dec 8 2016
The AWS virtual machine shares two iSCSI Volumes with your on-premises Windows server and the cloud. If you’re proficient with VMWare ESXi, you can easily sync local files with Amazon AWS and sync your on-premises data with the cloud.
Getting started ^
This article will assume that you already have an AWS account and you have VMWare ESXi running with a Windows VM also running on your host. Your Storage Gateway VM will need 7.5 GB of memory, and its two additional disks will require their own datastore. Storage Gateways aren’t exactly cheap at $125/month, but the first 60 days after activation are free. While it may sound like a lot of money, a service such as Iron Mountain offsite storage will cost a similar amount but with less flexibility.
First, we’re going to log into AWS, and then start the Storage Gateway setup. While there are several storage options available (for example, a virtual tape disk), we’ll stick to setting up synched volumes, which are called “Gateway-stored volumes.” Do this by navigating to the Storage & Content Delivery section of AWS, and select Storage Gateway:
Select Storage Gateway
The following page will describe the Storage Gateway and the general procedure for setting up the gateway. A wizard will ask you to Setup and Activate a New Gateway.
Set up and activate a new gateway
We’re now presented with three options: Gateway-Cached volumes, Gateway-Virtual Tape Library, and Gateway-Stored Volumes: Schedule off-site backups to Amazon S3 for your on-premises data. Select the third option, and click Continue.
Select gateway-stored volumes
After a warning, we’re presented with three different virtual machines to download. Select the first option, “I want to run the AWS Storage Gateway on VMware ESXi,” and then click Continue.
Select VMware ESXi
On the following page, click Download to download the AWS Storage Gateway VM software. The download is a zip file approximately 900 MB in size.
Download the VM
After unzipping the .ova file, be sure to place the file on a computer where the VSphere Client is installed. Log into your VSphere Client with administrative privileges. Similarly, click Continue in the Storage Gateway setup wizard.
Building the virtual machine ^
Log into vSphere
In vSphere, click on the File drop-down menu, and select Deploy OVF Template. Point the installer to your downloaded and unzipped .ova file. Name the storage gateway.
Name the storage gateway
Select a datastore where you want to store the .ova.
Specify datastore
When prompted for Disk Format, choose Thick provisioned format, and then click Finish.
Complete OVF deployment
It could take several minutes for the deployment to complete. Once you have completed the deployment, the new VM will appear on your ESXi host.
It’s important that your ESXi host has its clock synchronized with a Network Time Protocol (NTP) server. Additionally, we want to be certain that the new virtual machine uses the host to synchronize its own clock. Right-click on the Storage Gateway VM, and click Edit Settings. In the Options tab, select the VMware Tools setting in the Settings column. In the Advanced section, select Synchronize guest time with host, and click OK.
Sync guest time with host
Provision local disks ^
Here we’ll provision local storage with two new volumes. For more detailed step-by-step instructions, refer to the documentation here. In short, we’ll create two disks inside the Storage Gateway VM settings. Once again, use Thick Provisioning, and ensure that the disks use Paravirtualized Disk Controllers. Keep in mind that you’ll first create the two disks and then enable the Paravirtualized disk controllers. The disks can be anywhere from 1 GB to 1 TB in size. It is recommended that you use a datastore separate from the one in which the virtual machine is located, as demonstrated in the following screenshot.
Specify datastore
Once the disks have been created, right-click on the Storage Gateway VM, and click Edit Settings. Select the SCSI controller, and click Change Type.
Change SCSI controller type
Select VMware Paravirtual
Activate the Storage Gateway ^
Back in the AWS Setup and Activate Gateway wizard, after configuring your disks, click Continue to continue with activating the gateway.
AWS allocate local disks
Power up your Storage Gateway VM, go to the Summary tab, and copy the IP address.
Copy new IP address
Paste the VM’s IP address into the AWS wizard. It’s OK that the IP address is one that came from your local network—you do not need to enter your external IP here or mess around with a firewall. Then, click Proceed to Activation.
Enter IP address and proceed
You’ll now be prompted to enter your time zone, provide a Gateway Name, and then Activate My Storage Gateway.
Activate storage gateway
Now that the Storage Gateway has been activated, we want to Create Volumes in the AWS console to begin using the AWS Storage Gateway. Note that if we want to shut down or delete our Storage Gateway, we’ll do that under the Gateway tab in this screen.
Storage gateway settings
Create AWS storage volumes ^
Under the Volumes tab, click Create Volumes, select one of your SCSI disks, name the volume, and click Create Volume.
Create first volume
Next, we will configure the iSCSI settings. Find your unique initiator name in your Windows Server iSCSI Initiator Configuration tab and enter it, enter an initiator secret and a Mutual CHAP secret, and click Save. If you’re unsure how to configure CHAP for iSCSI, consult the documentation here. In the interest of experimentation, you can skip CHAP authentication altogether.
Enter IP address and proceed
While creating your volumes, select which volume will be used as the upload buffer and which will be the cache volume.
Configure local upload buffer
When we’ve completed the creation of our storage volumes, AWS will notify us that our volume is available.
Volume available
Configure Windows iSCSI Initiator ^
Back on your Windows server, configure your CHAP secret.
Enter IP address and proceed
When your volumes and CHAP are ready, connect to your volumes from the Discovery tab, then click Discover Portal, enter your IP address of your Storage Gateway, and then click the Advanced button.
iSCSI Discover Portal
Place a checkmark in Enable CHAP login, specify the Name, Target Secret, and place a checkmark in Perform mutual authentication.
Connect to the Storage Gateway volume ^
Next we’ll go into the Targets tab and connect to the IP address of our local Storage Gateway. If all goes well, you should see you’re now connected.
iSCSI discovered target
We are now able to go into the iSCSI Initiator Volumes and Devices tab and click the AutoConfigure button, which will populate the volume in the Volume List.
Auto configure iSCSI volumes
Initialize the iSCSI volume in Windows ^
We can finally head into Windows Computer Management tool, click on Disk Management, initialize our new disk, provide a drive letter, and begin synching data with our AWS volume.
Subscribe to 4sysops newsletter!
Initialize AWS volume
Where to go from here? ^
Now that we’ve got our Storage Gateway configured, we can manage our activated gateway, set bandwidth rate limits, optimize performance, take snapshots, and use those snapshots to build new EC2 instances. If your organization is already using AWS, it may prove worthwhile to bridge the storage gap between your on-premises datacenter and AWS S3 with Storage Gateways.
How do you integrate with S3 buckets?
There are 2 types of gateways, Volume and File. The differences between these two can be found in the Q&A FAQ found here and perhaps this may answer your question:
Q: When I look in Amazon S3 why can’t I see my volume data?
Your volumes are stored in Amazon S3 and accessible through AWS Storage Gateway. You cannot directly access them by using Amazon S3 API actions. You can take point-in-time snapshots of gateway volumes that are made available in the form of Amazon EBS snapshots. Use the file interface to work with your data natively in S3.