Using Amazon SQS with PowerShell

Amazon Web Services (AWS) offers a Simple Queue Service (SQS). Whether you are using PowerShell on Windows or PowerShell Core, you can use Amazon SQS within your application service.

Amazon SQS is a flexible queue service that has many uses cases, from scalable web applications and processing systems to inter-application communications. This is useful if you need to process lots of data at once or have threads or workers to do a specific action to a large pool of data.

With the explosion of use cases for PowerShell and Docker in production systems, you can easily scale using queue systems like Amazon SQS. SQS is beneficial when your service or application has job workers that need to perform actions based on a large pool of data.

For example, let's say you had a web application to manage your Active Directory accounts. You would have a self-service web application that manages those accounts to make changes as needed. With this example, a queue is a perfect example of how to scale this application. Let's break this down further:

  1. The user submits a requested change to an Active Directory account's properties via a webform.
  2. The web application would place this request into an Amazon SQS queue.
  3. You would have workers (or Docker containers) responsible for pulling data off the queues to complete the requested change.
Web application with a queue and worker backend

Web application with a queue and worker backend

Using Amazon SQS in this situation can scale your workers in and out depending on the current size of your SQS queue. This allows your application service to be flexible but also have a specific purpose.

Using PowerShell in this situation is a no-brainer. Active Directory and PowerShell (or .NET) go hand in hand. There are definitely other cases for using PowerShell to interact with Amazon SQS. But to keep this post simple, I will continue with this example.

The first thing you need to do is install the AWS Tools for PowerShell or PowerShell Core:

  1. PowerShell on Windows:
  2. PowerShell Core:

After you have installed AWSPowerShell, you will have access to all the available AWS functions, including the following list of SQS functions:

List of Amazon SQS functions

List of Amazon SQS functions

To continue, you will need all the necessary tokens from your AWS account. Accessing and creating these tokens and keys is out of the scope of this post.

Let's first create a new SQS queue. As you might guess, we will be using the New-SQSQueue cmdlet to create a new queue. This is how you create a new queue using PowerShell:

This will create a basic SQS queue. If you want or need to, you can specify certain attributes for your queue. These attributes range from encryption to VisibilityTimeout to setting the queue to be first in, first out (FIFO). The VisibilityTimeout attribute sets the duration to hide a queue message temporarily when a worker pulls an item from the queue. The neat thing about this attribute is that if the worker does not tell the queue it has processed the data successfully, the queue will automatically mark the data as available, as if the worker had never received the request originally.

Now that we've set up our new SQS queue, we now add data to it. To do this, we will use the Send-SQSMessage function. In our example, let's say a web application has a form that allows a user to change the DisplayName attribute of an Active Directory account. Once the user clicks Submit, the webform would add the following ($jsonData) to an Amazon SQS queue:

To mimic the webform, we will go ahead and add this data to our new queue:

Now that we have data in our SQS queue, we could have our worker or Docker container continually check the queue for new data to process. Our example could have a worker take the information submitted to our Amazon SQS queue and run some PowerShell code to process the data. In our example, our worker is just responsible for changing the DisplayName on a given Active Directory account. Our worker would first have to check the SQS queue for any tasks or jobs. Here is a small snippet of code that checks the SQS queue for a single request. This code then creates a new PowerShell object of the requested data:

Now that we have our data from our SQS queue, we can perform the action requested:

If the task completes successfully, we may have to tell the original queue that the message we received is complete, and the SQS queue should remove it. To do this, we would call the Remove-SQSMessage function like this:

That's it! Amazon SQS queues allow you to design your applications or service infrastructure in a scalable manner. Whether you need to create an internal service, like a self-service Active Directory management tool, or a customer-facing scalable application, combining SQS and PowerShell is a possibility. This means developers and IT professionals can create scalable infrastructure and services using their favorite language, PowerShell!

Join the 4sysops PowerShell group!

Your question was not answered? Ask in the forum!

1+
Share
0 Comments

Leave a reply

Your email address will not be published. Required fields are marked *

*

© 4sysops 2006 - 2020

CONTACT US

Please ask IT administration questions in the forums. Any other messages are welcome.

Sending

Log in with your credentials

or    

Forgot your details?

Create Account