One of the features Amazon Simple Storage Service (S3) provides to storage buckets is lifecycle rules. You can use a lifecycle rule to remove or archive objects. I will be looking at creating a lifecycle rule with PowerShell to remove objects from a S3 bucket.

Graham Beer

Graham is an experienced system engineer with a passion for PowerShell and automation. You can follow his blog or reach him on Twitter at @GKBeer.

Creating a lifecycle rule is a good way to automate the housekeeping of your S3 bucket. By using lifecycle rules, you can remove objects after so many days, by a certain date, or use Amazon's S3 Glacier for archiving. It's worth noting that you can set lifecycle rules by bucket, prefix, or tags, and you can also set them for current and non-current (previous) versions. I will demonstrate a simple example of removing all objects by a specified date.

Creating a bucket and configuring the lifecycle rule ^

I will be creating the lifecycle rule with PowerShell; however, it is possible to do this through Amazon's GUI interface. I will start by creating a new S3 bucket with PowerShell:

New bucket creation

New bucket creation

Building the rule with PowerShell requires you to make use of AWS classes. In this example, I won't be looking to move any objects to S3 Glacier. To do this, you would need to use the Amazon.S3.Model.LifecycleTransition class. If this is an area you want to look at, please review Amazon's documentation.

Set an expiration date through the Amazon.S3.Model.LifecycleRuleExpiration class. To do this, create an instance of the class:

To mark a deletion by a number of days, use the day property. Use the Date property to set a date for objects to remove:

Setting an expiration date

Setting an expiration date

After setting the expiration part, initiate an instance of the Amazon.S3.Model.LifecycleRule class:

Using Get-Member allows us to view the properties available to us to configure our rules:

Viewing properties available to the LifecycleRule class

Viewing properties available to the LifecycleRule class

To get more details on these properties and their uses, look at the AWS documentation.

Now we can attach the expiration rule:

To complete the creation of the lifecycle rule, change the status to Enabled. Also, if you don't add anything to the Prefix rule, it will fail.

Applying a lifecycle rule to an S3 bucket ^

With the configuration lifecycle rule complete, it's time to add to our S3 bucket. We do this with the Write-S3LifecycleConfiguration cmdlet:

We can pipe Get-S3Bucket to Write-S3LifecycleConfiguration, and with the Configuration_Rules parameter, we can add our object containing the lifecycle rules. The S3 bucket now has a lifecycle rule set up and enabled.

Viewing lifecycle rules on S3 buckets ^

We've configured and added the rules to our S3 bucket. But how can we check the bucket has a rule configured or see what rules are set? Amazon provides the Get-S3LifecycleConfiguration cmdlet to do this. Get-S3LifecycleConfiguration will return a property named Rules, which you will need to expand to view lifecycle rules:

Looking at S3 rules with Get S3LifecycleConfiguration

Looking at S3 rules with Get S3LifecycleConfiguration

With additional rules defined (in this case, we have the expiration rule), we must expand this property also. To help view all rules defined, I have created a function called Get-BucketLifeCycle. The function will display a custom object output, like so:

Using Get BucketLifeCycle to view lifecycle rules

Using Get BucketLifeCycle to view lifecycle rules

You can pipe or add the bucket name as a parameter to view any lifecycle rules quickly.

Summary ^

Amazon provides an excellent way to manage your objects with lifecycle rules. Should you wish to have a monthly removal process or move to S3 Glacier to archive and keep costs down, you have a way to automate this. You can do this programmatically with PowerShell or through the GUI.

All of my code in this article, including the Get-BucketLifeCycle function, is on my GitHub page.

Join the 4sysops PowerShell group!

1+
Share
0 Comments

Leave a reply

Your email address will not be published. Required fields are marked *

*

© 4sysops 2006 - 2019

CONTACT US

Please ask IT administration questions in the forums. Any other messages are welcome.

Sending

Log in with your credentials

or    

Forgot your details?

Create Account