AWS lifecycle rules on S3 buckets with PowerShell

One of the features Amazon Simple Storage Service (S3) provides to storage buckets is lifecycle rules. You can use a lifecycle rule to remove or archive objects. I will be looking at creating a lifecycle rule with PowerShell to remove objects from a S3 bucket.

Author
Recent Posts

Graham Beer

Graham is an experienced system engineer with a passion for PowerShell and automation. You can follow his blog or reach him on Twitter at @GKBeer.

Creating a bucket and configuring the lifecycle rule

I will be creating the lifecycle rule with PowerShell; however, it is possible to do this through Amazon's GUI interface. I will start by creating a new S3 bucket with PowerShell:

New-S3Bucket -BucketName demo-life-cycle

New bucket creation

Building the rule with PowerShell requires you to make use of AWS classes. In this example, I won't be looking to move any objects to S3 Glacier. To do this, you would need to use the Amazon.S3.Model.LifecycleTransition class. If this is an area you want to look at, please review Amazon's documentation.

Set an expiration date through the Amazon.S3.Model.LifecycleRuleExpiration class. To do this, create an instance of the class:

$Expiration = [Amazon.S3.Model.LifecycleRuleExpiration]::new()

To mark a deletion by a number of days, use the day property. Use the Date property to set a date for objects to remove:

$Expiration.Date = [DateTime]::new(2018,12,28)

Setting an expiration date

After setting the expiration part, initiate an instance of the Amazon.S3.Model.LifecycleRule class:

$LifecycleRule = [Amazon.S3.Model.LifecycleRule]::new()

Using Get-Member allows us to view the properties available to us to configure our rules:

$LifecycleRule | Get-Member -MemberType Properties

Viewing properties available to the LifecycleRule class

To get more details on these properties and their uses, look at the AWS documentation.

Now we can attach the expiration rule:

$LifecycleRule.Expiration = $Expiration

To complete the creation of the lifecycle rule, change the status to Enabled. Also, if you don't add anything to the Prefix rule, it will fail.

$LifecycleRule.Prefix = $null$LifecycleRule.Status = 'Enabled'

Applying a lifecycle rule to an S3 bucket

With the configuration lifecycle rule complete, it's time to add to our S3 bucket. We do this with the Write-S3LifecycleConfiguration cmdlet:

Get-S3Bucket -BucketName demo-life-cycle |    Write-S3LifecycleConfiguration -Configuration_Rules $LifecycleRule

We can pipe Get-S3Bucket to Write-S3LifecycleConfiguration, and with the Configuration_Rules parameter, we can add our object containing the lifecycle rules. The S3 bucket now has a lifecycle rule set up and enabled.

Viewing lifecycle rules on S3 buckets

We've configured and added the rules to our S3 bucket. But how can we check the bucket has a rule configured or see what rules are set? Amazon provides the Get-S3LifecycleConfiguration cmdlet to do this. Get-S3LifecycleConfiguration will return a property named Rules, which you will need to expand to view lifecycle rules:

Get-S3Bucket -BucketName demo-life-cycle |    Get-S3LifecycleConfiguration | Select-Object -ExpandProperty Rules

Looking at S3 rules with Get S3LifecycleConfiguration

With additional rules defined (in this case, we have the expiration rule), we must expand this property also. To help view all rules defined, I have created a function called Get-BucketLifeCycle. The function will display a custom object output, like so:

Using Get BucketLifeCycle to view lifecycle rules

You can pipe or add the bucket name as a parameter to view any lifecycle rules quickly.

Summary

Amazon provides an excellent way to manage your objects with lifecycle rules. Should you wish to have a monthly removal process or move to S3 Glacier to archive and keep costs down, you have a way to automate this. You can do this programmatically with PowerShell or through the GUI.

All of my code in this article, including the Get-BucketLifeCycle function, is on my GitHub page.

Subscribe to 4sysops newsletter!

function Get-BucketLifeCycle {
    [cmdletbinding(DefaultParameterSetName = 'FromPipeline')]
    param (
        [Parameter(Mandatory, Position = 0, ValueFromPipeline, ParameterSetName = 'FromPipeline')]
        [Amazon.S3.Model.S3Bucket] $S3Bucket,
        [Parameter(Mandatory, Position = 0, ParameterSetName = 'ByBucketString')]
        [String] $BucketByString
    )
    begin {
        if ($pscmdlet.ParameterSetName -eq 'ByBucketString') {
            $S3Bucket = Get-S3Bucket -BucketName $BucketByString
        }
    }
    process {
        $bucketRules = $S3Bucket | Get-S3LifecycleConfiguration
        foreach ($bucketRule in $bucketRules.Rules) {
            [PSCustomObject]@{
                RuleName          = $bucketRule.Id
                PrefixPath        = $bucketRule.Filter.LifecycleFilterPredicate.Prefix
                DateTimeExpire    = $bucketRule.Expiration.Date
                DaysUntilArchieve = $bucketRule.Expiration.Days
                NoncurrentExpire  = $bucketRule.NoncurrentVersionExpiration.NoncurrentDays
                Transitions       = $bucketRule.Transitions.Transitions
                StatusOfRule      = $bucketRule.Status.Value
            }
        }
    }
}

Join the 4sysops PowerShell group!

Your question was not answered? Ask in the PowerShell forum!

Learn PowerShell with our PowerShell guides!