Create and manage append blobs with PowerShell

Before diving into how to manage append blobs using PowerShell, let's have a quick look at the blob types (block blobs, page blobs, and append blobs) available on Azure storage accounts. Storage accounts on Azure support three types of blobs, each of which is used for a different purpose. Therefore, it's important to determine the type of blob you need when you create it, as it is not possible to change the type once the blob is created. The supported blob types are as follows.

Author
Recent Posts

Baki Onur Okutucu

Onur is a DevOps Engineer and Subject Matter Expert for Azure, and PowerShell. He is the founder of Clouderz Ltd, a cloud consultancy based in London. For 15 years in a row, Microsoft has recognized him as an MVP (Most Valuable Professional). You can follow Onur on Twitter: @BakiOnur.

Latest posts by Baki Onur Okutucu (see all)

Manage Azure PowerShell global settings - Fri, Sep 22 2023
Create and manage append blobs with PowerShell - Wed, Oct 12 2022
Permanently delete a Key Vault in Azure using PowerShell - Fri, Feb 4 2022

Block blobs: This type of blob can have 50,000 blocks. Each supports up to 4 GB of data, which makes the maximum size of a blob 190.7 TB. These limits vary depending on service version. If you need to upload a large amount of data to a storage account, block blobs are very suitable for such tasks.

Page blobs: The maximum size for this type is 8 TB. Page blobs comprise 512 byte pages designed for random read and write operations. Azure virtual machine disks are mainly based on this type of blob.

Append blobs: As the name suggests, append blobs are used for appending operations that are suitable for logging, where a log file is continuously appended. So, it is only supported to append a file by adding the content to the end of the file; this operation is called an append block. Therefore, updating or deleting an existing append blob is not supported. Each block can have a size of up to 4 MB, and each blob can have up to 50,000 blocks.

Maximum block size of an Append Blob can be 4 MB

In our scenario, we will create and manage an append blob using PowerShell to update a log file in an automated way.

Creating an append blob using PowerShell

To create an append blob using PowerShell, we can either use the Set-AzStorageBlobContent cmdlet or the REST API.

Option 1: Upload a local file and put it in a storage account container as an append blob.

#CREATE AND UPLOAD
$storageAccount = Get-AzStorageAccount -ResourceGroupName AppInsights -Name forfuncapp
$storageContext = $storageAccount.Context
Set-AzStorageBlobContent -BlobType Append -Container 4sysops -Blob appendBlobTest.txt -Context $storageContext -File "c:\temp\appendBlobTextContent.txt"

We can upload a file and create an append blob straight into the container using PowerShell

Option 2: Create an empty append blob in a storage account container.

#CREATE
$storageAccount = Get-AzStorageAccount -ResourceGroupName AppInsights -Name forfuncapp
$storageContext = $storageAccount.Context
$container = (Get-AzStorageContainer -Name 4sysops -Context $storageContext).CloudBlobContainer
$createAppendBlobToBeCreated = "https://forfuncapp.blob.core.windows.net/4sysops/appendBlobTest.txt"
$SAStoken = New-AzStorageAccountSASToken -Service Blob -ResourceType Object,Service,Container -Permission rac -Protocol HttpsOnly -Context $storageContext
$createAppendBlob = $("$createAppendBlobToBeCreated"+"$SAStoken")
$HEADERS= @{
"x-ms-blob-type" = "AppendBlob"
}
invoke-restmethod -method PUT -Uri $createAppendBlob -ContentType 'application/json' -Headers $HEADERS

image3

An empty append blob can also be created using a REST API for specific scenarios

Adding data to an append blob

Again, we have two options for adding data to an existing append blob. As mentioned above, append blobs can only have new data at the end of the file, and no data can be deleted from the blob.

Option 1

#UPDATE
$storageAccount = Get-AzStorageAccount -ResourceGroupName AppInsights -Name forfuncapp
$storageContext = $storageAccount.Context
$container = (Get-AzStorageContainer -Name $blobContainer -Context $storageContext).CloudBlobContainer

$encoding = [System.Text.Encoding]::UTF8
$blob = Get-AzStorageBlob -Container 4sysops -Blob appendBlobTest.txt -Context $storageContext
$contentToAdd = [System.IO.File]::OpenRead("c:\temp\appendBlobTextContent.txt")
$blob.BlobBaseClient.AppendBlock($contentToAdd)
$contentToAdd.Close()

Option 2

Subscribe to 4sysops newsletter!

#UPDATE
$appendBlobName =  "https://forfuncapp.blob.core.windows.net/4sysops/appendBlobTest.txt?comp=appendblock&"
$SAStoken = (New-AzStorageAccountSASToken -Service Blob -ResourceType Object,Service,Container -Permission rac -Protocol HttpsOnly -Context $storageContext)
$SAStoken = $SAStoken.Substring(1,($SAStoken.length-1))
$AppendBlob = $("$appendBlobName" +"$SAStoken")
$HEADERS= @{
"x-ms-blob-type" = "AppendBlob"
}
$body = gc "c:\temp\appendBlobTextContent.txt" | foreach{"`r$_"}
invoke-restmethod -method PUT -Uri $AppendBlob -ContentType 'application/json' -Headers $HEADERS -body $body

New data has been added to the append blob

Conclusion

Append blobs in Azure storage are a convenient way to manage operations, such as logging, where new data needs to be continuously added to the file. Creating and managing log files in a storage account using PowerShell is a very common practice for day-to-day admin tasks.