Before diving into how to manage append blobs using PowerShell, let's have a quick look at the blob types (block blobs, page blobs, and append blobs) available on Azure storage accounts. Storage accounts on Azure support three types of blobs, each of which is used for a different purpose. Therefore, it's important to determine the type of blob you need when you create it, as it is not possible to change the type once the blob is created. The supported blob types are as follows.
Avatar

Block blobs: This type of blob can have 50,000 blocks. Each supports up to 4 GB of data, which makes the maximum size of a blob 190.7 TB. These limits vary depending on service version. If you need to upload a large amount of data to a storage account, block blobs are very suitable for such tasks.

Page blobs: The maximum size for this type is 8 TB. Page blobs comprise 512 byte pages designed for random read and write operations. Azure virtual machine disks are mainly based on this type of blob.

Append blobs: As the name suggests, append blobs are used for appending operations that are suitable for logging, where a log file is continuously appended. So, it is only supported to append a file by adding the content to the end of the file; this operation is called an append block. Therefore, updating or deleting an existing append blob is not supported. Each block can have a size of up to 4 MB, and each blob can have up to 50,000 blocks.

Maximum block size of an Append Blob can be 4 MB

Maximum block size of an Append Blob can be 4 MB

In our scenario, we will create and manage an append blob using PowerShell to update a log file in an automated way.

Creating an append blob using PowerShell

To create an append blob using PowerShell, we can either use the Set-AzStorageBlobContent cmdlet or the REST API.

Option 1: Upload a local file and put it in a storage account container as an append blob.

#CREATE AND UPLOAD
$storageAccount = Get-AzStorageAccount -ResourceGroupName AppInsights -Name forfuncapp
$storageContext = $storageAccount.Context
Set-AzStorageBlobContent -BlobType Append -Container 4sysops -Blob appendBlobTest.txt -Context $storageContext -File "c:\temp\appendBlobTextContent.txt"
We can upload a file and create an append blob straight into the container using PowerShell

We can upload a file and create an append blob straight into the container using PowerShell

Option 2: Create an empty append blob in a storage account container.

#CREATE
$storageAccount = Get-AzStorageAccount -ResourceGroupName AppInsights -Name forfuncapp
$storageContext = $storageAccount.Context
$container = (Get-AzStorageContainer -Name 4sysops -Context $storageContext).CloudBlobContainer
$createAppendBlobToBeCreated = "https://forfuncapp.blob.core.windows.net/4sysops/appendBlobTest.txt"
$SAStoken = New-AzStorageAccountSASToken -Service Blob -ResourceType Object,Service,Container -Permission rac -Protocol HttpsOnly -Context $storageContext
$createAppendBlob = $("$createAppendBlobToBeCreated"+"$SAStoken")
$HEADERS= @{
"x-ms-blob-type" = "AppendBlob"
}
invoke-restmethod -method PUT -Uri $createAppendBlob -ContentType 'application/json' -Headers $HEADERS
image3

image3

An empty append blob can also be created using a REST API for specific scenarios

Adding data to an append blob

Again, we have two options for adding data to an existing append blob. As mentioned above, append blobs can only have new data at the end of the file, and no data can be deleted from the blob.

Option 1

#UPDATE
$storageAccount = Get-AzStorageAccount -ResourceGroupName AppInsights -Name forfuncapp
$storageContext = $storageAccount.Context
$container = (Get-AzStorageContainer -Name $blobContainer -Context $storageContext).CloudBlobContainer

$encoding = [System.Text.Encoding]::UTF8
$blob = Get-AzStorageBlob -Container 4sysops -Blob appendBlobTest.txt -Context $storageContext
$contentToAdd = [System.IO.File]::OpenRead("c:\temp\appendBlobTextContent.txt")
$blob.BlobBaseClient.AppendBlock($contentToAdd)
$contentToAdd.Close()

Option 2

Subscribe to 4sysops newsletter!

#UPDATE
$appendBlobName =  "https://forfuncapp.blob.core.windows.net/4sysops/appendBlobTest.txt?comp=appendblock&"
$SAStoken = (New-AzStorageAccountSASToken -Service Blob -ResourceType Object,Service,Container -Permission rac -Protocol HttpsOnly -Context $storageContext)
$SAStoken = $SAStoken.Substring(1,($SAStoken.length-1))
$AppendBlob = $("$appendBlobName" +"$SAStoken")
$HEADERS= @{
"x-ms-blob-type" = "AppendBlob"
}
$body = gc "c:\temp\appendBlobTextContent.txt" | foreach{"`r$_"}
invoke-restmethod -method PUT -Uri $AppendBlob -ContentType 'application/json' -Headers $HEADERS -body $body
New data has been added to the append blob

New data has been added to the append blob

Conclusion

Append blobs in Azure storage are a convenient way to manage operations, such as logging, where new data needs to be continuously added to the file. Creating and managing log files in a storage account using PowerShell is a very common practice for day-to-day admin tasks.

avataravatar
1 Comment
  1. Avatar
    Martin 3 months ago

    In the “Adding data to an append blob > Option 1” section, what is this line used for :
    “$encoding = [System.Text.Encoding]::UTF8”

Leave a reply

Please enclose code in pre tags: <pre></pre>

Your email address will not be published. Required fields are marked *

*

© 4sysops 2006 - 2023

CONTACT US

Please ask IT administration questions in the forums. Any other messages are welcome.

Sending

Log in with your credentials

or    

Forgot your details?

Create Account