- Create and manage append blobs with PowerShell - Wed, Oct 12 2022
- Permanently delete a Key Vault in Azure using PowerShell - Fri, Feb 4 2022
- Restore Azure Files with PowerShell - Fri, Jan 28 2022
Block blobs: This type of blob can have 50,000 blocks. Each supports up to 4 GB of data, which makes the maximum size of a blob 190.7 TB. These limits vary depending on service version. If you need to upload a large amount of data to a storage account, block blobs are very suitable for such tasks.
Page blobs: The maximum size for this type is 8 TB. Page blobs comprise 512 byte pages designed for random read and write operations. Azure virtual machine disks are mainly based on this type of blob.
Append blobs: As the name suggests, append blobs are used for appending operations that are suitable for logging, where a log file is continuously appended. So, it is only supported to append a file by adding the content to the end of the file; this operation is called an append block. Therefore, updating or deleting an existing append blob is not supported. Each block can have a size of up to 4 MB, and each blob can have up to 50,000 blocks.
Maximum block size of an Append Blob can be 4 MB
In our scenario, we will create and manage an append blob using PowerShell to update a log file in an automated way.
Creating an append blob using PowerShell
To create an append blob using PowerShell, we can either use the Set-AzStorageBlobContent cmdlet or the REST API.
Option 1: Upload a local file and put it in a storage account container as an append blob.
#CREATE AND UPLOAD $storageAccount = Get-AzStorageAccount -ResourceGroupName AppInsights -Name forfuncapp $storageContext = $storageAccount.Context Set-AzStorageBlobContent -BlobType Append -Container 4sysops -Blob appendBlobTest.txt -Context $storageContext -File "c:\temp\appendBlobTextContent.txt"
We can upload a file and create an append blob straight into the container using PowerShell
Option 2: Create an empty append blob in a storage account container.
#CREATE
$storageAccount = Get-AzStorageAccount -ResourceGroupName AppInsights -Name forfuncapp
$storageContext = $storageAccount.Context
$container = (Get-AzStorageContainer -Name 4sysops -Context $storageContext).CloudBlobContainer
$createAppendBlobToBeCreated = "https://forfuncapp.blob.core.windows.net/4sysops/appendBlobTest.txt"
$SAStoken = New-AzStorageAccountSASToken -Service Blob -ResourceType Object,Service,Container -Permission rac -Protocol HttpsOnly -Context $storageContext
$createAppendBlob = $("$createAppendBlobToBeCreated"+"$SAStoken")
$HEADERS= @{
"x-ms-blob-type" = "AppendBlob"
}
invoke-restmethod -method PUT -Uri $createAppendBlob -ContentType 'application/json' -Headers $HEADERS
image3
An empty append blob can also be created using a REST API for specific scenarios
Adding data to an append blob
Again, we have two options for adding data to an existing append blob. As mentioned above, append blobs can only have new data at the end of the file, and no data can be deleted from the blob.
Option 1
#UPDATE
$storageAccount = Get-AzStorageAccount -ResourceGroupName AppInsights -Name forfuncapp
$storageContext = $storageAccount.Context
$container = (Get-AzStorageContainer -Name $blobContainer -Context $storageContext).CloudBlobContainer
$encoding = [System.Text.Encoding]::UTF8
$blob = Get-AzStorageBlob -Container 4sysops -Blob appendBlobTest.txt -Context $storageContext
$contentToAdd = [System.IO.File]::OpenRead("c:\temp\appendBlobTextContent.txt")
$blob.BlobBaseClient.AppendBlock($contentToAdd)
$contentToAdd.Close()
Option 2
Subscribe to 4sysops newsletter!
#UPDATE
$appendBlobName = "https://forfuncapp.blob.core.windows.net/4sysops/appendBlobTest.txt?comp=appendblock&"
$SAStoken = (New-AzStorageAccountSASToken -Service Blob -ResourceType Object,Service,Container -Permission rac -Protocol HttpsOnly -Context $storageContext)
$SAStoken = $SAStoken.Substring(1,($SAStoken.length-1))
$AppendBlob = $("$appendBlobName" +"$SAStoken")
$HEADERS= @{
"x-ms-blob-type" = "AppendBlob"
}
$body = gc "c:\temp\appendBlobTextContent.txt" | foreach{"`r$_"}
invoke-restmethod -method PUT -Uri $AppendBlob -ContentType 'application/json' -Headers $HEADERS -body $body
New data has been added to the append blob
Conclusion
Append blobs in Azure storage are a convenient way to manage operations, such as logging, where new data needs to be continuously added to the file. Creating and managing log files in a storage account using PowerShell is a very common practice for day-to-day admin tasks.
