- Scalability and availability for Azure Web Apps - Wed, Jun 28 2017
- Managing Azure Virtual Machine scale sets - Wed, May 31 2017
- Azure virtual machine scale sets - Mon, May 8 2017
Let’s start with the basics. Blob stands for “binary large object” and is a collection of binary data stored as a single entity. You can think of blobs as files we use every day. Any unstructured object data, such as text, binary, images, documents, video or audio, are accepted as blobs by Azure.
Blob storage is usually used for storing images, documents, video/audio for streaming, backup/logging data or contents for cloud applications. It’s also the default storage type for virtual machine disks (VHDs) created in Azure. ¸
Blob storage is also much cheaper compared to file storage in Azure and is an excellent way to have persistent data storage for your Internet-scale applications.
Blob types
In general, three types of blobs exist in an Azure environment: page, block and append.
Page blobs are commonly used to store VHDs and are designed for random read/write access. Azure provides two types of storage account for page blobs – Premium and Standard. Read my article about storage accounts for more information.
Block blobs are for everything you can think of as a file. Each block in one of these types of blobs can be a different size, up to a maximum of 4 MB, and a block blob can include up to 50,000 blocks. The maximum size of a block blob is therefore slightly more than 195 GB (4 MB X 50,000 blocks).
An append blob is an optimized blob for append operations. The main difference between a block blog and an append blog is that when you modify an append blob, new blocks are added to the end block. It is a good fit especially for logging scenarios.
Blob namespace
As I mentioned in previous parts, Azure Storage keeps a single global accessible namespace that leverages DNS so that it is addressable for storage actions. Blobs in the Azure Storage namespace are identified by URIs. An Azure Storage URI is divided into three parts: the endpoint (address of the storage service), the container name and the blob name. The endpoint consists of the Storage account name (anilerduran in the example) and the address of the storage cluster. In my example, the endpoint is https://anilerduran.blob.core.windows.net.
Storage endpoints
The second part of the URI is the container name. It corresponds to a folder in a file system namespace and allows you to organize your blobs. In my example, the container names are test and test2.
Containers
Finally, the third part is the name of the blob.
Azure Storage blobs
Before reaching out to the final content (blob), you need a Storage account and containers to be created to store individual blobs.
Containers provides a simple grouping of different blobs and also allows you to set the access control at the container level.
Container access levels
Access levels
Three types of access levels can be set per container.
Private: Blob within the container will be accessible by the account owner
Blob: Gives the public read access for blobs
Container: Gives the public read and list access to the entire container
You can choose either to create a bunch of containers and store your blobs in them or to create one single container for storing all of your blobs. It really depends on your application’s requirements. Having multiple containers provides more granular access control over your content.
As partitioning in Azure blobs is done at the blob level (URL for blob itself is the smallest object that will be load balanced by Microsoft), and as containers can grow as large as you need them to grow within the storage account limit, it really doesn’t matter from a scalability perspective whether you have multiple or single containers for hosting blobs.
Get started with Blob storage
You can create a storage account and a container on the Azure Portal, and you can also use the Web interface to upload your blob to Microsoft’s cloud. This is fine as long as you are dealing with only a few blobs. If you want to automate blob management, you can work with the Azure PowerShell module.
Azure PowerShell is distributed through the Microsoft Web Platform Installer (Web PI) and PowerShell Gallery. To install Azure PowerShell, run the following commands:
Install Azure PowerShell module
I have created a very basic PowerShell menu script for taking initial storage actions on an Azure subscription.
function CreateMenu ($Title,$MenuItems,$TitleColor,$LineColor,$MenuItemColor)
{
CLS
[string]$Title = "$Title"
$TitleCount = $Title.Length
$LongestMenuItem = ($MenuItems | Measure-Object -Maximum -Property Length).Maximum
if ($TitleCount -lt $LongestMenuItem)
{
$reference = $LongestMenuItem
}
else
{$reference = $TitleCount}
$reference = $reference + 10
$Line = "═"*$reference
$TotalLineCount = $Line.Length
$RemaniningCountForTitleLine = $reference - $TitleCount
$RemaniningCountForTitleLineForEach = $RemaniningCountForTitleLine / 2
$RemaniningCountForTitleLineForEach = [math]::Round($RemaniningCountForTitleLineForEach)
$LineForTitleLine = "`0"*$RemaniningCountForTitleLineForEach
$Tab = "`t"
Write-Host "╔" -NoNewline -f $LineColor;Write-Host $Line -NoNewline -f $LineColor;Write-Host "╗" -f $LineColor
if($RemaniningCountForTitleLine % 2 -eq 1)
{
$RemaniningCountForTitleLineForEach = $RemaniningCountForTitleLineForEach - 1
$LineForTitleLine2 = "`0"*$RemaniningCountForTitleLineForEach
Write-Host "║" -f $LineColor -nonewline;Write-Host $LineForTitleLine -nonewline -f $LineColor;Write-Host $Title -f $TitleColor -nonewline;Write-Host $LineForTitleLine2 -f $LineColor -nonewline;Write-Host "║" -f $LineColor
}
else
{
Write-Host "║" -nonewline -f $LineColor;Write-Host $LineForTitleLine -nonewline -f $LineColor;Write-Host $Title -f $TitleColor -nonewline;Write-Host $LineForTitleLine -nonewline -f $LineColor;Write-Host "║" -f $LineColor
}
Write-Host "╠" -NoNewline -f $LineColor;Write-Host $Line -NoNewline -f $LineColor;Write-Host "╣" -f $LineColor
$i = 1
foreach($menuItem in $MenuItems)
{
$number = $i++
$RemainingCountForItemLine = $TotalLineCount - $menuItem.Length -9
$LineForItems = "`0"*$RemainingCountForItemLine
Write-Host "║" -nonewline -f $LineColor ;Write-Host $Tab -nonewline;Write-Host $number"." -nonewline -f $MenuItemColor;Write-Host $menuItem -nonewline -f $MenuItemColor;Write-Host $LineForItems -nonewline -f $LineColor;Write-Host "║" -f $LineColor
}
Write-Host "╚" -NoNewline -f $LineColor;Write-Host $Line -NoNewline -f $LineColor;Write-Host "╝" -f $LineColor
}
function WrongOption{
Write-Host "Please select one of the options available" -Fore Red;Start-Sleep 1
}
function PressKey{
Write-Host "Press any key to return main menu"
$HOST.UI.RawUI.ReadKey("NoEcho,IncludeKeyDown") | OUT-NULL
$HOST.UI.RawUI.Flushinputbuffer()
}
function ImportMenu(){
$Menu1=0
while ( $Menu1 -NotIn 1..4 ){
CLS
CreateMenu -Title "Azure Storage Activities" -MenuItems "Add Azure Account/Subscription","Create Container","Upload Blob","Download all Blob from container" -TitleColor Green -LineColor Magenta -MenuItemColor Cyan
$Menu1 = Read-Host "Enter Menu Option Number"
}
Switch ($Menu1){
1{
Write-Host "Use this menu to add an azure account" -ForegroundColor black -BackgroundColor Green;Start-Sleep 1
Add-AzureAccount
$SubsName = (Get-AzureSubscription).SubscriptionName
Select-AzureSubscription -SubscriptionName $SubsName
Write-Host "Listing Storage Accounts" -ForegroundColor black -BackgroundColor Green;Start-Sleep 1
(Get-AzureStorageAccount).StorageAccountName
$SelectedStorageAccount = Read-Host "Enter Storage Account Name to set context"
Write-Host "Setting context" -ForegroundColor black -BackgroundColor Green;Start-Sleep 1
Set-AzureSubscription -CurrentStorageAccountName $SelectedStorageAccount -SubscriptionName (Get-AzureSubscription).SubscriptionName
PressKey
ImportMenu
}
2{
Write-Host "Use this menu to create a new container" -ForegroundColor black -BackgroundColor green;Start-Sleep 1
$ContainerName = Read-Host "Enter container name"
New-AzureStorageContainer -Name $ContainerName -Permission Off -Verbose
Write-Host "$ContainerName created" -ForegroundColor black -BackgroundColor green;Start-Sleep 1
PressKey
ImportMenu
}
3{
Write-Host "Use this menu to upload a blob into a container" -ForegroundColor black -BackgroundColor green;Start-Sleep 1
$ContainerName = Read-Host "Enter container name"
$ImagePath = Read-Host "Enter local path of the file"
Set-AzureStorageBlobContent -Container $ContainerName -File $ImagePath
Write-Host "Uploaded.." -ForegroundColor black -BackgroundColor green;Start-Sleep 1
PressKey
ImportMenu
}
4{
$DestinationFolder = Read-Host "enter local destination to download"
$container = read-host "enter container name to download all blobs"
$blobs = Get-AzureStorageBlob -Container $container
Write-Host "Downloading.." -ForegroundColor black -BackgroundColor green;Start-Sleep 1
$blobs | Get-AzureStorageBlobContent –Destination $DestinationFolder
PressKey
ImportMenu
}
}
}
ImportMenu
Azure Storage activities
Uploading a blob into a container
Blobs on Azure Portal
Downloading all blobs
Please note that the script is quite basic; several actions can be taken to demonstrate how easy it is to use PowerShell in the Azure Blob storage service.
Some important points must be considered when using PowerShell to manage blob storage. Let’s take a look at them:
- Azure PowerShell allows you to encapsulate your storage credentials in an object so that you can easily authenticate without specifying account details each time. We call this “Storage Context.”
$StorageAccountName = "accountname" $StorageAccountKey = Get-AzureStorageKey -StorageAccountName $StorageAccountName $StorageContext = New-AzureStorageContext $StorageAccountName -StorageAccountKey $StorageAccountKey.Primary
- We have talked about storage keys before; you can retrieve/generate keys using Azure PowerShell cmdlets.
Get-AzureStorageKey -StorageAccountName "accountname"(Get-AzureStorageKey -StorageAccountName $StorageAccountName).Primary(Get-AzureStorageKey -StorageAccountName $StorageAccountName).Secondary
- While creating a new container for your blobs, you should also set the level of access. As mentioned above, three levels of anonymous access to blobs exist: Off, Blob, and Container. You can establish the access level using the -Permission
$ContainerName = "containername"New-AzureStorageContainer -Name $ContainerName -Permission Blob
- It’s possible to create snapshots to back up a blob as it appears at a moment in time. A blob can have any number of snapshots. To create a snapshot, you should call the ICloudBlob.CreateSnapshot method on a blob reference you have created before.
$blobReference = Get-AzureStorageBlob -Context $StorageContext -Container $ContainerName -Blob $BlobName$snaphot = $blobReference.ICloudBlob.CreateSnapshot()
Subscribe to 4sysops newsletter!
- Some additional tasks cannot be completed on blobs.
Available storage commands
In the text post of my Azure Storage series I will discuss Table Storage.
