Azure - BLOB Storage

-Balaji Ramasamy

Windows Azure offers four core storage services namely Blob, Table, Queue and Drive, which are accessible via multiple instances / applications simultaneously representing a dedicated storage instead of temporary storage. The simplest way to store with Windows Azure is Binary Large Object (BLOB) Service. In this blog, I provide highlights of storage services offered by MS Azure and compared the same with AWS S3 (Simple Storage Service) that enables you to store data in the AWS cloud.

We can store the files in three ways. The file size with 64MB or lesser size can be uploaded as a single BLOB. If the file size exceeds 64MB then we can go for BLOCK Blob or PAGE Blob.

The BLOCK Blob supports 200GB of total file size and each block should be maximum of 4MB and its identified by BlockID which is unique within the Blob. Block ID is Base64 encoded string of same length and its values can be duplicated in different blobs.

When using base-64 encoding, the pre-encoded string must be 64 bytes or less. The maximum of 50,000 blocks is allowed for each blob. The maximum of 1,00,000 uncommitted blocks is allowed, its total size should be 400GB and finally the uncommitted blocks will be garbage collected after a week. We have to call PutBlockList to commit the uploaded blocks.

The PAGE Blob supports 1TB of total size and each page should be maximum of 4MB and its collection of 512-bytes. To create a page blob, initialize the page blob and specify the maximum size. Writes to page blobs happens in-place and are immediately committed to the blob. It’s optimized for random read and writes operation.

BLOB Storage Local Testing

1. Local Storage is provided by Windows Azure to test cloud storage with help of Storage Emulator.
2. Blobs in the Windows Azure storage emulator are limited to 2 GB.
3. When making a request against the local storage service, specify the local hostname and Blob service port as 127.0.0.1:10000, followed by the local storage account name

The following is a comparison of Amazon Simple Storage Service and Azure BLOB Storage.

AMAZON Simple Storage Service	AZURE BLOB Storage
User Subscription
Single Account	Multiple Accounts
Security Keys
Two set of acceskey and secret key	Account name with Primary Access Key / Secondary Access Key
Authentication
With Acceskey and Secretkey	With Account name and Primary Access Key or Secondary Access Key
Domain Name
Bucket Name and its unique	Account Name and its unique
Contents are categorized by
Bucket	Container
Size Limitations on Storage
No size limit on storage	100TB storage size per account
Size Limitations on File-Size
Up to 5GB in a single PUT Up to 5 TB using multipart upload	Up to 64MB for single Blob. Up to 200GB for Block Blobs Up to 1TB for Page Blobs
Specifying API Version in Request Header
N/A	x-ms-version http://msdn.microsoft.com/en-us/library/windowsazure/dd894041.aspx
Offline Testing
N/A	Local Storage is provided by Windows Azure to test cloud storage with help of Storage Emulator.
Versioning
Enable/Disable/Suspend versioning at Bucket Level	Provided Snapshot functionality for object(s).
Logging
Bucket Level Logging No size limitations. Logs stored in configured Bucket and Prefix	Service(Blob, Table, Queue) Level Logging for storage account's 20TB size limitations other than 100TB storage size. Log data is stored as block blobs in a container called $logs in the blob namespace of your account Logs can be accessed thru $logs container using http://<accountname>.blob.core.windows.net/$logs Log name under the $logs container will have the following format:  <service name>/YYYY/MM/DD/hhmm/<Counter>.log Reference: http://blogs.msdn.com/b/windowsazurestorage/archive/ 2011/08/03/windows-azure-storage-logging-using-logs-to-track-storage-requests.aspx http://msdn.microsoft.com/en-us/library/hh343270.aspx

LIST API Comparison

	Amazon		Azure
	LIST Bucket		LIST Container
	Name	Description	Name	Description
Prefix	Prefix	Keys that begin with the indicated prefix.	prefix	Filters the results to return only blobs whose names begin with the specified prefix.
Marker	Marker	Indicates where in the bucket to begin listing.	Marker	Indicates where in the bucket to begin listing.
No.of Records	MaxKeys	1000	maxresults	5000
Next- Marker	NextMarker	NextMarker element has a value only if the list results are not complete.	NextMarker	NextMarker element has a value only if the list results are not complete.
To dentify list complete or not	IsTruncated	Specifies whether (true) or not (false) all of the results were returned. All of the results may not be returned if the number of results exceeds that specified by MaxKeys.	N/A	N/A
Delimiter	Delimiter	Causes keys that contain the same string between the prefix and the first occurrence of the delimiter to be rolled up into a single result element in the Common- Prefixes collection.	delimiter	When the request includes this parameter, the operation returns a BlobPrefix element in the response body that acts as a placeholder for all blobs whose names begin with the same substring up to the appearance of the delimiter character. The delimiter may be a single character or a string.
Folders	Common-Prefixes		BlobPrefix
FileName	Key	The object's key.	Name	name of the object
Size	Size	Size in bytes of the object.	Content-Length / Size	Size in bytes of the object.
Metadata	N/A	N/A	include	snapshots,metadata, uncommittedblobs
ETag	ETag	MD5 hash of the object. The ETag only reflects changes to the contents of an object, not its metadata.	Etag	identifier
Content Type	N/A	N/A	Content-Type	blob-content-type
Modified Date	LastModified	Date and time the object was last modified.	Last-Modified /LastModified	date-time-value

Hope this post helped you to understand Azure BLOB Storage.