Friday, December 9, 2011

Azure - BLOB Storage

-Balaji Ramasamy
Windows Azure offers four core storage services namely Blob, Table, Queue and Drive, which are accessible via multiple instances / applications simultaneously representing a dedicated storage instead of temporary storage. The simplest way to store with Windows Azure is Binary Large Object (BLOB) Service. In this blog, I provide highlights of storage services offered by MS Azure and compared the same with AWS S3 (Simple Storage Service) that enables you to store data in the AWS cloud.

We can store the files in three ways. The file size with 64MB or lesser size can be uploaded as a single BLOB. If the file size exceeds 64MB then we can go for BLOCK Blob or PAGE Blob.

The BLOCK Blob supports 200GB of total file size and each block should be maximum of 4MB and its identified by BlockID which is unique within the Blob. Block ID is Base64 encoded string of same length and its values can be duplicated in different blobs.
When using base-64 encoding, the pre-encoded string must be 64 bytes or less. The maximum of 50,000 blocks is allowed for each blob. The maximum of 1,00,000 uncommitted blocks is allowed, its total size should be 400GB and finally the uncommitted blocks will be garbage collected after a week. We have to call PutBlockList to commit the uploaded blocks.

The PAGE Blob supports 1TB of total size and each page should be maximum of 4MB and its collection of 512-bytes. To create a page blob, initialize the page blob and specify the maximum size. Writes to page blobs happens in-place and are immediately committed to the blob. It’s optimized for random read and writes operation.

BLOB Storage Local Testing
  1. Local Storage is provided by Windows Azure to test cloud storage with help of Storage Emulator.
  2. Blobs in the Windows Azure storage emulator are limited to 2 GB.
  3. When making a request against the local storage service, specify the local hostname and Blob service port as, followed by the local storage account name
The following is a comparison of Amazon Simple Storage Service and Azure BLOB Storage.

Simple Storage Service
BLOB Storage
User Subscription
Single Account Multiple Accounts
Security Keys
Two set of acceskey and secret key Account name with Primary Access Key / Secondary Access Key
With Acceskey and Secretkey With Account name and Primary Access Key or Secondary Access Key
Domain Name
Bucket Name and its unique Account Name and its unique
Contents are categorized by
Bucket Container
Size Limitations on Storage
No size limit on storage 100TB storage size per account
Size Limitations on File-Size
Up to 5GB in a single PUT
Up to 5 TB using multipart upload
Up to 64MB for single Blob.
Up to 200GB for Block Blobs
Up to 1TB for Page Blobs
Specifying API Version in Request Header
Offline Testing
N/A Local Storage is provided by Windows Azure to test cloud storage with help of Storage Emulator.
Enable/Disable/Suspend versioning at Bucket Level Provided Snapshot functionality for object(s).
Bucket Level Logging

No size limitations.

Logs stored in configured Bucket and Prefix
Service(Blob, Table, Queue) Level Logging for storage account's

20TB size limitations other than 100TB storage size.

Log data is stored as block blobs in a container called $logs in the blob namespace of your account

Logs can be accessed thru $logs container using http://<accountname>$logs

Log name under the $logs container will have the following format:
 <service name>/YYYY/MM/DD/hhmm/<Counter>.log


LIST API Comparison


LIST Bucket
LIST Container

Prefix Prefix Keys that begin with the indicated prefix. prefix Filters the results to return only blobs whose names begin with the specified prefix.
Indicates where in the bucket to begin listing.
Indicates where in the bucket to begin listing.
No.of Records MaxKeys
NextMarker NextMarker element has a value only if the list results are not complete. NextMarker NextMarker element has a value only if the list results are not complete.
To dentify list complete or not IsTruncated Specifies whether (true) or not (false) all of the results were returned. All of the results may not be returned if the number of results exceeds that specified by MaxKeys. N/A N/A
Delimiter Delimiter Causes keys that contain the same string between the prefix and the first occurrence of the delimiter to be rolled up into a single result element in the Common-
Prefixes collection.
delimiter When the request includes this parameter, the operation returns a BlobPrefix element in the response body that acts as a placeholder for all blobs whose names begin with the same substring up to the appearance of the delimiter character. The delimiter may be a single character or a string.
Folders Common-Prefixes
FileName Key The object's key. Name name of the object
Size Size Size in bytes of the object. Content-Length / Size Size in bytes of the object.
Metadata N/A N/A include snapshots,metadata,
ETag ETag MD5 hash of the object. The ETag only reflects changes to the contents of an object, not its metadata. Etag identifier
Content Type N/A N/A Content-Type blob-content-type
Modified Date LastModified Date and time the object was last modified. Last-Modified /LastModified date-time-value

Hope this post helped you to understand Azure BLOB Storage.


Erick Jhonson said...
This comment has been removed by the author.
Erick Jhonson said...

Thank you for influencing so many readers with your great information. Your article has sparked a lot of thought for me. I will share your info with my friend and class fellows. Keep it up.
dropped hard drive