Examining lower-frequency-access cloud storage tiers

A new tier of object storage that is particularly useful for data protection is now available on both AWS and Azure: AWS S3 Infrequent Access (“S3-IA”) and Azure Cool Blob Storage (“CBS”). Since CTERA backs-up cloud servers, on-prem servers, and other endpoints to object storage as a native target, we are offering up a comparison of these object storage tiers that covers granularity, availability, performance, scalability, price, and more.

AWS S3-IA and Azure CBS are best for lower frequency access where latency and performance are still important. They have a lower price for storage but a higher price for access. This makes the tier appropriate for backup, disaster recovery, and longer-term media storage.

The original/higher-frequency tiers are now specified as AWS S3 Standard and Azure Hot Blob Storage.

Let’s see how they compare!

Table of Contents:
Granularity
Availability
Performance
Scalability
Security
Regional Availability
Changing Access Tier
Quirks
Pricing
Conclusions

Granularity

Both AWS and Azure let you set some information to lower-frequency-access and others at normal-frequency. But the granularity at which you can set it varies.

In AWS an individual object can be set to use S3 Standard or S3-IA.

In Azure the entire storage account must be set to use Cool Blobs or Hot Blobs. That means that all objects (“blobs”) within a storage account must be set to cool.

Changing access-frequency tier may incur costs, and that is covered later.

Availability SLA

Both Azure and AWS offer an SLA that provides a billing credit if the service is not available beyond a certain threshold in a given month. The availability SLA is lower than the higher-frequency-access tier which is why they are able to offer lower prices.

Both AWS and Azure offer a 10% credit if availability is below 99.0% and a 25% credit if availability is below 98.0% in most configurations.

Azure also raises the threshold to 99.9% for Read-Access-Geo-Redundant-Storage (RA-GRS) configurations.

For both the AWS S3 Standard and AWS Hot Blob Storage, the 10% and 25% credit thresholds are 99.9% and 99.0% availability respectively (and 99.99%/99.0% for RA-GRS).

Performance

Latency and Bandwidth for S3-IA and Cool Blob Storage are claimed to not be significantly different than the higher-frequency tier.

Amazon has not published performance targets for AWS S3.

Azure publishes some performance targets for storage:

• For single blob: Up to 60 MB per second, or up to 500 requests per second
• Per storage account total request rate: Up to 20,000 IOPS @ 1KB block size
• LRS: 20 Gbps ingress, 30 Gbps egress per account
• GRS: 10 Gbps ingress, 20 Gbps egress per account

Scalability

Scalability of AWS S3-IA and Azure CBS appear to be the same as the higher-frequency tier scalability. The hierarchy of maximums are shown here:

S3 has a maximum of 100 buckets per account, 5 TB per object and 5GB per PUT operation. Azure has a maximum of 100 x 500 TB accounts per subscription, with 500 TB per blob container, about 195 GB per block blob, and 4MB per block. (Note that Azure’s 100 accounts per subscription might be a soft limit that you can request to increase.)

Note that a transfer of a large object would be broken down into smaller transactions, each of which you are charged for.

Note: Since lower-frequency-tiers tiers of storage are used for backup data, it is valuable for the backup software to minimize the number of transactions and amount of data stored by deduplicating data prior to transferring it to the lower-frequency-access storage. This is something that the CTERA platform does already.

Security

Security of AWS S3-IA and Azure CBS is no different than their higher-frequency-tiers, so let’s see how they compare.

The biggest difference between AWS and Azure security is that Azure currently does not offer server-side encryption for blob storage. It is on the roadmap in preview as of May 23.

AWS and Azure both offer a client-side encryption library that can use a customer-managed master key or a cloud-managed (AWS KMS/Azure Key Vault) master key.

Both AWS and Azure have the option of accessing encrypted data in-flight using TLS (using https).

For encrypting data-at-rest, AWS offers Server-Side Encryption that can use keys managed by AWS S3 (using AES-256), AWS KMS, or the customer.

Regional Availability

AWS S3-IA is available at all AWS regions:

Azure Cool Blob Storage is where blob storage is available, which is at more than half of the Azure regions. Currently: Central US, East US 2, North Central US, South Central US, North Europe, West Europe, Southeast Asia, Japan East, Japan West, Central India, South India, and West India:

Changing Access Tier

It is possible in both AWS and Azure to change from a higher-frequency access tier to the infrequent tier.

In AWS you can change an object from S3 Standard or Reduced Redundancy to S3-IA. The change has a fee, and the object must have existed for at least 30 days in the current access tier to avoid a further fee. AWS further explains that “if you are transitioning noncurrent objects (versioned bucket scenario), you can transition to [IA] only objects that are at least 30 days noncurrent.”

AWS also offers an automated tier migration tool (Object Lifecycle Management) that lets you set policies for automatically migrating objects from S3 Standard to S3-IA to Glacier based on a time period or date. The policies are set at the S3 bucket level.

In Azure it is possible to change a blob storage account from Hot to Cool. The change applies to all objects within the account. This change is free of charge. And it is not possible to change a general purpose storage account to a Cool Blob account.

It is not possible in AWS to change an object directly from S3-IA to a higher-frequency access tier. You must copy the object into a S3 Standard or S3 Reduced Redundancy object. Copying the object incurs charges: an S3-IA copy request and S3-IA data retrieval. It is possible to change from S3-IA to Glacier.

Azure you can change an account from Cool Blob Storage to Hot Blob Storage, but you are charged for it. The change applies to all objects within the account and you are charged the same as reading all data in the storage account.

Quirks + Considerations

Both AWS S3-IA and Azure Cool Blob Storage have some nuances worth knowing when it comes to backup.

When you take EC2 snapshots, they automatically go to AWS S3 Standard and not into S3 Infrequent Access. Those objects must stay in in S3 Standard for 30 days before they can be changed to Infrequent Access tier. So even if you replicate the snapshot immediately, you can not transfer the snapshot to a lower tier right away.

Azure Cool Blob Storage is currently not integrated with Azure Backup Vault, although that is something that Microsoft says they are working on. The Azure classic deployment model (System Center Data Protection Manager + Azure Backup and Azure Site Recovery) is also not supported by Azure Cool Blob Storage.

Another quirk is that AWS S3-IA has a minimum object size of 128KB. Objects that are smaller than 128 KB are billed for 128 KB of storage.

Pricing

Note: Pricing is normalized to 10,000 requests for easier comparison (“per 10,000 requests” shortened to “p10kr”). Pricing is for the AWS US East N. Virginia and Azure East US 2 regions as of this writing.

AWS S3-IA and Azure Cool Blob Storage pricing has three parts: Data Stored + Data Transferred + Requests.

Azure has different pricing depending on redundancy that you select. LRS = Locally redundant storage; GRS = Geo Redundant Storage; RA-GRS = Read Access Geo Redundant Storage. Pricing for ZRS is not specified but I suspect it is the same as LRS.

For the cost of Data Stored, AWS S3-IA is a little more expensive than Azure Cool Blob Storage LRS, but AWS S3-IA is nearly 50% cheaper than Azure Cool Blob Storage GRS and RA-GRS. Objects have a minimum of 30 days in AWS S3-IA and if you delete, overwrite, or transition the object to a different storage class before 30 days, you are charged a fee equivalent to the cost of the time remaining in the 30 day minimum.

Data Transferred costs have two parts: Data retrieved or written to storage + Transfer in or out of the cloud or region. The only costs consistent across AWS and Azure are that data retrieval from storage is $0.01 per GB and that transferring data into the cloud is free (but actually writing the data to storage may have an associated cost). Azure has a specific charge for the built-in Geo-Replication Data Transfer that is different from transferring to another region.

AWS S3-IA and Azure Cool Blob Storage are not always cheaper or more expensive across the various types of Requests. Generally speaking, AWS S3-IA and Azure Cool Blob Storage LRS have similar request prices, but specifics can vary widely.

Conclusions

AWS S3 Infrequent Access and Azure Cool Blob Storage are both object storage tiers that are appropriate for backup and similar uses where you store a lot of data, access it infrequently, but can’t wait a long time to access it when needed. They both lower the (already low) costs associated with object storage. So they are both good options to reduce backup costs by using CTERA to back-up cloud servers and remote servers/endpoints natively to object storage.

AWS lets you set the access-frequency-tier for each object, whereas Azure requires all objects in a storage account to be at the same tier. So if you are using Azure Blob Storage as your CTERA backup target, it would be wise to create a dedicated Cool Blob storage account for backup data and never use it for primary storage.

AWS S3-IA and Azure CBS offer similar Availability SLAs, with Azure RA-GRS read requests having an additional 0.9% advantage.

Performance and scalability of AWS S3-IA and Azure CBS match that of their respective higher-frequency-access-tier, which is good news for short backup windows and rapid restores when you need it most. Note that CTERA deduplicates backup data prior to transferring it, which further reduces backup window and restore time.

Both AWS S3 and Azure Blob Storage offer encryption and key management for data pre-flight and in-flight, with Azure a little behind in not offering post-flight encryption. CTERA also encrypts backup data prior to transferring it to the object storage (pre-flight), but CTERA goes further by deduplicating the data before encrypting it to minimize the transfer and storage costs.

AWS S3-IA is available at all AWS regions, whereas Azure CBS is currently available in only some of the Azure regions: South America, Western US, Australia, and Singapore are missing out.

It’s possible to reduce the frequency-access-tier of an AWS S3 object (fee per request and must have existed for 30 days at current tier) or an Azure Hot Blob Storage account (free). AWS does offer an automatic lifecycle management tool. For many of CTERA’s on-prem server/endpoint backup customers, it would be best to back-up directly to the lower-frequency-access tier; whereas for cloud server backup scenarios it might be best for CTERA to create a higher-frequency-access snapshot for replication purposes and then migrate that to a lower-frequency-access tier.

Whether AWS S3-IA or Azure CBS provides more effective pricing for you should be determined on a case-by-case basis. But it’s certain that both further lower the cost of using their object storage for backup. CTERA users should consider which public cloud service offers the right capabilities and cost for use as a backup target and long-term retention.

AWS region image source: Amazon.com