I’ve been working through some cost optimization in our cloud environment and am currently focused on Object Storage, tiers and lifecycle management and wanted to post my collective thoughts and observations around it.\u00a0 You can find more in depth analysis in the sources listed at the bottom of the post.<\/p>\n
The 4 storage tiers in Azure Blob Storage are Hot, Cool, Cold and Archive.\u00a0 Hot, Cool and Cold are all online tiers, meaning the data is available instantly.\u00a0 Archive is an offline tier, and retrieval time can take a few hours.<\/p>\n
Performance characteristics across Hot, Cool and Cold are very similar, so the use case for what tier your objects belong in is largely determined by access patterns, specifically how frequently the data will be accessed, how long it needs to be accessible and how long you plan to keep it accessible.\u00a0 Cool, Cold and Archive all have minimum retention periods.\u00a0 30 days for Cool, 90 days for Cold and 120 days for Archive.\u00a0 Deleting or moving objects outside of a tier before it’s retention period will carry an early deletion penalty.\u00a0 Cost is calculated both by how much it costs to store objects and read\/write operations.\u00a0 It costs the most to store objects in Hot but read\/write operations are the least, whereas in Archive, storage costs are the cheapest but read\/write operations is the most expensive.<\/p>\n
Migrating objects in or out of a tier costs money via read\/write operations and that cost can be significant.\u00a0 For example, I’ve seen storage accounts with a default tier of cool, but if we migrated those objects to cold we could save $300k\/year, but to move that data into cold would cost approximately $200k, meaning you’d need basically 3 quarters to see an ROI.\u00a0 For this reason it’s extremely important that the default storage tier of a storage account is appropriate for your use case.\u00a0 You can only set the default storage tier at the storage account level, not the container level, meaning that if you do have different use cases for different containers, it could be worthwhile to keep them in a different storage account with a different default tier. At the biotech company I currently work for, we store very large amounts of genetic sequencing data.\u00a0 We have to keep this data around for something like 7 years per FDA regulations.\u00a0 Once this data is written into object storage, it’s accessed via a pipeline, which takes that data and produces clinical results.\u00a0 After that, sometime in the not so distant future, those results are statistically validated by a biostats team.\u00a0 Because this data is infrequently accessed but needs to be available, the cool storage tier is our default tier, with a lifecycle management policy that might move it to archive at 180 days or 1 year.<\/p>\n
Sources: <\/p>\n","protected":false},"excerpt":{"rendered":" Synopsis I’ve been working through some cost optimization in our cloud environment and am currently focused on Object Storage, tiers and lifecycle management and wanted to post my collective thoughts and observations around it.\u00a0 You can find more in depth analysis in the sources listed at the bottom of the post. Storage Tiers The 4…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[45],"tags":[],"class_list":["post-84","post","type-post","status-publish","format-standard","hentry","category-azure"],"_links":{"self":[{"href":"https:\/\/inbaudwetrust.com\/wp\/index.php?rest_route=\/wp\/v2\/posts\/84","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/inbaudwetrust.com\/wp\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/inbaudwetrust.com\/wp\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/inbaudwetrust.com\/wp\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/inbaudwetrust.com\/wp\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=84"}],"version-history":[{"count":5,"href":"https:\/\/inbaudwetrust.com\/wp\/index.php?rest_route=\/wp\/v2\/posts\/84\/revisions"}],"predecessor-version":[{"id":90,"href":"https:\/\/inbaudwetrust.com\/wp\/index.php?rest_route=\/wp\/v2\/posts\/84\/revisions\/90"}],"wp:attachment":[{"href":"https:\/\/inbaudwetrust.com\/wp\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=84"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/inbaudwetrust.com\/wp\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=84"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/inbaudwetrust.com\/wp\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=84"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\nhttps:\/\/learn.microsoft.com\/en-us\/azure\/storage\/blobs\/access-tiers-overview<\/a>
\nhttps:\/\/learn.microsoft.com\/en-us\/azure\/storage\/blobs\/access-tiers-best-practices<\/a><\/p>\n