Provisioning Storage in GCP

GCP
by Scott Lapish

When provisioning storage in GCP, careful consideration needs to be given to object versioning and lifecycle management. Costs can quickly balloon if multiple obsolete versions of objects are being stored with no consideration to the number of versions required and to how to manage the lifecycle of those object versions through the various storage classes (Nearline, Coldline, Archive) and multiple regions.

A team new to Google cloud developing a large scale data warehouse system had missed changing the object versioning strategy on a set of data that was generated during ingestion. While they thought they were cleaning up old objects by deleting them, based on the versioning strategy, these objects were not actually removed resulting in continued ballooning costs attributed to a large and continually growing volume of unwanted data. After adjusting for the desired behaviour the costs in the non-production environment dropped by 60%.

Related Member Stories

A Guide for Adopting FinOps in Your Organization

AWS
Azure
GCP
Industry: Internet
Persona: FinOps Practitioner
by F2 Working Group, FinOps Foundation

One of the biggest challenges in starting a FinOps practice is getting broad executive support and buy-in to dedicate the time and resources needed for the cultural change.

Read more

Architecting Cloud Workloads for Financial Reporting

AWS
Azure
GCP
Industry: Information Technology & Services
Persona: FinOps Practitioner
by Rich Hoyer, SADA

A list of best practices for cloud architects to design systems to optimize FinOps.

Read more

Runaway Cost in BigQuery Capacity Commitments

GCP
Industry: Telecommunications
Persona: FinOps Practitioner
by Scott Lapish, Telus

Failure to purchase org level capacity commitments for BigQuery can result in runaway costs due to on-demand query costs. Purchasing an org level capacity commitment and enabling idle capacity at the org level can ensure stable BigQuery costs across the organization. Consideration also needs to be given to whether the...

Read more