This work is licensed under CC BY 4.0 - Read how use or adaptation requires attribution

Resource Utilization & Efficiency

Help by contributing content for this capability

Community collaboration is at the heart of the Foundation, why not create the initial content for this page so others can benefit and build onto it?

Definition

In context to FinOps, resource utilization is about making sure you are getting sufficient business value for the cost. It is necessary to observe a resource’s utilization over time to understand if the performance, availability or other quality metrics are of value for the expense incurred.

For example, there may be times when it is deemed that for performance or availability gains, average utilization may need to decrease and the extra expense incurred is worth the value creation the resource provides. Or the opposite may be true and performance expectations can be lowered to improve cost. For these decisions to be made, resource utilization, efficiency and cost must be looked at together.

The management of resource utilization and efficiency translates into identifying whether there is scope to reduce resource costs while maintaining the required performance and, if there is, making the changes required where it is economically worthwhile to do so.

Maturity Assessment

Crawl

  • Has some visibility of resource utilization using one or more sources such as cloud billing data, infrastructure monitoring tools, cloud provider insights/tools.
  • Define a business efficiency metric - i.e. a metric that speaks to your business that can be used to measure how efficient a resource is

Walk

  • Is able to put a $ value against costs that can be avoided by rightsizing underutilized resources.
  • Measure the cost required in performing the action - i.e. it’ll cost 50 man hours to make this change at an hourly figure of xyz.
  • Takes manual action to review recommendations and take appropriate action(s) to increase efficiencies.

Run

  • Uses cost and utilization data to drive automated processes to either:
  • Alert humans to analyze. Outcomes could be to update the architecture/sizing of resources deployed or suppress the notifications for these resources as there are good reasons for running at levels that look suboptimal on the surface.
  • Resize or stop/start the resources

Functional Activity

As someone in a Business/Product role, I will…

  • Clearly define service KPIs so that engineering are able to design efficient services within the defined boundaries
  • Provide demand forecasts and information on the demand pattern profiles (daily/weekly/monthly/cyclic)
  • Establish the business goals for the objective - i.e. release to customers as quickly as possible, release to customers w/ an availability of 99.99%, etc. (aka Business Value Creation!)

As someone in a Finance/FinOps role, I will…

  • Highlight any opportunities to increase utilization and efficiency and work with the teams to review feasibility of alternative options
  • Help create the reporting to track and report on the impact on value of underutilization and inefficiencies
  • Partner with the Engineering organization to establish budgetary & efficiency targets

As someone in an Engineering/Operations role, I will…

  • Architect services with the KPIs and forecasts guiding decisions
  • Use elasticity best practices to automatically scale resources with the workload demands
  • Build automation to output measure and metrics needed to measure utilization and efficiency
  • Constantly regularly review utilization of resources

As someone in an Executive role, I will…

  • Deliver the business value creation vision and strategy
  • Provide executive level support in the defined KPIs, establishing credibility in the FinOps efficiency program

Measure(s) of Success

at least one measure of success; should be described in a context of cost; this could be an efficiency KPI or an agreed upon threshold or target. for example:

  • idle resource costs will not exceed 3% of total monthly cloud spend
  • anomaly costs will not exceed $150/month

Inputs

the information used that contributes to the measure(s) of success listed above; information here may include specific datasources, reports or any relevant input

Real World Resources

Runaway Cost in BigQuery Capacity Commitments

GCP
Industry: Telecommunications
Persona: FinOps Practitioner
by Scott Lapish, Telus

Failure to purchase org level capacity commitments for BigQuery can result in runaway costs due to on-demand query costs. Purchasing an org level capacity commitment and enabling idle capacity at the org level can ensure stable BigQuery costs across the organization. Consideration also needs to be given to whether the...

Read more

DynamoDB Backups Gone Wild

AWS
Industry: Consumer Goods
Persona: Engineering & Operations
by AJ Bothe, Proctor & Gamble

I recently joined a team and one of our accounts was spending an insane amount on DynamoDB (close to $20k / mo) for a set of tables that was under 20GB. Come to find out we had a process to backup the tables every hour of every day with no...

Read more

Eliminating Waste in BigQuery

GCP
Industry: Information Technology & Services
Persona: FinOps Practitioner
by Rich Hoyer, SADA

BigQuery (BQ) is a powerful tool compared to traditional hosted database management systems. Because BQ is serverless, has high reliability and speed, and offers on-demand querying services, among other features, the tool is top-rated. The pricing structure of BiqQuery is very different from competitors’ offerings, however. Rather than charging for...

Read more

Managing EBS Snapshots

AWS
Industry: Computer Software
Persona: FinOps Practitioner
by Eric Mulartrick, Dell Boomi

AWS EBS (and other services) snapshots are backups of a system at a current state. Without using a policy, snapshots will continue to grow and grow in quantity. Before you know it, you could be sitting on thousands of unused snapshots (waste?) that have no business value. Do you need...

Read more

The Case of too Many EBS Volumes

AWS
Industry: Financial Services
Persona: FinOps Practitioner
by Joe Daly, Nationwide

Forgetting to delete EBS volumes is extremely common for engineers just starting their cloud journey. At a small scale, they accumulate slowly and most third party cloud cost management platforms can identify and even terminate them for you. However, all it takes is one buggy autoscaling group that doesn’t properly...

Read more

When Do Cloud Savings Start to Kick in?

AWS
Azure
Industry: Internet
Persona: FinOps Practitioner
by Peter Shi, Goldiserv

In this post I’ll share why cloud can both save, and not save, depending on your stage of cost optimization maturity. For illustration purposes, the stages of maturity in this post are split into crawl, walk, and run.

Read more

Migrating 15 Million Lines of Code from Six Data Centers Worldwide in Six Months

GCP
Industry: Internet
Persona: FinOps Practitioner
by Scott Snell, OpenX

Our initial challenge: migrating 15 million lines of code from six data centers worldwide in six months. Our team immediately saw a contrast in how to optimize cloud finance and solved this challenge by focusing on building strong governance, cost visibility, and cost optimization policies.

In our original cloud model:...

Read more

FinOps Platforms & Service Providers

Reference of cloud cost management platforms, tooling and service providers related to this Capability coming soon.

FinOps Training

Reference of courses and training partners related to this Capability coming soon.

Get Involved

Get involved and contribute to the community by sharing your real world experiences related to this Capability in the form of a story or providing a playbook for how you have implemented best practices in your organization. Your real world experiences can be provided in the context of:

  • one or more cloud providers
  • the types of cloud services used (compute, storage, database, etc…)
  • describe a combination of tooling, platform or vendor, and processes including KPIs
  • the industry the organization belongs to
  • the complexity of the organization (global enterprise, start-up, etc…)
  • the FinOps personas involved / organizational roles

Join the conversation about this Capability in the Slack channel #wg-resource-utilization-efficiency . You can submit stories, how-tos and suggest improvements using one of the options for contributing here.