Planning & Estimating

Framework / Domains / Quantify Business Value / Planning & Estimating

Estimation and exploration of potential cost and value of workloads if implemented in an organization’s cloud environment in a particular model or models.

Explore scenario(s) in the cloud

Define scope of estimate
Define detail required for estimate
Model parameters for future state

Estimate cost for defined scenario(s)

Use calculators and estimation tools
Compare to similar applications and systems
Extrapolate from past costs
Document expected impact of future plans
Estimate using trial runs
Incorporate pricing, policy, shared services, and other support costs

Implementation plan

Plan Proof of Concept
Make recommendations to adjust forecasting models

Definition

Because of the variety of services available in the cloud, frequent updates, new services, managed services, and the variety of models in which applications can be built in or migrated to the cloud, a robust set of practices is required to be able to estimate the future costs of a workload or system. Estimation can be done for any scope from a single service change to an entire application migrating to cloud from the data center. Oftentimes, multiple estimates will be made to compare future costs under a variety of scenarios.

Estimating is primarily supported by Engineering personas, supported by FinOps teams. Input from Product, Finance or Leadership may be required when estimates are particularly important, impactful or complex; or when trial budget might be required to estimate.

Planning & Estimating is closely related to Forecasting. Estimating is done to understand what potential future costs might be under various scenarios or use cases, in order to create a plan for migration, implementation, or modernization. Estimates will be an input to Forecasting, where a more detailed forecast model for the planned changes will be created and maintained. Forecasting represents anticipated spending and value creation an engineering or product team will be responsible to deliver.

By contrast, Planning & Estimating is exploratory ideation. It will produce inputs to cloud cost forecasts, but also for other reasons. Estimating is performed frequently in support of Optimize Cloud Usage & Cost domain activities, like Architecting for Cloud, Workload Optimization, or even Onboarding Workloads.

When estimating future cloud costs, organizations should define the scenarios which are appropriate to estimate for. This includes understanding the service(s), architectures or other changes that should be estimated, the build patterns used by the organization, and the parameters of the estimate that will be important to communicate to others. A variety of scenarios might be created for a specific change. For example, an engineering team might estimate the impact of moving a workload from a virtual machine to a managed service, or to a Kubernetes environment, or to a serverless compute model, looking at the cost, effort, and impact of each for comparison.

A variety of techniques are available to estimate costs in cloud, including:

Cost Calculators – provided by most cloud providers and some third parties, allow teams to estimate the cost of anticipated services used. Calculators work well to estimate simple service substitutions, but do not work as well when estimating whole applications or environments which do not currently exist in the organization’s cloud environment.
Similar Applications – for organizations which have consistent architectures or applications, new systems may follow similar patterns, giving engineers a way to compare the cost and performance of similar systems to at least start the estimate for a new change or workload.
Extrapolation from past cost – estimating a change in an existing system may be easier by isolating the individual service changes or usage volumes and extrapolating from current costs. This works well when estimating isolated service changes in large systems.
Expectations of future plans – using the variable use model of cloud allows organizations to save money by not implementing in cloud until such time as resources are actually needed. Larger scale estimating of whole systems may need to incorporate the phased implementation of all of the resources and environments required. For example, a new system being created in the cloud might have development and testing environments launched first, but production environments launched several months later when development nears completion. The inclusion of future plans and future implementation over time should be incorporated into these estimates rather than the entirety of all environments all at once.
Trial-run estimation – Because cloud services can be created and then removed quickly, often using infrastructure-as-code scripting, it is possible to estimate the cost of an environment accurately by actually creating it in the cloud, allowing it to run for some short period of time (1-3 days, perhaps) and then removing the environment. This trial run creates a small amount of cost, which must be accounted for from R&D or previously budgeted funds, but can provide a very realistic estimate, in addition to testing of the scripting to create the environment.

In all these cases Engineering personas should work with FinOps teams to ensure that estimates adhere to policies (where resources should be created, what types of resources are used, appropriate architectural models, etc.), that pricing estimates are appropriate (on-demand pricing, discounted rates, expected types of commitment levels, etc.), and that scenarios include estimates of shared costs, platform adjustments, or other support costs and impacts. These impacts may need to include both financial cost and other elements such as sustainability impact or operational impacts of making the considered change.

Estimating scenarios can then be used to provide input back to the Forecasting process, or to the Optimization process that triggered the estimating work. If Proof of Concept budget is needed to estimate, or get more specific cost information, Finance may be involved to provide that.

Unfortunately, there is no one estimating method that fits all situations. Cloud spend is variable which is inherently difficult to predict, and Engineers can create environments and workloads at any time, typically without having to go through a procurement process. This is why it is important to have an established Estimating capability with well-understood parameters, scenario planning, tooling, and documentation expectations.

Maturity Assessment

Crawl

Simple or consistent application build patterns are used, requiring fewer or more straightforward estimating techniques
Estimates are not required to be extremely granular or extremely accurate, perhaps because the organization’s cloud spend is low or immature
A variety of cloud cost data-sources and tools are used for forecasting by stakeholders across the organization, and consistency is not required
Estimates are documented manually and/or ad-hoc due to low volumes or simple scenario planning
Higher reliance on trend-based, extrapolation than on more complex mechanisms
Limited need for aggregate estimate visibility (perhaps by business unit or cost center)
Few shared costs or added cost elements required to complete estimates
Engineering teams are involved with the creation of estimates but little need to track discrepancies against actuals

Walk

Wider variety of application build patterns used, or complex cloud use, requires more estimating techniques to be used
Estimates are more granular, at the service level, and higher accuracy of estimates is required organizationally
Estimates are inclusive of cloud rate optimization and commitment-based discounts
Estimates are done on a regular cadence, documented consistently, but not automated
Stakeholder teams (Product, Leadership, Engineering, Finance) have access to cloud cost estimate data
Regular review cadence by FinOps team of estimate accuracy, and trends with stakeholder teams

Run

Complex and large scale collection of build patterns and cloud use requires complex estimating
Estimates are extremely granular and include several types of shared cost elements, platform costs, support costs, and policy compliance elements
Estimates are created using a variety of pricing models, discount-adjusted, amortized cloud usage pricing
Estimating scenarios support rolling, trend-based and driver-based Forecasting capability
Estimates are inclusive of usage optimization opportunities
Estimates aligned to the organization’s allocation constructs being used across the organization for reporting cloud costs
Granular estimating visibility (by business unit, cost center, team, product, service, etc …) in the context of organizational KPIs
Stakeholder teams (Product, Leadership, Engineering, Finance) have real-time visibility into a single source of truth for how cloud usage is impacting estimates, forecasts, trends, and budgets
integration and automated data flow between cloud cost forecast data and back-end accounting systems used for broader organizational reporting

Functional Activities

FinOps Practitioner

As someone in the FinOps team role, I will…

Establish requirements and parameters for estimating done by product and engineering teams
Generate granular estimates with reasonable accuracy
Help to establish estimating KPIs that are in alignment with business goals
Provide estimates of cloud costs to enable stakeholders to create forecast models
Participate in implementation planning and scenario estimating pricing, policy, shared cost, and other impacts
Provide granular reporting to teams on estimated spend by different business-centric dimensions
Manage collaboration with Allied persona groups and other stakeholders throughout estimation process
Provide reporting on estimate accuracy trends

Product

As someone in a Product role, I will…

Track product centric KPIs to measure the cost effectiveness in achieving my desired business outcomes
Use these KPIs to inform estimating scenarios in collaboration with my engineering counterparts
Establish estimating parameters that are in alignment with the product-line that i own

Finance

As someone in a Finance role, I will…

Establish requirements for estimate accuracy and level of detail
Help to establish estimating KPIs that are in alignment with business goals
Provide data for cloud costs to enable stakeholders to create forecasts to allow me to create budgets
Provide granular reporting to teams on forecasted spend by different business-centric dimensions
Provide reporting on budgets vs actuals vs forecast vs estimates to establish trends and compare against variance KPIs
Provide access to trial funding when appropriate to create more accurate estimates

Engineering

As someone in an Engineering role, I will…

Monitor for cloud spend forecasts exceeding budgets to trigger estimating work
Monitor optimization, onboarding, architecture for triggers to estimate new resources
Work with FinOps and Product personas to create meaningful estimating scenarios in each case where they are called for
Work to understand the various estimating scenarios I should consider in each case
Get approval for planned changes which impact cloud spend projections / budgets
Lead estimating work in areas under my control and responsibility
In cases where forecast models are in place, understand when new estimates are required, and how those estimates will impact existing forecast models I’m responsible for

Leadership

As someone in a Leadership role, I will…

Be aware of cloud cost and impact estimation processes and support engineering personas in that work
Establish estimating threshold variances that are in alignment with business goals
Manage competing priorities for active project timelines when estimated spend impacts budgets for high-priority initiatives
Foster successful communication between the FinOps team and Business Units
Manage tradeoffs and planning decisions when comparisons are required between different estimated scenarios

Measures of Success & KPIs

Estimating models leverage discount-adjusted, amortized cloud usage data
Estimate cost vs actual cost trends within established percentage threshold of variance.
Estimates are conducted quickly and with appropriate mechanisms that tie to business objectives
Estimates include shared costs, appropriate pricing metrics, sustainability impacts, and other appropriate elements
Meeting Cadence established (time specific) – ??
Unit costs specific to your company are established (using PPAs, EDPs, etc) – assists in forecasting

Inputs & Outputs

Inputs

Triggers to perform estimating from Workload Optimization
Triggers to perform estimating from Rate Optimization
Triggers to perform estimating from Reporting & Analytics
Triggers to perform estimating from Onboarding Workloads
Triggers to perform estimating from Forecasting requirement
System or workload details as currently built
System or workload functional and non-functional requirements in to-be location, model
Scenario details to be modeled
Desired state of system or workload, or future plans for modification
Current architectural guidelines (Architecting for Cloud)
Parameters of the estimate desired (time frame, level of accuracy, etc.)

Outputs

Scenario cost and delivery estimate(s) for the system or workload
Recommendations based on estimates among competitive scenarios
Likely risks and level of confidence

Filter:

Framework

Planning & Estimating

Definition