Token Economics: Managing AI Value in SaaS Model Token Costs

Executive Summary

Unlike cloud compute or SaaS seat licenses, SaaS-Model API consumption of AI tokens operates on a different economic model. Organizations pay per token, a unit of measurement that most finance and business stakeholders have not previously encountered and that defies easy budgeting with traditional tools. Tokens are an abstraction of both cost and value, but provide one of the only concrete and portable mechanisms to account for either.

The FinOps Foundation’s practitioner survey identified managing the cost and use of tokens in SaaS-model AI as the top challenge facing practitioners today. The root causes are structural: developer-led purchasing, opaque billing, no native allocation mechanisms, and pricing models that vary dramatically across model tiers and use cases.

FinOps teams can build the knowledge and frameworks needed to bring this spending under control. It begins by mapping the AI procurement landscape and explaining why direct model provider APIs represent the hardest category to manage. It then builds from token economics fundamentals through a complete cost management framework covering visibility, allocation, optimization, and governance.

Key Recommendations

• Establish API key governance immediately: this is the foundational control layer for everything else.
• Deploy a proxy or observability layer to inject tagging and allocation metadata that providers do not supply natively.
• Implement unit cost metrics (cost per query, cost per user, cost per workflow) to make token spend legible to business stakeholders.
• Right-size model selection: most workloads do not require frontier models; routing to smaller, cheaper models is the single highest-impact optimization.
• Engage procurement and legal early: model provider contracts, volume commitments, and enterprise agreements require the same rigor as any major vendor relationship.
• Build toward a maturity model: organizations that treat AI cost management as a crawl-walk-run journey will reach sustainable practices faster than those attempting comprehensive governance from day one.

How Organizations Buy AI Today

Before focusing on the hardest category to manage, it is worth mapping the ways organizations acquire AI capabilities. There are five primary procurement models, each with distinct cost structures, visibility characteristics, and FinOps implications.

SaaS Model Provider APIs

Organizations access foundation models directly from the companies that build them: Anthropic, OpenAI, Google (Gemini), Cohere, Mistral, and others. Access is via API, billing is consumption-based per token, and accounts are typically created by individual developers with a credit card before any procurement team becomes aware.

The appeal is immediate access to state-of-the-art models, no infrastructure to manage, and pricing that starts at zero. The risks are no native cost allocation, rapid model proliferation, and the potential for spend to scale faster than any other category in the technology portfolio.

Cloud Hyperscaler Marketplaces

AWS Bedrock, Azure OpenAI Service, and Google Vertex AI Model Garden wrap model API access inside existing cloud billing constructs. The underlying token pricing is often similar to or identical to direct provider pricing, but the spend flows through a cloud account the organization already manages, has tagging infrastructure for, and may have committed spend against through an EDP or MACC.

For FinOps teams, this is a significantly easier category to govern: the billing data lands in existing tools, reserved capacity can sometimes be applied, and the organizational muscle for cloud cost management transfers directly. The tradeoff is potential model availability lag and a dependency on the hyperscaler’s integration choices.

Self-Hosted and Open-Source Models

Organizations with sufficient ML engineering capacity can run open-weight models such as Meta’s Llama family, Mistral, Falcon, or Qwen on their own infrastructure, whether in a cloud VPC or on-premises. In this model, there is no per-token charge. Cost is expressed entirely in compute: GPU instance hours, storage, and networking.

This shifts the FinOps challenge from token management back to familiar cloud compute territory, but introduces new complexity: GPU right-sizing, utilization optimization across model serving infrastructure, and the total cost of the ML platform team required to operate it. For most organizations, this model only makes economic sense at significant scale or where data sovereignty requirements preclude sending data to external providers.

Embedded AI in SaaS Products

Microsoft 365 Copilot, Salesforce Einstein, ServiceNow Now Assist, and dozens of other enterprise SaaS products now bundle AI capabilities into seat-based licenses or consumption add-ons. In these products, token economics are abstracted entirely: the organization pays a per-seat fee or a platform-level add-on, and the vendor manages model selection, infrastructure, and token cost internally.

From a FinOps perspective, this is the most familiar model (it looks like any other SaaS contract) but introduces its own challenges: measuring value delivered per seat, ensuring adoption justifies license cost, and governing which embedded AI features employees are using and for what purposes.

AI Developer Tools

AI coding tools (Cursor, GitHub Copilot, Windsurf, Claude Code, OpenAI Codex) are now a material category of AI spend in most engineering organizations and frequently rival direct API spend within the first year of adoption. They do not fit cleanly into the four models above.

Two billing architectures coexist, with very different FinOps implications.

• In seat plus usage (Cursor, Copilot, Windsurf), the vendor mediates all model API calls and the organization pays the vendor. Cost visibility is limited to what the vendor exposes, and request-level metadata injection is not possible.
• In bring-your-own-key (Claude Code, Codex), the tool acts as a client and calls the model provider directly using the organization’s API key. Spend lands in the direct-provider billing relationship, and the attribution and governance levers described elsewhere in this paper apply.

Some tools support both modes. Claude Code can run on an Anthropic subscription or on a direct API key. Codex can run on a ChatGPT plan or in API key mode. The same product therefore appears in either billing architecture depending on how it is deployed, and the FinOps treatment changes accordingly.

Adoption typically follows a viral pattern: a few developers try the tool, productivity gains spread by word of mouth, and within months the engineering organization is using it at scale. Spend follows the same curve. Visibility does not.

Procurement Model Comparison

Procurement Model	Cost Unit	Billing Visibility	FinOps Tooling Support	Allocation Difficulty	Scalability Risk
Direct Model Provider API	Per token	Provider dashboard only	Minimal native; proxy required	High	Very High
Cloud Hyperscaler Marketplace	Per token (via cloud bill)	Cloud billing tools	Good (existing cloud tools)	Medium	Medium-High
Self-Hosted / Open-Source	Compute (GPU hours)	Cloud billing tools	Good (standard compute)	Low	Low-Medium
Embedded SaaS AI	Per seat or platform fee	SaaS invoice	Standard SaaS governance	Low	Low
AI Developer Tools	Per seat plus usage, or per token (BYOK)	Vendor admin tools, or direct provider dashboard	Mixed: standard SaaS for seat-priced, full proxy stack for BYOK	Medium-High (mode-dependent)	Medium-High

Why SaaS Model Provider APIs Are Hard to Manage

Why do FinOps practitioners consistently identify direct model provider APIs as the most difficult category? The answer is not a single problem but a confluence of structural characteristics that undermine every traditional cost management mechanism simultaneously.

Opaque Billing Without Native Attribution

When an organization receives an invoice from OpenAI or Anthropic, it typically shows aggregate token consumption across the account, broken down at most by API key or project. There is no native concept of business unit, cost center, application, or workload. The data that FinOps teams need to perform showback and chargeback does not exist in the provider’s billing export unless the organization builds the instrumentation layer itself.

Developer-Led Purchasing Bypasses Procurement

Model provider accounts are designed to be opened by individuals with a credit card. A developer can be calling the GPT-4o or Claude 3.7 Sonnet API within minutes of deciding to experiment, with no procurement gate, no security review, and no cost estimate attached. By the time Finance sees the first invoice, the application may already be in production. This is the same shadow IT dynamic that plagued early cloud adoption, but with a faster onramp and less institutional awareness of the risk.

Rapid Model Proliferation Complicates Comparisons

Model providers release new models continuously, each with distinct pricing structures. A team that benchmarked and budgeted for GPT-4 may discover that GPT-4o, GPT-4o-mini, o1, and o3-mini all have different price-performance characteristics and that the right model for a given workload changes every few months. Keeping unit economics current requires ongoing model evaluation that most teams do not have capacity for.

Usage Spikes Are Difficult to Predict and Cap

Unlike a server that costs a predictable amount per hour, token consumption can spike dramatically based on user behavior, prompt design, or application bugs. An agentic workflow that loops unexpectedly, a system prompt that was inadvertently doubled in length, or a viral feature that drives 10x the expected usage can all produce invoices that exceed monthly budgets in a single day. Native rate limits exist but are expressed in tokens-per-minute, not dollars-per-month.

The Token as a Foreign Unit of Measure

Finance teams, business stakeholders, and even many senior engineering leaders have no intuition for what a token costs, how many tokens a typical request consumes, or how to translate a token budget into a business outcome. This creates a communication gap that makes it difficult to set meaningful budgets, evaluate ROI, or justify spend to leadership without significant translation work.

Output Tokens Cost More, and Are Harder to Control

Most providers charge more for output tokens than input tokens, sometimes by a factor of three to five. Output length is influenced by prompt design, model instruction, and user behavior but is ultimately determined by the model at runtime. A significant portion of cost is generated by a system the organization does not directly control, and small changes in model behavior across versions can shift costs materially.

Token Economics 101 for FinOps Teams

Effective cost management of model provider APIs requires a working understanding of how token pricing works at a mechanical level. This section provides the foundational knowledge FinOps practitioners need to communicate with engineering teams, evaluate invoices, and design effective governance. This is current as of May 2026. Token economics, or Tokenomics, changes rapidly.

What Is a Token?

A token is the basic unit of text that language models process. Roughly speaking, one token corresponds to approximately four characters of English text, meaning that 1,000 tokens is approximately 750 words. Tokenization is model-specific and varies by language: code, technical terminology, and non-English languages often tokenize less efficiently, consuming more tokens per character.

Every API call involves both input tokens (the text sent to the model, including the system prompt, conversation history, and user message) and output tokens (the text the model generates in response). Both are billed, typically at different rates.

Input vs. Output Pricing

All major providers charge separately for input and output tokens. Output tokens consistently cost more, reflecting the additional compute required to generate text compared to processing it. Price ratios between input and output vary by model and provider but commonly fall in the range of 1:3 to 1:5. For cost modeling purposes, understanding the input/output ratio of a specific application is essential: a chatbot that generates long responses has a very different cost profile than a classification API that returns a single word.

The Context Window and Its Cost Implications

Language models process all input within a context window, which defines the maximum amount of text the model can consider at once. For multi-turn conversations, the entire conversation history is re-sent with each API call. Costs grow with conversation length: a ten-turn conversation may cost ten times as much per turn as a single-turn query, because each subsequent turn includes all prior turns as input.

Agentic applications that retrieve documents, browse the web, or accumulate tool outputs are particularly susceptible to context window cost explosion. A single agent run that fills a 128K token context window with retrieved documents will cost orders of magnitude more than a simple query-response exchange.

Model Tier Pricing

Every major provider now offers a tiered model portfolio: frontier models for the most demanding tasks, standard models for everyday work, and mini or nano variants optimized for cost and speed on simpler tasks. Price differences across tiers can be dramatic: a frontier model may cost 50 to 100 times more per token than the smallest available model from the same provider. Selecting the appropriate model tier for each workload is one of the highest-leverage optimization decisions available.

Batch vs. Real-Time Pricing

Most providers offer a batch processing API for workloads that do not require a synchronous response. Batch pricing typically offers a 50% discount relative to real-time pricing. Workloads suitable for batch processing include data enrichment pipelines, document classification, content moderation, and any other high-volume task where latency is not critical. Identifying and migrating eligible workloads to batch APIs is often the quickest path to meaningful savings.

Prompt Caching

Several providers, including Anthropic and OpenAI, offer prompt caching mechanisms that allow repeated prefixes, such as long system prompts or frequently referenced documents, to be cached server-side and billed at a significantly reduced rate on subsequent calls. For applications with stable, lengthy system prompts, enabling prompt caching can reduce input token costs by 80 to 90% for the cached portion. This is a particularly high-value optimization for customer-facing applications with consistent system prompts across thousands of daily sessions.

Fine-Tuned Model Pricing

Fine-tuned models, which have been trained on organization-specific data to perform better on a specific task, carry both a training cost (charged per token processed during fine-tuning) and a higher per-token inference cost than the base model. Organizations considering fine-tuning should model the full economics: whether the performance improvement justifies both the training cost and the ongoing inference premium, compared to prompt engineering the base model to achieve similar results.

Pricing Lever	Typical Savings Potential	Implementation Effort	Applicability
Model right-sizing	60-90%	Medium	Most workloads
Batch API	50%	Low-Medium	Non-real-time workloads
Prompt caching	50-90% on cached tokens	Low	Stable system prompts
Context window management	20-60%	Medium-High	Conversational / agentic apps
Output length control	10-40%	Low-Medium	All workloads
Volume / commitment discounts	10-30%	Low (procurement)	High-volume accounts

See the How Token Pricing Really Works paper for a more summarized view.

Building a Token Cost Management Framework

The FinOps lifecycle of Inform, Optimize, and Operate applies directly to token cost management. This section builds a complete framework from zero visibility to active governance, covering tagging and attribution, showback and chargeback, budgets and alerts, anomaly detection, and benchmarking.

Tagging and Attribution

The foundational challenge is that model providers do not natively support the tagging structures FinOps teams rely on for allocation. An OpenAI or Anthropic invoice will show spend by API key or project, not by business unit, cost center, application, or team. Solving this requires a deliberate instrumentation strategy.

API Key Governance as the Minimum Viable Control

The simplest form of attribution is a disciplined API key structure. Each key should map to a single team, application, or use case, and key provisioning should require a named owner, a designated cost center, and an approved use case. This alone provides rough allocation data without any additional tooling, but it is the ceiling of what the provider will give you natively.

Provider-Native Attribution Features

Provider-native attribution capabilities have advanced significantly since 2024 and now sit between bare API key governance and a full proxy deployment.

• AWS Bedrock supports Application Inference Profiles, IAM Principal Cost Allocation, and Bedrock Projects, which together push tags into the Cost and Usage Report at the inference call level.
• OpenAI offers project-scoped API keys and a Costs API that returns spend by project and cost category.
• Anthropic provides workspace-level keys and an Admin API that exposes per-workspace usage.

For organizations standardized on a single provider, these primitives are often sufficient and avoid the operational overhead of running a proxy.

They do not, however, solve cross-provider attribution, custom metadata dimensions beyond the provider’s tag schema, or real-time enforcement of model and spend policies. The recommended sequence is native first, proxy when native is insufficient: a proxy is the right investment for multi-provider portfolios, for feature- or user-level attribution beyond what the provider exposes, and for organizations that need policy enforcement in the request path.

Proxy Layers for Rich Metadata

For granular attribution, organizations should consider deploying an LLM proxy or gateway between their applications and the model provider. Tools such as LiteLLM, Portkey, Helicone, and similar platforms sit in the API call path and allow organizations to inject arbitrary metadata (user ID, session ID, application name, feature flag, cost center) that is then available in the proxy’s logging and reporting layer. This approach enables workload-level attribution that is not possible through provider APIs alone.

• Each API call is tagged with the calling application, team, and environment (dev/staging/prod).
• Aggregate spend by dimension in the proxy’s analytics layer.
• Export tagged usage data to your data warehouse for integration with other cost data.

The Cost Surface Beyond the Model Call

Token attribution is a necessary but partial view of AI feature cost. In production deployments, particularly Retrieval-Augmented Generation and agentic architectures, the infrastructure surrounding the model call routinely represents 40 to 60% of total feature spend. This harness typically includes vector databases, embedding generation, reranker calls, orchestration runtime (such as Lambda, Fargate, or Step Functions), key-value and semantic caches, cross-region data egress, and observability ingestion. None of these surface in the model provider invoice, and most do not flow through the same proxy layer that handles token attribution. Practitioners building unit cost metrics should plan for two parallel attribution streams: token spend captured at the proxy, and harness spend captured from cloud and SaaS billing, joined on a shared application or workload identifier.

Cost-per-query figures that exclude the harness will systematically underreport and skew optimization priorities toward token reduction when the larger savings sit elsewhere.

Showback and Chargeback

Once attribution data exists, the next step is making token spend legible to business stakeholders. Raw token counts and dollar amounts are not sufficient: business leaders need unit economics that connect AI spend to business outcomes.

Recommended Unit Cost Metrics

• Cost per query or API call: the average cost of a single model invocation, useful for capacity planning.
• Cost per user per month: total AI spend divided by active users, useful for comparing against seat-based alternatives.
• Cost per workflow completion: total cost of a multi-step agentic process, useful for ROI calculation.
• Cost per business transaction: AI cost embedded in a broader transaction (e.g., cost of AI-assisted customer ticket resolution).

Publishing these metrics in a shared dashboard, ideally integrated with existing FinOps reporting, moves token management from an engineering concern to a shared business responsibility. Teams that can see their AI unit economics make more thoughtful optimization decisions than teams that receive a monthly invoice line item.

Budgets and Alerts

Setting meaningful budgets for token spend requires first understanding the cost profile of each application. A budget set without a baseline will be either too conservative (blocking legitimate usage) or too permissive (allowing runaway spend). The recommended approach is to instrument, measure for 30 to 60 days, establish a baseline, and then set budgets at 110 to 120% of baseline with alerts at 80% and 100%.

Provider-native budget tools typically allow spend alerts at the account level, which is useful as a backstop but insufficient for workload-level governance. Proxy and observability tools generally support more granular budget enforcement, including the ability to block or rate-limit specific API keys or applications when thresholds are reached.

Anomaly Detection

Token spend anomalies have several common root causes, each with different remediation paths:

• Runaway agentic loops: an agent that calls itself or tool chains that recurse indefinitely can consume millions of tokens in minutes. Detection requires monitoring tokens-per-minute at the application level, not just daily spend.
• Prompt injection or abuse: malicious inputs designed to extract long model responses or cause the model to perform unintended work can drive unexpected cost spikes in customer-facing applications.
• Context window accumulation: conversational applications that do not implement context pruning will see costs grow linearly with session length, producing anomalous patterns in long-lived user sessions.
• Model version changes: a deployment that inadvertently switches from a mini to a frontier model will produce a step-change cost increase that looks like a usage spike but is actually a pricing change.
• Development and testing bleed: developer testing in production environments or non-production accounts without spend controls is a common source of anomalous spend that has no business value.

Benchmarking and Rate Baselines

Without a reference point, it is difficult to know whether token costs are reasonable for a given application type. Organizations that have instrumented multiple AI applications can begin building internal benchmarks by application archetype:

Application Archetype	Typical Input:Output Ratio	Avg. Tokens per Session	Cost Benchmark (indicative)
Simple Q&A / RAG	3:1 to 5:1	500-2,000	Low
Multi-turn chatbot	4:1 to 8:1	2,000-10,000	Medium
Document analysis	10:1 to 20:1	5,000-50,000	Medium-High
Agentic workflow	2:1 to 4:1	10,000-200,000+	High-Very High
Code generation / review	3:1 to 6:1	1,000-8,000	Medium
Content generation	1:1 to 2:1	1,000-5,000	Medium

Optimization Levers: Reducing Token Spend Without Reducing Value

Cost optimization in the token economy is different from cloud compute optimization. There is no instance type to resize, no reserved capacity to purchase, and no utilization percentage to tune. Optimization operates at the level of how AI capabilities are designed, configured, and consumed.

Model Right-Sizing

Matching model capability to task complexity is the single highest-impact optimization available to most organizations. The instinct of many engineering teams is to use the most capable model available, reasoning that quality risk is greater than cost risk. In practice, the majority of enterprise AI workloads do not require frontier model capability and can be handled effectively by standard or mini tier models at a fraction of the cost.

Effective right-sizing requires a deliberate evaluation process: defining quality criteria for the task, testing candidate models against those criteria, and selecting the smallest model that meets the quality threshold. This evaluation should be repeated periodically, as smaller models improve rapidly and may become viable for workloads that previously required a larger model.

For organizations running multiple applications, model routing frameworks can dynamically select the appropriate model based on query characteristics: routing simple, high-confidence requests to a mini model and escalating ambiguous or complex requests to a frontier model. This hybrid approach can reduce average cost per query by 60 to 80% while maintaining quality for the requests that require it.

Prompt Engineering for Cost Efficiency

System prompts are processed on every API call, meaning that unnecessary length in a system prompt is a tax paid on every request. Organizations should audit system prompts across their applications for verbosity, redundancy, and instructions that could be expressed more concisely without reducing effectiveness.

Output length is equally important. By default, models generate responses of whatever length seems appropriate to the task. Explicit instructions to be concise, to respond in a specific format, or to limit responses to a defined length can substantially reduce output token consumption without degrading quality for most use cases. Structured output formats such as JSON can also reduce output tokens compared to natural language responses when downstream systems only need structured data.

Caching Strategies

Caching operates at multiple levels and is often the most accessible optimization for engineering teams because it requires no change to model selection or prompt design.

Prompt Caching

Provider-side prompt caching (available from Anthropic and OpenAI) allows stable prefixes to be cached server-side and billed at a reduced rate. Enabling this for applications with long, stable system prompts requires only a simple API parameter change and typically delivers 80 to 90% cost reduction on the cached portion of input tokens.

Semantic Caching

Application-side semantic caching stores previous model responses and returns cached results for semantically similar queries without calling the model API at all. Tools such as GPTCache and similar libraries embed incoming queries, perform a similarity search against cached query-response pairs, and return a cached response when similarity exceeds a threshold. For applications with repetitive or templated queries, cache hit rates of 20 to 50% are achievable.

Response Memoization

For deterministic or near-deterministic queries where the same input reliably produces the same output, traditional key-value caching of exact input-output pairs eliminates API cost entirely for repeated requests. This is most applicable to classification, extraction, or formatting tasks on fixed input corpora.

Context Window Management

For conversational and agentic applications, managing what goes into the context window is one of the most impactful and frequently neglected optimization areas.

• Conversation summarization: rather than including the full conversation history in each API call, summarize earlier turns into a compact summary that captures the essential context at a fraction of the token cost.
• Sliding window approaches: limit context to the most recent N turns, discarding older turns that are unlikely to be relevant.
• Retrieval-Augmented Generation (RAG): instead of including entire documents in context, retrieve only the relevant passages using a vector search and include only those. A well-implemented RAG architecture can reduce context token consumption by 80 to 95% compared to full-document inclusion.
• Tool output compression: agentic applications that retrieve data from external sources should compress or summarize tool outputs before passing them back to the model, rather than passing raw API responses that may contain extensive irrelevant content.

Batching and Asynchronous Processing

The 50% cost reduction available through batch processing APIs is one of the most straightforward optimizations available for eligible workloads. Any AI task that does not require a real-time response is a candidate: document classification, content moderation, data enrichment, report generation, and similar high-volume, latency-tolerant tasks.

Beyond the pricing discount, batching also provides more predictable cost profiles. A batch job with a defined input set has a calculable cost before it runs, enabling accurate budget forecasting in a way that real-time consumption does not.

Commitment and Volume Discounts

For organizations with predictable, sustained AI API consumption, commitment-based pricing offers meaningful savings. The specific mechanisms vary by provider and are evolving rapidly, but generally include prepaid credit packages that offer a discount over pay-as-you-go, enterprise agreements with negotiated rates for committed volumes, and throughput reservations that guarantee capacity and often include a pricing benefit.

Before pursuing commitments, model the break-even arithmetic explicitly. Provisioned throughput is purchased in capacity units (PTUs, throughput units, or scale tier units depending on the provider) and is billed continuously regardless of actual consumption. To compare against pay-as-you-go, normalize the reservation cost to cost per million tokens at the utilization rate the workload can realistically sustain. The result is not always favorable: for several current frontier and reasoning models, provisioned capacity costs more per token than pay-as-you-go even at 100% utilization, meaning the reservation is a performance and SLA purchase rather than a discount. The break-even utilization rate, commonly between 50 and 80% depending on the model and provider, should be calculated and documented before any commitment is signed. Where the provider supports it (Azure OpenAI natively, Vertex AI through request headers, AWS Bedrock through custom failover logic), spillover routes excess traffic to pay-as-you-go capacity instead of returning throttle errors.

Engaging in commitment discussions with model providers requires the same preparation as any enterprise software negotiation: a clear picture of current and projected consumption, an understanding of the cost of alternatives, and organizational authority to make a multiyear spending commitment. FinOps practitioners are well positioned to lead this analysis and bring it to procurement.

For more perspectives on Committed vs. On Demand pricing in AI tokens, see the Navigating GenAI Capacity Options paper.

Governance and Operating Model

Technology and tooling can address the measurement and optimization dimensions of token cost management, but sustainable governance requires clarity on organizational roles, processes, and policy.

Who Owns AI Spend?

The ownership question is often the first obstacle to effective AI cost governance, because token spend sits at the intersection of Finance (who owns budgets), Engineering (who makes architectural decisions), Security (who governs API access), and Procurement (who manages vendor relationships). Without a clear owner, each function defers to the others and nothing gets done.

Three models have emerged in practice:

• FinOps-led: the existing FinOps team or Cloud Center of Excellence extends its scope to include AI spend, treating model provider APIs as a new cost category alongside cloud compute and SaaS. This is the most common path for organizations with mature FinOps practices.
• Platform Engineering-led: a centralized AI platform team manages the infrastructure layer, including API gateway, cost observability, and model governance, with FinOps providing the financial reporting overlay. This model works well where engineering has the appetite and capacity to build and operate shared infrastructure.
• AI Center of Excellence: a dedicated cross-functional team owns AI strategy, tooling, and governance end-to-end. This provides the most coherent ownership but requires organizational investment that is only justified at significant AI maturity and scale.

The critical requirement regardless of model is a named individual who is accountable for AI cost visibility and who has the organizational authority to enforce governance decisions.

API Key Governance

API keys are the primary control surface for model provider access. Effective key governance encompasses the full lifecycle:

• Provisioning: keys should be issued through a request process that captures the requesting team, the intended application, the expected use case, and the designated cost center. Self-service key creation should be disabled wherever possible.
• Naming and tagging: keys should follow a naming convention that encodes the team, application, and environment, enabling attribution even without a proxy layer.
• Scope limitation: where providers support it, keys should be scoped to specific models or endpoints rather than granted full account access.
• Rotation: keys should be rotated on a defined schedule, and rotation should be automated where possible to prevent key hygiene from creating operational burden.
• Revocation: there should be a documented, tested process for immediately revoking a key in response to a suspected compromise, runaway spend event, or team offboarding.
• Audit: key usage should be audited regularly to identify orphaned keys (associated with discontinued applications), over-permissioned keys, and keys whose spend patterns have deviated from their declared use case.

Policy Guardrails

Governance policies should define the boundaries within which engineering teams operate autonomously. Effective policies are specific enough to be enforceable but general enough not to require constant revision as models and use cases evolve.

• Approved model list: define which models are approved for use in production, development, and experimental contexts. This prevents teams from defaulting to the most expensive frontier model without justification.
• Maximum context length by use case type: set guardrails on maximum tokens per request for different application categories, preventing inadvertent context window bloat.
• Data classification guardrails: define which data categories may and may not be sent to external model providers, aligned with the organization’s data classification framework.
• Agentic workflow review requirements: require architectural review and spend modeling before deploying autonomous agent workflows, which carry disproportionate cost and security risk.
• Expense threshold for procurement involvement: define a dollar threshold above which model provider spend must flow through a formal procurement process rather than a developer credit card.

Developer Enablement

Governance that slows engineering teams will be circumvented. The goal is to give developers the information they need to make cost-conscious decisions without adding friction that reduces their effectiveness.

Engineering & Operations Teams: Cost visibility in development is not a surveillance tool; it is a design aid. When you can see that a particular prompt design costs three times more than an alternative without affecting quality, the decision to optimize becomes obvious. Ask your FinOps team to surface cost data in the environments you already work in, rather than in a separate dashboard you will never open.

• Cost feedback in development environments: surface estimated cost of API calls during development so engineers can see the cost implications of their design choices before they reach production.
• Internal pricing guides: publish a clear, current reference of approved model pricing, including the cost differential between model tiers, to support design decisions.
• Optimization playbooks: document the organization’s preferred caching, batching, and model selection patterns in a developer-accessible format, reducing the need for each team to rediscover the same optimizations.
• Cost attribution in CI/CD: integrate cost estimation into pull request reviews for AI-intensive changes, making cost a first-class concern in the review process.

Procurement Integration

Model provider spend that exceeds trivial thresholds should be treated with the same rigor as any material vendor relationship:

• Vendor risk assessment: model providers handle potentially sensitive data and are critical dependencies for production applications. They should be assessed under the organization’s standard third-party risk framework.
• Contract review: provider terms of service, data processing agreements, and acceptable use policies should be reviewed by legal before production deployment at scale.
• Spend visibility in procurement systems: model provider invoices should flow through accounts payable with proper cost center coding, not be paid on personal or team credit cards.
• Renewal and negotiation calendar: enterprise agreements with model providers have renewal cycles. Procurement should track these and initiate renegotiation conversations with sufficient lead time to leverage alternatives.

Tooling Ecosystem

The tooling landscape for AI token cost management is evolving rapidly. What follows is a snapshot of the categories available and their relative strengths, organized from provider-native tools through third-party observability platforms to the emerging FinOps platform integrations.

This section covers tooling specifically for API token tracking, but for a broader perspective on tooling in the AI space see the FinOps for AI Tools and Services Considerations paper.

Data, Analytics & AI Teams: You are likely the team that will evaluate, build, or integrate the tooling other personas depend on. Pay particular attention to the proxy platforms and build-versus-buy sections. Your data pipeline and warehouse capabilities are an asset here: the organizations with the richest AI cost insights are typically those that route token usage data through their existing analytics stack.

Provider-Native Dashboards

Every major model provider offers a usage dashboard within their web console. These dashboards provide aggregate spend, token consumption over time, and basic breakdowns by API key or project. For organizations just beginning to instrument their AI spend, these dashboards are a useful starting point.

Their limitations are significant: they offer no integration with external cost management systems, no support for organization-defined tagging, and no workload-level attribution. Budget alerts are available but coarse. For any organization with more than a handful of AI applications, provider dashboards are necessary but insufficient.

LLM Proxy and Observability Platforms

A growing category of specialized tools sits between applications and model providers, providing rich observability, attribution, and control capabilities:

• LiteLLM: an open-source proxy supporting 100+ model providers with spend tracking, rate limiting, and team-level attribution. Highly configurable and well suited to organizations that want full control over their instrumentation stack.
• Portkey: a commercial gateway with strong model routing, cost analytics, and fallback capabilities. Provides a unified interface across providers with built-in cost attribution.
• Helicone: an observability platform focused on logging, cost tracking, and prompt management, with a lightweight integration path that requires minimal code changes.
• LangSmith: primarily a debugging and evaluation platform for LangChain-based applications, but with growing cost tracking capabilities useful for teams already in the LangChain ecosystem.
• Braintrust: evaluation and observability with cost tracking, particularly strong for teams that want to connect cost data to quality metrics.

The choice among these tools should be driven by the organization’s technical architecture, existing observability investments, and the specific attribution and control requirements identified in the framework above.

FinOps Platform Support

As of 2026, support for AI model provider spend in mainstream FinOps platforms is nascent but growing. Most cloud cost management platforms (Apptio Cloudability, CloudHealth, Vantage, Spot by NetApp) have announced or begun shipping integrations for OpenAI and Anthropic billing data. The maturity of these integrations varies: some provide only spend import, while others are beginning to support allocation and anomaly detection.

Organizations should evaluate their existing FinOps platform’s AI cost capabilities against the attribution and reporting requirements outlined earlier. In many cases, a proxy layer combined with a data warehouse integration will provide richer capabilities than a native FinOps platform integration for the foreseeable future.

Tool Category	Attribution Depth	Real-Time Alerts	FinOps Integration	Build Effort	Best For
Provider Dashboards	Key/Project level	Basic spend alerts	None	None	Getting started
LLM Proxy (LiteLLM, Portkey)	Workload/User level	Configurable	Via export	Low-Medium	Engineering-led orgs
Observability (Helicone, LangSmith)	Request level	Some	Limited	Low	Dev-focused teams
FinOps Platform Integration	Key/Project level	Yes	Native	None	Finance-led governance
Custom Data Warehouse	Full custom	Via BI tooling	Full custom	High	Data-mature orgs

Maturity Model: Crawl, Walk, Run

AI cost management capability does not develop overnight, and organizations that attempt comprehensive governance before establishing foundational visibility will find the effort unsustainable. The following maturity model provides a self-assessment framework and a sequenced roadmap for building capability progressively.

Organizations at Any FinOps Maturity Stage: Use this section to locate where your organization is today and to identify the two or three actions that will have the most impact at your current stage. Resist the temptation to skip ahead: the Run-stage capabilities listed below are only sustainable if the Crawl-stage foundations are in place.

FinOps Practitioners & Analysts: The maturity model is also a stakeholder communication tool. Use it to set realistic expectations with leadership about what is achievable in the next 90 days versus the next year, and to frame the organizational investment required to advance stages.

Crawl: Foundational Visibility (Months 1-3)

Characteristics

• AI spend is visible at the provider account level but not attributed to applications or teams.
• API keys exist but were created ad hoc; no governance process in place.
• Engineering teams make model selection decisions without cost input.
• No budget or alert thresholds set for AI spend.

Actions

• Conduct an AI spend inventory: identify all active model provider accounts, API keys, and associated applications.
• Implement API key governance: establish a provisioning process, naming convention, and key-to-team mapping.
• Deploy a lightweight proxy or enable provider-native tagging where available.
• Publish a basic AI spend dashboard showing spend by team/application.
• Set account-level budget alerts as a backstop.

Walk: Allocation and Optimization (Months 3-9)

Characteristics

• Workload-level attribution is in place; spend can be allocated to business units.
• Unit cost metrics (cost per query, cost per user) are calculated and shared.
• Model right-sizing analysis has been completed for top-spend applications.
• Basic optimization levers (prompt caching, batch API) are being used where applicable.

Actions

• Implement showback reporting to application and team owners.
• Conduct model right-sizing review for all applications spending above a defined threshold.
• Enable prompt caching across eligible applications.
• Identify and migrate batch-eligible workloads to batch API.
• Begin context window optimization for conversational and agentic applications.
• Establish anomaly detection for spend spikes.

Run: Active Governance and Continuous Optimization (Month 9+)

Characteristics

• Chargeback is implemented; teams own their AI budget.
• Policy guardrails are enforced via technical controls (proxy rate limits, model allowlists).
• Commitment discussions with providers are based on modeled consumption forecasts.
• Cost-quality tradeoff analysis is embedded in the AI development lifecycle.
• AI cost metrics are reported alongside other technology cost metrics in leadership reporting.

Actions

• Implement chargeback where organizational maturity supports it.
• Build model routing for dynamic right-sizing across applications.
• Engage model providers in commitment or enterprise agreement discussions.
• Integrate AI cost estimation into the CI/CD pipeline.
• Contribute to FinOps Foundation FOCUS working groups to advance community standards for AI cost data.

Dimension	Crawl	Walk	Run
Visibility	Account-level spend	Workload-level attribution	Request-level with unit metrics
Allocation	None	Showback to teams	Chargeback with team budgets
Optimization	Ad hoc	Right-sizing and caching	Dynamic routing and commitments
Governance	No policy	Policy defined	Policy enforced via controls
Tooling	Provider dashboard	Proxy + basic FinOps	Integrated observability platform
Culture	Engineering awareness	Shared visibility	Cost-conscious AI development

Conclusion and Call to Action

Token management is the cloud unit economics challenge of the AI era. The practitioner community that spent the last decade mastering reserved instances, commitment-based discounts, and showback methodologies for cloud compute is facing a structurally similar challenge with a new set of primitives. The core discipline transfers: instrument, understand unit costs, optimize, govern, and repeat.

The tooling ecosystem is immature, provider APIs are evolving faster than governance frameworks, and the organizational awareness needed to govern AI spend effectively is still being built. The practitioners who move now to establish foundational visibility, implement API key governance, and begin the model right-sizing conversation with their engineering counterparts will be significantly ahead of those who wait for the tools to mature.

Four calls to action for practitioners at different stages:

• If you do not know what your organization is spending on model provider APIs today: stop reading and conduct an inventory. Shadow AI spend is real and it is almost certainly larger than anyone in Finance is aware of.
• If you have visibility but no attribution: the proxy layer is your next investment. The engineering effort required is low relative to the governance value delivered.
• If you have attribution and are ready to optimize: start with model right-sizing. It is the highest-leverage intervention and the one most engineering leaders will support once they see the unit economics.
• If you have a maturing practice with quality measurement capability, proxy infrastructure, and the capacity to run open-source models, consider mixing in open-source models with direct SaaS model usage to lower overall spending for use cases that do not require the most sophisticated models.

This is a domain where the practitioner community needs to build shared standards together. The FinOps Foundation’s FOCUS specification provides a model-agnostic billing data schema for cloud; extending FOCUS to encompass AI token spend is a natural and necessary evolution. Practitioners who engage with this work will help shape the standards that the industry coalesces around.

The organizations that treat AI cost management as a strategic capability rather than a finance hygiene exercise will be better positioned to scale AI confidently, allocate budget to the highest-value use cases, and demonstrate the ROI that earns continued investment.

Acknowledgments

We’d like to thank the following people for their work on this Paper: