TokenOps: How to Measure, Allocate and Control AI Token Costs

Two applications can use the same model at the same published price and still produce very different bills. One may send short prompts with high cache reuse; another may carry a growing context through an agent loop, generate longer outputs and invoke tools repeatedly. TokenOps is about measuring those differences, understanding where the spend comes from and assigning it to an owner.

This guide treats AI token cost management as a measurement problem first. It breaks an LLM bill into token types and billing modifiers, works one hypothetical example through with arithmetic, and lists the metrics worth tracking. It keeps general TokenOps practice separate from the metrics CloudQuell computes from provider billing data, and it states what billing data cannot show. The provider behavior described here was checked against current provider documentation on October 27, 2026. Exact rates are left out on purpose because they change, so the worked example uses hypothetical numbers and is illustrative arithmetic, not a benchmark.

What is TokenOps?

TokenOps is the discipline of measuring, allocating and controlling the cost of LLM tokens. It applies FinOps for AI practices to token spend: visibility into what was consumed, allocation to the team or product that consumed it, optimization of the drivers, and guardrails such as budgets and alerts. The FinOps Foundation’s FinOps for AI overview recommends adding token-based units and drivers alongside standard cloud metrics, and notes that the billing meters differ by provider.

The term is new and used mostly by vendors, so definitions vary. The underlying work does not: break the bill into token types, find what moves each one, and attach every dollar to an owner.

AI token economics: a token price is a rate, not a cost

Providers publish prices per million tokens. A bill is the sum of several of those rates, each applied to a different quantity:

cost = uncached input tokens x input rate
     + cache-read tokens     x cache-read rate
     + cache-write tokens    x cache-write rate
     + output tokens         x output rate
     + non-token charges     (for example, per-search tool fees)

Each term has its own driver:

• ◦Token type. Input, output, cache reads and cache writes are priced differently.
• ◦Caching. The share of input served from cache changes the blended input rate without changing the list price.
• ◦Context growth. Conversation history, documents and tool definitions are input tokens on every call that includes them.
• ◦Repeated calls. A retry, a validation pass or an agent step is a new billed call.
• ◦Model and service tier. The same prompt costs different amounts on different models, and on the batch tier versus the standard tier.
• ◦Source of the usage. Which team, project or key generated the tokens decides who owns the cost.

Two systems on the same model at the same list price can have very different effective rates. Cost per token, as published, is a property of the price list. Effective cost per token, the dollars divided by the tokens actually billed, is a property of your system. It is the number that describes your real LLM token cost, and the one to track.

Input vs output token cost

Providers bill input tokens, cached input tokens and output tokens at separate rates, and output tokens are commonly priced higher than input tokens. The exact ratio varies by provider and by model and changes over time, so read it from the provider’s current pricing page or from our LLM API pricing comparison rather than from a fixed rule of thumb.

A feature that generates long responses has a different cost shape from one that sends long prompts and returns a short label. A total token count hides that difference, so the input and output split belongs on the first dashboard.

Prompt caching cost: writes, reads and break-even

Prompt caching lets a provider reuse an already-processed prefix across calls, so repeated input costs less. Prompt caching cost has up to two parts: a charge when the prefix is first written to the cache, which some providers price above ordinary input and some do not charge separately, and a discounted rate each time the cached prefix is read. Caching pays off when the read savings exceed any write charge, which depends on how often the prefix is reused before it expires.

The specifics differ by provider and by model and change over time. For each model you use, take these from the provider’s current pricing and caching documentation:

• ◦The cached-read rate compared with the ordinary input rate.
• ◦Whether cache writes are charged separately, and at what rate.
• ◦How long a cached prefix lasts, and whether the lifetime is selectable.
• ◦The minimum prompt length before a prefix can be cached.
• ◦Whether caching is automatic or has to be requested.

Because these terms differ by provider and by model, do not apply one savings multiplier across a mixed bill. Read cache-read and cache-write token counts per model from billing data, and track the share directly:

cache-read share = cache-read tokens / (uncached input tokens + cache-read tokens)

A low share on a workload with a large repeated prefix is the signal to look at how prompts are structured and whether the prefix meets the provider’s minimum cacheable length.

Why LLM token cost per task diverges from cost per token

Tokens accumulate in ways the price list does not show. Anthropic’s pricing page describes the baseline: without caching, the same large system prompt, document or conversation history is reprocessed on every request. Tools add more. Anthropic’s pricing page lists three sources of extra tokens: the tools parameter (names, descriptions and schemas), tool_use content blocks and tool_result content blocks. When tools are provided, the API also adds a tool-use system prompt automatically. Server-side tools can add usage-based charges on top, such as a per-search fee.

The arithmetic below holds the rates constant and changes only how the system behaves. The rates are hypothetical and normalized. They are not any provider’s prices.

The rate per token is identical in every row. The loop sends about 27 times the input tokens of the single call and costs about 19 times as much, and caching removes most of the input cost but none of the output cost. Your steps, tool results, rates and cache hits will differ. The point is that the cost changed while the rates did not, which is why cost per task or per outcome is the unit that matches how the system behaves.

Model choice and service tier

Providers price their models in tiers, and the gap between a smaller and a larger model can be large. Routing work to a cheaper model lowers cost only if quality holds, and billing data cannot tell you whether it does. Measure quality on your own evaluation set before moving traffic.

Service tier is the other lever. Both Anthropic and OpenAI offer batch, or asynchronous, processing at lower rates than standard processing. It applies only to work that can wait, and the discount and eligible models vary by provider, so the measurement is how much of your volume is eligible and how much already runs on the batch tier.

The TokenOps metrics worth tracking, and which ones CloudQuell provides

The metrics fall into two groups. The first can be computed from provider billing and usage reports. The second needs data about your own application: outcomes, features and individual requests. CloudQuell reads the first group. The second group needs instrumentation in your code or an observability tool.

The effective rate uses token-metered dollars, not total spend. When part of the spend did not buy tokens, such as tools or sessions, CloudQuell shows what share of spend the rate covers.

Allocate AI token usage to the teams that generated it

Attribution is only as good as the boundary the provider records on each charge. OpenAI reports spend per project and Anthropic per workspace, and spend follows the project or workspace the key belongs to. The general practice is one project or workspace per team, with that team’s keys created inside it, and production kept apart from load tests so a test reads as a test.

Spend made in OpenAI’s Default project or Anthropic’s Default workspace carries no ID, so it cannot be matched to a team. Move team keys out of the defaults.

In CloudQuell, allocation rules match the project_id tag on OpenAI rows or the workspace_id tag on Anthropic rows and route that spend to a cost center, the same cost centers used for cloud spend. Spend no rule claims reports as Unallocated, which is the signal that a new project or workspace appeared. Rules match IDs, not display names. Connecting OpenAI and Anthropic works on every plan, including Free. Cost centers, allocation rules, budgets and alert rules are paid features.

Control AI token costs with budgets and alerts

A monthly budget can tell you when spend is approaching its limit, but it may not explain what caused the increase. Controls work better scoped to the same boundaries used for allocation, with a trend or baseline check beside the fixed limit.

In CloudQuell, a budget can be scoped to a cost center, to specific OpenAI projects or Anthropic workspaces, or to a tag. Default thresholds are 80% and 100%, delivered to email, Slack or Teams. Alert rules include Spend Threshold, Trend Spike and Anomaly Scan, and Anomaly Scan compares each model with its own baseline, so a loop on one model can show even when the team’s total looks ordinary. The AI Spend view also shows a projected month-end figure at the current run rate. Budgets and alert rules are paid features.

The limit is timing. Provider data syncs daily, each sync re-pulls a trailing three-day window so late billing corrections settle, and alert rules run daily, weekly or monthly. This is not real-time monitoring. For a runaway loop in the last hour, provider-native spend or usage limits, where available, are the real-time backstop.

Token usage optimization: where to start

Anthropic’s pricing page lists four cost levers for agent workloads: use an appropriate model for each task, implement prompt caching, batch work that is not time-sensitive, and monitor token consumption. For token usage optimization, apply them in order of measured impact rather than guessing:

• ◦Measure the split first. Token mix by type, cache-read share and model mix show which bucket is largest.
• ◦Fix the largest bucket. If output dominates, look at response length and format. If uncached input dominates and the prefix repeats, look at caching. If one expensive model dominates, test a cheaper one on that workload.
• ◦Move work that can wait to the batch tier, where the provider documents a discount.
• ◦Review tool definitions and tool results. They are billed as input tokens on every call that includes them.
• ◦Re-measure with the same metrics. Effective $ per 1M tokens and cache-read share show whether the change worked.

This guide quotes no savings percentages beyond the providers’ published multipliers. Savings from any lever depend on the workload, so measure your own before and after.

What billing data cannot tell you

CloudQuell reads provider cost and usage reports. It does not read requests. These limits follow from that and are worth knowing before you plan around them:

• ◦No request tracing. Cost per prompt, per feature, per end customer or per trace is finer than either provider’s billing API reaches. An application-level observability tool that instruments the call site does that job.
• ◦No per-key dollars. OpenAI and Anthropic report token and request counts per API key, not cost.
• ◦The Default project and Default workspace cannot be attributed to a team by rule, budget or alert scope.
• ◦Seat subscriptions such as ChatGPT Plus or Team and Claude Pro, Max or Team are not billed per API call and do not appear in either provider’s cost report.
• ◦Data is not real-time, as described above.

To get cost per feature or per task, log the token counts the provider returns on each response at the call site, along with a feature or task identifier, and reconcile the totals against billing data. That is general practice, and it requires changes in your application. CloudQuell does not do it.

Where CloudQuell fits

CloudQuell connects to OpenAI and Anthropic with a read-only admin key. Nothing is added to your application code, and no SDK or proxy is involved. It backfills up to 12 months on connection and then syncs daily. The AI Spend view provides the metrics marked Yes in the table above. Amazon Bedrock usage arrives through the AWS integration and appears in the same view as a read-only lens over AWS rows, so it is counted once, in cloud spend.

AI token cost management: frequently asked questions

How is TokenOps different from FinOps for AI?

TokenOps is a name for the token-spend part of FinOps for AI. The FinOps Foundation’s overview describes FinOps for AI more broadly, covering training and inference cost, and recommends adding token-based units alongside standard cloud metrics.

How do I calculate cost per token?

The published cost per token is the price per million tokens divided by 1,000,000, and it differs for input, output, cache reads and cache writes. Your effective cost per token is the token-metered dollars divided by the tokens actually billed. The effective figure reflects your token mix, cache-read share and model choice, so it is the one to track over time.

Are output tokens more expensive than input tokens?

Commonly, yes. Providers bill input and output tokens at separate rates and usually price output higher, but the ratio varies by provider and model and changes over time. Check each model’s current rates before assuming a ratio.

How much does prompt caching save?

It depends on the cache-read share, on how the cached-read rate compares with the ordinary input rate, and on any charge for writing the cache. Those terms vary by provider and model. Read cache token counts per model from billing data and compute your own figure.

How do I reduce LLM token costs?

Measure the token mix first, then work on the largest bucket: response length for output-heavy workloads, caching for repeated prefixes, a cheaper model where quality holds, and the batch tier for work that can wait. Re-measure effective cost per token afterward to confirm the change.

Can CloudQuell show cost per feature, prompt or customer?

No. CloudQuell reads provider billing and usage reports, which do not go below project or workspace, model, token type and API key. Cost per prompt, per feature, per agent run or per end customer needs call-site instrumentation in your application or an observability tool.

Is CloudQuell token cost data real-time?

No. OpenAI and Anthropic data syncs daily with a trailing three-day re-pull, and alert rules run daily, weekly or monthly. For a runaway loop in the last hour, provider-native spend or usage limits, where available, are the real-time backstop.