Skip to content

Cost Tracking

symfonic-core ships a model-pricing registry, a per-call cost computer, and an adopter-side override knob so deployment owners can correct rate-card drift or register custom models without waiting for a framework release.

The pieces

Surface Where Purpose
MODEL_PRICING pricing.py Built-in registry: model id → {input, output, cache_read, cache_write} in USD per 1M tokens
_resolve_pricing(model) pricing.py Lookup with exact-match → strict-prefix precedence. Used by calculate_cost and calculate_cost_with_meta
calculate_cost(model, ...) pricing.py Numerical cost for a single LLM call. Returns 0.0 for unknown models with a one-time WARNING log
calculate_cost_with_meta(...) pricing.py Same as above plus pricing_unknown: bool discriminator. Use when "known model, $0" must be distinguished from "unknown model, fell back to $0"
register_pricing_overrides(overrides) pricing.py v7.8.0 public: install adopter overrides consulted BEFORE the built-in registry
clear_pricing_overrides() pricing.py Test isolation helper
FrameworkConfig.model_pricing_overrides agent/config.py v7.8.0 knob: agent installs these at construction time

Lookup precedence (v7.8.0+)

For a given model string, lookup order is:

  1. Override exact match — overrides[model]
  2. Override prefix match — overrides[k] where model.startswith(k)
  3. Built-in exact match — MODEL_PRICING[model]
  4. Built-in prefix match — MODEL_PRICING[k] where model.startswith(k)
  5. Cloud/gateway normalization (v8.10.0) — if the id is a Bedrock namespaced id or an OpenRouter vendor/model id, retry steps 1–4 on the normalized SKU (see below)
  6. Unknown — return None, emit a one-time WARNING, cost reported as 0.0

Override precedence is total: a prefix override at level 2 wins over a built-in exact match at level 3. This is deliberate — adopters who have negotiated enterprise rates on an existing model id need their override to fire even when the built-in entry would have matched.

Cloud/gateway id normalization (v8.10.0)

AWS Bedrock and OpenRouter wrap first-party models under their own id schemes, which match none of the bare-SKU keys in MODEL_PRICING. Because both gateways bill at the underlying provider's per-token list rate, _resolve_pricing normalizes these ids to their first-party SKU and reuses the existing rows instead of reporting $0:

Wrapped id Normalizes to Billed as
us.anthropic.claude-sonnet-4-5-20250929-v1:0 (Bedrock) claude-sonnet-4-5 Anthropic list rate
anthropic/claude-sonnet-4.5 (OpenRouter) claude-sonnet-4-5 Anthropic list rate
openai/gpt-4o (OpenRouter) gpt-4o OpenAI list rate
google/gemini-2.5-pro (OpenRouter) gemini-2.5-pro Google list rate

Non-first-party vendors are still unknown. A Bedrock/OpenRouter id whose underlying model has no MODEL_PRICING row — Meta Llama, Amazon Nova, Mistral, Cohere, etc. — falls through to the WARNING + $0 path. Install a rate for those the normal way:

FrameworkConfig(model_pricing_overrides={
    "us.meta.llama3-3-70b-instruct-v1:0": {
        "input": 0.72, "output": 0.72, "cache_read": 0.0, "cache_write": 0.0,
    },
})

When to use overrides

Scenario Recipe
Anthropic published a rate change and you can't wait for the next release Add an exact-match override for the affected model id
You deployed a custom in-house model Add an exact-match (specific build) or prefix-match (family) override
You negotiated enterprise rates that differ from the public table Override the affected model ids
You want $0 cost for sandbox/test runs Override with all-zero rates for the test model ids

What overrides do NOT do

  • They do not change the model the agent calls. The override only affects the cost number reported by the framework.
  • They do not cascade to budget enforcement automatically. The TokenBudgetTracker consults calculate_cost so the overrides flow through, but if you set a budget ceiling in absolute dollars the ceiling stays in dollars regardless.
  • They do not remove the "unknown model" WARNING for novel ids that match no override and no built-in entry. Use the explicit override pattern for new model ids.

Validation contract

register_pricing_overrides validates at registration time:

  • Model id must be a non-empty string.
  • Pricing row must be a dict.
  • Pricing row must include all four keys: input, output, cache_read, cache_write.
  • Each value must be a non-negative int or float.

Partial-row overrides (only input and output set) are rejected with ValueError. This is deliberate — a missing cache_read would silently default to 0.0 at compute time, masking the gap.

Example: full override pipeline

from symfonic.agent.config import FrameworkConfig
from symfonic.agent.engine import SymfonicAgent

cfg = FrameworkConfig(
    model_pricing_overrides={
        # Correct an Anthropic rate change before the next framework release.
        "claude-haiku-4-5": {
            "input": 0.5, "output": 2.5,
            "cache_read": 0.05, "cache_write": 0.625,
        },
        # Register a custom in-house model family.
        "acme-internal-": {
            "input": 0.0, "output": 0.0,
            "cache_read": 0.0, "cache_write": 0.0,
        },
    },
)

agent = SymfonicAgent(model_provider=..., config=cfg)
# Overrides are now installed process-wide.  Any caller of
# calculate_cost(model="claude-haiku-4-5", ...) sees the new rates.

Process-scoping note

The override registry is module-level (_PRICING_OVERRIDES in pricing.py). This is one set of overrides per Python process. If you spin up two SymfonicAgent instances with different overrides in the same process, the second __init__ wins.

The expected deployment shape is one agent process per service. If you have a stronger isolation requirement (e.g. multi-tenant SaaS where each tenant has its own pricing contract), thread an override table at call time rather than via FrameworkConfig — the calculate_cost function is pure and can be wrapped.

Pre-v7.8.0 path

Before v7.8.0 the only way to correct a stale rate was to ship a patch release. The most prominent example was v7.4.5 (an adopter request) where claude-opus-4-6 had been mis-mapped to the legacy Opus 4 tier — a 3.0× cost overcharge. The override knob removes this release-cadence dependency.

References