Skip to content

Cache-tier trade-offs (5-minute vs 1-hour)

Status: stable as of v7.24.1. Audience: adopters tuning prompt-cache cost vs hit-rate for intermittent traffic flows.

1. The two TTL tiers Anthropic exposes

Anthropic's prompt-cache API exposes two TTL tiers via cache_control markers on system blocks, tool definitions, and messages-region blocks:

Tier Wire marker Lifetime
5m {"type": "ephemeral"} 5 minutes
1h {"type": "ephemeral", "ttl": "1h"} 1 hour

The 5-minute tier is the default the framework emitted from v7.13 through v7.23. v7.24.0 adds an opt-in 1-hour tier under explicit adopter control.

2. Pricing differential

Anthropic prices cache writes higher than uncached input, and the 1-hour tier is priced higher than 5-minute:

Operation Price multiplier (vs uncached input)
Cache read 0.10x
Cache write 5m 1.25x
Cache write 1h 2.00x

Reads cost the same regardless of tier; the differential is entirely on the cold write. A 1-hour cache write is 60% more expensive than a 5-minute cache write on the same prefix.

3. The framework's surfaces

v7.24.0 ships four opt-in knobs that flip blocks from 5m to 1h:

3.a Global framework knobs

  • FrameworkConfig.system_prefix_cache_ttl: Literal["5m","1h"] (default "5m") Stamps the L1 stratigraphic breakpoint at agent/prompts/stratigraphic.py:191-198.

  • FrameworkConfig.tools_cache_ttl: Literal["5m","1h"] | None (default None) Post-processes the bound tools array in core/nodes/react.py:_bind_tools to stamp cache_control on the LAST tool. Anthropic's rule: marker on last tool = entire tools array advertised at the declared tier.

3.b Plugin-level knobs

  • PluginSection.cache_ttl: Literal["5m","1h"] | None Section-level opt-in for adopter-defined plugins (precedes contribution-level value when both set).

  • PluginContribution.cache_ttl: Literal["5m","1h"] | None Per-contribution opt-in; bubbles up to the section's wire marker via PluginPromptSection.resolve_cache_ttl() (strongest value wins: "1h" > "5m" > None).

All four knobs default to a value that produces the byte-identical v7.23 wire shape when left untouched -- v7.24.0 is a zero-breakage release for current adopters.

4. When to opt in

Opt into 1h when your traffic exhibits one or more of:

  • Intermittent idle gaps. Brand-author UIs, ops dashboards, human-in-the-loop review queues, multi-minute pauses between user turns. The 5m default expires before the next turn arrives, so every interaction pays the 1.25x write premium on a cold prefix.
  • Stable static prefix. Plugin contributions or domain prompts that don't drift across turns within a session. The 1h tier amortises the 2x write premium over more reads.
  • High-cost stable prefix. When your L1 system prefix is large (5k+ tokens on Opus), even a single saved cold-write per hour outweighs the 2x premium.

5. When NOT to opt in

Keep the 5m default when:

  • High-throughput chat. Turns arrive within 5m of each other; the 5m tier already caches every turn.
  • Per-turn-mutating tools array. Multi-tenant routing, lazy_tooling adopters, or ToolPaletteResolver adopters whose tools array changes across turns. A 1h-cached tools array that mutates pays the 2x cold-write premium on every change -- worse than the 1.25x baseline.
  • Drifting messages region. Path E messages-region marker is intentionally pinned at 5m by the framework (closed-decision #3 in the v7.24.0 plan). The marker sits on settled-history bytes that drift across turns under slicing; 1h there underperforms 5m on hit rate by the same drift mechanism.

6. R-V7.22-B math shift addendum (1h × multi-scope mutations)

Load-bearing for v7.22 multi-scope procedural retrieval adopters.

R-V7.22-B's v7.22 verdict ("one cache miss per edit is acceptable") was computed against the 5m baseline. The math shifts at 1h.

Setup: an adopter uses v7.22 multi-scope procedural retrieval to brand-author per-turn rules. The procedural section sits inside the cached L1 prefix. Each brand-author edit (a rule mutation) busts the cache on the next turn.

TTL Cold-write cost Per-edit cache miss cost Net vs uncached
5m 1.25x input 1 miss saves ~5min of reads Net positive at 1-2 reads/edit
1h 2.00x input 1 miss saves ~60min of reads Net positive only at ~7+ reads/edit

The crossover point at 1h is roughly 7 reads per cache write. Below that threshold, the 1h opt-in COSTS more than the 5m baseline on mutation-prone prefixes -- you pay the 2x premium without amortising over enough reads.

Mitigation: measure your actual reads-per-edit ratio in production before opting into 1h on a multi-scope-mutating prefix. For typical brand-author flows (1-3 edits per author session, 5-20 turns) the 5m default outperforms 1h.

6.5. Empirical minimum-prefix thresholds for the 1h tier

Load-bearing for adopters on the Haiku family.

The 5-minute baseline thresholds documented in src/symfonic/core/prompt/messages_cache.py:73-78 (sonnet/haiku ~1100 tokens, opus-4.5 ~5500 tokens) were measured against 5-minute caching only. v7.24.0 R-V7.24-C verified empirically (2026-06-11) that the 1-hour tier does NOT carry over the same threshold uniformly across the model lineup.

Model family 5m minimum-prefix 1h minimum-prefix Status
Sonnet 4.5 ~1100 tokens ~1100 tokens (verified at 1200t) Safe to opt in
Haiku 4.5 ~1100 tokens >1200 tokens (exact unknown) Marker silently ignored below threshold
Opus 4.5 / 4.6 / 4.7 / 4.8 ~5500 tokens Unverified by us Assume ≥5500 tokens; measure your prefix

The Haiku 4.5 result is the load-bearing one. At 1200 tokens of cached prefix (which DOES create a 5m cache when the marker is bare ephemeral), Anthropic returned cache_creation_input_tokens=0 for the {"type":"ephemeral","ttl":"1h"} marker. The cached prefix was NOT created — the 1h marker was silently ignored as if absent.

Adopter impact: Haiku-family adopters who opt into EITHER 1h knob with a cached prefix below the unknown-but-larger-than-1200-tokens Haiku 1h threshold will see:

  • No cache_creation_input_tokens on the cold turn.
  • No cache_read_input_tokens on warm turns.
  • No cost reduction. The opt-in does nothing.
  • No error or warning from Anthropic — the behaviour is byte-for-byte identical to running without the opt-in.

Both 1h opt-in knobs share the failure mode. Anthropic caches the tools-array region as part of the cached prefix alongside the system prompt — the minimum-prefix-size threshold is on the entire cached prefix region, not per-block. So:

  • system_prefix_cache_ttl="1h" with a small L1 prefix on Haiku → silent miss.
  • tools_cache_ttl="1h" with a small tools array on Haiku → same silent miss for the same reason.
  • Setting both 1h on Haiku does NOT compound — the combined cached region still has to clear the same single threshold.

Mitigation paths:

  1. Verify your prefix and tools-array sizes. Measure both the L1 stratigraphic prefix and the bound tools-array region in tokens (use tiktoken or the Anthropic count_tokens API). If either is below ~1500 tokens on Haiku, the corresponding 1h opt-in is speculative — there is no guarantee Anthropic will honour the marker.
  2. Stay on the 5m default when the cached region is small. 5m on a small prefix outperforms 1h-silently-dropped on the same prefix at every metric.
  3. The framework emits a UserWarning at FrameworkConfig construction when EITHER system_prefix_cache_ttl="1h" OR tools_cache_ttl="1h" is combined with a Haiku-family model. The warning lists every affected knob in one message (no separate warnings to triage when both are set). Filter the warning category UserWarning with stacklevel 2 to suppress if you know your region size exceeds the threshold.

Opus families: the verification test did not cover Opus because the prefix size required to clear the 5500-token 5m threshold would have inflated the live-API spend beyond the budget. Adopters on Opus should measure their own production prefix and run a 2-turn cold/warm probe before committing to 1h on those models.

Reference: R-V7.24-C verification test at tests/integration/live_llm/test_cache_ttl_threshold.py. Re-run pytest tests/integration/live_llm/test_cache_ttl_threshold.py -m live_llm against any new model family before opting them into 1h. Expected spend per run: ~$0.06 (Sonnet + Haiku) per the 2026-06-11 measurement; ~$0.20-0.40 if Opus tests are added.

7. Path E messages-region stays 5m

The Path E messages-region breakpoint (v7.13.0, core/prompt/messages_cache.py) emits the bare 5m marker regardless of the system-prefix knob. This is intentional:

  • The Path E anchor sits on settled-history bytes that drift across turns when max_conversation_messages slices.
  • A 1h marker there would survive longer in wall-clock time but the underlying bytes change before 1h elapses -- the cached entry is reborn on every slice. The 2x cold-write premium is paid for nothing.

This was a closed decision at the v7.24.0 architect verdict. Adopters who want a 1h messages-region marker must justify the architectural exception in writing.

8. Migration story

Pre-v7.24.0 adopters: install v7.24.0; no behavior change. All defaults produce the v7.23 wire shape.

Adopters with idle-prone flows:

config = FrameworkConfig(
    system_prefix_cache_ttl="1h",
    # Optionally:
    tools_cache_ttl="1h",  # only when tools array is stable
)

Adopters with custom plugins:

PluginSection(name="Stable Domain Rules", cache_ttl="1h")
# or, per-contribution granularity:
PluginContribution(content="...", cache_ttl="1h")

High-volume production adopters: static tool-usage guidelines + BASE_INSTRUCTIONS map cleanly to a single 1h-tagged L1 section. The legacy CACHE_BOUNDARY sentinel approach was rejected at the architect verdict in favour of the L0/L1 split (closed-decision #2 in the v7.24.0 plan).

9. Telemetry loop

When an adopter opts into 1h via any of the four knobs:

  1. Wire-level cache_control carries ttl: "1h".
  2. Anthropic's response surfaces cache_creation.ephemeral_1h_input_tokens > 0 on the cold turn.
  3. AnthropicProvider.extract_cache_ttl (v7.20.0 §6) returns "1h".
  4. TokenUsage.cache_ttl == "1h" flows through callbacks.
  5. _compute_cost branches to the row's cache_write_1h rate.
  6. Cost telemetry reflects 2x rate on the cold write -- matching what Anthropic actually charged.

The v7.20.0 §5 pricing scaffolding was read-path only; v7.24.0 closes the loop with the send-side knobs that produce the wire markers.

10. Descending-TTL invariant (v7.24.1 hotfix)

Anthropic's messages endpoint enforces a non-increasing TTL ladder across the cumulative prompt prefix. In wire terms: when scanning cache_control markers from system[0] forward, the TTL tier must never INCREASE. A 5m marker followed by a 1h marker is rejected with HTTP 400 (cache_control: ttl values must be in descending order).

Why this matters for the framework: PromptBuilder.build_cached auto-promotes the FIRST emitted block to cached regardless of its cache_breakpoint attribute. In stratigraphic mode that's the L0 Brain Kernel (which carries no cache_ttl -- collapses to 5m on the wire). When an adopter sets system_prefix_cache_ttl="1h" the L1 breakpoint carries 1h, producing 5m -> 1h ASCENDING. Anthropic 400s every brain turn.

F2 algorithm: right-to-left tier propagation

The fix is a single right-to-left pass over rendered CacheBlocks that propagates the highest-tier-seen backward onto earlier cached blocks (src/symfonic/core/prompt/caching.py :: _normalise_descending_ttl). Uncached blocks (cache_control is None) are skipped -- they do not participate in the wire ladder. The result is a uniformly non-increasing TTL sequence Anthropic accepts.

Adopter-facing implication: when you set system_prefix_cache_ttl="1h" in stratigraphic mode, L0 is implicitly promoted to the 1h tier on the wire. The cost impact is real but bounded: L0 is small (~200-400 tokens, the kernel template) and the 2x cache_write premium applies only to the cold turn that populates the entry. Warm-turn reads are still 0.10x regardless of tier.

The v7.23 byte-identity guarantee is preserved for the default path: when every cached block carries cache_ttl in (None, "5m"), the scan sees a uniform rank-0 ladder and promotes nothing -- output is bitwise-identical to pre-v7.24.1.

Incident: 2026-06-12 production failure

A production adopter flipped system_prefix_cache_ttl="1h" shortly after upgrading to v7.24.0. The brain hit HTTP 400 on every turn with Anthropic's descending-order rejection message. Root cause traced to PromptBuilder.build_cached's first-block auto-promotion: L0 (cache_ttl=None -> 5m wire) followed by L1 (cache_ttl="1h") produced ascending ladder.

Lesson and guard: every send-side cache_ttl knob the framework exposes must respect the descending-TTL contract. The v7.24.1 normalisation pass is the framework-wide enforcement point; plugin authors writing new cache-tier knobs do NOT need to coordinate with each other -- the normalisation pass is the funnel. Regression coverage:

  • tests/unit/prompt/test_descending_ttl_invariant.py (parametrised across the full knob matrix).
  • tests/agent/test_stratigraphic_l1_cache_ttl.py ::test_1h_propagates_to_L0_block.
  • tests/integration/live_llm/test_stratigraphic_1h_real_anthropic.py (R-V7.24.1-C live verification).

11. References

  • v7.24.0 architect plan: .agent/team/plans/2026-06-11-v7.24-cache-ttl-tiers.md
  • v7.24.1 hotfix algorithm: src/symfonic/core/prompt/caching.py :: _normalise_descending_ttl
  • force_last_cache docstring: src/symfonic/core/prompt/builder.py:84-95
  • Empirical threshold notes (5m baseline): src/symfonic/core/prompt/messages_cache.py:73-78
  • R-V7.22-B (5m verdict): .agent/team/plans/2026-06-08-v7.22-multi-scope-procedural.md