symfonic.core.prompt.messages_cache

messages_cache

v7.13.0 Path E — messages-region cache_control breakpoint placement.

Background

v7.12.0 shipped tool-result compaction based on the wrong mental model of Anthropic's prompt cache. v7.12.1 default-flipped it OFF after Jarvio's $1.75 -> $3.42 cost regression. v7.13.0 ships the correct architectural fix: place a SECOND cache_control breakpoint in the messages region so Anthropic's lookback can cache settled history (including big ToolMessages) instead of replaying them uncached every turn.

Empirical foundations (verified against Anthropic API 2026-05-31)

• Opus 4.5+ and Sonnet 4.6+ KEEP prior-turn thinking blocks in context by default. The cache key auto-filters them server-side. See .claude/temp/v713_thinking_cache_smoke.py: both unstripped and stripped paths produced cache_read=6442.
• Earlier model families (opus-4, sonnet-3.5, haiku) strip thinking blocks; the framework MAY strip defensively but Jarvio's opus-4-6 (which Anthropic's docs alias as opus-4.5+) gets full benefit without strip.
• Opus 4.5+ minimum cacheable prefix is empirically ~5000 tokens. Below that threshold, cache_control annotations are ignored by Anthropic. See tests/integration/live_llm/test_cache_mechanics.py.

Architectural pattern

Mirrors v7.10 ForcedToolChoiceResolver, v7.11 ToolPaletteResolver, v7.12 ToolResultLedger: Protocol on BaseAgentDeps + framework- registered default adapter + adopter-overridable via deps.register(MessagesRegionCacheStrategy, custom_impl).

Concerns addressed (C1-C4 per architect ADR 2026-05-31)

• C1: Gate by prefix size -- below model-specific threshold, place no breakpoint (don't waste one of Anthropic's 4-breakpoint budget).
• C2: Boundary anchored on a CLOSED AIMessage (all tool_use IDs satisfied), never on a ToolMessage whose bytes drift per turn.
• C3 Policy A (v7.13.0): stateless strategy recomputes boundary each turn. When _pair_aware_history_slice drops messages, the NEW oldest-survivor becomes the breakpoint; one cold-write turn after a slice is the tolerable cost.
• C3 Policy B (v7.13.1, opt-in): window-aware preemptive placement for very long conversations. NOT shipped in v7.13.0.
• C4: thinking-block strip helper for pre-4.5 model families. Defensive on opus-4.5+/sonnet-4.6+ (the empirical test confirmed no strip needed); load-bearing for older families.

LastStableTurnBoundaryStrategy

LastStableTurnBoundaryStrategy()

Default v7.13.0 implementation (v7.15.1 sticky-anchor memoization): anchor on the most recent closed AIMessage / HumanMessage (C2) when the prefix exceeds the model-family threshold (C1).

Lifecycle: ONE instance per agent (registered in engine.py via deps.register(MessagesRegionCacheStrategy, ...)). The same instance is consulted on every React iteration of every turn. We exploit that lifecycle to memoize the anchor's IDENTITY (not its index) across iterations.

Why sticky (v7.15.1): v7.15.0 telemetry showed the stateless walk picked the NEWEST closed AIMessage every iteration. On a multi-iteration React turn, the anchor advanced on every iteration, invalidating Anthropic's cached prefix on each advance and re- billing cache_creation_input_tokens (~$1.7K/mo/brand silent overpay measured at Jarvio). Sticky behavior pins the anchor until it's either (a) dropped from messages (history slice) or (b) no longer closed under C2, at which point we re-anchor.

Wire-neutrality safeguards (all auto-disable, no flag needed):

• Non-Anthropic provider -> None (other providers ignore cache_control but cleanliness matters).
• Prefix below model threshold -> None (don't waste an Anthropic breakpoint budget slot).
• No stable boundary found -> None.

Concurrency (v8.7.1 H1): the sticky anchor is isolated PER CONVERSATION in a bounded LRU keyed by conversation_id (the react run_id). ONE agent serves ALL concurrent requests in create_agent_router and its BaseAgentDeps (and this strategy instance) are shared across them, so a single scalar anchor was clobbered by interleaved React loops on different conversations — moving the marker and re-billing the full prefix (the v7.15.0 regression class). Keying per conversation removes the shared mutable state; the never-move math is unchanged. Each place_breakpoint call is synchronous (no await inside), so a given conversation's read- modify-write of its own bucket is atomic.

Adopter constraint: sticky behavior depends on AIMessage.id being stable across React iterations. LangChain assigns these automatically. Adopters constructing AIMessage directly with id=None degrade silently to the v7.13.0 fresh-walk semantic on every call (no correctness bug; no cost win).

Configuration is read from FrameworkConfig.messages_cache_policy elsewhere; this strategy is the default whenever the policy is "default".

Source code in src/symfonic/core/prompt/messages_cache.py

def __init__(self) -> None:
    # v8.7.1 (H1): sticky anchor id isolated PER CONVERSATION in a
    # bounded LRU keyed by ``conversation_id`` (react run_id). ``None``
    # maps to a single shared bucket for callers that don't thread an
    # id. Cleared when the anchor is no longer present in ``messages``
    # or no longer closed under C2.
    self._sticky_anchor_ids: OrderedDict[str, str] = OrderedDict()

MessagesRegionCacheStrategy

Bases: Protocol

v7.13.0 Path E -- decides where (if anywhere) to place a cache_control breakpoint in the messages region of an Anthropic request.

Registered on :class:BaseAgentDeps. The react node invokes place_breakpoint once per request after _consolidate_messages assembles the wire-accurate message list; when the strategy returns an index, the framework annotates the trailing content block of messages[index] with cache_control={"type":"ephemeral"}.

Implementations MUST be pure: same inputs => same index. The cache key on subsequent turns depends on the placement being deterministic.

place_breakpoint

place_breakpoint(
    messages: Sequence[Any],
    *,
    provider_family: ProviderFamily,
    model_id: str,
    thinking_enabled: bool = False,
    prior_breakpoint_indices: Sequence[int] = (),
    conversation_id: str | None = None,
) -> int | None

Return the index of the message AFTER which the breakpoint belongs, or None to skip placement this turn.

Parameters:

Name	Type	Description	Default
messages	Sequence[Any]	The wire-accurate message list (post- _consolidate_messages).	required
provider_family	ProviderFamily	Detected from the model provider; only "anthropic" actually honors cache_control.	required
model_id	str	e.g. "claude-opus-4-6" -- used to look up the minimum cacheable prefix threshold.	required
thinking_enabled	bool	True when ModelConfig.thinking is set. Strategies may apply C4 defensive handling.	False
prior_breakpoint_indices	Sequence[int]	Indices of any other breakpoints already placed (currently always empty; reserved for future composability with adopter-placed breakpoints).	()
conversation_id	str | None	v8.7.1 (H1) -- the per-conversation key (the react run_id) under which stateful strategies isolate their sticky/rolling marker state. None maps to a single shared bucket (legacy behaviour for callers that don't thread it).	None

Source code in src/symfonic/core/prompt/messages_cache.py

def place_breakpoint(
    self,
    messages: Sequence[Any],
    *,
    provider_family: ProviderFamily,
    model_id: str,
    thinking_enabled: bool = False,
    prior_breakpoint_indices: Sequence[int] = (),
    conversation_id: str | None = None,
) -> int | None:
    """Return the index of the message AFTER which the breakpoint
    belongs, or ``None`` to skip placement this turn.

    Args:
        messages: The wire-accurate message list (post-
            ``_consolidate_messages``).
        provider_family: Detected from the model provider; only
            ``"anthropic"`` actually honors ``cache_control``.
        model_id: e.g. ``"claude-opus-4-6"`` -- used to look up
            the minimum cacheable prefix threshold.
        thinking_enabled: True when ``ModelConfig.thinking`` is set.
            Strategies may apply C4 defensive handling.
        prior_breakpoint_indices: Indices of any other breakpoints
            already placed (currently always empty; reserved for
            future composability with adopter-placed breakpoints).
        conversation_id: v8.7.1 (H1) -- the per-conversation key
            (the react ``run_id``) under which stateful strategies
            isolate their sticky/rolling marker state. ``None`` maps to
            a single shared bucket (legacy behaviour for callers that
            don't thread it).
    """
    ...

MultiBreakpointCacheStrategy

Bases: Protocol

v8.7.0 -- a strategy that can place a LADDER of messages-region cache_control breakpoints (not just one), within a caller- supplied marker budget.

The react node prefers this method when the registered strategy satisfies this Protocol (isinstance check), passing the number of messages-region marker slots still free after the system prefix and tools array have claimed theirs. Strategies that only implement the single-breakpoint :class:MessagesRegionCacheStrategy continue to work unchanged (the node wraps their result in a 1-element list).

Implementations MUST be pure w.r.t. their inputs PLUS their own persisted rolling state, and MUST NOT return more than budget indices (a 5th total marker on the wire is an Anthropic 400).

place_breakpoints

place_breakpoints(
    messages: Sequence[Any],
    *,
    provider_family: ProviderFamily,
    model_id: str,
    budget: int,
    thinking_enabled: bool = False,
    prior_breakpoint_indices: Sequence[int] = (),
    conversation_id: str | None = None,
) -> Sequence[int]

Return ASCENDING message indices to annotate (at most budget of them), or an empty sequence to skip placement.

Parameters:

Name	Type	Description	Default
budget	int	Messages-region marker slots still free within Anthropic's 4-marker/request cap. <= 0 means the system prefix + tools array already consumed the whole budget; the strategy MUST return ().	required
conversation_id	str | None	v8.7.1 (H1) -- per-conversation isolation key for the ladder's held-rung state (the react run_id).	None

Remaining arguments are as :meth:MessagesRegionCacheStrategy.place_breakpoint.

Source code in src/symfonic/core/prompt/messages_cache.py

def place_breakpoints(
    self,
    messages: Sequence[Any],
    *,
    provider_family: ProviderFamily,
    model_id: str,
    budget: int,
    thinking_enabled: bool = False,
    prior_breakpoint_indices: Sequence[int] = (),
    conversation_id: str | None = None,
) -> Sequence[int]:
    """Return ASCENDING message indices to annotate (at most
    ``budget`` of them), or an empty sequence to skip placement.

    Args:
        budget: Messages-region marker slots still free within
            Anthropic's 4-marker/request cap.  ``<= 0`` means the
            system prefix + tools array already consumed the whole
            budget; the strategy MUST return ``()``.
        conversation_id: v8.7.1 (H1) -- per-conversation isolation key
            for the ladder's held-rung state (the react ``run_id``).

    Remaining arguments are as
    :meth:`MessagesRegionCacheStrategy.place_breakpoint`.
    """
    ...