Per-dispatch model resolver (v7.23)¶

Adopters drive cost-vs-quality trade-offs by routing each react iteration to a different model. The ModelResolver Protocol is the v7.23 first-class hook.

v8.1.0 — declarative per-intent model: if you want per-intent model selection together with deterministic tool routing from a single declaration, see tool-call-policy.md. A ToolCallPolicy with a model set is evaluated at this same PRE-model seam before the role_model_resolver (policies-first precedence): the first policy whose match(ctx) is True and that carries a model supplies the model for the turn; otherwise control falls through to the resolver described below, then the snapshot default. The resolver remains the free-form catch-all; the policy list is the declarative, inspectable table for specific intents.

This page covers the seven topics every adopter must internalise before wiring a resolver:

Why per-dispatch dispatch
The ModelResolver Protocol + closure-factory pattern
DispatchContext — locked 7-field MVP + four explicit non-fields
Cache invariants under per-turn switching
Additive narrowing — react brain LLM dispatch only
Cross-worker cache_state — out-of-scope-for-framework
The v7.8.5 force-resolver synchronisation

1. Why per-dispatch dispatch¶

Pre-v7.23, model selection happened ONCE at agent build time via FrameworkConfig.role_models["action"]. Static role-to-model mapping covered "all action turns use gpt-4.1-nano" but couldn't express "this specific turn is a complex reasoning task, route to Opus; the next turn is a simple lookup, route to Haiku."

v7.23 closes that gap. The adopter supplies a callable that runs on EVERY react iteration and picks the model for THIS turn. Typical adopter shape:

A classifier (gpt-4.1-nano cost ~$0.002/turn) on the latest message decides "simple" vs "complex".
A complexity hint on the procedural skill metadata routes the procedural-rule turn to a high-compliance model.
Cache-state awareness (see §4) keeps the resolver on the warmest model when both candidates are close in cost.

Cost-vs-quality is the lever. The framework stays out of the policy; it just exposes the seam.

2. The `ModelResolver` Protocol + closure-factory pattern¶

from typing import Optional
from symfonic.core.contracts.resolvers import (
    DispatchContext, ModelResolver,
)
from symfonic.core.config import ModelConfig


@runtime_checkable
class ModelResolver(Protocol):
    def __call__(
        self, ctx: DispatchContext,
    ) -> Optional[ModelConfig]:
        ...

A bare callable (function, lambda, or class with __call__) satisfies the Protocol. Returning None is the abstain signal — the framework falls through to the snapshot default (AgentConfig.model).

The closure-factory pattern is the recommended construction shape:

def make_resolver(
    agent: SymfonicAgent,
    classifier: Classifier,
    pricing_table: dict[str, float],
) -> ModelResolver:
    """Capture adopter state at agent-build time."""

    OPUS = ModelConfig(model_name="claude-opus-4-5", thinking={"budget_tokens": 1024})
    HAIKU = ModelConfig(model_name="claude-haiku-4-5")

    def resolve(ctx: DispatchContext) -> Optional[ModelConfig]:
        # Adopter-side classifier picks the right model for THIS turn.
        # Read whatever state the closure captured.
        last_message = ctx.messages[-1] if ctx.messages else None
        if last_message is None:
            return None  # snapshot default

        if classifier.is_complex(last_message.content):
            return OPUS
        return HAIKU

    return resolve


framework_config = FrameworkConfig(
    role_model_resolver=make_resolver(agent, classifier, pricing_table),
)

The closure carries whatever adopter state the resolver needs (agent reference, classifier handle, postgres connection, pricing table). The framework stays oblivious to the adopter's policy — it just calls the closure.

3. `DispatchContext` — 7-field MVP + `turn_hop_index` (v8.3.0)¶

Round-2 architect verdict (2026-06-08) LOCKED seven fields; v8.3.0 added turn_hop_index (the 8th) on a production adopter's hop-aware- routing request:

Field	Type	Meaning
`role`	`str`	Always `"action"` in v7.23.
`messages`	`Sequence[BaseMessage]`	Conversation about to be sent.
`scope`	`TenantScope`	v7.21 `from_state_dict` rebuild.
`forced_tool_choice`	`Optional[str]`	Bare tool name when force is on.
`resolved_tool_names`	`Optional[tuple[str, ...]]`	Intent-narrowed tool list.
`cache_state`	`Mapping[str, datetime]`	Per-engine last cache_creation.
`run_id`	`str`	Run UUID for correlation.
`turn_hop_index`	`int` (v8.3.0)	In-turn react hop index: `0` on the first model call of the current user turn, +1 per hop, reset to `0` on a new user turn.

Four explicit non-fields (closed decisions; do NOT reopen):

Non-field	Why rejected	Adopter workaround
`procedure_id` / `skill.node_id`	`winning_procedure_id` is a transient local; not in state.	Closure captures the agent reference; call `agent.get_active_skills()` inside the resolver.
`turn_index`	Engine-instance GLOBAL turn counter, NOT LangGraph state. (Distinct from `turn_hop_index`, which is the IN-turn hop counter and IS a field.)	Compute adopter-side from message count or external counter.
`intent_verdict`	Computed at `engine.py:6580` but never propagated into react state.	Compute adopter-side via classifier on `ctx.messages[-1].content`.
`intent_tags`	Phantom field; doesn't exist in `IntentVerdict`.	Not a real engine concept.

Future addition path: v8.0 backlog §5 ("DispatchContext field expansion"). First adopter ask with a concrete use case reopened it for turn_hop_index.

3.1 `turn_hop_index` — hop-aware routing (v8.3.0)¶

turn_hop_index is the number of assistant (AIMessage) turns the model has already produced since the start of the current user turn: 0 on the first model call of a turn, incrementing by one per react hop within the turn, reset to 0 when a new user turn begins.

The turn boundary is the last HumanMessage in ctx.messages (the genuine current user turn). Tool results come back as ToolMessage, and the ask_user / interrupt resume flows (v7.1.0) also inject a ToolMessage on resume — so neither falsely resets the hop count mid-turn. This is distinct from the v7.15.0 iteration_index, which counts global AIMessages across the whole conversation and therefore accumulates across turns (so "iteration 0" there is NOT "first hop of this turn" on a multi-turn conversation).

Use case — think on the first hop, skip thinking on mechanical hops:

THINKING_ON = ModelConfig(model_name="claude-opus-4-5", thinking=True)
THINKING_OFF = ModelConfig(model_name="claude-opus-4-5", thinking=False)

def hop_aware_resolver(ctx: DispatchContext) -> Optional[ModelConfig]:
    # First model call of the turn: let the model reason.
    if ctx.turn_hop_index == 0:
        return THINKING_ON
    # Later mechanical hops (tool-result consumption): skip thinking
    # to shave latency where the model is just threading a result.
    return THINKING_OFF

Documented boundary behaviour: the boundary is defined as the last HumanMessage. No node on the normal react path appends a HumanMessage to state["messages"] mid-turn (verified 2026-06-14); the other HumanMessage(...) uses in the engine build one-off .ainvoke([...]) sub-prompts that never touch graph state. If a custom node were to inject a HumanMessage mid-turn, the hop count would reset there by definition — this is the documented contract, not an accident.

DispatchContext is dataclass(frozen=True). Mutations raise. cache_state is typed Mapping, not dict — adopters can read but cannot mutate the engine's internal state.

4. Cache invariants under per-turn switching¶

The Anthropic prompt cache is per-model. Switching from Opus to Sonnet between turns 5 and 6 forces a cold cache_creation on Sonnet because the cache key includes the model identifier. Naive per-turn switching collapses the cache benefit.

v7.23 ships Option 3.4 sticky-with-TTL. The resolver consults ctx.cache_state.get(target_model, EPOCH) and applies the rule:

TTL = timedelta(minutes=5)
EPOCH = datetime(1970, 1, 1, tzinfo=UTC)

def sticky_resolver(ctx: DispatchContext) -> Optional[ModelConfig]:
    target = pick_quality_optimal(ctx)
    age = datetime.now(UTC) - ctx.cache_state.get(target.model_name, EPOCH)
    if age < TTL:
        return target  # cache still warm; switch is free
    # Cache expired; staying on the previous model is also fine.
    return None  # snapshot default (typically the cheaper baseline)

The framework stamps cache_state[model_name] on every response with cache_creation_input_tokens > 0 (see SymfonicAgent._on_llm_response). Stamps are in-memory per-engine-instance.

Per-model cache scoping (Option 3.2) is deferred to v8.0 backlog §4. Wire-protocol constrained; not solvable inside framework boundaries.

5. Additive narrowing — react brain LLM dispatch ONLY¶

BOLD note for adopter mental models:

The ModelResolver applies to the react brain LLM dispatch site (react.py:678) and the v7.23-swapped force-introspection site (engine.py:2861). It does NOT apply to:

Reflection at engine.py:8284 — consults static role_models["reflection"].

Public agent.get_chat_model("action") at engine.py:2426 — consults static role_models["action"].

Why narrowed?

get_chat_model is stateless + safe-to-call-from-any-thread (v7.16.0 contract). Background workers (Celery, APScheduler) call it without a DispatchContext — they have no messages, no scope, no run_id. Consulting the resolver would either crash or require fabricated context that's misleading.
Reflection runs in a different lifecycle (post-conversation) where the "decide model for THIS turn" framing doesn't apply. The static role_models["reflection"] lookup is the right shape.

Future extension via the same ModelResolver Protocol with role="reflection" is v8.0 backlog §6.

Invariant tests (tests/agent/test_resolver_additive_narrowing.py) pin the current contract so future sessions can't silently extend it.

6. Cross-worker `cache_state` — out-of-scope-for-framework¶

cache_state is in-memory per-engine-instance. Multi-worker deployments (Celery, gunicorn workers, APScheduler workers) see DIFFERENT cache_state per worker. Worker A's cache_creation stamp is invisible to worker B.

The framework explicitly does NOT coordinate cross-worker cache_state. v8.0 backlog §7 documents the decision. Adopters with multi-worker deployments have two recommended patterns:

Pattern 1 — Session affinity¶

Route each conversation (session_id) to a single worker. The same worker handles every turn for that session; in-memory cache_state is correct because every turn's stamp lands on the same engine.

Implementation: a load balancer that hashes session_id to worker. Most session-aware web frameworks support this natively.

Pattern 2 — External Redis store¶

Replace the in-memory dict with a Redis-backed wrapper. The resolver reads from Redis; _on_llm_response writes to Redis. Cross-worker visibility is real-time.

class RedisCacheState:
    def __init__(self, redis_client):
        self._redis = redis_client

    def __getitem__(self, model_name: str) -> datetime:
        raw = self._redis.get(f"cache_state:{model_name}")
        return datetime.fromisoformat(raw) if raw else EPOCH

    def get(self, model_name: str, default: datetime = EPOCH) -> datetime:
        try:
            return self[model_name]
        except KeyError:
            return default

The adopter wires this into the resolver closure (the framework's in-memory view is ignored).

7. The v7.8.5 force-resolver synchronisation¶

v7.8.5 (a thinking+forcing 400 crash fix) established the refuse-to-force-under-thinking contract. When a provider refuses to honour tool_choice for a given ModelConfig (Anthropic + thinking enabled), the engine MUST abstain — adding the tool_choice block to the wire request would 400-crash.

Pre-v7.23 the introspection consulted resolve_model_config("action") (static role_models["action"]). v7.23 per-iteration dispatch breaks this: when the resolver picks Opus+thinking for THIS turn while static says Sonnet+no-thinking, the introspector returns True (Sonnet accepts) but the actual LLM call targets Opus+thinking and Anthropic 400-crashes.

v7.23 T-7.23.8 swaps the action_model source to the per-iteration _AgentModelResolver result. The supports_forced_tool_choice(config) + refusal_reason(config) introspection shape is byte-identical; only the SOURCE of the ModelConfig argument changes.

Double-call avoidance: the react node stashes the resolved ModelConfig on state["_resolved_action_model"] (see react.py:678). The force-introspection site reads from the stash first; only on out-of-react direct callers does it re-invoke the resolver. Single resolve per iteration — adopter's classifier cost (~$0.002/turn) doesn't double.

Implication for adopters: if your custom ModelProvider implements supports_forced_tool_choice(config), the function now receives the PER-ITERATION ModelConfig, not the static role_models one. Implementations that check config.thinking (the canonical Anthropic pattern) keep working unchanged.

Per-dispatch model resolver (v7.23)¶

1. Why per-dispatch dispatch¶

2. The ModelResolver Protocol + closure-factory pattern¶

3. DispatchContext — 7-field MVP + turn_hop_index (v8.3.0)¶

3.1 turn_hop_index — hop-aware routing (v8.3.0)¶