Skip to content

Model Routing

symfonic-core treats model selection as an adopter-owned concern. The framework orchestrates the agent loop; the adopter assigns models per role and picks the provider that serves each model. This concept doc explains how the four knobs that touch model selection compose, and why the architectural line sits where it does.

The four knobs

Knob Layer Default What it controls
AgentConfig.model core/config.py empty ModelConfig The DEFAULT model for every site in the framework that calls an LLM.
FrameworkConfig.role_models (v7.9.0) agent/config.py:1017 {} Per-role ModelConfig overrides. v7.9.0 wires "action"; v7.9.1+ wires the remaining roles.
FrameworkConfig.procedural_force_first_action_tool (v7.8.3) agent/config.py:909 False Proactive forcing — set tool_choice to the bound action tool on turn 1 when one bare-id precondition resolves unambiguously.
ModelProvider.supports_forced_tool_choice(config) (v7.8.5) core/providers.py:51 True Per-provider, per-config introspector. Lets adopters declare "this model rejects forced tool_choice under these conditions" so the engine abstains cleanly.

The load-bearing architectural principle

The framework may constrain WHEN and WITH WHAT TOOLS the model is invoked, but the framework MUST NOT author the model's tool arguments or modify model-owned reasoning surfaces. Procedural compliance is the framework's job; reasoning and arguments remain the model's. Model assignment per role is the adopter's job.

This principle was codified in v7.8.3 and has held through every subsequent release. It draws a hard line in three places:

  1. Forcing is OK — constraining the choice set to a specific tool name is a framework responsibility.
  2. Arg authorship is NOT OK — pre-injecting an AIMessage with tool_calls already populated would make the framework the agent and the model a passenger.
  3. Reasoning toggles are NOT OK — auto-disabling config.thinking to make forcing succeed would cross from choice-set constraint into reasoning-suppression.

The four knobs above all sit inside this line.

The canonical recipe: asymmetric routing

The pattern emerged from a production 3-engine A/B/C probe (2026-05-27) which documented a model-class compliance ceiling: the adopter's legacy gpt-4.1-nano agent scored 6/6 on the WLM probe vs the symfonic claude-opus-4-6 + thinking agent's 1/6, on the same prompt with the same tool catalog. The variable was the model class — opus' RLHF prefers verbal narration on explanatory prompts; nano was trained on a tighter "see tool → call tool" prior.

The framework can't retrain a model. It CAN give adopters the asymmetric-routing knob:

from symfonic.core.providers import (
    AnthropicProvider, MultiProviderRouter, OpenAIProvider,
)
from symfonic.core.config import AgentConfig, ModelConfig
from symfonic.agent import FrameworkConfig, SymfonicAgent

provider = MultiProviderRouter(
    default=AnthropicProvider(),
    routes={
        "gpt-": OpenAIProvider(),
        "o1-":  OpenAIProvider(),
        "o3-":  OpenAIProvider(),
    },
)

agent = SymfonicAgent(
    model_provider=provider,
    config=FrameworkConfig(
        agent=AgentConfig(
            model=ModelConfig(
                model_name="claude-opus-4-6",
                thinking={"type": "enabled", "budget_tokens": 4096},
            ),
        ),
        role_models={
            "action": ModelConfig(model_name="gpt-4.1-nano"),
        },
        procedural_force_first_action_tool=True,
        procedural_render_preflight_in_l1=True,
        # ... your other knobs
    ),
)

What happens at runtime:

Turn type Model Forcing eligible? Why
General reasoning claude-opus-4-6 + thinking NO AnthropicProvider.supports_forced_tool_choice returns False when config.thinking is set. Engine abstains; reactive precondition_gate (v7.7.4) remains.
Action turn (the react node brain LLM) gpt-4.1-nano YES MultiProviderRouter dispatches to OpenAIProvider; supports_forced_tool_choice returns True; v7.8.3 forcing engages.

The composition reads "opus thinks about everything; nano acts on procedural rules." That's the asymmetric-routing pattern.

Why each piece is necessary

Without procedural_force_first_action_tool=True: the model emits zero tool_use blocks under load, narrates compliance instead of acting (a documented failure mode under claude-opus-4-6 + thinking).

Without MultiProviderRouter: role_models["action"]= ModelConfig(model_name="gpt-4.1-nano") with AnthropicProvider crashes at the API boundary with 404 model_not_found — AnthropicProvider.get_chat_model hard-codes ChatAnthropic regardless of model_name.

Without supports_forced_tool_choice returning False on Anthropic + thinking: the engine's tool_choice payload reaches the Anthropic API and the call crashes with 400 (Anthropic's published constraint: forced tool_choice is incompatible with extended thinking).

Without role_models["action"] override: the action turn runs on the default model — claude-opus-4-6 + thinking — which hits the model-class compliance ceiling and emits zero tool_use blocks.

All four pieces are necessary; remove any one and the composition breaks at a known wire-layer or API-layer failure mode.

Inspecting your composition before going live

A reference configuration prints a routing table for any composition you build:

  role=action         model=gpt-4.1-nano    provider=OpenAIProvider    forces_ok=True
  role=summary        model=claude-opus-4-6 provider=AnthropicProvider forces_ok=False
  role=critic         model=claude-opus-4-6 provider=AnthropicProvider forces_ok=False
  ...

forces_ok=True means the engine's v7.8.3 force lever will engage on that role's turns; forces_ok=False means it will abstain (refuse-to-force pattern). Use this output to verify your composition before pointing real API keys at it.

Role taxonomy

Role What it routes v7.9.0 status
action The react node brain LLM (the model that consumes tool definitions and emits tool_use blocks) WIRED
summary Context-window summarisation (core/nodes/context_window.py:128) Reserved, wires in v7.9.x
critic Metacognition critic (agent/middleware/metacognitive.py:270) Reserved, wires in v7.9.x
router CapabilityRouter (memory/layers/procedural/router.py) Reserved, wires in v7.9.x
reflection Self-reflection (engine.py:6481) Reserved, wires in v7.9.x
consolidation Phase 12 promotion (core/learning/consolidation.py) Reserved, wires in v7.9.x

Adopters can SET reserved roles today — the resolver returns them correctly via SymfonicAgent.resolve_model_config(role) — the consuming sites just don't read them yet. The wirings land in subsequent v7.9.x patches.

When to use which knob

Symptom Knob
Model narrates "I tried to call X but couldn't" instead of emitting tool_use procedural_force_first_action_tool=True
Above PLUS using opus + thinking Above + asymmetric routing to a non-thinking model for the action role
Anthropic API returns 400 on forced turns Either disable thinking on ModelConfig OR route the action role to a non-Anthropic model
Want one provider for cheap/fast tool calls and another for expensive reasoning MultiProviderRouter + role_models["action"]=ModelConfig(model_name=...)
Custom provider (in-house model server) Implement the ModelProvider protocol; MultiProviderRouter accepts any conforming provider as default or as a route

What model routing does NOT do

  • Does not retrain the model. Model-class compliance ceilings are real; the framework routes around them but cannot eliminate them.
  • Does not modify ModelConfig instances. The resolver returns the adopter's exact instance; the router dispatches without copy or mutation.
  • Does not auto-disable thinking. Adopters who want thinking off should set it off in ModelConfig. v7.8.5 refuse-to-force is the principled response when thinking forbids forcing.
  • Does not author tool arguments. v7.8.3 declared this off- limits; v7.8.5 and v7.9.x preserve the line.
  • Does not cache per-role chat model instances. get_chat_model is called per turn by the framework; adopters who need instance-level caching wrap the provider themselves.

Out-of-graph consumers (v7.16.0)

Background workers (Celery, RQ, APScheduler) often need to invoke a chat model outside the agent's run() / stream() graph — the canonical example is a scheduled job that summarises a batch of new memories overnight, or a cron task that critiques a finished conversation asynchronously.

Pre-v7.16 the adopter pattern was:

chat = agent._model_provider.get_chat_model(agent._config.agent.model)

That works, but it bypasses resolve_model_config(role) entirely — any role_models[role] overrides the adopter has configured get silently ignored, and the worker silently runs on the default AgentConfig.model instead of the role's bound model. Adopters hit this when role routing worked in the foreground but not in the worker, with no error to point at.

The supported pattern from v7.16.0 is:

chat = agent.get_chat_model("action")    # or "summary", "critic", ...
response = await chat.ainvoke([HumanMessage(content="...")])

SymfonicAgent.get_chat_model(role) is stateless and safe to call from any thread or process — it reads only self._config and dispatches to the provider's get_chat_model. Adopters instantiate one SymfonicAgent per worker process and call this from task bodies; role routing flows through unchanged.

Default role="action" matches the v7.9.0 wired role (the tool-using brain LLM archetype). Pass a different role string when your worker plays a different cognitive role — for example, a nightly summariser would call agent.get_chat_model("summary") and an offline critic would call agent.get_chat_model("critic"). Unknown roles fall back to AgentConfig.model (the same fallback resolve_model_config documents).

The old agent._model_provider.get_chat_model(...) pattern keeps working — the framework does not force migration — but role-aware adopters should prefer the public get_chat_model(role) method.

Pre-v7.8.x context

Before v7.8.3, model selection was implicit: the agent used config.agent.model for every LLM call site. There was no per-role override, no forcing knob, and no provider introspection. Adopters who wanted different models per site wrote their own dispatcher logic outside the framework.

The v7.8.x and v7.9.x line shipped the four knobs above as canonical surfaces, with the v7.8.3 architectural principle as the governing line. Adopters can still write custom dispatchers — the ModelProvider protocol is runtime_checkable — but the common patterns now have first-class framework support.

References