Skip to content

Model Providers

A ModelProvider turns a ModelConfig into a LangChain chat model. Every provider is optional and lazy-imported — install only the extra you use. Pass a provider to SymfonicAgent(model_provider=...); the model itself comes from ModelConfig.model_name.

from symfonic.agent.engine import SymfonicAgent
from symfonic.agent.config import FrameworkConfig
from symfonic.core.config import AgentConfig, ModelConfig
from symfonic.core.providers import AnthropicProvider

agent = SymfonicAgent(
    model_provider=AnthropicProvider(),
    config=FrameworkConfig(agent=AgentConfig(model=ModelConfig(model_name="claude-sonnet-4-5"))),
)

Catalog

Provider Extra LLM_MODEL example Wire family Notes
AnthropicProvider [anthropic] claude-sonnet-4-5 anthropic Extended thinking, prompt cache. ANTHROPIC_API_KEY
OpenAIProvider [openai] gpt-4o openai OPENAI_API_KEY; custom base_url for OpenAI-compatible endpoints
DeepSeekProvider [deepseek] deepseek-chat openai OpenAI-compatible; R1 reasoning
KimiProvider [kimi] kimi-k2-0905-preview openai Moonshot AI, OpenAI-compatible
GoogleProvider [google] gemini-2.5-pro google Gemini 2.5 thinking
OllamaProvider [ollama] llama3 openai Local; no cloud API
AWSBedrockProvider [aws] us.anthropic.claude-sonnet-4-5-20250929-v1:0 auto Any Bedrock model; see below
OpenRouterProvider [openrouter] anthropic/claude-sonnet-4.5 unknown Gateway to many vendors; see below
AnthropicOAuthProvider [anthropic] claude-haiku-4-5 anthropic dev/test only — Claude Code subscription token; see the OAuth guide
CodexOAuthProvider [openai] gpt-5.6-sol openai dev/test only — ChatGPT subscription token (OpenAI Codex backend)
KimiOAuthProvider [kimi] kimi-for-coding openai dev/test only — kimi.com subscription token (coding backend)

The three *OAuthProvider classes replay a personal subscription OAuth token instead of a metered API key. They are a local development convenience only — each refuses to construct in a production-like environment. See the Subscription OAuth providers guide.

The wire family decides which content-block dialect and cache/thinking breakpoints the engine uses. unknown is wire-neutral: no provider-specific cache_control or thinking blocks are stamped onto the request.

AWS Bedrock

Run any Bedrock model — Claude, Llama, Nova, Mistral, Titan, Cohere — through the Converse API. LLM_MODEL must be a Bedrock model id or inference-profile id (not a bare Anthropic SKU).

pip install "symfonic-core[aws]"

from symfonic.core.providers import AWSBedrockProvider

# Region + credentials resolve via the standard boto3 chain
# (env vars, a shared profile, or an EC2/ECS/EKS instance role).
provider = AWSBedrockProvider(region_name="us-east-1")

Key behaviors:

  • Family is auto-detected per model (bedrock_family_for_model): Anthropic ids get the anthropic family (extended thinking via additional_model_request_fields); every other vendor is treated wire-neutral (unknown), so no Anthropic-specific block is sent to a non-Claude model. Override with AWSBedrockProvider(provider_family="unknown").
  • temperature and top_p are mutually exclusive. Bedrock's Converse API rejects both together for Claude. The provider sends top_p when set (nucleus sampling), otherwise temperature.
  • Multi-tenant credentials: pass credentials_resolver=lambda: {...} — a zero-arg callable returning per-request region_name / aws_access_key_id / aws_secret_access_key / aws_session_token. The result is threaded into the client and never stored on the instance.
  • Cost telemetry for Claude-on-Bedrock uses Anthropic list rates automatically (see Cost Tracking); non-Anthropic vendors need a pricing override.
  • Model access: newer Anthropic models on Bedrock require submitting the "Anthropic use case details" form in the Bedrock console (Model access) before the account can invoke them.
  • Prompt cache: Bedrock's cachePoint wire is not identical to Anthropic-native cache_control; leave the messages-region cache disabled on Bedrock until a cachePoint adapter lands.

Scaffold it end-to-end: symfonic init myapp --llm-provider aws (sets LLM_PROVIDER=aws, a default Bedrock LLM_MODEL, and AWS_REGION in .env).

OpenRouter

OpenRouter is a gateway that routes to models from many vendors behind one OpenAI-compatible endpoint. LLM_MODEL is a fully-qualified id like anthropic/claude-sonnet-4.5, openai/gpt-4o, or meta-llama/llama-3.3-70b-instruct.

pip install "symfonic-core[openrouter]"

from symfonic.core.providers import OpenRouterProvider

provider = OpenRouterProvider(
    # api_key defaults to the OPENROUTER_API_KEY env var; fails closed if unset
    # (it never falls back to OPENAI_API_KEY).
    site_url="https://your-app.example.com",  # optional HTTP-Referer
    app_name="Your App",                       # optional X-Title
)

Key behaviors:

  • Auth via OPENROUTER_API_KEY (or explicit api_key=). The provider raises rather than let the underlying client inherit OPENAI_API_KEY and transmit an OpenAI credential to openrouter.ai.
  • Wire-neutral unknown family — the underlying model varies per request, so no Anthropic-/OpenAI-specific cache or thinking assumptions are made.
  • Cost telemetry normalizes vendor/model ids to first-party rates when the underlying model is known (Anthropic / OpenAI / Google); other vendors need a pricing override.

Scaffold it: symfonic init myapp --llm-provider openrouter.

Custom providers and routing

Implement the ModelProvider protocol for an in-house model server, and declare a wire family with _symfonic_provider_family: ClassVar[str] = "anthropic" (or openai / google / unknown). To assign different providers/models per role or per model-id prefix, see Model Routing and the MultiProviderRouter.

Provider-family-aware memory extraction (9.2+)

The HMS memory-extraction directive is tuned per provider family, because instruction-following models parse it differently:

  • Anthropic (and unknown/custom families): the default delimiter-based <MEMORY_EXTRACT> directive.
  • OpenAI-family and Google: a stricter <GRAPH_OPERATIONS> JSON protocol, auto-selected because these models ignore the delimiter format more often and silently drop memory writes. Wire-neutral for Anthropic.

The family is taken from the resolved model config, so a router serving a different vendor than its default still gets the right template. To override the directive entirely — e.g. for a self-hosted Qwen/DeepSeek model that classifies as unknown but needs the JSON protocol — point at your own template:

FrameworkConfig(extraction_template_path="/path/to/my_extraction.txt")

An explicit extraction_template_path always wins over the family default. Both the built-in and custom templates support the additive ops plus retract_node (see Consolidation & Deep Sleep).

References