Binding a turn's tool palette without recompiling the plan.
Lazy tool routing selects a subset of the palette from the turn's own input, so
what the model is offered differs per turn. The plan binds tools once, at
compile time, and is immutable by contract (IPL-4) — rebinding requires a new
plan and there is no setter.
Both are kept. The projector is built at compile time, holds the unbound chat
model and the full palette, and hands back a model bound to whichever subset a
turn resolved. The plan's identity never moves: equality_key excludes
bindings, which is the same reason event_sink and stage_handlers live
there — a reference frozen at compile time whose behaviour stays live.
Cached by palette identity, not by turn. Two turns that resolve the same
subset share one bound model, and a provider that charges for binding pays once.
The identity is the sorted tool names, so it is stable under a resolver that
returns the same set in a different order — which the procedural layer does,
since it ranks.
None means the full palette and returns the binding the plan already
made, byte for byte. That is the no-router case and the disabled case, and it
must stay indistinguishable from the behaviour before this module existed.
PaletteProjector
PaletteProjector(model: Any, tools: Any, bound: Any, *, wrap: Any = None, forcing: Any = None, bind_with: Any = None)
Binds the model to a subset of the compiled palette, on demand.
Parameters:
| Name |
Type |
Description |
Default |
model
|
Any
|
the unbound chat model, kept so a subset can be bound.
|
required
|
tools
|
Any
|
the compiled palette, in the order the plan holds it.
|
required
|
bound
|
Any
|
what the plan already bound and the caller already calls --
the adapter, not the raw model. Returned verbatim for the full
palette so the unrouted path is not merely equivalent to the
previous behaviour but identical to it.
|
required
|
wrap
|
Any
|
turns a freshly bound model into the same shape as bound.
Injected because the adapter is the caller's concern: this
module knows how to narrow a palette and nothing about how the
kernel prefers to call one.
|
None
|
Source code in src/symfonic/kernel/palette.py
| def __init__(
self,
model: Any,
tools: Any,
bound: Any,
*,
wrap: Any = None,
forcing: Any = None,
bind_with: Any = None,
) -> None:
"""
Args:
model: the *unbound* chat model, kept so a subset can be bound.
tools: the compiled palette, in the order the plan holds it.
bound: what the plan already bound and the caller already calls --
the adapter, not the raw model. Returned verbatim for the full
palette so the unrouted path is not merely equivalent to the
previous behaviour but identical to it.
wrap: turns a freshly bound model into the same shape as ``bound``.
Injected because the adapter is the *caller's* concern: this
module knows how to narrow a palette and nothing about how the
kernel prefers to call one.
"""
self._model = model
self._tools = tuple(tools or ())
self._bound = bound
self._wrap = wrap or (lambda bound_model: bound_model)
self._cache: dict[str, Any] = {"*": bound}
#: What the provider said about honouring a forced tool choice, asked
#: once when the plan compiled. ``None`` means nobody asked, which is
#: how a caller that predates this argument keeps working.
self._forcing = forcing if forcing is not None else _unasked()
#: ``(model, tools, forced) -> (bound_model, reason)``. Injected, like
#: ``wrap``: binding is provider-shaped work and the capability that
#: owns its degradations lives on the other side of the kernel's
#: layering. The default is the bare call, which is what a caller that
#: predates this argument already got.
self._bind_with = bind_with or _plain_bind
|
The compiled palette, for a caller checking a force against it.
bind
bind(names: frozenset[str] | None, forced: str | None = None) -> Any
The model to call this turn, for the palette selected and required.
names of None — no router, a router that declined, or a
deployment with the feature off — is the full palette.
forced is bound as the provider's tool_choice. A forced turn
binds the full palette when nothing narrowed it, rather than
narrowing to the forced tool alone: forcing says which call must
happen, not which calls exist, and collapsing the palette would change
what the model is told it can do on the round after the force is spent.
Source code in src/symfonic/kernel/palette.py
| def bind(
self, names: frozenset[str] | None, forced: str | None = None
) -> Any:
"""The model to call this turn, for the palette selected and required.
``names`` of ``None`` — no router, a router that declined, or a
deployment with the feature off — is the full palette.
``forced`` is bound as the provider's ``tool_choice``. A forced turn
binds the *full* palette when nothing narrowed it, rather than
narrowing to the forced tool alone: forcing says which call must
happen, not which calls exist, and collapsing the palette would change
what the model is told it can do on the round after the force is spent.
"""
key = palette_identity(names, forced)
cached = self._cache.get(key)
if cached is not None:
return cached
selected = [tool for tool in self._tools if _name_of(tool) in (names or ())]
# An empty selection is *not* an empty binding. A resolver that matched
# nothing has told us nothing about what the model may call, and binding
# zero tools would silently remove the palette rather than leave it
# alone -- the difference between "route to these" and "route to none".
if not selected and forced is None:
self._cache[key] = self._bound
return self._bound
projected = self._wrap(self._bind(selected or list(self._tools), forced))
self._cache[key] = projected
return projected
|
model_for
model_for(plan: Any, ctx: Any, *, transcript: Any = None) -> Any
The model to call this turn, narrowed to a resolved palette if there is one.
Reads the snapshot rather than being told, because the resolution is a
capability's contribution and the kernel's job is to honour what was
contributed, not to know who contributed it.
Source code in src/symfonic/kernel/palette.py
| def model_for(plan: Any, ctx: Any, *, transcript: Any = None) -> Any:
"""The model to call this turn, narrowed to a resolved palette if there is one.
Reads the snapshot rather than being told, because the resolution is a
capability's contribution and the kernel's job is to honour what was
contributed, not to know who contributed it.
"""
projector = getattr(plan.bindings, "palette", None)
if projector is None:
return plan.bindings.model
resolved = getattr(ctx, "resolved", None)
entry = resolved.of(_TOOLS_CAPABILITY) if resolved is not None else None
value = getattr(entry, "value", None) if entry is not None else None
# Two shapes, and the bare frozenset is kept rather than migrated. It is
# what a deployment composing only the router resolves, it means exactly
# "these tools, model chooses", and rewriting every such contribution to
# carry a ``forced=None`` would be churn in return for one fewer branch.
if isinstance(value, ToolPalette):
value.check(frozenset(_name_of(tool) for tool in projector.tools))
forced = value.forced
# The resolved palette is immutable for this invocation, but a forced
# *first* action is spent once its matching result reaches the live
# transcript. Re-reading this at every model round avoids an infinite
# forced loop after a successful tool call.
if forced is not None and _forced_tool_answered(transcript, forced):
forced = None
return projector.bind(value.names, forced)
return projector.bind(value if isinstance(value, frozenset) else None)
|
palette_identity
palette_identity(names: frozenset[str] | None, forced: str | None = None) -> str
A stable key for one resolved palette.
Sorted, because a resolver that ranks returns the same set in different
orders and two orders of one palette must not bind twice.
forced is part of the identity, not a detail beside it: the same tools
bound with and without a required choice are two different bindings, and
sharing a cache entry between them would hand a forced turn the model an
unforced turn had already bound.
Source code in src/symfonic/kernel/palette.py
| def palette_identity(
names: frozenset[str] | None, forced: str | None = None
) -> str:
"""A stable key for one resolved palette.
Sorted, because a resolver that ranks returns the same set in different
orders and two orders of one palette must not bind twice.
``forced`` is part of the identity, not a detail beside it: the same tools
bound with and without a required choice are two different bindings, and
sharing a cache entry between them would hand a forced turn the model an
unforced turn had already bound.
"""
key = "*" if names is None else "\x00".join(sorted(names))
return key if forced is None else f"{key}\x01!{forced}"
|