Skip to content

symfonic.services.observability.callbacks

callbacks

The compatibility observer: shipped callback schemas, new event source.

An adopter's CallbackHandler is public API. This task is allowed to change where those events come from — the kernel event stream instead of emission sites scattered through the engine — and is not allowed to change a field name, a default, or the order a handler sees them in.

So this observer is the seam. It implements three narrow ports and renders them into the shipped dataclasses, which is what lets the OTel callback bridge and ConversationMetricsCollector keep working unmodified: both are CallbackHandler implementations, and both now reach the event stream through here rather than through their own wiring.

Three honest limits, recorded rather than papered over:

  • The kernel stream reports usage per run, not per model round. One LLMEndEvent is therefore emitted per run and it is an aggregate. That is a true statement about what the stream knows; synthesising per-round events would be a false one.
  • node_name is "invocation". The stream has no node concept, and reusing "react" would attribute kernel-driven runs to a graph node that did not run.

  • The tool hooks are not rendered here. This observer binds the run, cost and error ports and deliberately not the tool port, so on_tool_call_dispatch, on_before_tool_call and on_tool_result do not fire from this source. Two independent reasons, both structural:

  • on_tool_call_dispatch and on_before_tool_call are not observations. Their return value decides what the engine dispatches — a rewritten name and args, or a refusal with a synthetic result. An observer sits downstream of the kernel's dispatch decision and cannot honour either, so firing them would tell a handler its rewrite took effect when it did not. A silently ignored rewrite is worse than a hook that never fired.

  • ToolResultCallbackEvent requires tool_name and call_id to correlate a result to its call. The kernel's tool_result event carries only the opaque outcome (KernelEvent.tool_outcome); the join key lives on the preceding tool_call event, and pairing them by arrival order would be an assumption about dispatch concurrency that the kernel contract does not make.

Adopters who need the pre-dispatch seams keep registering them on the engine path that owns dispatch. Closing the result gap needs the kernel to carry the ToolRequest on its tool_result event — a contract change, not a rendering one, and out of this task's scope.

CallbackHandlerObserver 

CallbackHandlerObserver(manager: Any)

Renders observations into the shipped callback schemas.

Dispatch goes through the supplied CallbackManager, so per-handler error isolation, partial-handler tolerance and the has_hook fast path stay exactly as they are today — this observer adds a source, not a second dispatcher.

Source code in src/symfonic/services/observability/callbacks.py 
106
107
def __init__(self, manager: Any) -> None:
    self._manager = manager