Skip to content

symfonic.agent.cutover.rehydration

rehydration

Rebuilding a paused turn's messages, with the answer folded in (HK2).

TA8.19 recorded session_id's role 3 -- rehydration of prior state -- as absent on the kernel route: SymfonicAgent._maybe_rehydrate_working was the legacy consumer and it early-returns when no checkpointer is wired, which on the migrated path was every turn. This module is that role, made present, and it is next door to :mod:~symfonic.agent.cutover.kernel_resume for the line-budget reason typed_route is next door to delegate: one module per shape, and this one owns exactly the translation from a recorded state back into the messages a provider is sent.

From the checkpoint, and only from the checkpoint. Nothing here reads a process-local cache, and the value it translates was read from the checkpoint port on this call. That is what makes a worker that never saw the pause able to continue the run -- and pause/resume spans worker lifecycles by definition, which is why SymfonicAgent.resume opens its own checkpointer pool before it touches anything.

rehydrate 

rehydrate(outcome: ResumeOutcome) -> tuple[BaseMessage, ...]

The paused turn's messages, with the answer folded in as its observation.

From the checkpoint, and only from the checkpoint. Every message here came out of outcome.turn, which :class:~symfonic.capabilities.human.turnstate.TurnCheckpointStore read from the checkpoint port on this call. Nothing is recovered from a cache keyed by run id, and nothing is remembered between processes -- which is what makes a worker that never saw the pause able to continue it.

The answer arrives as a tool message joined on the reserved call id, because that is what the paused round was waiting for: the model asked for a call, the kernel stopped before dispatching it, and the person's answer is that call's observation. Anything else -- a fresh user turn, say -- would leave the assistant's tool call unanswered in the transcript, which is the orphaned-call state every strict provider rejects on the next request.

Source code in src/symfonic/agent/cutover/rehydration.py 
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
def rehydrate(outcome: ResumeOutcome) -> tuple[BaseMessage, ...]:
    """The paused turn's messages, with the answer folded in as its observation.

    **From the checkpoint, and only from the checkpoint.** Every message here
    came out of ``outcome.turn``, which
    :class:`~symfonic.capabilities.human.turnstate.TurnCheckpointStore` read
    from the checkpoint port on this call. Nothing is recovered from a cache
    keyed by run id, and nothing is remembered between processes -- which is
    what makes a worker that never saw the pause able to continue it.

    The answer arrives as a ``tool`` message joined on the reserved call id,
    because that is what the paused round was waiting for: the model asked for
    a call, the kernel stopped before dispatching it, and the person's answer is
    that call's observation. Anything else -- a fresh user turn, say -- would
    leave the assistant's tool call unanswered in the transcript, which is the
    orphaned-call state every strict provider rejects on the next request.
    """
    checkpoint = outcome.turn
    if checkpoint is None:
        raise InteractionConfigurationError(
            "this resume has no recorded turn state to rebuild from; a "
            "continuation assembled from nothing would answer the person's "
            "question into a conversation the model has never seen"
        )
    messages = [_message(dict(body)) for body in checkpoint.messages]
    reserved = _reserved_call_ids(messages)
    answered = outcome.tool_call_id or outcome.interrupt_id
    messages.append(
        ToolMessage(content=_answer_text(outcome.response), tool_call_id=answered)
    )
    # Every other call the round reserved gets an observation too. The pause
    # stopped the whole round at the first registered interaction, so the rest
    # never ran -- and a provider handed an assistant message with three tool
    # calls and one tool result rejects the request outright.
    messages.extend(
        ToolMessage(content=NOT_DISPATCHED, tool_call_id=call_id)
        for call_id in reserved
        if call_id and call_id != answered
    )
    return tuple(messages)