Skip to content

Consolidate without hydrate (CORE-404)

Two operator recipes for running memory consolidation with no hydration in the loop. Recipe 1 is the explicit act: an operator invokes a cycle directly, outside any turn ("sleep-time"), over a composition that contains no retrieval stage, no JIT prompt collaborator, and no model provider -- there is nothing that could hydrate. Recipe 2 is the driven act: turns deepen each scope's cadence backlog, the operator reads the depth telemetry, and due scopes consolidate.

Every step below executes. The executable form is tests/capabilities/memory/test_consolidate_without_hydrate_recipes.py, run by CI with the rest of the suite; each recipe names the test that performs its steps. If a step and its test disagree, the test is right and this page is stale -- fix the page, not the test.

Vocabulary used here

  • Backlog: per-scope turns_since_quick on the cadence cursor. Each completed turn deepens it by one; a consolidation resets it to zero.
  • Depth telemetry: the cursor readings (turns_since_quick per scope) plus the cycle's terminal record (background_consolidation rows with outcome and counts).
  • JIT hydration off: no just-in-time prompt collaborator is composed (the summary/reflection collaborators in memory_capabilities are None) and, in recipe 1, no turn loop exists at all. Consolidation phases read the graph; they never retrieve for a prompt.
  • Sleep-time: any invocation outside a turn -- an operator command, a scheduler tick, a maintenance window. The coordinator's run is the seam.

Recipe 1 -- prompted sleep-time consolidation

Invoke one QUICK cycle explicitly over seeded material. No agent, no retrieval, no model provider.

Executed by test_recipe_1_prompted_sleep_time_cycle.

  1. Build the pieces over an in-memory backend: JournalledGraph, quick_phases with exact session ownership, a ConsolidationRuntime over those phases, and a ConsolidationCoordinator with InProcessLeases(single_process=True) and a terminal sink that keeps the completion records. Note what is absent: no Agent, no MemoryCapability retrieval stages, no extractor, no summary collaborator, no model provider.
  2. Seed the scope's material: semantic nodes with access_count >= 2 (the strengthen phase's input). Also seed a sibling scope's node to prove the cycle touches only its own scope.
  3. Invoke the cycle directly: await coordinator.consolidate(scope, ConsolidationCycle.QUICK). This is the "prompted" half: an explicit call, not a cadence firing. (consolidate is run plus terminal reporting; run alone suits callers with their own reporting. In production the same call arrives from the operator's scheduler; the coordinator's schedule_cycle is the background variant with identical reporting.)
  4. Read the cycle state: state.mutations["strengthen"] counts the strengthened nodes, and the seeded node's importance rose by the phase's step. The sibling scope's node is untouched.
  5. Read the terminal record from the sink: one background_consolidation row, outcome == "completed", cycle == "quick".
  6. Confirm nothing hydrated: the runtime's roster holds consolidation phases only. There is no retrieval stage to call, no JIT collaborator to materialize, and no provider to spend a model call on -- the composition cannot hydrate by construction.

Recipe 2 -- backlog consolidation driven by depth telemetry

Let turns deepen the backlog, read the depth, consolidate what is due.

Executed by test_recipe_2_backlog_depth_drives_cycles.

  1. Build the coordinator with ConsolidationSchedule(quick_every_turns=3) and background=False (the turn waits, so the recipe is deterministic; production uses the background registry with the same reporting), plus a terminal sink. Seed one strengthenable node.
  2. Simulate three turns. Each turn produces its exchange rows with turn_records(query, answer, scope, turn_id): four rows per exchange (user + assistant, working + episodic), one shared turn_id, origin auto_turn. Assert the shape -- well-formed rows are the backlog's content. (Persisting those rows is the write half's job inside a turn; this recipe covers the consolidation half.)
  3. After each turn, advance the backlog with coordinator.after_turn(scope) -- the same call the nap stage makes -- and read the depth with coordinator.cursor(scope).turns_since_quick. Depths read 1, 2, then due: the third call returns ConsolidationCycle.QUICK.
  4. Run what came due: await coordinator.consolidate(scope, ConsolidationCycle.QUICK). The terminal sink gains one background_consolidation row with outcome == "completed", and the strengthened node shows the work.
  5. Read the depth again: turns_since_quick is back to 0. A fourth turn advances it to 1 and nothing is due -- the backlog drained honestly, not by skipping.
  6. Confirm scope isolation: a second scope that saw no turns still reads depth 0 with nothing due. Backlogs never leak across tenants sharing one coordinator.