Deep Sleep Phases

Deep Sleep is symfonic's offline memory consolidation pipeline: an 11-phase routine that runs on a schedule (or on explicit trigger) to compact, strengthen, and prune the tenant-scoped memory graph.

The orchestrator is SleepConsolidator.run in src/symfonic/core/learning/consolidation.py. Each phase is a pure function that takes a GraphMemoryStore, a TenantScope, and a pre-fetched node list, then returns a count of mutations.

Phase index

Phase	File	Function	Counter
1 Strengthen	phases.py	strengthen	nodes_strengthened
1.5 Co-occurrence edges	phases.py	create_cooccurrence_edges	edges_created
2 Tag risk nodes	phases.py	tag_risk_nodes	risk_nodes_tagged
2.5 Pending edges	phases.py	consolidate_pending_edges	edges_created
3 Prune orphans	phases.py	prune_orphans	nodes_pruned
4 Meta-nodes	phases.py	generate_meta_nodes	meta_nodes_created
5 SOUL corrections	phases.py	apply_soul_corrections	soul_updates
8 Working TTL cleanup	phases_maintenance.py	cleanup_working_ttl	working_ttl_pruned
9 Importance decay	phases_maintenance.py	decay_importance	nodes_decayed
10 Episodic summarisation	phases_episodic.py	summarize_episodic	episodes_summarized
11 Synthetic linking	phases_synthetic.py	build_synthetic_links	synthetic_edges_created

Phase numbering is historical (phases 6 and 7 were merged into earlier passes). The brief numbering in the Deep Sleep design doc maps to code numbering with an offset: brief Phase 3 = code Phase 2, brief Phase 5 = code Phase 3, etc. Test filenames follow the brief convention (test_phase_03_risk_tagging.py exercises tag_risk_nodes).

Phase 1 — Strengthen

Repeatedly accessed nodes (appearing >= 2 times in the recent-nodes window) have their importance bumped by IMPORTANCE_BOOST (0.5), capped at MAX_IMPORTANCE (10.0).

• Trigger: Counter(access_counts)[node_id] >= 2
• Config: lookback_hours on SleepConsolidator (default 24h)
• Constants: IMPORTANCE_BOOST = 0.5, MAX_IMPORTANCE = 10.0
• Tests: tests/core/learning/test_phase_01_strengthen.py
• Runtime: O(N_recent), <1ms per 100 nodes

Phase 1.5 — Co-occurrence edges

Creates CO_OCCURRED edges between pairs of recent nodes whose updated_at timestamps are within a 15-minute window. Repeated co-occurrence increments edge weight (via upsert_edge). Degree-capped at MAX_CO_OCCUR_DEGREE (5) to prevent clique explosions.

• Trigger: abs((a.updated_at - b.updated_at).seconds) <= 900
• Constants: CO_OCCUR_WINDOW_HOURS = 0.25, MAX_CO_OCCUR_DEGREE = 5
• Tests: covered by existing tests/unit/test_sleep_consolidation.py
• exercised in the Deep Sleep integration test
• Runtime: O(N_recent^2) worst case, degree cap keeps edges bounded

Phase 2 — Tag risk nodes

Marks nodes whose label or string properties contain negative-sentiment markers with properties["risk_node"] = True plus a risk_tagged_at ISO timestamp. Idempotent: already-tagged nodes are skipped.

• Markers: risk, danger, failure, loss, negative, decline, warning, critical, error, threat (case-insensitive)
• Tests: tests/core/learning/test_phase_03_risk_tagging.py
• Runtime: O(N_recent)

Phase 2.5 — Pending edges

Writes pending_connections (inferred edges from spreading activation) into real graph edges, but only when both source + target labels map to existing nodes and no edge already exists.

• Trigger: both endpoint labels resolvable in all_nodes
• Called with: SleepConsolidator.run(scope, pending_connections=...)
• Runtime: O(|pending|)

Phase 3 — Prune orphans

Deletes semantic nodes that are simultaneously: 1. Importance < ORPHAN_MIN_CONFIDENCE * 10 (= 4.0), 2. updated_at > 7 days old, AND 3. Has <= ORPHAN_MAX_EDGES (1) neighbours.

Separate unconditional path: nodes with a properties["ttl_hours"] whose TTL has elapsed (relative to updated_at) are pruned regardless of importance or neighbour count.

• Constants: ORPHAN_MAX_EDGES=1, ORPHAN_STALE_DAYS=7, ORPHAN_MIN_CONFIDENCE=0.4
• Tests: tests/core/learning/test_phase_05_prune_orphans.py
• Runtime: O(N_all) + O(E) for neighbour lookups

Phase 4 — Meta-nodes

Clusters nodes by label prefix (first colon-split token, or first whitespace-delimited token). Clusters of >= MIN_CLUSTER_SIZE (3) members produce a META:{prefix} node with a summary and up to 10 CLUSTERS edges to the cluster members.

• Constant: MIN_CLUSTER_SIZE = 3
• Optional LLM: pass llm_summariser=async callable(labels) -> str to SleepConsolidator for richer summaries
• Tests: tests/core/learning/test_phase_06_meta_nodes.py
• Runtime: O(N_all) + one query_nodes(label=...) per prefix

Phase 5 — SOUL corrections

Applies user-manual edits (properties["_last_edited_by"] == "user_manual_edit") in recent nodes to the in-memory SOUL schema dict. Only the four canonical SOUL keys are honoured: name, role, personality, language.

• Trigger: _last_edited_by == "user_manual_edit"
• Called with: SleepConsolidator.run(scope, soul_schema=...)
• Tests: tests/core/learning/test_phase_07_soul_corrections.py
• Runtime: O(N_recent)

Phase 8 — Working TTL cleanup

Deletes MemoryLayer.WORKING nodes whose properties["ttl_hours"] has elapsed relative to updated_at. Nodes without ttl_hours are exempt.

• Trigger: updated_at + ttl_hours <= now()
• Tests: tests/core/learning/test_phase_08_working_ttl.py
• Runtime: O(N_working)

Phase 9 — Importance decay

Reduces importance by DECAY_AMOUNT (0.5) on any semantic node whose updated_at is older than stale_days (default 30). Floored at MIN_IMPORTANCE (1.0).

• Constants: STALE_DAYS_DEFAULT=30, DECAY_AMOUNT=0.5, MIN_IMPORTANCE=1.0
• Tests: tests/core/learning/test_phase_09_importance_decay.py
• Runtime: O(N_all)

Known gap (documented, not fixed)

The Deep Sleep brief specified that critical nodes (importance >= 8.0 or labels prefixed SOUL: / AGENT_IDENTITY:) should be EXEMPT from decay. The current implementation has no such exemption and decays all stale nodes uniformly. Tests pin the current behaviour (test_high_importance_nodes_not_exempt_gap, test_soul_prefix_not_exempt_gap) so a future exemption patch flips the assertions intentionally.

Phase 10 — Episodic summarisation

When the episodic event count exceeds max_entries (hard-coded 100), the oldest summarize_batch (50) entries are retrieved, concatenated or LLM-summarised, and deleted from the vector store. The summary text is written back as a META:episodic_summary semantic node.

• Thresholds: max_entries=100, summarize_batch=50 (NOT plumbed through SleepConsolidator.run kwargs today -- see gaps below)
• Optional LLM: reuses the llm_summariser passed to SleepConsolidator
• Tests: tests/core/learning/test_phase_10_episodic_summarization.py
• Runtime: O(count + batch)

Phase 11 — Synthetic linking

Scans recent episodic entries for [semantic:X] / [SOUL:X] markers and creates low-weight SYNTHETIC edges between pairs of referenced nodes that co-occur more than min_co_count times. Capped at max_pairs. Edges carry provenance="synthetic" plus up to 20 source_episodic_ids for traceability.

• Tests: tests/core/learning/test_phases_synthetic.py + tests/core/learning/test_consolidation_phase11.py
• Runtime: O(E_episodic * avg_refs^2)

Quick nap

SleepConsolidator.quick_nap(scope) runs only phase 1 (strengthen) + phase 10 (episodic summarisation) as a lightweight every-N-turns consolidation. Target runtime < 1s.

End-to-end validation

tests/core/learning/test_deep_sleep_integration.py spins up a realistic 30+-node, 120-episode-entry graph with backdated timestamps spanning 90 days and runs the full pipeline. The test asserts that every relevant phase counter is positive, errors are empty, and total runtime is under 5 seconds.

Gaps surfaced during Gate 5

1. Phase 9 exemptions unimplemented. Brief required importance >= 8.0 / SOUL: / AGENT_IDENTITY: exemption. Not enforced today.
2. Phase 10 thresholds not configurable from orchestrator. max_entries / summarize_batch are hard-coded. Callers cannot override without calling summarize_episodic directly.
3. Phase 5 silently drops unknown SOUL keys. A correction for a schema key that is not already present is ignored rather than inserted.
4. Meta-cascade on second run. When a META:* node is created by phase 4 and another by phase 10 (episodic summary), the two share prefix META and become candidates for a META:META cluster on the next consolidation. Pinned by test_idempotence_second_run_is_stable.

These are tracked as follow-up tickets rather than blocking Gate 5.

Phase 12 — Episodic-to-procedural promotion

Added in v5.5.2 (Sprint 2, Autonomous Learning Roadmap).

SleepConsolidator calls promote_episodic_to_procedural() from src/symfonic/core/learning/phases_procedural_promotion.py as the final step of each consolidation run (before returning the report).

It scans recent episodic entries for repeated action patterns using a two-pass heuristic: 1. Explicit metadata["action_type"] field (preferred). 2. Action-verb prefix regex: phrases like "I ran X", "Then I deployed Y".

Patterns that cross the frequency threshold within the recency window are written to ProceduralLayer as status="draft" skills. Phase 12 always stores draft — human approval via the Sprint 1 endpoints is the only path to an active skill.

• Counter: ConsolidationReport.draft_skills_promoted
• Dedup: SemanticMerge inside store_skill (cosine threshold 0.65)
• Skipped silently when episodic_layer or procedural_layer is absent

Config knobs (all on FrameworkConfig, forwarded by /deep-sleep route):

Field	Default	Effect
promotion_min_pattern_count	3	Minimum occurrences before promotion
promotion_recency_days	30	Episodic window to scan (days)
promotion_max_drafts_per_run	5	Per-run draft creation cap

Tests: tests/core/learning/test_phases_procedural_promotion.py (unit) + tests/core/learning/test_consolidation_phase12_integration.py (integration, 16 tests covering pattern extraction, recency filtering, threshold + cap semantics, dedup, and report integration).

See Autonomous Learning for the full human-review workflow that gates Phase 12 drafts into active skills.