The migration-window persistence contract: what legacy still has to read.
The wave plan's rollback story rests on one assumption — state written by the
legacy path stays readable by the migrated services, and vice versa. For the
memory capability that means three formats, and this module is all three:
- Where a memory lives. The legacy row carries a materialised
properties['scope_path']: unit-separated kind\x1fid pairs, compared
by exact set membership against a query's ancestor prefixes. The capability
carries MemoryScope with /-joined segments. They are the same
hierarchy in two spellings, and
test_visibility_agrees_with_the_legacy_prefix_isolation_filter proves the
two visibility rules do not disagree by even one row.
- What a memory is. A
MemoryNode's label is the text, its
importance is a 1–10 grid, and everything else this capability knows
about a record rides in the properties bag every backend already
persists — so a migrated write needs no schema change and a legacy reader
needs no new column.
- What a consolidation run did.
ConsolidationReport.to_dict() is
published surface. :meth:ConsolidationState.to_legacy_dict emits exactly
its key set, so a dashboard reading the legacy shape keeps working while the
runtime underneath it changes.
Nothing here imports the legacy packages — capability has no edge to them
in the dependency matrix. The formats are restated, and
tests/capabilities/memory/test_legacy_persistence_compat.py is where the
restatement is checked against the shipped types themselves.
importance_to_salience
importance_to_salience(importance: float) -> float
Map the legacy 1–10 importance grid onto [0, 1] salience.
Clamped rather than refused: an out-of-range importance is a row legacy
already stored (its own validator only fires on construction, not on read),
and refusing to read it would make one bad row poison a whole retrieval.
Source code in src/symfonic/capabilities/memory/compat.py
| def importance_to_salience(importance: float) -> float:
"""Map the legacy 1–10 importance grid onto ``[0, 1]`` salience.
Clamped rather than refused: an out-of-range importance is a row legacy
already stored (its own validator only fires on construction, not on read),
and refusing to read it would make one bad row poison a whole retrieval.
"""
clamped = min(max(float(importance), IMPORTANCE_FLOOR), IMPORTANCE_CEILING)
# Twelve places, not six: the grid step is 1/9, and rounding a repeating
# fraction at six places loses the round trip (``2 -> 0.111111 -> 1.999999``),
# which would drift a memory's importance a little on every migration hop.
return round((clamped - IMPORTANCE_FLOOR) / (IMPORTANCE_CEILING - IMPORTANCE_FLOOR), 12)
|
legacy_entry_payload
legacy_entry_payload(record: MemoryRecord, *, metadata: Mapping[str, Any] | None = None) -> dict[str, Any]
Build the kwargs a legacy MemoryEntry is constructed from.
Source code in src/symfonic/capabilities/memory/compat.py
| def legacy_entry_payload(
record: MemoryRecord,
*,
metadata: Mapping[str, Any] | None = None,
) -> dict[str, Any]:
"""Build the kwargs a legacy ``MemoryEntry`` is constructed from."""
scope = record.scope
# The record's own metadata first, so a caller's bag can still override it
# -- the pending markers are written that way and must win.
# The producer's own keys first and stripped of the capability's, so a
# record cannot name its own scope, identity or durability; the caller's
# bag then wins, because the pending markers are written that way.
bag: dict[str, Any] = {
**producer_metadata(record.metadata),
**dict(metadata or {}),
}
bag[SCOPE_PATH_KEY] = legacy_scope_path(scope)
bag[ORIGIN_KEY] = record.origin
# TA8.74. Stamped in the properties as well as in ``id``, which is what
# ``legacy_node_payload`` already did and this builder did not.
# ``record_from_legacy_node`` reads the bag first and falls back to the
# node's own id, so a store that mints its own identity used to hand back a
# record nobody could match to what was written -- the write said ``m1``
# and the read said a fresh uuid. The layers honour the id now; this keeps
# the round trip true even where one does not.
bag[RECORD_ID_KEY] = record.record_id
return {
"id": record.record_id,
"layer": record.layer.value,
"tenant_id": scope.tenant,
"content": record.text,
"metadata": bag,
"importance": salience_to_importance(record.salience),
}
|
legacy_node_payload
legacy_node_payload(record: MemoryRecord, *, durability: str = 'durable', provenance: Mapping[str, Any] | None = None, properties: Mapping[str, Any] | None = None) -> dict[str, Any]
Build the kwargs a legacy MemoryNode is constructed from.
label carries the text because that is where legacy keeps it — its
commit_pending writes content=op.node.label. Everything this
capability knows and legacy does not rides in properties, which every
graph backend persists as an opaque bag.
Source code in src/symfonic/capabilities/memory/compat.py
| def legacy_node_payload(
record: MemoryRecord,
*,
durability: str = "durable",
provenance: Mapping[str, Any] | None = None,
properties: Mapping[str, Any] | None = None,
) -> dict[str, Any]:
"""Build the kwargs a legacy ``MemoryNode`` is constructed from.
``label`` carries the text because that is where legacy keeps it — its
``commit_pending`` writes ``content=op.node.label``. Everything this
capability knows and legacy does not rides in ``properties``, which every
graph backend persists as an opaque bag.
"""
scope = record.scope
bag: dict[str, Any] = dict(properties or {})
bag[SCOPE_PATH_KEY] = legacy_scope_path(scope)
bag[DURABILITY_KEY] = durability
bag[RECORD_ID_KEY] = record.record_id
bag[ORIGIN_KEY] = record.origin
# Written only when the record claims one, so a row that makes no claim
# carries no key rather than an empty string a reader has to interpret.
# Top level and nowhere else: ``producer_metadata`` strips the same name
# from the nested bag, so there is one home and never two copies to
# disagree.
if record.edited_by:
bag[EDITED_BY_KEY] = record.edited_by
# TA-3-1-9. The field was public, accepted and discarded: this builder read
# its ``properties`` keyword and never ``record.metadata``, while
# ``legacy_entry_payload`` next door merged it -- so one direction honoured
# the field and the other dropped it, and the store used the one that
# dropped it. Written only when there is something to write, so a record
# with no metadata does not grow an empty key in every stored row.
if (own := producer_metadata(record.metadata)):
bag[METADATA_KEY] = own
if provenance is not None:
bag[PROVENANCE_KEY] = dict(provenance)
return {
"id": record.record_id,
"layer": record.layer.value,
"tenant_id": scope.tenant,
"label": record.text,
"properties": bag,
"importance": salience_to_importance(record.salience),
}
|
legacy_scope_path
legacy_scope_path(scope: MemoryScope) -> str
Materialise scope into the string legacy stores and filters on.
Source code in src/symfonic/capabilities/memory/compat.py
| def legacy_scope_path(scope: MemoryScope) -> str:
"""Materialise ``scope`` into the string legacy stores and filters on."""
parts: list[str] = []
for kind, segment in zip(LEGACY_SCOPE_KINDS, scope.segments, strict=False):
parts.append(kind)
parts.append(segment)
return LEGACY_SCOPE_DELIMITER.join(parts)
|
record_from_legacy_node
record_from_legacy_node(payload: Mapping[str, Any]) -> MemoryRecord
Read a stored legacy node back into a record.
A pre-v8 row with no scope_path reads as tenant-global — the same
conservative backfill legacy's own dual-read applies, because those rows
were tenant-global before the key existed.
Source code in src/symfonic/capabilities/memory/compat.py
| def record_from_legacy_node(payload: Mapping[str, Any]) -> MemoryRecord:
"""Read a stored legacy node back into a record.
A pre-v8 row with no ``scope_path`` reads as tenant-global — the same
conservative backfill legacy's own dual-read applies, because those rows
*were* tenant-global before the key existed.
"""
bag = dict(payload.get("properties") or {})
stored_path = bag.get(SCOPE_PATH_KEY)
scope = (
scope_from_legacy_path(str(stored_path))
if isinstance(stored_path, str) and stored_path
else MemoryScope(str(payload["tenant_id"]))
)
layer = payload.get("layer", MemoryLayer.SEMANTIC)
stored_metadata = bag.get(METADATA_KEY)
origin = str(bag.get(ORIGIN_KEY, ""))
metadata = dict(stored_metadata) if isinstance(stored_metadata, Mapping) else {}
label = str(payload.get("label", ""))
# Expand a truncated display prefix, but preserve independent identifiers
# such as Entity:person:mara_venn (whose content is a readable name).
content = bag.get("content")
text = (
content if isinstance(content, str) and content.strip() and content.startswith(label)
else label
)
return MemoryRecord(
record_id=str(bag.get(RECORD_ID_KEY) or payload.get("id") or ""),
layer=resolve_layer(str(getattr(layer, "value", layer))),
text=retrievable_text(text, metadata) if origin.startswith("extraction:") else text,
scope_path=scope.path,
salience=importance_to_salience(payload.get("importance", 5.0)),
origin=origin,
# The other half of TA-3-1-9: this constructor took no ``metadata``
# argument at all, so even a bag that carried it read back bare.
metadata=metadata,
# A legacy row may carry an authority this vocabulary does not know --
# it was a free-form string there. Dropped rather than raised on the
# read: refusing would make a scope with one such row unreadable, and
# this is a reader, not the door that admits the claim.
edited_by=_known_authority(bag.get(EDITED_BY_KEY)),
)
|
salience_to_importance
salience_to_importance(salience: float) -> float
Map [0, 1] salience back onto the legacy 1–10 grid.
Source code in src/symfonic/capabilities/memory/compat.py
| def salience_to_importance(salience: float) -> float:
"""Map ``[0, 1]`` salience back onto the legacy 1–10 grid."""
clamped = min(max(float(salience), 0.0), 1.0)
return round(
IMPORTANCE_FLOOR + clamped * (IMPORTANCE_CEILING - IMPORTANCE_FLOOR), 6
)
|
scope_from_legacy_path
scope_from_legacy_path(path: str) -> MemoryScope
Read a stored scope_path back into a scope.
The kinds are discarded rather than validated: an adopter who named their
levels org/brand/conversation has the same three-level
hierarchy under different labels, and refusing their rows would make the
migration a data conversion instead of a re-read.
Source code in src/symfonic/capabilities/memory/compat.py
| def scope_from_legacy_path(path: str) -> MemoryScope:
"""Read a stored ``scope_path`` back into a scope.
The kinds are discarded rather than validated: an adopter who named their
levels ``org``/``brand``/``conversation`` has the same three-level
hierarchy under different labels, and refusing their rows would make the
migration a data conversion instead of a re-read.
"""
parts = path.split(LEGACY_SCOPE_DELIMITER) if path else []
if not parts or len(parts) % 2:
raise MemoryContractError(
f"legacy scope path {path!r} has {len(parts)} segments; the stored form is "
"alternating kind/id pairs, so an odd count is a truncated write."
)
return MemoryScope(*parts[1::2])
|