Procedural shadow inheritance

v7.22.0 — multi-scope procedural retrieval via TenantScope.inherits_from

v8.0 — SUBSUMED by the hierarchical path. The inherits_from chain described below was a procedural-only ancestor-override primitive. In v8.0 it is subsumed by the general N-level TenantScope.path: query_skills now fans out over the scope's path ancestor prefixes (most-specific first) and merges with the same first-wins shadow-by-identifier rule — a brand skill shadows an org skill of the same identifier; distinct skills blend in. The shadow merge rules (§3), cache trade-offs (§4), and depth cap (§5) below carry over unchanged; only the source of the ancestor chain moved from the explicit inherits_from tuple to the path's prefixes.

inherits_from is retained and still honoured for legacy callers (deprecated). New code should express the hierarchy as a path (TenantScope.root("org", ...).child("brand", ...)) and let the procedural layer derive the ancestor chain from it. See hierarchical-tenant-scope.md §5.

Migration. The §6 "before/after" rewrite below still applies, but the "after" form is now:

from symfonic.core.scope import TenantScope
# org-level defaults seeded at the org root:
org_scope = TenantScope.root("org", "acme")
# brand inherits via the path, NOT via inherits_from:
brand_scope = org_scope.child("brand", "acme_brand_42")

A brand-scoped query_skills sees the org defaults (ancestor prefix) plus brand-specific skills, brand shadowing org on identifier — no inherits_from wiring, no explicit ancestor tuple.

The ProceduralLayer is the one HMS layer where an OOP-style inheritance model is structurally coherent: identifier-keyed authored content where a brand-tier procedure can override (shadow) a global-tier default. This document explains the field semantics, the merge rules, the cache trade-offs adopters need to know, and why the four non-procedural layers deliberately ignore inherits_from.

---

1. Why only procedural

TenantScope.inherits_from is a layer-agnostic field, but only ProceduralLayer honours it. The other four layers — SemanticLayer, EpisodicLayer, WorkingLayer, ProspectiveLayer — emit MultiScopeIgnoredWarning on first observation of a scope carrying inherits_from and continue with single-scope semantics.

The architectural verdict behind this asymmetry:

• Episodic: events are speaker-attributed and timestamped. Global inheritance has no speaker, would corrupt per-tenant cost accounting, and would break GDPR scope guarantees (v5.5.0 Gate 3).
• Working: per-(tenant, sub_tenant) FIFO buffer with max_entries cap. Inheritance would either silently leak cross-tenant conversation context or require a merge policy with no defensible default.
• Semantic: facts are tenant-bound by entity construction. The one inheritance candidate (AGENT_IDENTITY persona) is a single-node case adopters solve by writing one node per tenant — not worth a framework primitive.
• Prospective: retrieve() returns [] by design. Inheritance has nothing to compose with.
• Procedural: identifier-keyed authored content with the v7.5+ metadata['identifier'] field. The use case of "tier 0 default rules, tier 1 brand overrides" maps cleanly to class-resolution semantics. The one coherent case.

The field stays on TenantScope so adopters can build the scope once at the edge and pass it everywhere; the layer honour-the-field contract is explicit and warned on miss. Adopters who only pass multi-scope scopes into procedural never see the warning.

---

2. Field semantics and adopter wiring

TenantScope.inherits_from: tuple[TenantScope, ...] defaults to (). The common case (one scope per request, no inheritance) is the fast path — zero traversal cost in the validator, no fanout cost in ProceduralLayer.query_skills.

When set, it represents the ordered ancestor chain. The ProceduralLayer.query_skills method walks [self, *inherits_from] in BFS order, queries each scope individually, and merges results with shadow semantics (see §3).

Adopter wiring example. A two-tier setup (brand → global):

from symfonic.core.scope import TenantScope

global_scope = TenantScope(tenant_id="__GLOBAL__")

def brand_scope_for_request(brand_id: str, user_id: str) -> TenantScope:
    brand = TenantScope(
        tenant_id=brand_id,
        inherits_from=(global_scope,),
    )
    # narrow() preserves inherits_from (R-V7.22-A — v7.21 silent-bypass
    # closure invariant carries forward).
    return brand.narrow(user_id)

A three-tier setup (brand → org → global):

global_scope = TenantScope(tenant_id="__GLOBAL__")
org_scope = TenantScope(tenant_id="acme_org", inherits_from=(global_scope,))
brand_scope = TenantScope(tenant_id="acme_brand_42", inherits_from=(org_scope,))

Depth is capped at _MAX_INHERIT_DEPTH = 8 (see §5).

---

3. Shadow merge rules

Under multi-scope, ProceduralLayer.query_skills fans out to each scope in the chain, then merges the per-scope results with first-wins shadow semantics:

1. For each entry, compute the shadow key:
2. metadata['identifier'] (the v7.5+ authored-tier handle) — first priority.
3. metadata['label'] (the v7.8.2 projected label) — fallback.
4. None — entry has no key; participates in the union but never shadows another entry.
5. Iterate scopes in chain order ([self, *ancestors]). The first scope to claim a key wins; later entries with the same key are dropped.
6. Entries with no shadow key pass through to the union unconditionally.
7. Final ordering: sort(key=lambda e: str(e.node_id or "")) for cross-backend determinism (matches the _query_skills_single v7.19.2 contract).

Blank-string keys are not keys. An empty identifier or whitespace-only label does NOT count as a shadow key — preventing a silent collision where an empty-label seeder accidentally drops a real entry from another scope via the "" key.

Top-k headroom. Each per-scope query asks for top_k * len(scope_chain) entries, not top_k. When self shadows N global entries and self has fewer than top_k results, the merged top_k must still be top_k; per-scope queries surface enough headroom to fill it. Worst-case fanout = top_k × 8 = 40 entries scanned per query — acceptable for procedural (typically <100 entries per tenant).

---

4. Cache invariants under multi-scope retrieval

Multi-scope procedural retrieval interacts with two Anthropic prompt-cache surfaces:

1. L1 PRE-FLIGHT block (when procedural_render_preflight_in_l1=True). The block content is derived from the active-skills snapshot. When a brand-side procedure is added/removed/edited and that procedure shadows or unshadows a global one, the L1 cached prefix key changes and the next request misses cache.
2. tools[] array (when lazy_tooling=True or tool_routing_mode != "off"). The narrowed tool set is derived from the active-skills snapshot. Same shadow mutation → different tools[] membership → cache collapse, same class as tool_routing_mode=enforce in v7.11.0.

Both surfaces collapse together if both knobs are active. MEMORY_CONTEXT, by contrast, is L2/volatile (see preflight_l1.py:9-14) and is NOT in the cached prefix — shadowed procedures landing in MEMORY_CONTEXT do NOT bust cache.

Recommended policy

• Adopters with rare brand-side procedure mutations: accept the collapse. A brand admin editing a procedure is a deliberate human action; one cache miss per edit is acceptable.
• Adopters with frequent brand-side mutations (e.g. brand-template authoring flows): batch mutations outside active sessions, or set procedural_render_preflight_in_l1=False AND tool_routing_mode="off" for sessions where mutations are expected.
• Adopters who care about cache hit rate above shadow semantics: stay on the per-brand seed pattern (today's adopter shape) and skip multi-scope entirely. The v7.22 feature is opt-in; default scope inheritance is empty.

The v7.11.0 nuance translated for v7.22: manifest_cache_position="volatile" only fixes the L1 TEXT manifest, not the wire-level tools[]. For v7.22: procedural_render_preflight_in_l1=False keeps the L1 PRE-FLIGHT block out of the cached prefix, but lazy_tooling tools[] derivation still busts cache on per-turn membership changes. Adopters wanting full cache stability under multi-scope need BOTH procedural_render_preflight_in_l1=False AND lazy_tooling=False (or tool_routing_mode="off").

---

5. Cycle detection and depth cap

The TenantScope model_validator rejects cyclic chains and chains deeper than _MAX_INHERIT_DEPTH = 8.

Cycle detection. DFS walk over inherits_from with a per-path ancestry set. Identity key is the (tenant_id, sub_tenant_id, namespace) triple. Visiting the same key twice on a single path raises:

ScopeValidationError(
    "TenantScope.inherits_from forms a cycle through: A -> B -> A",
    code="SCOPE_INHERITS_CYCLE",
)

Diamonds (A → B, A → C, B → D, C → D) are valid — the same identity reached via distinct ancestor paths is NOT a cycle. The linearisation step deduplicates by identity key so a diamond visits D once.

Depth cap rationale.

Depth	Real-world coverage
1	Brand → global (phase 1)
2	Brand → org → global (phase 2)
3	Brand → sub-org → org → global (large enterprise)
4–7	Buffer for unforeseen patterns (geographic, regulatory tiers)
8	Hard cap before fanout cost dominates: top_k=5 × 8 = 40 entries scanned per query

Not 4 — kills legitimate enterprise hierarchies. Not 16 — fanout × top_k explosion ≈ procedural layer's total entry count, makes ranking redundant. 8 is the median.

Exceeding the cap raises:

ScopeValidationError(
    "TenantScope.inherits_from exceeds max depth 8 ...",
    code="SCOPE_INHERITS_TOO_DEEP",
)

Field-level type rejection runs first. Non-TenantScope elements in the inherits_from tuple (a dict, a string, a list) raise a Pydantic ValidationError at the field validator, BEFORE the cycle walk runs. The walk's current.tenant_id access is therefore safe — type-soundness is a hard prerequisite for cycle detection coverage.

---

6. Migration from per-brand seeding

Adopters running the per-brand seeding pattern (the Path-A spike shape — write the same global default into every brand tenant separately) can migrate to multi-scope without a data backfill on day 1.

Before (per-brand seeding):

# Seed every brand's procedural layer with the global defaults.
for brand in all_brands():
    brand_scope = TenantScope(tenant_id=brand.id)
    for default_skill in global_default_skills:
        await proc_layer.seed_authored_skill(
            brand_scope, default_skill, source_tag="authored:global",
        )
    for brand_skill in brand.custom_skills:
        await proc_layer.seed_authored_skill(
            brand_scope, brand_skill, source_tag="authored:brand",
        )

After (multi-scope):

# One tenant for globals, one per brand.
global_scope = TenantScope(tenant_id="__GLOBAL__")
for default_skill in global_default_skills:
    await proc_layer.seed_authored_skill(
        global_scope, default_skill, source_tag="authored:global",
    )

for brand in all_brands():
    brand_scope = TenantScope(
        tenant_id=brand.id,
        inherits_from=(global_scope,),
    )
    for brand_skill in brand.custom_skills:
        await proc_layer.seed_authored_skill(
            brand_scope, brand_skill, source_tag="authored:brand",
        )

The brand-side query_skills now sees the union (global defaults + brand-specific) with brand-side overrides shadowing globals on identifier.

Migration safety: the per-brand seed entries can stay in place. The multi-scope path queries both the brand scope and the global ancestor; existing brand-seeded entries continue to work, and over time adopters can backfill-delete the brand copies once they've validated the global ancestor surfaces the right defaults. The v7.22 plan parks (does not delete) the Path-A spike for adopters wanting a phased migration.

Pitfall: narrow() preserves inherits_from. If the brand scope narrows to a per-user scope (brand.narrow("user_42")), the inheritance chain carries through. This is correct behaviour — a per-user sub-scope should still see the brand and global procedures — but it's worth knowing if you previously used narrow() to scope away from inheritance. There is no un-inherit() helper; build the narrower scope explicitly:

isolated = TenantScope(tenant_id="acme", sub_tenant_id="user_42")
# vs. brand.narrow("user_42") which carries inherits_from

---

7. Warning emit-once semantics

Adopters wiring inherits_from into a scope that is then passed to non-procedural layers will see exactly one MultiScopeIgnoredWarning per (layer-instance, tenant_id, sub_tenant_id) triple per process. This is by design, not a bug.

The warning fires at first-query-time, not constructor-time, because:

• The TenantScope is built at request-edge (typically a FastAPI dependency) and inheritance is set there. The layer instance has no visibility into scopes until a query lands.
• An adopter might build a multi-scope scope and pass it ONLY to procedural — no warning is needed in that case.
• The right signal-to-noise ratio: fire when the user actually attempts a non-honouring retrieve, not for every scope construction.

Each non-procedural layer holds a private _multi_scope_warned: set[tuple[str, str | None]] of (tenant_id, sub_tenant_id) pairs that have already emitted. Per-layer state, not central dispatcher — central dispatch would require either a singleton (avoided per v5.0 type unification work) or a global keyed by id(layer) which leaks across layer lifecycle.

The WorkingLayer wires the warning in retrieve() ONLY. push() and get_context() have no scope-merge semantics to ignore, so emitting from them would be a false-positive in adopter mental models.

To silence the warning once you understand the design constraint:

import warnings
from symfonic.memory.warnings import MultiScopeIgnoredWarning

warnings.filterwarnings("ignore", category=MultiScopeIgnoredWarning)

Concurrency. The non-atomic check-then-add on _multi_scope_warned means two coroutines racing on the same uncached (tenant, sub_tenant) may BOTH observe the missing key and BOTH emit. We accept the duplicate as benign — locking the hot path would penalise the common single-scope case for no compensating safety improvement.

---

Related work

• v7.21 — TenantScope.from_state_dict round-trip + seven-site reconstruction refactor (closes the silent-bypass class that v7.22's narrow() fix preserves through the new field).
• v7.5+ — authored-tier metadata['identifier'] field (shadow-key source).
• v7.8.2 — label projection (shadow-key fallback source).
• v7.7.5 — L1 PRE-FLIGHT synthesiser (the surface that makes §4 cache trade-offs load-bearing).
• v7.11.0 — tool_routing_mode=enforce cost review (cache-collapse precedent translated for v7.22).
• v7.18.0 — precondition_gate state-predicate DSL (the runtime gate that consumes the multi-scope getter output).