The reference ErasureFence (CON-S-4), and the race it is built to lose.
Picture the writer the acceptance criteria describes. It reads
(generation=4, tombstoned=False), is descheduled, and wakes up after an
erasure published. Two designs:
- Generation only. The writer re-reads, sees generation 5, retries with the
new value, and commits. The subject it just recreated was deleted a
microsecond ago.
- Generation and tombstone-absence, in the same operation. The retry
fails on the tombstone condition. Which side of the generation advance the
writer landed on stops mattering, and that is the entire point: the window
between "tombstone published" and "generation advanced" cannot be exploited
because there is no such window — PRIV-4 makes them one transition.
Two properties are easy to get subtly wrong here and both are load-bearing:
- The lookup is subtree-aware. Erasure matches by
:meth:
SubjectScope.narrows, so a tenant tombstone has to fence every scope
underneath it. An exact-key lookup would let a sub-scope invocation recreate
what the tenant-level erasure removed, and the subtree-scoped export would
then hand it straight back. :mod:symfonic.services.privacy.lineage owns the
walk.
- The linearizable section is a thread lock, not a per-loop one. This
class's whole contract is that publish and conditional-write are indivisible.
An
asyncio.Lock is only mutual exclusion inside one event loop, and a
fence shared by a sidecar, a worker thread, or an asyncio.run in a
background thread would have its lock swapped out from under a holder — which
reopens the exact time-of-check/time-of-use window this port exists to close.
Nothing awaits inside the section (apply must be synchronous), so holding
a plain lock across it cannot deadlock.
A real backend gets the same guarantee from a conditional update or a predicate
transaction; what it may never get it from is two round trips and good
intentions.
InMemoryErasureFence
InMemoryErasureFence(state: MutableMapping[str, ErasureGeneration] | None = None, *, on_denial: DenialRecorder | None = None, denial_buffer: int = _DENIAL_BUFFER)
Registry row 14, reference implementation.
state is injectable so a durable backend can own the bytes while this
class owns the rule. A restart over the same mapping keeps refusing, which
is what makes the tombstone persistent rather than a process fact.
Source code in src/symfonic/services/privacy/fence.py
| def __init__(
self,
state: MutableMapping[str, ErasureGeneration] | None = None,
*,
on_denial: DenialRecorder | None = None,
denial_buffer: int = _DENIAL_BUFFER,
) -> None:
self._state: MutableMapping[str, ErasureGeneration] = (
state if state is not None else {}
)
self._denials: deque[FenceDenial] = deque(maxlen=denial_buffer)
self._on_denial = on_denial
self._lock = threading.Lock()
|
has_denial_recorder
property
has_denial_recorder: bool
Whether a host has already claimed this fence's denial stream.
complete_erasure
async
complete_erasure(scope_key: str) -> ErasureGeneration
Advance once more, and keep the tombstone forever (PRIV-7).
Source code in src/symfonic/services/privacy/fence.py
| async def complete_erasure(self, scope_key: str) -> ErasureGeneration:
"""Advance once more, and keep the tombstone forever (PRIV-7)."""
with self._lock:
governing = self._effective(scope_key)
if not governing.tombstoned:
raise ErasureFenceDenied(
"cannot complete an erasure that was never published; the "
"tombstone is what bars the writers this completion claims "
"to have outlived",
reason="generation_mismatch",
scope_key=scope_key,
observed_generation=governing.generation,
)
base = self._exact(scope_key)
if base is None or not base.tombstoned:
# An inherited tombstone: give this scope its own row so the
# completion advance belongs to the scope that completed rather
# than bumping an ancestor's generation on its behalf.
base = replace(governing, scope_key=scope_key)
advanced = replace(base, generation=base.generation + 1)
self._state[scope_key] = advanced
return advanced
|
conditional_write
async
conditional_write(scope_key: str, *, expected_generation: int, apply: Callable[[], Any]) -> WriteOutcome
The atomic dual-condition commit (EFX-ER-3).
apply must be synchronous. An awaitable would suspend inside the
very section whose indivisibility is the contract, and "atomic except
while it awaits" is not atomic.
Source code in src/symfonic/services/privacy/fence.py
| async def conditional_write(
self,
scope_key: str,
*,
expected_generation: int,
apply: Callable[[], Any],
) -> WriteOutcome:
"""The atomic dual-condition commit (EFX-ER-3).
``apply`` must be synchronous. An awaitable would suspend inside the
very section whose indivisibility is the contract, and "atomic except
while it awaits" is not atomic.
"""
if inspect.iscoroutinefunction(apply):
raise TypeError(
"conditional_write applies its effect inside the linearizable "
"section, so `apply` must be synchronous; an awaited effect "
"would reopen the time-of-check/time-of-use window this port "
"exists to close"
)
with self._lock:
current = self._effective(scope_key)
if current.tombstoned:
denial = self._record(
scope_key, "subject_tombstoned", expected_generation, current.generation
)
elif current.generation != expected_generation:
denial = self._record(
scope_key, "generation_mismatch", expected_generation, current.generation
)
else:
result = apply()
if inspect.isawaitable(result):
raise TypeError(
"`apply` returned an awaitable; see the synchronous-effect "
"rule above — the effect has already run and cannot be undone, "
"so fix the caller rather than awaiting here"
)
return WriteOutcome(
committed=True, generation=current.generation, result=result
)
# Outside the section: the seam is somebody else's I/O and the decision
# is already made. Emitting under the lock would let a slow audit sink
# serialize every writer in the process.
await self._emit(denial)
return WriteOutcome(
committed=False,
generation=denial.observed_generation,
denial_reason=denial.reason,
)
|
denials
denials() -> tuple[FenceDenial, ...]
The retained denial window (bounded; see :data:_DENIAL_BUFFER).
Source code in src/symfonic/services/privacy/fence.py
| def denials(self) -> tuple[FenceDenial, ...]:
"""The retained denial window (bounded; see :data:`_DENIAL_BUFFER`)."""
return tuple(self._denials)
|
drain_denials
drain_denials() -> tuple[FenceDenial, ...]
Take the window and clear it, for a host that batches its own emit.
Source code in src/symfonic/services/privacy/fence.py
| def drain_denials(self) -> tuple[FenceDenial, ...]:
"""Take the window and clear it, for a host that batches its own emit."""
with self._lock:
drained = tuple(self._denials)
self._denials.clear()
return drained
|
publish_tombstone
async
publish_tombstone(scope_key: str, *, reason: str) -> ErasureGeneration
One transition: tombstone planted, generation advanced (PRIV-4).
Idempotent, because a resumed saga re-publishes and a second advance
would invalidate every generation an in-flight reader holds for no
reason at all.
Source code in src/symfonic/services/privacy/fence.py
| async def publish_tombstone(self, scope_key: str, *, reason: str) -> ErasureGeneration:
"""One transition: tombstone planted, generation advanced (PRIV-4).
Idempotent, because a resumed saga re-publishes and a second advance
would invalidate every generation an in-flight reader holds for no
reason at all.
"""
if not reason:
raise ErasureFenceDenied(
"an erasure needs a reason; an unexplained tombstone is not "
"reviewable evidence",
reason="generation_mismatch",
scope_key=scope_key,
)
with self._lock:
current = self._exact(scope_key) or ErasureGeneration(scope_key=scope_key)
if current.tombstoned:
return current
published = ErasureGeneration(
scope_key=scope_key,
generation=current.generation + 1,
tombstoned=True,
tombstoned_at=time.time(),
reason=reason,
)
self._state[scope_key] = published
return published
|
set_denial_recorder
set_denial_recorder(recorder: DenialRecorder | None, *, replace: bool = False) -> None
Late-bind the audit seam (EFX-ER-4). Once, unless told otherwise.
A fence is usually built before the service that audits it — the host
wires storage first — so the recorder is attachable rather than
constructor-only. What it is not is silently replaceable: two hosts in
one process can share a fence, and a second attachment would redirect
the first host's denials into the second host's sink without either of
them saying so. replace=True is how a host that means it says so.
Source code in src/symfonic/services/privacy/fence.py
| def set_denial_recorder(
self, recorder: DenialRecorder | None, *, replace: bool = False
) -> None:
"""Late-bind the audit seam (EFX-ER-4). Once, unless told otherwise.
A fence is usually built before the service that audits it — the host
wires storage first — so the recorder is attachable rather than
constructor-only. What it is *not* is silently replaceable: two hosts in
one process can share a fence, and a second attachment would redirect
the first host's denials into the second host's sink without either of
them saying so. ``replace=True`` is how a host that means it says so.
"""
if recorder is not None and self._on_denial is not None and not replace:
raise FenceAuditBindingError(
"this fence already has a denial recorder; attaching a second "
"one would send the first host's EFX-ER-4 events to the second "
"host's audit sink. Pass replace=True if that is the intent, or "
"leave the binding to the host that made it"
)
self._on_denial = recorder
|