Skip to content

symfonic.capabilities.prompting.budget

budget

Deterministic, offline token budgeting and admission.

Two properties matter more than accuracy here. Offline: the default estimator is arithmetic over the string, so a cold container with no network and no tokenizer cache compiles the same prompt as a warm one — a downloaded tokenizer would make prompt assembly fail on the machine least able to debug it. Deterministic: admission is a total order over the rows, so the same request always drops the same contributions, and a golden prompt corpus is possible at all.

Dropping runs most-volatile-first: a per-turn block is the cheapest thing to lose and the one most likely to be re-derivable next turn. Pinned rows are never dropped; when they alone exceed the budget the compile fails, because a prompt missing its boundaries is worse than a prompt that refused to build.

BudgetRow

Bases: NamedTuple

One admission candidate: identity, position, cost, and pinned-ness.

HeuristicTokenEstimator dataclass 

HeuristicTokenEstimator(chars_per_token: int = 4)

Characters-per-token arithmetic — the offline default.

Deliberately an estimate with a name that says so. A provider-exact counter is a legitimate substitution (T3.2.2 owns that seam), but it must arrive as a bound port rather than as a network call this module makes, which is why the estimator is a protocol and this is only its default.

PromptBudget dataclass 

PromptBudget(max_total_tokens: int | None = None)

The ceiling one compiled prompt runs under.

None means unbounded, which is the honest default: inventing a ceiling the adopter did not ask for would silently truncate prompts that fit.

narrows 

narrows(parent: PromptBudget) -> bool

True when this budget is the parent's or a strictly tighter one.

Source code in src/symfonic/capabilities/prompting/budget.py 
71
72
73
74
75
76
77
def narrows(self, parent: PromptBudget) -> bool:
    """``True`` when this budget is the parent's or a strictly tighter one."""
    if parent.max_total_tokens is None:
        return True
    if self.max_total_tokens is None:
        return False
    return self.max_total_tokens <= parent.max_total_tokens

TokenEstimator

Bases: Protocol

Estimates the token cost of a rendered string.

admit_within_budget 

admit_within_budget(rows: Sequence[BudgetRow], budget: PromptBudget) -> tuple[tuple[BudgetRow, ...], BudgetReport]

Admit as much as the budget allows, dropping in a fixed, total order.

The returned rows keep their original (compiled) order; only membership is decided here. Reordering the prompt to fit would change what the model reads for reasons the adopter never declared.

Source code in src/symfonic/capabilities/prompting/budget.py 
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
def admit_within_budget(
    rows: Sequence[BudgetRow], budget: PromptBudget
) -> tuple[tuple[BudgetRow, ...], BudgetReport]:
    """Admit as much as the budget allows, dropping in a fixed, total order.

    The returned rows keep their original (compiled) order; only membership is
    decided here. Reordering the prompt to fit would change what the model
    reads for reasons the adopter never declared.
    """
    total = sum(row.tokens for row in rows)
    limit = budget.max_total_tokens
    if limit is None or total <= limit:
        return tuple(rows), BudgetReport(
            limit=limit,
            total_tokens=total,
            admitted=tuple(row.contribution_id for row in rows),
        )

    pinned_total = sum(row.tokens for row in rows if row.pinned)
    if pinned_total > limit:
        pinned = ", ".join(row.contribution_id for row in rows if row.pinned)
        raise BudgetExceededError(
            f"pinned contributions ({pinned}) need {pinned_total} tokens but the budget is "
            f"{limit}. Pinned content is never dropped and never truncated: raise the budget "
            "or unpin something, but the compiler will not ship a prompt missing it."
        )

    dropped: set[str] = set()
    running = total
    for row in _most_droppable_first(rows):
        if running <= limit:
            break
        if row.pinned:
            continue
        dropped.add(row.contribution_id)
        running -= row.tokens

    admitted = tuple(row for row in rows if row.contribution_id not in dropped)
    return admitted, BudgetReport(
        limit=limit,
        total_tokens=sum(row.tokens for row in admitted),
        admitted=tuple(row.contribution_id for row in admitted),
        dropped=tuple(row.contribution_id for row in rows if row.contribution_id in dropped),
    )