Skip to content

symfonic.services.budget.allocation

allocation

Splitting one context window across the kinds of content that share it.

Prompt, tools, and memory compete for the same window, and before this module each of them decided its own allowance from its own config knob. Three independent ceilings that were never checked against each other add up to more than the window whenever the workload is unusual — which is precisely when the window matters.

Two rules make the split safe. Shares round down: three lines at one third of a hundred are 33 each, not 34, because rounding up three lines is how a 100% split ships at 101%. The remainder is reported, not redistributed: unallocated is visible in the plan, so an operator can see the two tokens nobody claimed instead of wondering which kind quietly absorbed them.

The plan is emitted in :class:~.contracts.BudgetKind ladder order rather than in declaration order, so two configurations that mean the same split are the same plan — which is what lets a golden corpus compare them at all.

allocate 

allocate(policy: BudgetPolicy) -> BudgetPlan

Derive the per-kind ceilings policy implies.

A pure function of the policy: no clock, no environment, no counter. The plan for a given policy is the same on every machine and in every process, which is what makes it comparable across a deploy.

Source code in src/symfonic/services/budget/allocation.py 
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
def allocate(policy: BudgetPolicy) -> BudgetPlan:
    """Derive the per-kind ceilings ``policy`` implies.

    A pure function of the policy: no clock, no environment, no counter. The
    plan for a given policy is the same on every machine and in every process,
    which is what makes it comparable across a deploy.
    """
    usable = policy.usable
    minimums = sum(line.minimum for line in policy.lines)
    if minimums > usable:
        raise BudgetPolicyError(
            f"declared minimums total {minimums} tokens but only {usable} are usable "
            f"({policy.context_window} window less {policy.output_reserve} reserved "
            "for output). Lower a minimum or widen the window."
        )

    limits: dict[BudgetKind, int] = {}
    for line in policy.lines:
        limits[line.kind] = max(line.minimum, int(usable * line.share))

    allocated = sum(limits.values())
    if allocated > usable:
        forced = ", ".join(
            f"{kind.value}={limit}" for kind, limit in sorted(
                limits.items(), key=lambda item: item[0].value
            )
        )
        raise BudgetPolicyError(
            f"per-kind minimums raise the total allocation to {allocated} tokens "
            f"against {usable} usable ({forced}). Minimums are floors, not "
            "priorities: the budget cannot honour all of them at once."
        )

    allocations = tuple(
        BudgetAllocation(kind=kind, limit=limits[kind])
        for kind in BudgetKind
        if kind in limits
    )
    return BudgetPlan(
        context_window=policy.context_window,
        output_reserve=policy.output_reserve,
        usable=usable,
        allocations=allocations,
        unallocated=usable - allocated,
    )