Skip to content

Handling Large Data: Tool Outputs & Attachments

Agents routinely pull in far more data than a turn needs: a catalog dump, a big query result, a multi-megabyte CSV, a high-resolution image. Sent naively, that payload is re-prefilled into the model on every react iteration and every follow-up turn — you pay full input price to re-attend data the model already saw, and the extra tokens dilute its attention.

This guide is the map. It separates three distinct problems, tells you which the framework manages and which is your job, and points to the deep-dive guides for each.

Three problems, not one

Problem Example Owner
Re-sending across turns The same 200K-token catalog re-prefilled every turn Framework (opt-in) — compaction & offload
Over-sending on first delivery A tool returns 10,000 rows when 20 + a count would do Tool author — preview + paginate
Large media repeated every turn A 4MB screenshot re-attached each turn You choose — extract vs. send natively

Conflating the first two is the common trap. Compaction handles don't re-send; a good tool contract handles don't over-send the first time. They compose.

1. Re-sending across turns — the framework manages this (opt-in)

When a tool returns a genuinely large result, symfonic can collapse it to a compact stub on older turns and let the model pull back exactly the slice it needs on demand via the framework-provided recall_tool_result tool. The full body is never deleted — it stays in a ledger.

Results are handled by size tier:

Tier Size Handling
Hot the keep_last_n most recent kept full, inline — freshest data is never stubbed
Small ≤ tool_result_compaction_size_chars (~8K) kept full, inline
Large ≥ tool_result_large_offload_threshold_chars (~60K) and the net-saving gate fires offloaded to a stub + recall_tool_result

Both switches are OFF by default — the feature ships default-off behind a cost gate because it only pays off on cached, tool-heavy, multi-turn sessions. Turn it on and measure:

from symfonic.agent import FrameworkConfig

config = FrameworkConfig(
    tool_result_compaction_enabled=True,   # umbrella compaction switch
    tool_result_offload_enabled=True,      # large-tier offload (default False)
    # optional tuning (defaults shown):
    tool_result_compaction_keep_last_n=1,        # most-recent results kept full
    tool_result_large_offload_threshold_chars=60_000,
)

The model reads offloaded data through a bounded tool (hard 50KB cap so one 5MB result can't blow the next window):

recall_tool_result(
    mem_id,                     # from the stub
    query="order-4821",         # substring → ±window around each match
    # or: offset=0, max_chars=2000        # a byte range
    # or: json_path="items[3].price"      # structural addressing
)

Retrieval is deterministic (no semantic search), so it never destabilizes the prompt cache. The stub also carries a structural index (keys, row counts, a head preview) so the model can often answer without recalling at all.

The offload net-saving gate only fires when it is actually cheaper than re-prefilling (priced from the model's rates); an unknown model never fires. That's what makes it safe to enable.

Deep dive: Guide 16 — Tool-Output Offload.

2. Over-sending on first delivery — design your tool to preview + paginate

The framework deliberately does not auto-truncate a tool's first return to a "sample." What a meaningful sample of arbitrary JSON / XML / CSV is — first N rows? schema + count? summary stats? — is domain-specific, and guessing wrong silently corrupts the agent's view of the world.

So the responsibility sits with the tool author, and the pattern is simple: return a bounded preview plus the shape of the whole, and expose a way to get more.

from symfonic.core import symfonic_tool

@symfonic_tool()
def search_products(query: str, offset: int = 0, limit: int = 20) -> dict:
    """Search the catalog. Returns a bounded page plus the total count.

    Args:
        query: Search terms.
        offset: Row offset for pagination.
        limit: Max rows to return (kept small so the model isn't flooded).
    """
    rows = db.search(query, offset=offset, limit=limit)
    return {
        "total": db.count(query),      # the model knows there's more
        "offset": offset,
        "returned": len(rows),
        "rows": rows,                  # a *page*, not the whole table
        "next_offset": offset + limit if offset + limit < db.count(query) else None,
    }

The model sees total vs returned, understands it has a slice, and calls the tool again with next_offset when it needs more. If a large result does get offloaded (§1), recall_tool_result's keyed offset / json_path recall is the framework-side pagination primitive that complements this.

Rule of thumb: a tool should return what a human would put in a summary, plus a handle to drill in — not a raw database dump.

3. Large images & documents — extract vs. send natively

Media is a different mechanism. Attachments arrive as content blocks, and image tokens repeat on every turn if you send them natively. symfonic ships attachment text extraction so you can convert a document or image to text once and carry the cheap, cacheable text forward instead.

Choose per case:

Case Approach
Visual content genuinely matters (charts, diagrams, UI screenshots) Send natively via run(attachments=[...]) — but know it repeats every turn
Only the text matters (invoices, contracts, text screenshots) Extract once to text — cheap, caches, and survives in HMS memory
Any multi-turn conversation with attachments Extract — don't re-attach raw media each turn

The extraction helpers live in symfonic.agent.attachments:

from symfonic.agent.attachments import extract_text, extract_texts, is_extractable

# One attachment (PDF / DOCX / XLSX / image-OCR)
if is_extractable(pdf):
    text = extract_text(pdf, max_pages=20)
    resp = await agent.run(f"Summarise this contract:\n\n{text}")

# Many, tolerant of per-file failures
texts = extract_texts(attachments, on_error="warn")

Install only the extractors you need:

pip install "symfonic-core[attachments-pdf]"    # PDF (pdfplumber + pypdf)
pip install "symfonic-core[attachments-docx]"   # DOCX
pip install "symfonic-core[attachments-xlsx]"   # Excel / XLSX
pip install "symfonic-core[attachments-ocr]"    # image OCR (needs tesseract)

is_extractable is a cheap capability probe; extract_text raises a typed error (UnsupportedAttachmentError, ExtractionDependencyMissingError, ExtractionFailedError) so you can fall back to sending natively.

Deep dive: Guide 07 — Attachment Text Extraction.

Decision cheat-sheet

  • Big tool result you'll revisit over several turns? Enable tool_result_compaction_enabled (+ tool_result_offload_enabled for ≥60K results). Measure the cost delta.
  • Tool that can return a lot? Return a bounded page + total + a cursor. Don't dump the whole set on the first call.
  • Document where only the text matters? extract_text before the run.
  • Image whose pixels matter? Send natively, and extract on later turns so it isn't re-attended forever.
  • Not sure it's worth it? The offload gate is cost-priced and default-off — turn it on, run a representative workload, keep it if cost drops without a quality regression.