[MarkusNeusinger]

Summary

Drag and drop, PR 2 of 2 (binary data files) for the admin agent chat. PR 1 (#12192) read text files in the browser; this PR reads Excel (.xlsx, .xlsm, .xlsb, .xls), OpenDocument (.ods), Parquet, Feather (v1 and v2), Arrow IPC (file and stream) and NumPy (.npy, .npz) files on the server, in the renderer's sandbox, and puts the text into the paste box. The user's .parse() then sends it through the ordinary paste route, unchanged (data/parse.py is untouched), with the file name and sheet as source (Bericht.xlsx · sheet "2024"), screened as in PR 1.

• Converter (agents/anyplot/render/convert.py, standalone like the harness, copied to /opt/anyplot/convert.py). Format from the first bytes only, never the name.
  • Every zip is checked from its central directory first: at most 2,000 entries, 100 MB declared uncompressed, 250 times the archive's size, no encrypted member.
  • An OLE container with EncryptedPackage is refused with "remove the password". Pickle, joblib and HDF5 are never read.
  • Workbooks go through python-calamine (MIT): stored values only, so no formula, external link, macro or DDE runs. The sheet goes out as its raw cell grid via csv.writer: ISO dates, shortest numbers, TRUE/FALSE, empty for nothing. So the parser's table_block finds a header below row 1 and keeps a title as a note (tested end to end with parse_dataset).
  • Merged cells keep their value in the top-left cell only. A formula without a saved value is left empty and counted (xlsx and xlsm).
  • Sheets are read row by row and refused past 20,200 raw rows or 50 columns, whatever dimension the file declares. Hidden sheets are skipped and counted; very hidden ones are never named.
  • Parquet, Feather and Arrow go through pyarrow (26.0.0, at least 14.0.1 for CVE-2023-47248). The footer row and column counts are checked before any page is read, at most 20,001 rows are read, and every read is single-threaded.
  • Structs become dotted columns, dictionaries are decoded, and timestamps with a time zone are written in UTC. Map, binary and union columns are dropped and counted.
  • List columns follow feat(agents): read space-aligned tables and API-wrapped JSON #12191's list rule. A table with a list column goes out as JSON records, so the parser's own expansion does the work (one row per value, record_id, thinning). There is no second implementation.
  • NumPy loads with allow_pickle=False after the header's shape is checked against the bytes present. Only 2-D or structured arrays are read; each .npz member is a "sheet", and a pickled member is skipped or refused.
  • Caps: 1 MiB of text (the paste cap), 20,000 rows, 50 columns. A file over a cap is refused with its numbers, never cut.
• Harness convert mode (harness.py --convert input.bin). As in data mode it reads the bytes, loads the converter, and preloads the readers (including a cp437 and deflate warm-up for zipfile) before it arms ConvertGuard. ConvertGuard is DataGuard with no exec at all. It allows one compile: NumPy's .npy header reader parsing its dict literal to an AST, matched by its exact caller chain. The harness writes out.csv, meta.json, and status.json with the refusal's exit code (10 + REFUSALS index), the measured value and the cap.
• Renderer POST /convert (wire.ConvertRequest: base64, sheet, at most 15 s). It uses the same slot, memory floor, watchdog, kill path and idempotency key (job_id, convert). executor._prepare writes bytes. The sandbox environment adds ARROW_DEFAULT_MEMORY_POOL=system. Nothing of the file is logged.
• Agents POST /v1/sessions/{sid}/dataset/convert:
  • Backends: ConvertJob and ConvertOutput in remote, local (mounts the converter), fake and sandbox (unsupported).
  • SerialRenderer.convert takes the toggle lane, behind every pipeline render. It runs with a 10 s timeout and a 20 s slot wait, then 503 capacity.
  • Limits: its own counter (5 a minute, 30 a day, AGENT_USER_CONVERSIONS_PER_*), checked after the daily budget and strikes. One conversion per user at a time (Runtime.converting).
  • Refusals answer 422 file_<reason> with measured and limit. A run the host stopped answers file_timeout or file_too_complex. A renderer without the route answers 503 convert_unavailable.
  • The data_convert attribution line carries kind, outcome, byte count and time only.
• BFF POST /debug/agent/sessions/{sid}/dataset/convert. The body is taken by hand, so the new router dependency require_body_within_cap refuses an oversized body by its Content-Length before it is read (413); a chunked conversion body gets 411. The BFF caps the file at 5 MB raw and passes only whole-number measured and limit through.
• Page. fileText.ts hands the four convertible kinds to the server, up to 5 MB, and its accept list gains the extensions and MIME types. In DataPanel, a conversion fills the box like a text file. While a file is read, .open(), .parse() and the drop zone wait. The line names the sheet, the shape and the warning counts, and a refusal names its numbers. A sheet picker reads another sheet again: over the file's own untouched text it replaces at once, over an edited box it asks .replace().
• Analytics. agent_data_file gains the server kinds (xlsx, ods, parquet, feather, arrow, npy, npz, ...) and the outcomes sheet and failed (docs/reference/plausible.md).
• Docs. Design doc (request flow, renderer, guardrail row, /v1 table), api.md (route and error section), agents/README.md settings and the changelog fragment. The dev BFF mock answers the conversion.

Decided and documented (within the brief)

• Conversion and paste count separately. The conversion counts on its own window and never as a paste. The paste that follows counts as one paste (settings.user_conversions_per_minute docstring).
• Lane. The conversion takes the toggle lane, so a waiting pipeline render always goes first.
• Auto-pick. Without a chosen sheet, .npz skips pickled members (refused only when nothing else is readable or the user picks one). A workbook reads the first visible sheet with a cell.
• Sheet list. Only visible sheets with cells are offered (owner decision 2026-10-15); the first 50 are checked, the rest counted.

Security review and owner decisions (fe3276a)

Security review (Fable with 60 crafted files through the real harness, Gemini by reading), all fixed with a test each:

• Quadratic regex. The xlsx scan now uses one possessive tag tokenizer, which is linear. The review's 30 MB run of <c previously ran to the 10 s kill; the scan now takes 0.39 s, and the crafted workbook converts in under 1 s (both asserted).
• Parquet row count. The pre-check adds up the row groups, and a footer total that disagrees is file_corrupt.
• Declared zip sizes. Every zip member is inflated (zlib, bounded by its declared size + 1, at most 100 MB in all). A member whose size differs from what it declares is corrupt.
• Bounding box. One linear pass measures an xlsx sheet's cell span before calamine loads it; past the caps it is refused with the count. A reader that dies without status.json (SIGABRT) is now file_too_complex.
• Memory. MemoryError and ArrowMemoryError now map to too_complex instead of corrupt.
• Small fixes:
  • npz and sheet names with controls, bidi characters or paths are never offered;
  • a broken .npy header (TokenError) is corrupt;
  • durations and intervals are dropped and named;
  • a busy renderer (503 busy or low_memory) answers capacity;
  • the jemalloc docstring is corrected;
  • ConvertGuard keeps none of the CODE_BUILDERS.
• Found by the new in-image smoke: the UTF-16 codec of the OLE sniffer loads lazily, and the guard refused it, so encrypted workbooks came back blocked. It is now preloaded and tested under the real guard.

Owner decisions of 2026-10-15:

• pyarrow is slimmed in the renderer Dockerfile. Removed: headers, Cython sources, tests, Flight, S3, Azure/GCS/HDFS, ORC, the Substrait Python module, Parquet encryption, the C++ test module and two duplicate libarrow_python copies. libarrow_substrait stays because lib links it. CI now converts every format inside the built image (tests/image/convert_smoke.py, new step "Convert every data format through the harness").
• A formula-only sheet is refused as file_formulas_only: "contains only formulas without saved values; open it in Excel and save it once". The formulas are counted even when the grid is empty.
• Empty sheets are hidden from the picker.
• Precision is exact, per the owner rule:
  • timestamps keep every digit (nanoseconds) and their offset in their own zone; UTC with a warning only where the zone cannot be read;
  • decimals and int64/uint64 are exact;
  • float32 is written in its shortest text;
  • dropped columns are named on the page (dropped).
• The parser (parse._as_iso_stamps) reads ISO timestamps with 1–9 fraction digits and Z/±HH/±HHMM/±HH:MM offsets as datetimes. Each cell is padded to the column's longest fraction, which keeps every value and lets pandas' read_csv(parse_dates=…) load the column as datetime64[ns, UTC+01:00]. A column with several offsets is written in UTC with a note. Cells with and without an offset in one column are text. prepare_data writes fractions and offsets back (isoformat()). The profile's min/max are the exact cells.
• Conversion limits and the sheet picker wording stay.

Measurements

• Renderer image size, from the CI image job:
  • with full pyarrow: 1,012,827,949 bytes;
  • slimmed: 944,079,490 bytes (−68.7 MB);
  • pyarrow itself: 167 MB → 99 MB.
  • No main run in the last 60 built the renderer image, so I have no without-pyarrow baseline; about 848 MB if pyarrow and calamine are subtracted.
• The crafted inputs: the 30 MB tag run scans in 0.39 s locally, the zip-inflation check is linear, and a cell at A1048576 or XFD1 is refused before loading.
• No binary fixture is committed. Workbooks, ODS files, zip bombs, OLE stubs, Parquet, Feather, Arrow and NumPy files are all built in memory (tests/unit/agents/runtime/convert_fixtures.py). New dependencies: python-calamine MIT, pyarrow Apache-2.0 (with its NOTICE in the wheel).

Test plan

• ruff check, ruff format --check, mypy api core agents clean; uv lock --check passes (pyarrow is added to the mypy ignore_missing_imports list because it ships no types).
• pytest tests/unit -n auto: 11,282 passed. The one failure is the known xdist flake test_pipeline_retry_gaps.py, which passes serially.
  • test_convert.py (49 tests), covering the brief's fuzz list:
    • zip bomb, entity expansion (never expanded), spoofed XFD1048576 dimension
    • formulas without cache, encrypted OOXML, .xls container sniff, ODS, multi-sheet with hidden and very hidden sheets, merged cells
    • header not in row 1 through parse_dataset, row, column and byte caps
    • hostile and garbage Parquet footers and Arrow streams, the footer row cap
    • Feather v2, Arrow stream, list columns through parse_dataset (6 rows)
    • .npy 2-D, structured, 1-D refused, object array refused, spoofed header shape, .npz with a pickled member
    • pins across converter, harness and wire; real-harness runs per format under the guard
  • Renderer: executor convert (argv, environment, real harness behind the fake launcher, SIGPIPE launcher, meta validation), service /convert (contract, invalid jobs not echoed, shared slot, replay and conflict, log without content), wire bounds, remote backend (round trip, end to end with the real harness, 404, busy).
  • Agents route: answer shape, nothing stored, refusals with numbers, timeout, memory and error mapping, convert_unavailable, bad bodies, own rate window that leaves the pastes alone, one conversion per user, busy slot, toggle lane behind a waiting render.
  • BFF: forwarding, body allowlist, 5 MB cap, Content-Length cap before reading, 411 for a chunked body, measured and limit filtering, CSRF header.
• cd app && yarn lint && yarn fm:check && yarn type-check && yarn test && yarn build on Node 24 (1,108 tests): fileText convert hand-off and caps, convertAnswer, toBase64, toAgentApiError counts. DataPanel tests cover base64 hand-off and source, warnings line, sheet picker re-read, ask-before-replace after an edit, refusal with counts, waiting while a file is read, pickle never sent.
• /verify-frontend against the BFF mock (3017/8017), checked at 1440 light and at 390 dark:
  • an .xlsx through .open() fills the box with the sheet, the warnings line and the picker;
  • choosing sheet "2025" reads it again;
  • a row-cap refusal shows its counts, wraps at 390 px, and the picker label is 13 px;
  • no horizontal overflow;
  • the console shows only the known font-CORS noise off port 3000 and the expected 422.
• Not verifiable locally: the real Cloud Run sandbox, and the image build and size (no Docker daemon). .xls and .xlsb parsing has no generated fixture (no writer exists); only the sniff and refusal path is tested.

Follow-ups (not in this PR)

1. Arrow/Feather row count before decoding (Copilot round 2, thread left open; the review's item 5). IPC and Feather batches, and Feather v1's read_table, decode before rows are counted. A decompression bomb is bounded by the sandbox and now answers file_too_complex instead of file_corrupt, but not with the row-cap message. Fix: a small flatbuffer reader of the RecordBatch metadata.
2. Bounding-box pre-scan for ODS, XLS and XLSB. The linear span scan covers xlsx/xlsm only. ODS number-rows/columns-repeated and the binary formats still rely on the sandbox limits and the new file_too_complex mapping.
3. Mixed offsets in one column are written in UTC with a note, because pandas reads one offset per column. Keeping each offset would need an object column in the plot loader.

There are no new Plausible registrations: the new kind and outcome values use properties that already exist.

Review rounds

• Round 1 (5 comments), all applied in 4411617:
  • a conversion without Content-Length is 411;
  • the rows of a table block keep their empty trailing fields;
  • sheet names travel whole;
  • repeated flattened names are renamed and counted;
  • UTC times are written in a form the parser reads.
• Round 2 (3 comments): two applied in 9a7c44d (ConvertGuard keeps none of the CODE_BUILDERS; meta.json is written unescaped). One is left open as follow-up 1.
• Security review: all applied in fe3276a (see above); per the coordinator, there is no new Copilot round.

Test plan update (fe3276a)

• ruff, format and mypy are clean; pytest tests/unit -n auto shows 11,324 passed, with only the known xdist flake. One CI run of the test job died at 79% with no failing test in its log; the rerun is green.
• New tests:
  • TestSecurityReview and TestExactValues in test_convert.py;
  • harness runs under the guard for zone-aware timestamps and an encrypted workbook;
  • TestPreciseTimestamps in test_parse.py;
  • fractions and offsets through prepare_data;
  • the route's mapping of a dead reader to file_too_complex and of file_formulas_only;
  • the remote backend's busy → RenderBusy;
  • the frontend's dropped-columns line.
• The CI renderer image converts all 14 cases inside the slimmed image.
• Frontend: 1,109 tests pass, and lint, format check, type-check and build pass.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M

[codecov]

Codecov Report

❌ Patch coverage is 89.40833% with 145 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
agents/anyplot/render/convert.py	89.07%	77 Missing ⚠️
agents/anyplot/render/harness.py	85.71%	14 Missing ⚠️
agents/anyplot/render/backends/remote.py	66.66%	13 Missing ⚠️
agents/anyplot/render/backends/local.py	75.00%	12 Missing ⚠️
app/src/sections/agent-chat/DataPanel.tsx	90.00%	9 Missing ⚠️
agents/renderer/executor.py	91.93%	5 Missing ⚠️
app/src/hooks/useAgentSession.ts	20.00%	4 Missing ⚠️
agents/anyplot/data/parse.py	95.77%	3 Missing ⚠️
api/routers/agent.py	94.33%	3 Missing ⚠️
agents/anyplot/render/contract.py	93.33%	2 Missing ⚠️
... and 2 more

📢 Thoughts on this report? Let us know!

[Copilot]

🟡 Changes recommended

The request-size guard can be bypassed, and conversion has unresolved data-loss and round-trip correctness issues.

5 open findings

• Require Content-Length or enforce streaming size limits · New
• Preserve trailing empty fields in parsed spreadsheet rows · New
• Keep lossless sheet selection identifiers · New
• Reject duplicate flattened column names · New
• Parse timezone-aware and fractional-second timestamps · New

What changed in this PR

Adds sandboxed binary-file conversion to the agent chat, extending the existing text-file import flow while keeping parsing and screening on the ordinary paste route.

Changes:

• Adds conversion APIs, renderer execution, backend support and separate usage limits.
• Connects binary-file imports and sheet selection to the data box.
• Adds conversion tests, dependencies, documentation and analytics updates.

File	Description
tests/​unit/​api/​test_agent_router.py	Tests conversion forwarding and request limits.
tests/​unit/​agents/​runtime/​test_convert.py	Tests formats, refusals and guarded conversion.
tests/​unit/​agents/​runtime/​test_convert_route.py	Tests conversion responses, limits and scheduling.
tests/​unit/​agents/​runtime/​convert_fixtures.py	Builds binary fixtures in memory.
tests/​unit/​agents/​renderer/​test_wire.py	Tests conversion contract validation.
tests/​unit/​agents/​renderer/​test_service.py	Tests renderer conversion requests and replay.
tests/​unit/​agents/​renderer/​test_remote_backend.py	Tests remote conversion integration.
tests/​unit/​agents/​renderer/​test_executor.py	Tests sandbox conversion execution and collection.
tests/​unit/​agents/​renderer/​helpers.py	Adds conversion test helpers.
pyproject.toml	Adds conversion dependencies and typing configuration.
docs/​reference/​plausible.md	Documents file-conversion analytics.
docs/​reference/​api.md	Documents conversion routes and errors.
changelog.d/​agent-file-drop-binary.md	Records binary-file import support.
app/​src/​sections/​agent-chat/​messages.ts	Adds conversion status and refusal messages.
app/​src/​sections/​agent-chat/​fileText.ts	Routes supported binary files to conversion.
app/​src/​sections/​agent-chat/​fileText.test.ts	Tests binary handoff and size limits.
app/​src/​sections/​agent-chat/​DataPanel.test.tsx	Tests conversion and sheet-picker interactions.
app/​src/​pages/​AgentChatPage.tsx	Connects the conversion callback.
app/​src/​lib/​agent.ts	Adds conversion client contracts and helpers.
app/​src/​lib/​agent.test.ts	Tests conversion client behavior.
app/​src/​hooks/​useAgentSession.ts	Exposes session-bound file conversion.
app/​scripts/​agent-bff-mock.mjs	Adds mock conversion responses.
api/​routers/​agent.py	Adds conversion forwarding and body limits.
agents/​renderer/​wire.py	Defines conversion requests and responses.
agents/​renderer/​main.py	Adds the shared-slot conversion endpoint.
agents/​renderer/​executor.py	Executes conversions and validates output.
agents/​renderer/​Dockerfile	Includes conversion readers and module.
agents/​README.md	Documents conversion settings.
agents/​main.py	Adds session conversion handling and limits.
agents/​evals/​matrix.py	Passes conversions through without recording plots.
agents/​anyplot/​settings.py	Defines conversion quotas and timeouts.
agents/​anyplot/​render/​serial.py	Schedules conversions behind waiting renders.
agents/​anyplot/​render/​runtimes/​python.py	Adds conversion commands and environment.
agents/​anyplot/​render/​harness.py	Adds guarded conversion mode.
agents/​anyplot/​render/​contract.py	Defines backend conversion interfaces.
agents/​anyplot/​render/​backends/​sandbox.py	Explicitly marks conversion unsupported.
agents/​anyplot/​render/​backends/​remote.py	Sends conversion jobs to the renderer.
agents/​anyplot/​render/​backends/​local.py	Runs conversions in isolated containers.
agents/​anyplot/​render/​backends/​fake.py	Supplies scripted conversion results.
agents/​anyplot/​plugins/​ledger.py	Tracks separate conversion usage windows.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

Guard exemptions, late allocation checks, and multi-sheet correctness issues remain unresolved.

3 open findings

• IPC and Feather decode oversized batches before enforcing row limits · New
• Conversion mode inherits unsafe DataGuard code exemptions · New
• ASCII JSON escaping overflows metadata cap for Unicode sheet names · New

5 resolved since last review

• Require Content-Length or enforce streaming size limits
• Parse timezone-aware and fractional-second timestamps
• Reject duplicate flattened column names
• Keep lossless sheet selection identifiers
• Preserve trailing empty fields in parsed spreadsheet rows

Previously missed (1)

In code that hasn't changed since last review

Selected sheet missing from options when beyond first ten sheets

app/​src/​sections/​agent-chat/​DataPanel.tsx:1331

The selected sheet can be absent from sheets: the converter searches every visible sheet but lists only the first ten. With ten empty sheets followed by a populated Data sheet, this select has value Data without a matching option, so the displayed choice does not identify the data in the box. Include an option for the current sheet when it falls outside the listed choices, and cover this auto-selection case in a panel test.

🧠 Review effort: Balanced