Let people send files to the agent - #567
Merged
Merged
Conversation
Attachments were outbound only. The agent could attach an image to a reply,
but someone with a screenshot, a scanned invoice, or a photo of a whiteboard
had nowhere to put it.
The upload goes through the agent, not the file share. Frontends co-mount the
shared volume read-only ("the agent owns writes") and the CLI does not mount
it at all, so a client cannot stage a file itself. AttachmentUploadRequest
carries the bytes to the agent, which writes them through its existing
IAttachmentStorage and returns an AgentAttachment path reference; the message
the user then sends carries only that reference.
Letting each client write directly would have meant flipping the Blazor mount
to read-write, duplicating the containment check, filename sanitisation, MIME
allowlist and size cap per client, and still leaving the CLI unable to attach
anything. Validation is authoritative agent-side: image/* plus PDF, an 8 MB
cap matching AnalyzeFileMaxBytes, and a check that the declared type agrees
with the extension so a file cannot be stored under a name the next reader
resolves differently.
This is the one place bytes cross the bus, and it is its own request/reply
rather than a field on UserMessage, so the conversation message, the persisted
turn and every history replay stay byte-free.
InboundAttachmentInjector appends the current turn's attachments to the last
user message — a DataContent when the attachment is an image and the tier
declares SupportsImageInput, otherwise a line naming the path and pointing at
analyze_file. A file someone deliberately attached is never silently dropped.
Replayed history turns keep only the marker: re-materialising every image in
the window would cost a disk read per image per request and leave thousands of
image tokens standing in context.
ConversationTurn.Attachments is additive, so per schema-migrations.md it needs
no migration and the conversation store is not enrolled in them anyway.
Also flips ClientCapabilityPresets.Blazor to declare ImageAttachment. The test
asserting it should not had already documented its own expiry — "until
AgentReply.Attachments lands", which it has.
Closes #565
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uf8a2Z1v9dY3YjjM42zJEj
…turn Two changes, both found by pushing on the first cut. An API alternative to the bus. Routing every upload through RabbitMQ meant multi-megabyte bodies sitting in broker memory until they were acked, to move a file onto a volume the agent already mounts. AttachmentUploadEndpoint is a small Kestrel listener in the agent process: POST /attachments writes it directly. Clients use it when AttachmentUploadUrl is configured and fall back to the bus when it is not or the attempt fails, so a CLI on a laptop keeps working. A rejection is not a fallback trigger — a file the agent refused over HTTP would be refused over the bus, and retrying would only make the user wait twice. The validation rules moved into InboundAttachmentService so both transports share them. An allowlist enforced on one path and not the other is no allowlist at all. The chart gains a ClusterIP Service for the agent (it had none) and points Blazor at it. The endpoint has no auth of its own — same posture as the introspection sidecar in that Pod — so the Service is deliberately ClusterIP and documented as not safe to expose without auth in front. An attachment now survives its own turn. On a seeing tier the image was injected as DataContent and the path never entered the persisted turn text, so a follow-up had nothing to reach. A live Blazor session found it: asked to look again, the agent answered "I don't have the image in this chat to re-check" and called file_list hunting for it. design/multimodal-input.md claimed replayed turns keep a marker; they did not. DescribeAttachments now appends the path to a replayed turn, and the same session answers correctly from the file afterwards. Every unit test passed through that defect because they all exercise a single turn. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uf8a2Z1v9dY3YjjM42zJEj
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Attachments were outbound only. The agent could attach an image to a reply (
attach_image→AgentAttachment→ Blazor's/attachmentsendpoint), but someone with a screenshot, a scanned invoice, or a photo of a whiteboard had nowhere to put it. This is the missing direction.#563 gave the agent
analyze_file; #564 made the context estimate honest aboutDataContent, which is what made inline images safe to land.The upload goes through the agent, not the file share
The issue sketched a Blazor upload writing into the shared directory. That would have broken the rule the chart states plainly — "Read-only: Blazor only serves attachment bytes; the agent owns writes" — so it isn't what shipped.
Instead:
AttachmentUploadRequestcarries the bytes to the agent, which writes them through its existingIAttachmentStorageand returns anAgentAttachmentpath reference. The message the user then sends carries only that reference.readOnly: trueThis is the one place bytes cross the bus, and it is deliberately its own request/reply rather than a field on
UserMessage— so the conversation message, the persisted turn, and every history replay stay byte-free. (AgentAttachment's "bytes never ride the bus" is about the outbound direction, where the frontend already mounts the volume and shipping bytes would be waste; inbound is the opposite case.)Validation is authoritative agent-side:
image/*plus PDF, an 8 MB cap matchingAnalyzeFileMaxBytes, and a check that the declared type agrees with the extension — so a file cannot be stored under a name the next reader resolves differently. Clients pre-check the same rules only to spare a pointless round trip.What the model sees
InboundAttachmentInjectorappends the current turn's attachments to the last user message — bytes can only enter a conversation as content parts on a user message on OpenAI-compatible APIs:SupportsImageInput→DataContent, the real thinganalyze_fileA file someone deliberately attached is never silently dropped. That is the behaviour with the most tests behind it.
Replayed history turns keep only the marker line. Re-materialising every image still in the context window would cost a disk read per image per request and leave several thousand image tokens standing in context; the model can still reach an older image deliberately through
analyze_fileon the path the marker names.Also here
UserMessage.AttachmentsandConversationHistoryTurn.AttachmentsreuseAgentAttachmentoutright rather than adding a parallel type, soAttachmentList.razorandAttachmentPlaceholderrender user attachments with no change.ConversationTurn.Attachmentsis additive — perdesign/schema-migrations.mdno migration, and the conversation store is not enrolled in them anyway.OpenReadStreamis passed an explicitmaxAllowedSize(the default is 512 KB and throws past it).--attach <PATH>, repeatable.ClientCapabilityPresets.Blazornow declaresImageAttachment. The test asserting it should not had already documented its own expiry: "until AgentReply.Attachments lands", which it has.design/multimodal-input.md— concern (D) marked landed, with the gateway decision and the current-turn-only rule recorded.Deliberate limits
--attachrequires--message(one-shot). Attaching per-message in the REPL needs an affordance it does not have; a clear error beats quietly attaching to whichever message happened to go first.image/*becomesDataContent; a PDF is announced with its path.Test plan
AttachmentUploadHandlerTests(8) — valid PNG writes and returns a relative reference; a second file of the same name does not clobber the first; disallowed type rejected with a message naming what is accepted; declared type disagreeing with the extension rejected; both JPEG extensions accepted; over-cap and empty rejected before touching disk; a traversal in the supplied filename stays inside the base directoryInboundAttachmentInjectorTests(9) — image on a seeing tier becomesDataContentand no other message is touched; blind tier gets the marker, no bytes, and no disk read; a PDF is announced even on a seeing tier; an unreadable file degrades to a marker that says so rather than failing the turn; several attachments all land; no attachments leaves the context byte-identical; no user message does not throwChatCommandAttachTests(4) —--attachwithout--messagefails before a host is built; the option binds asstring[]so it is genuinely repeatable; MIME guessing covers the accepted types and defers to the agent for anything elseChatStateServiceLoadHistoryTests(+3) — attachments survive a history reload, absent ones stay null, and a sent message shows its attachment immediatelyClientCapabilitiesTests— the Blazor preset assertion flippeddotnet test RockBot.slnx— 20 assemblies, 0 failuresNot yet exercised end-to-end against a running stack; the compose path (
ROCKBOT_SHARED_PATHas a local directory) is the cheap way to do that.Closes #565
🤖 Generated with Claude Code
https://claude.ai/code/session_01Uf8a2Z1v9dY3YjjM42zJEj