Interior views in view_scene, paint and search_materials with native scene materials

[Aymericr]

What

The E2 set from Adam's #1024, curated onto main. Two capabilities:

• view_scene can look inside a floor.
• paint and search_materials are shared agent tools, with native scene materials.

Interior views (view_scene)

• Contract (core/agent-tools/view-scene.ts): new input interior: { levelId }. The description only covers what main does; feat: Bundle 2 editing, interior captures and native materials #1024's request_upload wording is gone.
• Plan (core/agent-operations/scene-view.ts): sceneViewInterior(nodes, levelId) returns the floor's box, a cut at 0.65 of the storey, and the ids to hide (the other floors, plus this floor's roofs and ceilings).
  • With no target, eye or camera, the view looks down into the floor from 55°.
  • It refuses:
    • interior_level_invalid: not a level, or a level holding only roofs and ceilings.
    • interior_hidden: the level or a parent is hidden. A hidden site doesn't count, because it only hides its own ground.
    • interior_transform_unsupported: a tilted building.
    • interior_target_elsewhere: a target on another floor, or one of this floor's roofs or ceilings.
  • The contract says what is cut: everything on the floor (walls, openings, furniture) at 0.65 of the storey height.
• Editor capture (thumbnail-generator.tsx): interiorCutaway runs inside the capture's existing set-up and restore cycle, for one render.
  • It hides those ids and clips the level at the cut height. Shadows are clipped too, so light gets into the rooms.
  • Walls are drawn whole whatever the user's wall mode, and the user's solo and isolation on that floor are lifted (temporarilyShowPresentation, new in the viewer).
  • The live view doesn't change.
• Whole walls for a capture now come from the viewer's WallCutout. An authored pose or an interior view emits thumbnail:before-capture with { wholeWalls: true }, and wallsForCapture swaps each wall's palette using the resolver the renderer mounted, so a curtain wall keeps its glass. This replaces the capture's own swap, which used the plain wall palette.
• A capture that asks for its own perspective view (as view_scene does) stays perspective when the user's camera is orthographic (capturesOrthographically).
• MCP (view-scene.ts): passes interior on to its SceneViewHost.
• Two framing fixes from the same file in feat: Bundle 2 editing, interior captures and native materials #1024:
  • A building that is moved or rotated on its site is framed where it actually stands.
  • A roof, or a roof level holding only roofs, now frames instead of being refused with nothing_to_view.

paint and search_materials

• planWallPaint and plannedWallIds move unchanged from nodes/src/wall/paint-plan.ts to core/src/systems/wall/wall-paint-plan.ts. The editor's hand paint and the agent's paint share one plan instead of feat: Bundle 2 editing, interior captures and native materials #1024's copy.

• paint gives one finish to walls, columns, slabs, ceilings, fences, doors, windows and roofs. The finish is one of:

  • a colour (the nearest library colour, reported with how near it is);
  • a library: or scene: ref;
  • a native material;
  • erase.

  role picks the surface: a slot, a wall's exterior or interior face (worked out from the rooms on each side), or a roof's top, edge or wall.

  • A wall face is painted whole, the same way the editor's Paint writes a click on each part: the face's slot, plus an override in each room whose wall finish would otherwise show over it.
  • Painted parts (regions) keep their own finish, and the result's note names them.
  • An unknown target, role or material refuses the whole call. So does exterior or interior on a curtain wall. shadowed_field refuses a roof role that a segment's own finish would hide.

• Native materials are reused when the scene already holds one the material schema reads as equal; otherwise they become a new scene material through SceneChanges.materials. A host that can't store scene materials refuses them (materials_unavailable) but still paints refs and colours.

• search_materials finds library refs by words (timber is wood, render is stucco), by shade and by surface, and names the words nothing matched.

  • Words match whole words, or the start of a word from four letters (tile matches tiles).
  • Short and common words are dropped.
  • A flat colour passes every surface filter.

• Other core changes:

  • AgentContext.materials is new.
  • requireMaterialRef checks scene refs against the scene's materials, and its refusal now points to search_materials.
  • get_node returns the scene materials a node's finishes use.
  • mergeSceneChanges carries materials through.

• MCP:

  • SceneBridge takes upsert_material patches, written before the nodes in the same history step. This is an internal op, not part of apply_patch's schema.
  • supportsMaterialUpserts and getMaterials are added on SceneOperations.
  • paint is registered as destructive and search_materials as read-only.
  • Updated to match: the annotation policy and justifications, the MCP README, the agent guide, the pascal-3d skill and wiki/architecture/agent-surfaces.md.

Not ported, on purpose:

• feat: Bundle 2 editing, interior captures and native materials #1024's read-only capture transaction: per-node commit tracking in NodeRenderer, a frame-loop freeze, snapshots of camera controls and object state, and refusals on pending or failed assets. It also dropped the loader hooks that only fed it (model-loader resource failures, ktx2-loader load state). Interior views stay on the snapshot path view_scene already uses, which waits for the scene to build and holds the viewport's last frame. The asset-readiness and scene-revision gaps are on that path for every view_scene picture today (see Review).
• material-swatches.ts (only the facade elevation reads it; facades are held back).
• sceneViewFacing (Follow Pascal).
• The run_batch and plan-reference hooks.

Why

From the owner's feedback (plan plans/scene-agent/ai-architect-chat-feedback.md, slice E-materials):

C14 "let's look at the latest PRs from Adam (AxiomeCG) and Steven Tibbs to bring them in: I also added images in the reference folder with commentary on some Steven's feature for generative UI and DXF import."

G2 "we need to compile and harmonize tools developped by Wassim, Adam (AxiomeCG), and Steven Tibbs from the AI capabilities to avoid conflicts, explosation in complexity, and increase reliability and consistency of outputs."

Adam's interior views and native materials come in as shared tools: one contract and one operation in core, so the chat and the MCP get the same paint, search_materials and view_scene. The editor's hand paint and the agent's paint now share one wall paint plan.

How to test

From a checkout that has this branch as its editor/ submodule (the commands run from the repo root):

bun test editor/packages/core/src/agent-operations/paint.test.ts editor/packages/core/src/agent-operations/search-materials.test.ts editor/packages/core/src/agent-operations/scene-view.test.ts
bun test editor/packages/mcp/src/tools/paint.test.ts editor/packages/mcp/src/tools/view-scene.test.ts editor/packages/mcp/src/tools/read-tool-annotations.test.ts
bun test editor/packages/viewer/src/lib/scene-visibility.test.ts editor/packages/viewer/src/systems/wall/wall-cutout-cache.test.ts editor/packages/editor/src/components/editor/thumbnail-generator.test.ts
bun test editor/packages/nodes/src/wall   # the hand paint on the moved plan: preview equals commit, room scope, erase

Results on this branch (head b566369):

• core: 4061 pass.
• mcp: 704 pass.
• nodes: 4753 pass.
• viewer: 527 pass.
• editor: 2054 pass.
• check-types (--force): clean.
• biome: clean on the changed files.
• skills:validate and its policy tests: pass.

Viewer and editor show only the failures that happen without this branch too: the R3F version pin, and the history.test.ts subprocess stderr. Both come from the hosted install, not editor CI.

Checked in the running app on a two-storey test scene. The chat doesn't pass interior until the hosted bump, so this used a local hook that calls the chat's executeViewScene directly.

• First pass (before review):
  • Interior views of the ground floor, the upper floor and one room all came back. Each showed no other floor, no ceilings, walls cut low and light in the rooms.
  • A normal south-west view still shows the whole house.
  • interior_target_elsewhere and interior_level_invalid came back as refusals.
  • The live viewport was pixel-identical before and after the six calls.
• After review, on the same scene with its east ground-floor wall turned into a curtain wall:
  • The interior view of the ground floor shows the curtain wall's frame and glass, and so does a view from the east.
  • With the user's camera switched to orthographic, the interior view came back perspective from 55°. It was byte-identical to the same call made in perspective mode.
  • The live viewport matched before and after the calls: no pixel moved by more than 12/765.

Deployment notes

• Merge this before the hosted H-roofs PR. That PR bumps the editor and wires paint, search_materials and interior into the chat and the hosted MCP. Until then the hosted app doesn't offer them.
• With this branch pinned, hosted's inventory tests fail until H-roofs registers the tools: agent-surface-parity, hosted-mcp-tool-effects and the hosted MCP tool count. That's expected.
• This branch is rebased on main 80d7a2f. That includes Blender (bpy) modules as a second geometry-script language #1033, whose geometry-script API change (GEOMETRY_SCRIPT_MIME_TYPE, language) breaks hosted typecheck on its own. Any hosted bump past Blender (bpy) modules as a second geometry-script language #1033 needs that adaptation as well.
• This PR edits the same tool index files as the E-foundations editor PR (agent-tools/index.ts, agent-operations/index.ts, mcp/src/tools/shared-tools.ts). Whichever lands second rebases.
• For H-roofs and H-catalogue (G2), with this bump:
  • Pro's in-tab toolset calls registerSharedTools on a SceneBridge, so it gets paint and search_materials, with native materials. That puts them next to the Architect plugin's set_material and list_materials, which have different semantics: slots named interior/exterior, missing targets skipped. One of each pair has to be retired or routed to the other with the bump.
  • search_materials searches the built-in catalog only. Community and workspace materials (library:mtl_…, or registered at runtime) still need an owner before list_materials retires.
  • Nothing yet checks that interior reaches the capture on either host: the chat's executeViewScene and the hosted SceneViewHost both drop it today. Test it on both surfaces.
• thumbnail:before-capture now takes an optional { wholeWalls } payload. Existing listeners that ignore it are unaffected.
• Env vars: none. Migrations: none.

Review

Both reviews (Claude adversarial, GPT-6 Astra) were checked against the code.

Fixed:

• paint on a room-finished wall. It wrote a face slot that the room's wall finish hides, and still reported the paint. It now paints the face whole (see What). Tests check what the renderer resolves along the face after paint and after erase.
• search_materials matched substrings. for matched forest, red matched prepared, and a matched everything. It now matches whole words and drops short and common ones. A universal flat colour now passes a surface filter.
  • Surface words still only count materials made for that surface. Counting flat colours too flooded "light oak floor" with presets.
• Roofs on an interior view. An interior of a roof-only level, or a roof or ceiling target, framed something the picture leaves out. Both are now refused.
• Hidden site. A hidden site blocked interior views, but it only hides its own ground. It no longer counts.
• Contract wording. It said only walls are cut; everything on the floor is cut at 0.65. Fixed in the contract, README, agent guide, skill and wiki.
• Orthographic live camera. It replaced view_scene's requested perspective camera.
• Curtain walls in captures. An interior or authored-pose capture lost their glass. Whole walls now come from WallCutout's own resolver.
• Curtain wall faces. exterior and interior on a curtain wall wrote unused face slots. They are now refused.

Not changed:

• Asset readiness, and camera vs. cutaway revision (Astra).
  • Every view_scene and sheet capture already waits for the scene to build, then renders. The interior view adds no new path.
  • feat: Bundle 2 editing, interior captures and native materials #1024's capture transaction, which gated on asset readiness, is held back from this PR on purpose.
  • A pending or failed-asset refusal for view_scene as a whole is a follow-up.
• library:mtl_… refs pass without a check (Astra). That is main's behaviour, unchanged here: the host checks community refs against its catalog. Scene refs are now checked whenever the host lends its materials.
• MCP receipts under readOnly (Astra). This is in the shared runner and affects every mutating shared tool, not just paint, so it is out of scope for this PR.
• Other buildings' floors are hidden in an interior view (Bugbot). This is intended: the picture shows one floor, and a neighbouring building's floors, uncut, would block the look down into it.
• A fourth "equal scene material" matcher (Claude nit). Follow-up: make hand paint's three findMatchingSceneMaterial copies use core's normalising matcher, so agent-made materials aren't duplicated by hand paint.

Supersedes

Supersedes in part #1024 by Adam NAILI (@AxiomeCG): its E2 set (interior captures and native materials). Adam is co-author on every commit that ports his work. The rest of #1024's curated set (E1) comes in a separate PR. Plan reference, vectorize, facades, unit layouts, photo tools, coherence, batch/checkpoints, visual references and request_upload are held back.

🤖 Generated with Claude Code

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 45c5a14. Configure here.}