Per-node visual effects: blocked on paint_pass, with the patch sketched

[LeadcodeDev]

The "per-node visual effects" gap (High/L) from the re-scored Remotion differential. Investigation landed in #175 as a diagnostic with tests; this is the work that remains.

What is missing

effects is a field of Scene only (schema/scenario.rs:386), with four variants (Grain, Vignette, Pixelate, ProgressiveBlur). An effect covers the whole frame. There is no way to vignette a mockup, or grain a background image while sparing the text over it.

What is already possible today

style.filter's FilterFn::Noise { intensity, seed } is a real Skia ImageFilter, applied per node with correct layer isolation. This works right now:

"style": { "filter": [{ "fn": "noise", "intensity": 0.15, "seed": 42 }] }

It cannot do animated grain — noise_image_filter is deliberately frame-stable. So the grain half of this gap is narrower than the differential suggested. Vignette, Pixelate and ProgressiveBlur have no FilterFn equivalent, and are where the value is.

Why the obvious workaround is disqualified

Render normally, then crop the effect to a node's rect from render_scene_hits. Measured in crates/rustmotion/tests/node_effects_leak_proof.rs:

5904/10000 pixels inside an overlapping sibling's own box changed
sample at (300,300): [0,0,255,255] -> [155,0,99,255]

59% contamination of a sibling sitting visually on top — destroying exactly the text the feature is supposed to spare. It also costs x1.47 on mega-showcase scene 0, but the leak is the disqualifying part.

The three blockers

1. A node's device-space rect exists only inside paint_node. BoxLayout carries neither transform nor camera. And css/animation.rs resolves animation presets into CssStyle.transform overrides, so most animated content carries a transform absent from the raw JSON — a rect derived from run_layout alone is silently wrong for it.
2. Crate direction. post_effects.rs lives in rustmotion, which depends on rustmotion-core where paint_pass.rs lives. The pure effect functions must move to rustmotion-core first (the PostEffect type is already there); leave a thin re-export so tasks.rs keeps working.
3. CssStyle is the only per-node channel that already reaches BoxNode — via box_builder → BoxNode.css → paint_pass, the same route filter/transform/opacity take. Anywhere else means teaching box_builder.rs a new field.

Sketch

crates/rustmotion-core/src/css/style.rs, after backdrop_filter:

/// Node-scoped post-processing effects — same catalogue as `Scene::effects`,
/// applied only to this node's isolated layer (paint_pass step 4). Runs
/// BEFORE the `filter` ImageFilter chain — see paint_pass.rs.
pub effects: Option<Vec<crate::schema::scenario::PostEffect>>,

CssStyle carries deny_unknown_fields, so style.effects currently fails loudly (unknown field 'effects') rather than being silently ignored — no phantom-field window in the meantime.

paint_pass.rs step 4: when node.css.effects is non-empty, replace the automatic SaveLayerRec with an owned raster surface bounded to the node's box, paint steps 5-10 into it, read the raw pixels, run apply_post_effects, write back, then draw_image_rect with the same paint the automatic path would have used.

The real work is factoring steps 5-10 of paint_node into a function taking an interchangeable target canvas, and deciding how z-index recursion, the hit map, plane cameras and Ghost nodes behave inside an isolated layer. That is a judgement call for whoever owns the file.

Order relative to style.filter

Node effects (CPU) run before filter (GPU), always, and it is not configurable: the CPU pass needs the raw pixels of the isolated layer before Skia recomposes it, and canvas.restore() applies set_image_filter at exactly that moment. After that point the isolated layer no longer exists.

The constraint any implementation must respect

Bounding the effect pass to the node's real box rather than the full frame, at a 13.9x area ratio:

	Release
4 effects, full 1920x1080 frame	44.6 ms/frame
4 effects, bounded to a 1247x120 card	3.06 ms/frame

x14.57. The same argument PR #154 made for the save_layer it bounded (42-60s → 0.5s over 60 frames). crates/rustmotion/tests/node_effects_cost.rs is #[ignore]d and reproduces it:

cargo test -p rustmotion --release --test node_effects_cost -- --ignored --nocapture

Not explored

Transition and world-view frames composite two already-rendered scene buffers, so per-node provenance is gone by then. Applying node effects there needs a separate design.

Target syntax

{
  "type": "image",
  "src": "hero-bg.jpg",
  "style": {
    "width": "1080px", "height": "1920px",
    "effects": [
      { "type": "vignette", "intensity": 0.5, "radius": 0.7 },
      { "type": "grain", "intensity": 0.12, "animated": true }
    ]
  }
}