[Quill] Clamp brush texture sampling to an opt-in UV window to stop atlas bleed from the AA fringe

[Acissathar]

As mentioned in Discord, there is an issue with Paper using the SetTextureBrush to render a part of a spritesheet that causes the AA to bleed over some pixels, regardless of filtering on the sprite / clamping. Tested locally and can verify it fixes the issue, but the logic itself was largely handled by Claude because this is out of my knowledge space.

Problem

Textured shape draws bleed neighbouring atlas content along their edges whenever the brush maps a
sub-rect (UV window) of a texture onto a shape:

• RectFilled (and shape AA in general) emits its AA fringe half a physical pixel outside the
  nominal rect (Canvas.cs, outer ring at x - hp / x + width + hp). The overshoot itself is
  correct: the coverage ramp is centred on the edge, so a pixel centred on the boundary gets ~0.5
  coverage and abutting shapes join without seams.
• The canvas shader samples the brush texture by fragment position and applies coverage after the
  sample (Canvas.slang: float4 fill = color * texture0.Sample(applyTransform(..., input.fragPos))).
• Fringe fragments therefore map to UVs outside the sprite's window — into the next atlas cell. The
  sampler can't help: ClampToEdge acts at the texture border, not at u1.
• DrawImage has the same overshoot, but its window is [0,1], so ClampToEdge happens to hide it —
  and on Repeat samplers the fringe wraps and shows the opposite edge instead.

A transparent gutter in the atlas is not a substitute fix: the fringe is half a screen pixel,
which spans more than one texel whenever a sprite draws below native size.

Fix

An opt-in UV clamp window on the brush, applied in the shader before the sample:

• Brush.TextureClamp — (uMin, vMin, uMax, vMax), where uMax <= uMin means unclamped. The
  default is unclamped, so existing draws and tiling brushes are untouched. Deliberately not a
  (0,0,1,1) default: that would break Repeat wrapping and depend on every construction site
  remembering to set it.
• Canvas.SetBrushTextureClamp(uMin, vMin, uMax, vMax) stores the window inset half a texel
  toward its centre. The inset is load-bearing: the window boundary is the texel edge shared with
  the neighbouring cell, where bilinear blends 50/50 with the neighbour and nearest floors into it —
  clamping to the raw window still bleeds. Half a texel in, both filters resolve to the window's own
  edge texel. The inset needs the texture size, so call this after SetBrushTexture. Windows
  narrower than a texel collapse to their centre rather than inverting.
• Canvas.slang clamps the transformed UV to the window before sampling when a window is active —
  the behaviour ClampToEdge gives at a real texture border, produced at a sub-rect border.
• TextureClamp participates in Brush.Matches/ComputeHash, so adjacent sub-rect draws from one
  atlas can't merge into a single batch sharing the wrong clamp.
• DrawImage now self-clamps to (an inset) [0,1] — it knows its own window — which fixes the
  Repeat-sampler fringe wrap with no API change.

Why opt-in rather than automatic: the brush transform is a mapping, not an extent — nothing in
brush state says the paint ends at u1, and for tiling brushes it deliberately doesn't. Deriving
the window per-draw from the shape's bounds is only correct while the brush is axis-aligned with
the shape, and would silently bleed again under rotation. (This is the same trade-off Skia exposes
as SrcRectConstraint on drawImageRect.)

Notes: no ICanvasRenderer change — the inset uses the existing GetTextureSize. Renderers opt in
by uploading DrawCall.TextureClamp as the textureClamp uniform; one that doesn't keeps exactly
the current behaviour (zero = unclamped). Backend shaders regenerate from Canvas.slang via the
existing ShaderGen step.

[michaelsakharov]

Huh, this is not the solution i expected xD

I guess it works, not a huge fan, of it being shader-sided i was thinking of like just padding the Texture Brush or something.
But this works just fine.