Skip to content

feat(#1520): Selfie generative background + cartoonify (client-side) - #1565

Merged
jung-thomas merged 12 commits into
mainfrom
selfie-generative-bg-1520
Aug 9, 2026
Merged

feat(#1520): Selfie generative background + cartoonify (client-side)#1565
jung-thomas merged 12 commits into
mainfrom
selfie-generative-bg-1520

Conversation

@jung-thomas

Copy link
Copy Markdown
Contributor

Selfie Tier 3 (#1520) — client-side generative background + cartoonify

Adds two 100% client-side capabilities to the Devtoberfest selfie composer (/devtoberfest/selfie/), preserving the hard privacy promise — your photo never leaves the browser. No hosted inference, no photo egress; the only new network activity is loading same-origin assets (themed background PNGs + the ONNX model + onnxruntime-web's own bundled WASM).

What's new

  • Themed background swap — pick a scene (Terminal, Starfield, TechEd Stage, Pumpkin Patch, Autumn Gradient) rendered on a new bottom-most Konva layer behind the cut-out person. Picking a scene forces removeBg on (a themed background is meaningless without the cutout). Vector art generated at build time (SVG→PNG via sharp).
  • Cartoonify — a neural style-transfer effect (AnimeGANv2 face_paint_512_v2, ONNX, run in-browser via onnxruntime-web). New async effect path; the model + runtime are lazy-imported so neither touches the page-load path.

Architecture (Approach A — two independent bounded layers)

  • backgrounds.ts — scene definitions + backgroundUrl helper.
  • compose.tssetBackground(img|null) on a bottom-most bgLayer; exportPng now awaits the async effect dispatcher (effect bakes before the polaroid border).
  • stylize.tscartoonify(canvas); fail-soft (any failure returns the input canvas unchanged); a load failure clears the cached session so a later attempt can retry.
  • effects.ts'cartoon' effect id + applyEffectAsync (routes cartoon → cartoonify, delegates all other ids to sync applyEffect).
  • BackgroundPicker.vue + Composer.vue wiring (scene picker, cartoon busy spinner, Export disabled during bake).

Licensing

The AnimeGANv2 model binary ships with a verbatim MIT LICENSE (© 2021 Bryan Lee) + ATTRIBUTION.md (source URL, SHA-256, provenance) under hugo/static/vendor/animegan/. Vendored via scripts/vendor-animegan.mjs (idempotent, SHA-guarded, sentinel-tracked).

Testing

  • 139/139 selfie unit tests pass; vite build clean (onnxruntime-web lazy-splits into its own chunk).
  • Built via subagent-driven-development: fresh implementer per task + task review after each + a broad opus whole-branch review (clean — no Critical/Important findings).

Not covered / follow-up

  • Live browser QA at /devtoberfest/selfie/ is the post-merge step — the onnxruntime-web WASM resolution under the approuter CSP can only be confirmed in a real browser (unit tests mock ORT). The imgly precedent (fix(selfie): attach camera after DOM mount; scope 'unsafe-eval' CSP to /devtoberfest/* #1546) already proved the CSP permits 'wasm-unsafe-eval' + 'unsafe-eval'.
  • Cartoon output is opaque (alpha=255) — the niche borderless+background-less cartoon case loses composite transparency (in-design: the effect applies to the whole composite).

🤖 Generated with Claude Code

…ied)

- scripts/vendor-animegan.mjs: idempotent vendor script (sentinel +
  SHA-256 guard); model sourced from akhaliq/AnimeGANv2-ONNX on HF,
  upstream weights MIT-licensed (bryandlee/animegan2-pytorch © 2021
  Bryan Lee); SHA-256 1381b17ed988e14a1f3cf8954d88073c0884ff4b70128ebe599f0d6021bca63e
- hugo/static/vendor/animegan/model.onnx: face_paint_512_v2_0, 8.59 MB
- hugo/static/vendor/animegan/.animegan-vendored-version: sentinel
- hugo-apps/package.json: add onnxruntime-web ^1.20.0 (installed 1.21.0);
  add vendor:animegan script
- package.json: add vendor:animegan script; chain into build:apps after
  vendor:imgly
…AN model

MIT requires the copyright notice be included in copies. Adds:
- hugo/static/vendor/animegan/LICENSE — verbatim upstream MIT text
  (Copyright (c) 2021 Bryan Lee, from bryandlee/animegan2-pytorch)
- hugo/static/vendor/animegan/ATTRIBUTION.md — provenance note: model
  name, ONNX source URL, upstream repo, SHA-256, and statement that the
  ONNX is a mechanical serialisation so the MIT licence carries forward
@jung-thomas
jung-thomas marked this pull request as ready for review August 9, 2026 00:02
@jung-thomas
jung-thomas merged commit 650c291 into main Aug 9, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant