Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 28 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,25 +2,28 @@

Visual screenshot testing is an excellent control surface for directing coding agents. Using renders inside the coding loop can increase render counts by 100x to 1000x, which is wildly unaffordable with existing centralized render services such as Chromatic.

Selfie is a visual review gate built for that loop. It runs on your machine beside your agent, in CI, and on GitHub, and it's piloting with StorybookJS. PRs to support other testing models are welcome.
Selfie is a visual review gate built for that loop. It runs on your machine beside your agent, in CI, and on GitHub, and when you want to know why something changed, you can ask Muse Spark. It's piloting with StorybookJS, and PRs to support other testing models are welcome.

Go to [selfie.dev](https://selfie.dev) to follow development after the hackathon.

## Watch it

| [Why Selfie](https://youtu.be/8B3-v4DTwfU) | [Product demo](https://youtu.be/FiGuwfB13b0) |
| [Why Selfie](https://youtu.be/8B3-v4DTwfU) | [Product demo](https://youtu.be/ZrUv0Pn7yyc) |
| --- | --- |
| [![Watch: Why Selfie](https://img.youtube.com/vi/8B3-v4DTwfU/maxresdefault.jpg)](https://youtu.be/8B3-v4DTwfU) | [![Watch: Selfie product demo](https://img.youtube.com/vi/FiGuwfB13b0/maxresdefault.jpg)](https://youtu.be/FiGuwfB13b0) |
| From Storybook to Chromatic to Selfie: why snapshot testing needs to change for coding agents. | The real app: a two-line CSS change touches all 124 stories, and a person and an agent work through it together. |
| <a href="https://youtu.be/8B3-v4DTwfU"><img src="https://img.youtube.com/vi/8B3-v4DTwfU/maxresdefault.jpg" alt="Watch: Why Selfie" width="380" /></a> | <a href="https://youtu.be/ZrUv0Pn7yyc"><img src="https://img.youtube.com/vi/ZrUv0Pn7yyc/maxresdefault.jpg" alt="Watch: Selfie product demo" width="380" /></a> |
| From Storybook to Chromatic to Selfie: why snapshot testing has to change for coding agents. | The real app: two lines of CSS touch all 142 stories, and Muse Spark finds what broke dark mode. |

You can also look around the hosted review pages without signing in: here's [Selfie reviewing itself](https://hackathon.selfie.dev/diffplug/selfie-hackathon).

## What it does

- **Fits an agent's loop.** `selfie review` prints one JSON document naming the diff images, with an exit code that says whether anything blocks.
- **Runs where the agent runs.** On an M4 MacBook Air, Selfie's own 124 stories render in about 25 seconds, with no cloud and no bill per image. A larger Storybook of 541 stories takes about 70 seconds. Reviewing a build that's already rendered takes under half a second.
- **Says what kind of change it is.** Selfie separates what moved or gained spacing from what really changed, so you can approve the harmless ones together.
- **Runs where the agent runs.** On an M4 MacBook Air, Selfie's own Storybook of about 140 stories renders in under 30 seconds, with no cloud and no bill per image. A larger Storybook of 541 stories takes about 70 seconds. Reviewing a build that's already rendered takes under half a second.
- **Says what kind of change it is.** Selfie separates what moved or gained spacing from what really changed, and finds one change repeated across stories, so you can accept it once.
- **Tells you why, with Muse Spark.** On a changed story in the cloud, ask what caused it. Muse Spark reads the before, after, and diff images and the code from GitHub, and it can point out what nobody asked for.
- **Lets you point at the problem.** Write a note on an image, and your agent gets your words, the exact box, and a picture of what you saw. It answers on the same note, and you flip between before and after.
- **Follows Git.** An approval counts wherever history reaches it, so it isn't repeated after a commit, a rebase, or a merge.
- **Gives the same pixels everywhere.** Stories render in a pinned container, and `selfie flakes` finds any that still vary between renders.
- **Keeps the pixels reproducible.** Stories render in a container pinned by digest, and `selfie flakes` finds any that still vary between renders.
- **Reports on every commit.** A GitHub Action renders each push, and the commit gets a `selfie` status that links to its review.

## Quickstart
Expand All @@ -47,7 +50,7 @@ You'll need Node 26 or later, pnpm, and Docker. Selfie isn't on npm yet, which i

**To work together,** a person leaves notes on any story in `selfie ui`, and either copies them into any agent, or presses Send while the agent runs `selfie agent-await-feedback`. The agent acts on them, runs `selfie render`, and answers each with `selfie reply`. The person flips against the image they commented on, and resolves the note or answers back.

**In the cloud,** a GitHub Action renders every pushed branch and uploads its batch. Each commit gets a `selfie` status: red if a story failed or a person rejected a change, yellow if a change is undecided, and green if everything matches what's approved.
**In the cloud,** a GitHub Action renders every pushed branch and uploads its batch. Each commit gets a `selfie` status: red if a story failed or a person rejected a change, yellow if a change is undecided, and green if everything matches what's approved. On a changed story's page, anyone can ask Muse Spark about the change.

A project needs a `selfie.config.json` at its Git top level, which says where its built Storybook is and how to build it, and may also set its environment, init script, and how close changed hunks join.

Expand All @@ -65,6 +68,7 @@ From a checkout, `pnpm selfie <command>` runs these.
| `notes`, `agent-await-feedback`, `reply` | How a person's notes on the images reach a coding agent, and how it answers |
| `push`, `pull` | Sends a batch and its decisions to the cloud, and takes what a commit's history approved from it |
| `flakes` | Renders a built Storybook several times, and says what wasn't the same each time. `selfie skill fix-flakes` is a guide for a coding agent to fix what it finds |
| `skill` | Prints a guide for a coding agent: `quickstart` to set Selfie up in a repository, and `fix-flakes` to make every story render the same way |

## What's built, and what isn't

Expand All @@ -86,9 +90,23 @@ We previously launched [Selfie](https://lib.selfie.dev), a snapshot testing libr

### Meta models

Most of the event went on the foundation: rendering, storage, review, and sync. **Selfie calls a Meta model in one place so far.** On a changed story's page in the cloud service, anyone can ask Muse Spark about the change. It reads the before, after, and diff images, and the commits and code diff from GitHub between what was approved and this commit ([`docs/specs/chat.md`](docs/specs/chat.md)).
Most of the event went on the foundation: rendering, storage, review, and sync. **Selfie calls a Meta model in one place so far: asking Muse Spark about a change.**

Selfie's own code can say exactly which pixels differ and whether content moved, but not why, or whether anyone wanted it. So on a changed story's page in the cloud service, anyone can ask. For each question, `muse-spark-1.3` reads:

- the image last approved, the new image, and their diff
- the measurements of their differences
- the commits and code diff from GitHub, from what was approved to this commit

In the [product demo](https://youtu.be/ZrUv0Pn7yyc), we asked it the page's own starter question, "What code caused this change?", about a commit whose request was only for more room in a top bar. Muse Spark found the CSS, and then raised the bug by itself:

> What was not asked for in that message is the color: in this `--dark` story the old bar is dark and the new bar is white/light. The `background: #f6f8fa` line is fixed light instead of the theme variable, so check dark mode before accepting.

We wrote that bug on purpose for the demo, and that answer is real and unedited.

It's built to be safe to leave open on a public page. Each answer says which code it read and what it left out. The model has no tools, and its answer is plain text, so an instruction hidden in a branch's code has nothing to act with. An answer can never approve, reject, or block anything. See [`docs/specs/chat.md`](docs/specs/chat.md).

Selfie's own code can say exactly which pixels differ and whether content moved, but not why, or whether anyone wanted it. That matters at agent scale: one style tweak we measured changed 290 of a Storybook's 541 stories, which is why the next step, specified in [`docs/specs/system.md`](docs/specs/system.md) as `change-grouping`, is to group changed stories by the code change that caused them, and flag the changes nobody asked for.
That matters at agent scale: one style tweak we measured changed 290 of a Storybook's 541 stories. The next step, specified in [`docs/specs/system.md`](docs/specs/system.md) as `change-grouping`, is to ask about a whole batch, grouping changed stories by the code change that caused them, and flagging the changes nobody asked for.

## Architecture

Expand Down
2 changes: 1 addition & 1 deletion product-demo/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ A [HyperFrames](https://hyperframes.heygen.com) film of the real product, review
|---|---|
| `SCRIPT.md` | The words. One paragraph is one scene, and the bold line under it is what's on screen. |
| `scenes.json` | The scenes, in the same order, and how long the last one holds after its last word. |
| `assets/narration.mp3`, `assets/narration-timing.json` | The voiceover, and when each word of it happens, from ElevenLabs. |
| `assets/narration.mp3`, `assets/narration-timing.json` | The voiceover, and when each word of it happens, from ElevenLabs. Without them, timing is estimated. |
| `compositions/<id>.html` | Each scene. Beats are written as `w("word")`, so they follow the voice. |
| `assets/ui/` | Screenshots of the real app, 2x, each one a state it was really in. `points.json` is where the pointer was. |
| `assets/lib.js`, `assets/style.css` | The app window, its pointer and camera, and the terminal. |
Expand Down
5 changes: 3 additions & 2 deletions product-demo/REVIEW.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,10 @@

For whoever changes this film next, including an agent.

- **Real:** every screen is a screenshot of the real app in a state it really reached, running on an M4 MacBook Air against this repository's Storybook, in a throwaway clone: `selfie review`, the 124 changed, the 48 spacing-only, 85,143 pixels down to 13, the note, the reply, and the final review. Every terminal line is from those runs. GitHub and `hackathon.selfie.dev` are the live public pages, read only, for the real `main` commit, which isn't the commit in the story.
- **Real:** every screen is a screenshot of the real app in a state it really reached, on an M4 MacBook Air against this repository's Storybook, in a throwaway clone. That covers the 142 changed, the two repeated changes accepted (21 and 42 images, so 63 of 142), 85,143 pixels down to 13, the note, the reply, and the final review. Every terminal line is from those runs.
- **Real, in the cloud:** the buggy commit was pushed to a throwaway branch of this repository, so GitHub's status and `hackathon.selfie.dev` are the live pages for it. Muse Spark's answer is its real, unedited answer to the page's own starter question, "What code caused this change?". It raised the color and dark mode by itself.
- **Staged:** the bug. The agent's change was written to hard-code a light color, so that dark mode would break. The request, the fix, and the reply were written for the film too, though each really ran.
- **Added on top:** the pointer, the camera moves, and the callouts. The render log is sped up, and ends on what it really took. JSON is trimmed, with `…` where it is. The note's typing is uncovered from its finished screenshot.
- **Added on top:** the pointer, the camera moves, the callouts, and the highlights on the answer. The render log and Muse Spark's answer are sped up, and the note's typing is uncovered from its finished screenshot. Terminal output is trimmed, with `…` where it is.
- **Timing:** scenes, subtitles, and beats come from the word times ElevenLabs gave for this take.
- **Not checked:** the sound was measured, not heard.
- **Kept warnings:** `timeline_track_too_dense`, one clip per subtitle; `dead_stretch`, judged against freeze detection on the rendered file.
54 changes: 22 additions & 32 deletions product-demo/SCRIPT.md
Original file line number Diff line number Diff line change
@@ -1,55 +1,45 @@
## THE ASK

I asked my coding agent for one small thing: give the top bar a bit more room. It changed two lines of CSS. Looks fine, right?
**[Terminal: the request, and the real two-line diff]**
I asked my coding agent to give the top bar a bit more room. Two lines of CSS. Then it runs selfie render.
**[Terminal: the request, the real two-line diff, and the command]**

So the agent runs selfie review. That renders every story in our Storybook, 124 of them, in a pinned container, right here on this MacBook Air. Twenty-five seconds, with no cloud and no bill per image.
**[Terminal: the real render log, ending on its real timings]**

What comes back is one JSON document and an exit code, so any agent can read it. Exit 1, because that two-line change touched all 124 stories.
**[Terminal: the real JSON, trimmed, and the exit code]**
That renders all 142 stories in a pinned container, right here on this MacBook Air, in under 30 seconds, with no cloud and no bill per image.
**[Terminal: the real render log and its timings]**

## THE REVIEW

Now it’s my turn. The selfie ui command opens the same review for a person: 124 changed, and nothing decided yet.
**[Selfie UI: the batch, its counts]**
Now it’s my turn. The selfie ui command shows me what changed: those two lines touched every story. But Selfie has already found the changes repeated across them. I look through them, accept them, and 63 of the 142 are done.
**[Selfie UI: the batch and its Repeated changes. Look through the first, and accept the two biggest]**

That’s a lot to look at, but Selfie works out what kind of change each one is. 48 of them only moved or gained spacing, with nothing new in the image.
**[Show filter: Moved and spacing only, 48 of 124]**
But what about dark mode? I search for dark, and there it is: 85,000 changed pixels. Press F, and the images swap in place. The top bar went white.
**[Search for dark. The dark story, its numbers, and a flip between golden and current]**

Here’s one. Yellow is spacing that grew. Press F, and the two images flip in place, so you can see exactly what moved.
**[A spacing-only story. Stack it, and flip between golden and current]**
## ASKING MUSE SPARK

That looks right, so I approve those 48 together.
**[The bulk dialog, listing what it will approve]**
Selfie can tell me exactly which pixels changed, but not why. The commit is on GitHub, so I open the same story in the cloud, and ask Muse Spark what caused it.
**[The real GitHub status, the hosted story, and the question being asked]**

But what about dark mode? I search for dark, and there it is: 85,000 changed pixels. The agent hard-coded a light color, and the top bar went white.
**[Search for dark. The dark story, its numbers, and the white bar]**
It reads the before, the after, the diff, and the code, and it finds the line. Then it spots what I never asked for: a fixed light color, where the theme’s color used to be. It even tells me to check dark mode.
**[Muse Spark's real answer arriving, with the code it says it read]**

## THE LOOP

I don’t have to describe where. I click the change, write what’s wrong, and send it to the agent, which has been waiting.
So back in my review, I click the change, write a note, and send it to the agent, which has been waiting.
**[Click the hunk, type the note, press Send to agent]**

The agent gets my words, the story, the exact box, and a picture of what I saw.
**[Terminal: the real note it received, beside the real image with the box drawn on it]**
It gets my words, the exact box, and a picture of what I saw.
**[Terminal: the real note it received, beside the image with the box on it]**

It puts the theme’s color back, renders again, and answers my note.
**[Terminal: the real fix, render, and reply]**

Selfie tells me there’s a newer batch, and the reply is right there on the story. The bar is dark again, and 85,000 changed pixels are down to 13.
**[The newer batch banner, the reply, the white and fixed images, and the new number]**

I resolve the note, and approve the rest. The 48 I already approved are simply unchanged now, because an approval follows your Git history.
**[Resolve, Approve all, and the batch with nothing blocking and 48 unchanged]**

The agent commits and reviews once more. It’s the exact same build, so nothing renders. A third of a second, and exit 0.
**[Terminal: the real commit and final review, reusing the images]**
The bar is dark again, and 85,000 changed pixels are down to 13.
**[The reply on the story, the white and fixed images, and the new number]**

## THE CLOUD
I approve the rest, and the agent commits and reviews once more. It’s the exact same build, so nothing renders. Under half a second, and exit 0.
**[Approve all, then the terminal: the real commit and final review, reusing the images]**

Push it, and Selfie renders in CI and puts a status on the commit. Click through, and it’s the same review, in the cloud.
**[The real GitHub commit list, its selfie status, and the hosted review]**
## THE END

Selfie works with any coding agent. If you have a Storybook, start locally today at selfie.dev, and join the waitlist for the cloud.
Selfie works with any coding agent. If you have a Storybook, you can start locally today, and the link’s in the description.
**[End card]**
Loading
Loading