Bug regression comparison for Teable: give it a list of teable-ee commits, it runs the same bug cases against every one of them and answers, in a single table, "is this bug still there on this revision" — so you can see at a glance which two revisions a fix landed between, and whether anything that was fixed has come back.
| case | issue | status | `3f2a1c` | `8b4e77` | `a1c9f0` | transition |
| record/t1481-... | T1481 | fixed | ❌ | ❌ | ✅ | fixed 8b4e77..a1c9f0 |
| record/t1502-... | T1502 | open | ⬜ | ⬜ | ⬜ | |
| field/t1520-... | T1520 | fixed | ✅ | ✅ | ❌ | regressed 8b4e77..a1c9f0 |
The execution skeleton is teable-perf-lab's, proven in production there:
- Injection: this repository's
cases/,framework/,registry.ts, and spec are copied into ateableio/teable-eecheckout and run through teable-ee's own e2e harness (vitest,initApp(), seeded auth). Cases live here and evolve independently; historical commits of the product need no knowledge of them. - Pinning: refs are resolved to SHAs once, up front; every job checks out the pinned SHA.
- One job per commit: isolated database built from that commit's own migrations, every selected case run once per engine, one JSON payload per case per engine written before any assertion throws — the payloads are the source of truth, and failures carry the server's own error body.
- Fail-closed report: every planned (case × commit) cell must have exactly one payload. Missing evidence fails the run; it never renders as an empty cell someone might read as green.
Two engines, one of them guarded. v2 is where fixes land, so that is the
column the run is judged on. v1 is asked the same cases as a reference — what
does the engine our older customers are still on do with this — and nothing it
reports fails a run; it renders as its own table. Reaching v1 needs more than
an environment switch, and a case whose feature v1 does not have declares
skipV1 rather than failing there every run. Both are explained in
docs/operations/e2e-lab.md.
What is this repository's own: the verdict model. Each case declares the bug
it reproduces and its believed status (open / fixed); the run observes
(absent / present / error) and the comparison judges. Known-unfixed bugs
failing is expected and green; a fixed bug reproducing on the gating column
(the newest commit) is a regression and red; the same observation on an older
column is just the world before the fix. A case that errors is red everywhere,
because it observed nothing. Details: framework/verdict.ts
and docs/operations/e2e-lab.md.
gh workflow run e2e-lab.yml \
--repo teableio/teable-e2e-lab \
--ref main \
-f teable_ee_commits="<sha-before-fix>,<sha-after-fix>,develop" \
-f case_filter=allCommits go oldest-first; the last one is the gating column. The table lands in
the report job's GitHub summary; comparison.json in the
e2e-lab-comparison-* artifact. Local direction-finding runs follow the localrun skill, which is not published
with this repository; ask the team for it.
Read .agents/README.md — the short version: one
declarative cases/<group>/<name>.case.ts (data only, typed against its
runner), one same-name .md documenting the bug and the checkpoint, one
registry entry. Execution logic lives in framework/runners/. pnpm check
validates everything that can be validated without a teable-ee checkout,
including a full type check against the runner/config contracts.
Security-sensitive bugs are not accepted until their fix has shipped — see CONTRIBUTING.md.
cases/ declarative bug cases + per-case docs
framework/ types, verdict model, checkpoint seam, runners, artifacts
registry.ts explicit case registration
e2e-lab.e2e-spec.ts the single injected entry point
scripts/ planner, comparison table, acceptance gate, checks
.github/ e2e-lab.yml (the run), check.yml (secrets-free PR checks)
docs/ operations guide, cross-repo dispatch status