Skip to content

#1199 stage 17: quality tiers on the long-form templates #1218

Closed

Description

@dkackman

Stage of #1199, filed by model claude-opus-5-5 via provider anthropic. Already
built on feat/long-form-media (merged 3f8ed222). The authoritative spec is the
plan (docs/superpowers/plans/2026-11-03-long-form-media.md, task named below) and
the design record. If they differ, the design record says what shipped.

What shipped: a tier preset variable (draft / standard / best, default
standard) on minimax/chained-segments, chain-video-continuity,
chain-matched-to-audio, chain-matched-and-aligned, dialogue-short and
music-video. A bundle moves only width, height, num_inference_steps and
weights_dtype, and shares no variable with the shot presets. tier is in each
template's cost_drivers. standard is today's defaults, so no cost row moved and no
default changed (no breaking-change).

draft standard best
FL2VA chain 4 steps (0.66-0.72x wall) 960x544, 5 steps, int4 1344x768, int8 (1.68x)
Ref2VA (5 templates) 5 steps (0.56x) 960x544, 9 steps, int4 1344x768, int8 (1.32x)

Bench evidence (lem bench :8766 / :8767, bench/records/tiers.md): the template
end to end with shot: static_long at 120 s, aa80458ab4f7, 59.2 min. S1 standard
f5bb2add1911 14.1 min, 4 steps cc6e23eb555a 9.3 min, 1344x768 int8 6810b23c2e35
23.7 min. Draft keeps standard's staging at the same seed (segment-1 PSNR 32 dB on
S1). best needs about 65 GB of host RAM per job (one H3 job at a time on lem).

Observe over MCP:

  • validate_workflow on templates/minimax/chained-segments (and dialogue-short) with image, prompt, segments 2 and tier each of draft, standard, best, and with no tier: compare plan.estimate and the reported cost_drivers.
  • run_workflow with tier: draft (the cheap one), then get_job_workflow shows the realized width, height, num_inference_steps and weights_dtype.
  • A tier: best run needs about 65 GB of host RAM. Run it alone on lem, never beside another H3 job (bug: an int8 H3 job needs about 65 GB of host RAM, and two servers on one host OOM-kill each other #1227).

Acceptance intent:

  • tier: standard (or no tier) realizes exactly today's template values
    (get_job_workflow).
  • tier: draft and tier: best realize the bundle values in the table; a variable
    passed beside tier wins.
  • An unknown tier is refused at validate, naming the three values.
  • The estimate differs by tier (validate's plan.estimate), and tier is listed in
    cost_drivers.
  • No tier bundle sets a shot variable.

Owes: nothing. Deploy: done at 3f8ed222.

verify: run every pending: #1218 case.

Activity

  1. dkackman commented on Nov 7, 2026

    @dkackman
    OwnerAuthor

    Verified by tester model claude-opus-5-5 via provider anthropic, on lem develop @ 85bcd33, workspace default.

    No suite case carries pending: #1218 (grep across regression-suite-*.md found none), so acceptance is this issue's own intent list.

    Ran:

    • validate_workflow templates/minimax/chained-segments, segments: 2, with tier draft / standard / best / none. All valid. Estimates: 7.2 / 5.5 / 17.8 / 5.5 min. Non-standard tiers report differs: ["tier"] (best adds weights_dtype), and standard and no-tier share one fingerprint.
    • Same for templates/minimax/dialogue-short: draft / best / none give 11.8 / 41.9 / 38.6 min.
    • tier: "ultra" on all six templates (chained-segments, chain-video-continuity, chain-matched-to-audio, chain-matched-and-aligned, dialogue-short, music-video) is refused with arguments.tier: 'ultra' is not a 'tier' value; allowed: best, draft, standard.
    • Passing a variable beside the tier changes the plan, so the passed variable wins. draft + num_inference_steps: 6 is scaled as "steps 6 vs 5". best + width: 960, height: 544 drops the canvas term and keeps the weights_dtype difference.
    • run_workflow chained-segments {segments: 2, tier: draft}, job dc4c32402665, succeeded in about 3 min. get_job_workflow (realized) shows width 960, height 544, num_inference_steps 4, weights_dtype {int4} and shot: moving. presets.tier has draft {960x544, 4, int4}, standard {960x544, 5, int4} and best {1344x768, 5, int8}. No tier bundle sets any shot variable, and tier is first in cost_drivers. Output deleted.
    • I did not run best (65 GB host RAM, bug: an int8 H3 job needs about 65 GB of host RAM, and two servers on one host OOM-kill each other #1227). I checked its bundle values through the realized presets and validate's plan instead.

    One finding, filed separately as #1236: the tier estimates do differ, which is what the acceptance list asks. But they are scaled from the curated cost row while standard uses the observed figure. So chained-segments draft is quoted at 7.2 min, above standard's 5.5 (the real draft run took about 3 min). I filed it on its own because it comes from how the estimator chooses its basis, not from the tier bundles.

    Closing as verified.

  2. added
    status:verifiedTester confirmed the fix via a real MCP call
    and removed on Nov 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    owner:testerTester's turn to actstageA child issue: one stage of a featurestatus:verifiedTester confirmed the fix via a real MCP calltarget:lem

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions