Skip to content

fix: add VitPose config-driven dummy inputs - #1299

Merged
ssss141414 merged 1 commit into
microsoft:mainfrom
ssss141414:ssss141414/add-nielsr-vitpose-base-simple-codegen
Aug 14, 2026
Merged

fix: add VitPose config-driven dummy inputs#1299
ssss141414 merged 1 commit into
microsoft:mainfrom
ssss141414:ssss141414/add-nielsr-vitpose-base-simple-codegen

Conversation

@ssss141414

@ssss141414 ssss141414 commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds CPU fp32/fp16 recipes for nielsr/vitpose-base-simple and fixes recipe-free VitPose input generation.

Optimum's VitPoseDummyInputGenerator inherits a static Vision generator that reloads preprocessors and indexes the result with [-1]. Config-only resolution has no loaded processor, so that path raises IndexError. A WinML VitPoseIOConfig now preserves Optimum's VitPose I/O declarations and model patcher while replacing only its dummy generator with the config-driven DummyVisionInputGenerator.

The workaround is contained in the existing VitPose model adapter. Generic export I/O behavior is unchanged.

The generic Perf range-consumption change was split into #1309. The VitPose evaluator processor fallback was split into #1310.

Baseline and outcome

Without a recipe, current-main baseline export resolves VitPoseOnnxConfig but fails while its static dummy generator loads an empty processor list:

Error: Build failed: list index out of range

With this change, recipe-free input resolution produces pixel_values [1, 3, 256, 192]. The checked-in recipes represent verified CPU fp32/fp16 coverage.

Validation

  • Final head: 5635a06aee12a2dfe3d2f0a4e7c92dc8ac00f8f1
  • Recipe-free CPU fp32 winml build: passed; pixel_values [1, 3, 256, 192] float32 -> heatmaps; final model.onnx generated
  • Checked-in CPU fp32 recipe winml build: exit 0; completed in 73.5s; final model.onnx generated
  • Checked-in CPU fp16 recipe winml build: exit 0; completed in 82.7s (Export 38.0s, Optimize 35.3s, FP16 8.8s); final model.onnx generated
  • tests/unit/export/test_io.py + tests/unit/models/test_vitpose_mapping.py: 101 passed
  • Ruff check: passed
  • Ruff format check: passed
  • GitHub checks: 9/9 passed
  • Recipe README remains unchanged

Files

  • examples/recipes/nielsr_vitpose-base-simple/cpu/cpu/keypoint-detection_fp32_config.json
  • examples/recipes/nielsr_vitpose-base-simple/cpu/cpu/keypoint-detection_fp16_config.json
  • src/winml/modelkit/models/hf/vitpose.py
  • tests/unit/models/test_vitpose_mapping.py

@ssss141414 ssss141414 added the model-scale-by-skill Model support PR created or maintained by the adding-model-support skill label Aug 10, 2026
@ssss141414

Copy link
Copy Markdown
Contributor Author

REQUEST_CHANGES

Reviewed GitHub PR #1299 at exact head 88be42b4f4f68e2e299ffc58509ed190c154511e from a fresh detached checkout. The PR remains OPEN/DRAFT with model-scale-by-skill. Original PR #1195 remains unchanged at 26b538f96a05881b62ca334517b2eb6d18e3b7fc, draft, updated 2026-07-24T02:33:40Z.

Required change

  1. Current-main baseline provenance is stale. The frozen charter and public PR body cite baseline commit 79d7baad10f28cd79e65072f78494e8be758f3df, while independently fetched origin/main and GitHub's PR base tip are ac38b61e4501ffeb9573a1d37d1bf811e4a2c1d7. git rev-list --count 79d7baad10f28cd79e65072f78494e8be758f3df..origin/main is 1. This violates reviewer.md Outcome-L0 Baseline HEAD == current origin/main (_meta-052), which requires rebase and baseline rerun even when the intervening commit is non-overlapping.

    • planner: refresh/reissue the charter on ac38b61e4501ffeb9573a1d37d1bf811e4a2c1d7; rerun the Optimum probe, winml --version, exact git rev-parse HEAD, recipe-free baseline build, and same-model/task winml config comparison. Reassess Effort/Goal only from those refreshed results.
    • producer: rebase the candidate onto current origin/main; apply only changes required by the refreshed charter and keep examples/recipes/README.md untouched.
    • tester: use a new immutable evidence root and validate the new exact head, including fp32/fp16 L0-L2, focused offline behavior, models+commands partitions, lint/mypy, artifacts, named inputs, and CPU/DML analyze retries.
    • learner: re-audit and reseal findings against the refreshed baseline/new candidate; explicitly preserve or revise vitpose-017..019 rather than carrying the old candidate identity implicitly.
    • explainer: update the public Baseline section and all derived hashes/head references from refreshed source-owned evidence; keep the PR draft/labeled and re-enumerate checks/comments/reviews/threads after the push.

Verified in this pass

  • Live identity: head exact; base tip ac38b61e...; draft and label exact; original PR recipe(vitpose-base-simple): add CPU keypoint detection configs #1195 unchanged.
  • Diff: 7 paths, two recipes plus generalized export/perf/session code and tests; no README, skill, or agent change; no source model/checkpoint hardcoding.
  • Engineering review: metadata-driven single-input vision recovery is guarded by vendor vision declarations and unambiguous shape/processor metadata; incompatible/non-vision paths retain fallback behavior.
  • Independent focused check: repaired empty-cache/offline VitPose test passed, 1 passed in 0.81s.
  • Exact-head GitHub gate: 9/9 attached checks COMPLETED/SUCCESS, including lint, models, commands, remaining, optim, analyze, CodeQL, and CLA.
  • Fresh conversation inventory: 0 issue comments, 0 reviews, 0 inline comments, 0 review threads, 0 unresolved threads.
  • Evidence: fp32/fp16 structures and sizes, realized fp16, perf/memory, L2 values, portable named-input hash, complete CPU/DML analyze JSON and exact provider-bridge errors all match the PR report.
  • Local full-partition harness was not accepted because the VS Code terminal stopped launching long or trivial subprocesses after one queued pytest run; exact-head GitHub checks and sealed command outputs were directly verified instead. This is not the routed defect and does not weaken the baseline failure above.

No PR branch, PR body, evidence root, label, draft state, or original PR #1195 content was modified by this reviewer.

@ssss141414
ssss141414 force-pushed the ssss141414/add-nielsr-vitpose-base-simple-codegen branch 2 times, most recently from 0203b48 to e6d5aa4 Compare August 13, 2026 11:14
@ssss141414

Copy link
Copy Markdown
Contributor Author

APPROVE (content-only reviewer verdict; not a GitHub Review state)

Reviewed PR: #1299
Reviewed head SHA: 63bfd4a
Base SHA: 9f74f91

  1. Fresh-checkout and diff-scope verification
  • Independent detached worktree created at exact head SHA 63bfd4a.
  • merge-base(HEAD, origin/main) = 9f74f91.
  • Diff scope (origin/main...HEAD) is 9 files:
    • examples/recipes/nielsr_vitpose-base-simple/cpu/cpu/keypoint-detection_fp16_config.json
    • examples/recipes/nielsr_vitpose-base-simple/cpu/cpu/keypoint-detection_fp32_config.json
    • src/winml/modelkit/commands/perf.py
    • src/winml/modelkit/eval/keypoint_detection_evaluator.py
    • src/winml/modelkit/export/io.py
    • src/winml/modelkit/session/session.py
    • tests/unit/commands/test_perf_cli.py
    • tests/unit/eval/test_keypoint_detection_evaluator.py
    • tests/unit/export/test_io.py
  • Production README index untouched: examples/recipes/README.md not in diff.
  1. Code and invariants review
  • No model-id hardcoding introduced in touched source.
  • Evaluator fallback remains narrow and generalized:
    • AutoImageProcessor first path retained.
    • Fallback only on known "Unrecognized image processor" ValueError and vitpose-family metadata gate.
    • trust_remote_code forwarded in both primary and fallback branches.
  • model_id guard correctness verified: explicit early ValueError when model_id is missing before any processor loader call.
  • Regression test presence verified, including missing-model-id and non-vitpose no-regression cases.
  1. Planner/producer/tester/learner/explainer consistency and provenance
  • Charter revision 3 confirms:
    • baseline main_commit = 9f74f91,
    • baseline refresh decision = PARTIAL-RERUN,
    • optimum probe verdict = WINML-ONLY for vitpose keypoint-detection.
  • Producer deliverable revision 4 and tester verdict revision 4 are aligned on final head 63bfd4a and PARTIAL-RERUN invalidation logic.
  • Learner revision 3 remains a d5fa845-stage artifact as declared by explainer inputs; rev4 body explicitly documents final-head delta and rerun evidence.
  • PR body at live GitHub head 63bfd4a is updated, coherent, and labels functional smoke correctly as operability-only.
  1. Independent execution evidence
  • Mandatory final-SHA FP32 CPU functional smoke rerun executed independently (both modes):
    • without --trust-remote-code: processed num_images=1, map=0.8, exit 0
    • with --trust-remote-code: processed num_images=1, map=0.8, exit 0
  • Artifact precision realization independently verified from binary artifacts:
    • fp32 model.onnx sha256 = 0040b80cfb62adb2a78892cc4a8fdea39d68e92d95bc989d031971e5eae1ed72
    • fp16 model.onnx sha256 = e3cf530ab2ed8aa738f72026dbb6bb776085a39fe31c4e5c19ad23e470ae3f43
    • model.onnx.data sizes: fp32=343649280, fp16=171824640
    • fp16 FLOAT16 initializers: 201
  • Independent affected partition rerun (eval/export slice) PASS:
    • 729 passed, 1 warning.
  1. Local gate notes and pre-existing baselines
  • Local ruff check (src/ tests/) shows 5 existing RUF036 diagnostics; same 5 reproduced on clean base worktree at 9f74f91 (pre-existing, not introduced by this PR).
  • Local full models partition shows one existing failure:
    • tests/unit/models/wav2vec2/test_onnx_config.py::TestWav2Vec2EmotionModelClassMapping::test_model_initialization_sets_transformers_bookkeeping
    • same failure reproduced on clean base 9f74f91 (pre-existing, not introduced by this PR).
  1. GitHub final-sha gates and threads
  • Live PR metadata at review time:
    • headRefOid = 63bfd4a
    • Draft = true
    • label model-scale-by-skill = present
  • GraphQL reviewThreads enumeration on current head:
    • total threads = 0
    • unresolved threads = 0
  • gh pr checks on exact head:
    • 9 successful, 0 failing, 0 pending, 0 cancelled, 0 skipped.
  1. Coverage and verdict
  • Required tuples: CPUExecutionProvider/cpu/fp32 and CPUExecutionProvider/cpu/fp16.
  • Evidence status: full coverage; no deferred tuples.
  • Mandatory final-SHA FP32 CPU smoke: PASS (processed=1, map=0.8), clearly labeled functional-only.

Final reviewer verdict: APPROVE for head 63bfd4a.

@ssss141414

Copy link
Copy Markdown
Contributor Author

Scope update: this branch was narrowed from combined head \63bfd4ad\ to \8d3d74d3. It now contains only the VitPose CPU recipes and recipe-free Vision I/O recovery. Generic Perf input-range consumption moved to #1309; the VitPose evaluator processor fallback moved to #1310. The earlier skill reviewer APPROVE comment evaluated the combined head and should not be treated as review of this narrowed head. Focused validation on the narrowed slice: 103 export I/O tests passed, plus Ruff check and format check.

@ssss141414
ssss141414 force-pushed the ssss141414/add-nielsr-vitpose-base-simple-codegen branch from 8d3d74d to 5635a06 Compare August 14, 2026 04:42
@ssss141414 ssss141414 changed the title fix: recover declared vision model inputs fix: add VitPose config-driven dummy inputs Aug 14, 2026
@ssss141414

Copy link
Copy Markdown
Contributor Author

Implementation update: the generic \export/io.py\ recovery has been removed entirely. The branch now fixes the Optimum processor-list \IndexError\ in the existing VitPose model adapter by registering a \VitPoseIOConfig\ that preserves Optimum's declarations/patcher and replaces only its static dummy generator. Generic I/O files are byte-for-byte unchanged from \main. Validation: 101 focused tests passed; Ruff check and format check passed.

@ssss141414

Copy link
Copy Markdown
Contributor Author

Final-head end-to-end validation on \5635a06a: recipe-free CPU fp32 \winml build\ passed and generated \model.onnx\ with \pixel_values [1,3,256,192] float32 -> heatmaps; checked-in fp32 recipe build exited 0 in 73.5s; checked-in fp16 recipe build exited 0 in 82.7s and completed the FP16 stage. Both recipe builds generated final \model.onnx\ artifacts. GitHub checks are 9/9 successful.

@ssss141414
ssss141414 marked this pull request as ready for review August 14, 2026 06:41
@ssss141414
ssss141414 requested a review from a team as a code owner August 14, 2026 06:41
@ssss141414
ssss141414 merged commit 24cccfb into microsoft:main Aug 14, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

model-scale-by-skill Model support PR created or maintained by the adding-model-support skill

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants