recipe(multilingual-e5-small): add CPU fp32/fp16 feature-extraction and sentence-similarity recipes

[Qiong Wu (qiowu) (DingmaomaoBJTU)]

Summary

Adds complete CPU fp32/fp16 support for intfloat/multilingual-e5-small across feature extraction and sentence similarity. The Effort L2 contribution ships four task/precision recipes plus a metadata-driven shared input fix and regression coverage; Outcome L2 reached the committed Goal L3 with full coverage on candidate 8e7c621a2312a3ef45b08c43e7347301e992f842. The bounded L3 result is functional-smoke evidence only, not representative benchmark accuracy.

Model metadata

What the model does

intfloat/multilingual-e5-small is a multilingual text-embedding checkpoint. It encodes prefixed text with a 12-layer BertModel into token-level hidden states; the published SentenceTransformers pipeline applies attention-mask-aware mean pooling and L2 normalization to obtain 384-dimensional embeddings for retrieval, semantic similarity, clustering, and related feature use.

• Evidence: Hugging Face model card and metadata, including pinned modules.json, 1_Pooling/config.json, and model config; current-main winml inspect for feature extraction and sentence similarity.
• Confidence: verified.

Primary user stories

• A user supplies a prefixed query and candidate passages to obtain normalized embeddings for multilingual semantic retrieval. Evidence: pinned model-card usage and FAQ for query:/passage: asymmetric retrieval prefixes. Confidence: verified.
• A user supplies two query-prefixed sentences to obtain cosine-comparable embeddings for semantic similarity or paraphrase matching. Evidence: pinned model-card FAQ for the query: prefix on symmetric semantic-similarity tasks. Confidence: verified.
• A user supplies query-prefixed text to obtain reusable dense features for clustering or linear-probe classification. Evidence: pinned model-card FAQ for the query: prefix on embedding-as-feature use. Confidence: verified.

Supported tasks

• Feature extraction: supported through Transformers, Optimum ONNX, and WinML. The Optimum BERT vendor registry contains feature-extraction, and current-main winml inspect resolves AutoModel and BertIOConfig. Confidence: verified.
• Sentence similarity: declared by the pinned checkpoint (pipeline_tag=sentence-similarity, library_name=sentence-transformers) and supported by WinML while using the feature-extraction export contract. Confidence: verified.

Model architecture

BertModel
├── Embeddings (vocab 250037 × 384; positions 512 × 384; token types 2 × 384)
│   └── LayerNorm + dropout
├── Encoder stack × 12
│   ├── Multi-head self-attention (12 heads; head width 32)
│   ├── Feed-forward (384 → 1536 → 384, GELU)
│   └── Residual + LayerNorm
└── Raw export output: last_hidden_state [batch, sequence, 384]
SentenceTransformers postprocessing (outside BertModel/ONNX)
├── Attention-mask-aware mean pooling
└── L2 normalization → sentence embedding [batch, 384]

• Source/confidence: pinned model config (architectures=[BertModel]), Transformers 5.14.1 BertModel source, and pinned SentenceTransformers module/pooling configs (verified).

Validation and support evidence

Baseline

Baseline was fully rerun at current main e564a6375d6cd2b596fb3d21d918f07824b349e0 with WinML 0.2.0.

• Auto-config and export classification: current-main config resolves both tasks to AutoModel/BertIOConfig; sentence similarity deliberately uses the feature-extraction ONNX graph contract. winml config produced task-specific anchors and has no precision selector. The repaired final recipes are structurally identical to the corresponding frozen current-main fp32/fp16 auto-config anchors. This is not a new task-family registry.
• Optimum classification: VENDOR-ONLY. Optimum's BERT registry already supplies feature-extraction; WinML adds no vendor task, and sentence similarity reuses the feature-extraction export contract.
• Build: PASS in 67.6s (31.0s export, 31.1s optimize), quant=null, 448.4 MB float model. ONNX checker passed: IR 8, opset 17, 372 nodes, three int32 [1,512] inputs, and float32 [1,512,384] last_hidden_state. The historical default-quant failure is not current.
• Perf: mean 96.87 ms, p50 94.99 ms, p90 108.50 ms, p95 111.20 ms, p99 115.63 ms, 10.32 samples/s, total RSS delta +81.4 MB.
• Numeric parity: raw last_hidden_state max absolute error 1.996755599975586e-06 (1.997e-6), mean absolute error 2.1955554529995425e-07, cosine 0.9999999403953552; pooled normalized embedding cosine minimum 0.9999999403953552. PyTorch and ONNX retrieval rankings were identical for the pinned bilingual examples.
• Former Eval blocker: the bounded 2-source-row/4-pair sentence-similarity Eval loaded its data but failed before emitting a metric because the tokenizer omitted token_type_ids while the ONNX graph required it.

Goal

• Effort: L2.
• Goal ceiling: L3.
• Outcome: L2.
• Success definition: L0 four clean builds; L1 four CPU perf rows; L2 per-artifact PyTorch parity; one bounded final-SHA fp32 CPU functional Eval.

The final Goal ladder passed at every tier: L0 4/4, L1 4/4, L2 4/4, and L3 PASS. The ceiling was not downgraded.

Outcome

Outcome L2 shipped at final candidate 8e7c621a2312a3ef45b08c43e7347301e992f842; highest Goal verdict is L3 PASS, coverage is full, and there are no deferred tuples or blockers.

Shipped paths:

• examples/recipes/intfloat_multilingual-e5-small/cpu/cpu/feature-extraction_fp32_config.json
• examples/recipes/intfloat_multilingual-e5-small/cpu/cpu/feature-extraction_fp16_config.json
• examples/recipes/intfloat_multilingual-e5-small/cpu/cpu/sentence-similarity_fp32_config.json
• examples/recipes/intfloat_multilingual-e5-small/cpu/cpu/sentence-similarity_fp16_config.json
• src/winml/modelkit/models/winml/feature_extraction.py
• tests/unit/models/auto/test_feature_extraction.py

Model findings bert-011 through bert-016 were appended on the separate Lane A Draft ModelKitArtifacts PR #216, final head 8c67d9cb69ac1ca464beabe82bac1771c4edd912, labeled model-scale-by-skill. Methodology declaration: no_friction:false; reviewer-discovered doc-code drift captured as _meta-108 with paired tester/reviewer contract edits in pushed Lane A PR #216. _meta-108 records only that schema-v2 perf precision supersedes the stale literal console gate.

Per-EP/device/precision results and Functional smoke Eval

Goal ladder and CPU perf

Task	EP / device	Precision	L0 build	Build time	L1	Mean	p50	Throughput	RAM delta
feature-extraction	CPUExecutionProvider / cpu	fp32	PASS	70.2s	PASS	116.4 ms	115.92 ms	8.59 samples/s	+81.4 MB
feature-extraction	CPUExecutionProvider / cpu	fp16	PASS	72.8s	PASS	179.643 ms	178.686 ms	5.57 samples/s	+88.73 MB
sentence-similarity	CPUExecutionProvider / cpu	fp32	PASS	65.7s	PASS	118.74 ms	118.04 ms	8.42 samples/s	+81.3 MB
sentence-similarity	CPUExecutionProvider / cpu	fp16	PASS	70.7s	PASS	180.017 ms	179.447 ms	5.56 samples/s	+88.71 MB

Both fp16 builds used --precision fp16 without --no-quant. Current winml perf --format json emitted schema-v2 authoritative resolved model_info.precision="fp16" and corroborating requested-policy benchmark_info.precision="fp16" for both fp16 artifacts. Each contains 200 FLOAT16 initializers and 234,931,456 external-data bytes, versus 200 FLOAT initializers and 470,027,776 bytes for fp32, a ratio of 0.4998246231; float I/O is preserved.

L2 numeric and retrieval parity

Pinned PyTorch checkpoint revision: 614241f622f53c4eeff9890bdc4f31cfecc418b3.

Artifact	Max abs error	Mean abs error	Minimum embedding cosine	Retrieval ranking
feature-extraction fp32	1.7583370208740234e-06	2.223688824187775e-07	0.9999999403953552	identical
feature-extraction fp16	0.003210783004760742	0.0002058723330264911	0.9999998211860657	identical
sentence-similarity fp32	1.7583370208740234e-06	2.223688824187775e-07	0.9999999403953552	identical
sentence-similarity fp16	0.003210783004760742	0.0002058723330264911	0.9999998211860657	identical

Functional smoke Eval

Functional smoke only; this is not representative benchmark accuracy or model-quality evidence. One final-SHA FP32 CPU sentence-similarity run passed on candidate 8e7c621a2312a3ef45b08c43e7347301e992f842.

• Dataset: tester-owned, E5 model-card-derived multilingual smoke at source checkpoint revision 614241f622f53c4eeff9890bdc4f31cfecc418b3; split train; deterministic no-shuffle selection with seed 0.
• Scope: 2 bilingual source rows × 2 passages = 4 requested and 4 processed pairs; 8 wrapper inference calls.
• Fan-out caps: 2 source rows; 2 candidate passages per query; 4 expanded pairs; 2 inferences per pair; sequence length 512; beams 1; frames/crops not applicable.
• Schema fields sentence1, sentence2, score, source_query_row, source_passage_row, and relation were verified. Labels were 1.0 for same-source matches and 0.0 for cross-source negatives; predictions were attention-mask-aware mean-pooled embedding cosine similarities. Schema, label semantics, and prediction semantics all passed.
• The tokenizer omitted token_type_ids; all 8 wrapper feeds synthesized all-zero int32 [1,512] values. The former missing-input blocker was therefore removed through the shared metadata-driven input capability.
• Metric: cosine_spearman=89.4427 on the MTEB [-100, 100] scale.

Delta

All four recipe paths listed under Outcome were repaired and are structurally identical to their current-main auto-config anchors. Relative to the pre-repair recipes, all four changed /export/compatibility/transformers_attention from absent to "eager". The fp32 recipes retain /quant = null. Both fp16 recipes changed /quant from null to the full current auto-config block, including /quant/mode = "fp16", /quant/model_id = "intfloat/multilingual-e5-small", /quant/model_type = "bert", /quant/fp16_keep_io_types = true, and task-specific /quant/task values of "feature-extraction" or "sentence-similarity". These deltas are reducibility-consistent: eager attention is already class-wide current-source behavior, while precision and task are tuple intent rather than checkpoint hardcoding. examples/recipes/README.md is unchanged.

Shared source and regression paths are src/winml/modelkit/models/winml/feature_extraction.py and tests/unit/models/auto/test_feature_extraction.py. No-recipe acceptance passed for final-SHA FP32 feature extraction and sentence similarity; the change does not add or modify a task-family registry.

Bug fix explanation

1. Symptom and trigger: single-segment feature-extraction/sentence-similarity inference failed with Missing required inputs: {'token_type_ids'} when the tokenizer emitted input_ids and attention_mask but the exported ONNX graph also declared token_type_ids.
2. Root cause: strict session input validation correctly rejected the incomplete feed before inference. The single-text wrapper had no safe defaulting path for a graph-declared segment-ID input omitted by a tokenizer.
3. Symbols and mechanism: WinMLModelForFeatureExtraction.forward now synthesizes an all-zero token_type_ids tensor only when ONNX input metadata declares that input, the tokenizer omitted it, input_ids supplies the realized batch/sequence shape, declared dtype/shape metadata exists, and every static required dimension matches. The tensor uses the realized input_ids shape and exact declared ONNX dtype before normal strict validation.
4. Class-wide rationale: the rule is driven entirely by ONNX input metadata and tokenizer-produced input_ids; it contains no model ID or model_type branch and applies to the single-text feature-extraction abstraction.
5. Compatibility and blast radius: tokenizer-provided segment IDs are preserved; only token_type_ids receives the safe zero default; unrelated missing inputs and incompatible static shapes still fail; synthesized dtype/shape follows graph metadata; output names/shapes, attention-mask pooling, sentence-similarity postprocessing, and recipe-owned semantics are unchanged. The intentional change is that single-segment paths can now feed graphs requiring omitted segment IDs.
6. Regression and partition evidence: focused tests 13 passed in 16.92s; models/loader/datasets/export 1527 passed, 6 skipped, 2 xfailed; optim 711 passed, 16 skipped, 1 xfailed; commands/config/build/compiler/session/eval 3563 passed, 9 skipped, 1 warning; remaining core/ONNX/cache/utils/helpers/sysinfo/inspect/optracing/regression/CLI 869 passed, 2 skipped, 1 deselected, 1 warning; analyze 1526 passed, 45 skipped. Total non-overlapping completed tests: 8196. Ruff reported All checks passed!; mypy reported no issues in 435 source files.

Analyze summary — component level and op level

Static rule analysis PASS completed for all four artifacts. These classifications are static compatibility evidence, not runtime execution or a claim that accelerator inference was run.

Component-level summary

Artifact	Architecture regions	Mapping	Actionable EP findings
feature-extraction fp32	embeddings; 12× encoder attention/feed-forward; runtime mean pooling/normalize	372 mapped, 0 partial, 0 unmapped; runtime-only postprocessing explicitly excluded	QNN GPU/NPU: Gather partial
feature-extraction fp16	embeddings; 12× encoder attention/feed-forward; runtime mean pooling/normalize	373 mapped, 0 partial, 0 unmapped; runtime-only postprocessing explicitly excluded	QNN GPU/NPU: Gather partial
sentence-similarity fp32	embeddings; 12× encoder attention/feed-forward; runtime mean pooling/normalize	372 mapped, 0 partial, 0 unmapped; runtime-only postprocessing explicitly excluded	QNN GPU/NPU: Gather partial
sentence-similarity fp16	embeddings; 12× encoder attention/feed-forward; runtime mean pooling/normalize	373 mapped, 0 partial, 0 unmapped; runtime-only postprocessing explicitly excluded	QNN GPU/NPU: Gather partial

There are no mapping gaps.

Op-level summary

Artifact	Graph	Dominant operator counts	Rule-backed EP roll-up
feature-extraction fp32	372 operators / 15 types	Reshape 121; Gemm 72; Transpose 48; Add 38; LayerNormalization 25	NVIDIA TensorRT RTX GPU and OpenVINO CPU/GPU/NPU: 14 supported types, Where unknown; QNN GPU/NPU: same plus Gather partial
feature-extraction fp16	373 operators / 15 types	Reshape 121; Gemm 72; Transpose 48; Add 38; LayerNormalization 25	NVIDIA TensorRT RTX GPU and OpenVINO CPU/GPU/NPU: 14 supported types, Where unknown; QNN GPU/NPU: same plus Gather partial
sentence-similarity fp32	372 operators / 15 types	Reshape 121; Gemm 72; Transpose 48; Add 38; LayerNormalization 25	NVIDIA TensorRT RTX GPU and OpenVINO CPU/GPU/NPU: 14 supported types, Where unknown; QNN GPU/NPU: same plus Gather partial
sentence-similarity fp16	373 operators / 15 types	Reshape 121; Gemm 72; Transpose 48; Add 38; LayerNormalization 25	NVIDIA TensorRT RTX GPU and OpenVINO CPU/GPU/NPU: 14 supported types, Where unknown; QNN GPU/NPU: same plus Gather partial

The six rule-backed targets are NvTensorRTRTXExecutionProvider/GPU, OpenVINOExecutionProvider/{CPU,GPU,NPU}, and QNNExecutionProvider/{GPU,NPU}. No rule-backed target reports an unsupported type. Rule-less CPU, CUDA, MIGraphX, and DML groups classify all 15 operator types as unknown.

Reproduce commands

$OUT='temp/multilingual-e5-small-repro'
winml build -c examples/recipes/intfloat_multilingual-e5-small/cpu/cpu/sentence-similarity_fp32_config.json -m intfloat/multilingual-e5-small -o $OUT
winml perf -m $OUT/model.onnx --device cpu --ep cpu
winml eval -m $OUT/model.onnx --model-id intfloat/multilingual-e5-small --task sentence-similarity --dataset <fresh-e5-smoke-dataset> --split train --samples 4 --no-shuffle --ep cpu --device cpu --skip-build
winml analyze --model $OUT/model.onnx --ep QNNExecutionProvider --device npu --no-information --output $OUT/analyze-qnn-npu.json

[Qiong Wu (qiowu) (DingmaomaoBJTU)]

APPROVE

coverage: partial
reachable-verified: [cpu]
deferred: [gpu, npu, full-fp16-eval-matrix]

Checklist evidence:

• Fresh PR-head checkout: git rev-parse HEAD = 59530534c0aef5d65d2bb197301cefebce1daf39; merge-base/current origin/main = d9feefffccecfe47f08b7d2cfffb11fe57eb94df; git rev-list --count HEAD..origin/main = 0.
• Diff scope: git diff --name-only origin/main...HEAD is exactly 4 files: examples/recipes/intfloat_multilingual-e5-small/cpu/cpu/{feature-extraction_fp16_config.json,feature-extraction_fp32_config.json,sentence-similarity_fp16_config.json,sentence-similarity_fp32_config.json}; no src/, tests, README, or skill files.
• Float CPU recipe set: both fp32/fp16 present for feature-extraction and sentence-similarity; no w8a16/w8a8 under cpu/cpu/ (git diff --name-only list above).
• Blob identity: git rev-parse HEAD:<path> gives FE fp32==fp16 49f99b779cc4f475b04c5e1e84059419a25cd299; SS fp32==fp16 cda4592eb8f5452f8ccf135e890ea881c846af1a.
• Recipe schema: Python JSON audit reports keys_ok True, quant_is_None True, opset 17, batch 1, loader {task, model_class: AutoModel, model_type: bert}, inputs input_ids/attention_mask/token_type_ids int32 [1,512], output last_hidden_state, clamp True for all 4.
• Baseline gate: temp\e5_baseline_build.log shows Export ✅ 61.2s / Optimize ✅ 31.1s then ValueError: Required inputs (['token_type_ids']) are missing from input feed (['input_ids', 'attention_mask']); temp\e5_baseline has export.onnx + optimized.onnx and no model.onnx.
• L0 artifacts: Python ONNX audit on temp\e5_fe\model.onnx and temp\e5_ss\model.onnx reports IR 8, opset 17, inputs input_ids, attention_mask, token_type_ids all elem_type 6 [1,512], output last_hidden_state [1,512,384]; .data files are co-located.
• L1 perf sanity: winml perf -m temp\e5_fe\model.onnx --device cpu --ep cpu --iterations 3 --warmup 1 --no-analyze reports CPUExecutionProvider fp32 avg 201.34 ms, throughput 4.97 samples/sec, RAM total +179.3 MB, consistent with PR’s 256.26 ms / 3.90 samples/sec / +186.9 MB.
• L2 parity: python temp\e5_l2.py reports raw last_hidden_state max-abs 1.729e-06, sentence embedding cosine min/mean 1.000000, query top-3 rankings [0,1,3] and [3,4,5] match PyTorch==ONNX, VERDICT: PASS; script lines 45-52 feed zero token_type_ids for XLMRobertaTokenizer single-segment default.
• Optimum/Effort tier: probe reports feature-extraction in vendor: True | added_by_winml: [] for bert, so L0★ recipe-only is correct; sentence-similarity maps to the same AutoModel backbone.
• PR body: gh api repos/microsoft/winml-cli/pulls/1182 body contains all 12 explainer sections, baseline FAIL→recipe FIX delta, L0/L1/L2 table, methodology declaration “No methodology friction observed”, CPU perf row, and README row omitted with CPU-only justification.
• Knowledge capture: Lane A model_knowledge\bert.json has bert-001, validated_on entries for FE/SS fp32 CPU, baseline error, recipe shape, L1/L2 numbers, analyze 384 ops / 15 unique, and deferred GPU/NPU/fp16 notes; it is not in this PR diff.
• Build artifacts read directly: temp\e5_fe/temp\e5_ss contain analyze_result.json, export_htp_metadata.json, and winml_build_config.json; audit shows 117,653,760 parameters, 228 modules, 88 traced, quant None, loader tasks correct, and optim clamp/fusions.
• Static gate: GitHub lint check succeeded; log shows uv run pre-commit run insert-license --all-files Passed, uv run ruff check src/ tests/ All checks passed, and uv run mypy -p winml.modelkit Success.
• N/A checks: Outcome-L1/L2 code/task-family add-ons, quant-preserving checks, composite gates, L3, NPU/GPU --ep-options, and skill_meta dead-link checks are not applicable because the diff is recipe-only L0★ with quant:null, no code/task/skill changes, and Goal ceiling L2.

Residual note: the test (models) check is red on an unrelated existing export unit (tests/unit/export/test_pytorch_export.py::TestExportPytorch::test_mismatched_input_order_exports_successfully); this PR changes only recipe JSONs and the required static gate is green.

[ssss141414]

REQUEST_CHANGES

Finding

1. Tester/explainer: the mandatory fp16 perf precision line is absent. Both fp16 winml perf logs omit Model Precision: fp16; the tester explicitly records model_precision_source: material ONNX initializer/external-data evidence; perf CLI did not print Model Precision, and the PR body repeats that substitution. The current reviewer contract requires both an fp16 artifact and a perf log that prints Model Precision: fp16; material proof cannot silently waive the printed-line gate. This is CLI output-field/evidence drift, not an actual fp16 conversion failure: independent artifact inspection found 200 FLOAT16 initializers, 234,931,456 external bytes versus 470,027,776 for fp32 (ratio 0.4998246231303573), float I/O, and builds used --precision fp16 without --no-quant.

   Action: tester must refresh both fp16 L1 perf records with output that explicitly reports Model Precision: fp16 (or route the CLI output regression to its owning implementation and rerun after repair), then reseal the tester verdict/manifest. Explainer must regenerate the L1 table/body from that refreshed tester evidence. Do not replace the required line with initializer/size inference.

Independent review evidence

• Reviewed state: head 8e7c621a2312a3ef45b08c43e7347301e992f842; current origin/main, PR base, and merge-base e564a6375d6cd2b596fb3d21d918f07824b349e0; clean dedicated detached reviewer worktree.
• Exact six-file scope: four examples/recipes/intfloat_multilingual-e5-small/cpu/cpu/{feature-extraction,sentence-similarity}_{fp32,fp16}_config.json recipes, src/winml/modelkit/models/winml/feature_extraction.py, and tests/unit/models/auto/test_feature_extraction.py. Both READMEs are untouched; no skill/temp leakage; no TASK_REGISTRY change.
• Engineering soundness: the shared wrapper is metadata-driven and contains no model-ID/type branch. It only synthesizes graph-declared token_type_ids, uses realized input_ids shape plus declared dtype, checks every static dimension, preserves tokenizer-provided IDs, and leaves strict session validation authoritative for arbitrary missing inputs. Regression tests cover supplied nonzero IDs, unrelated missing inputs, incompatible static shape, and both task call patterns. Effort L2/Outcome L2 is coherent as shared task-pattern infrastructure even though no new task registry family is added.
• Recipes: all four are structurally identical to the frozen current auto-config task/precision anchors. Eager attention is present on all; fp32 has quant:null; fp16 has full task-correct mode:fp16 blocks with fp16_keep_io_types:true; locations, model identity/class/type, opset 17, static int32 [1,512] inputs, and float output are exact. Recipe runs use no contradictory semantic overrides.
• Planner/producer provenance: all supplied hashes verified. Baseline was a FULL-RERUN on current base; build/perf/parity passed and bounded Eval reproduced the missing-token_type_ids gap. Frozen BERT profile, pinned revision, VENDOR-ONLY Optimum decision, recipe reducibility, generalized no-recipe acceptance, and six-part bug explanation are internally consistent.
• L0 artifact inspection: 4/4 ONNX checker/runtime validations passed; IR 8, opset 17; fp32 has 372 nodes and 200 FLOAT initializers; fp16 has 373 nodes and 200 FLOAT16 initializers. All named inputs are int32 [1,512], outputs are float [1,512,384], external data is co-located, and synthesized token_type_ids is zero int32 [1,512].
• L1 raw evidence: feature-extraction fp32 116.4 ms, fp16 170.74 ms; sentence-similarity fp32 118.74 ms, fp16 165.73 ms; throughput 8.59/5.86/8.42/6.03 samples/s; RAM deltas 81.4/89.0/81.3/88.8 MB. The sole L1 defect is the missing printed model-precision line described above.
• L2 independent rerun: 4/4 artifacts passed against pinned PyTorch revision 614241f622f53c4eeff9890bdc4f31cfecc418b3. fp32 max abs 1.758337e-6, fp16 max abs 0.003210783; minimum embedding cosine >=0.999999821; all retrieval rankings identical. Reviewer harness made 16 direct artifact calls; tester L3 separately proves 8 wrapper calls for 4 pairs.
• L3 raw evidence: exactly one fresh retry run, 2 source rows, 4 requested/processed pairs, 8 wrapper calls, zero int32 [1,512] synthesis, and cosine_spearman=89.4427. Dataset schema, query/passage and label semantics, fan-out caps, prediction interpretation, and the operability-only/not-representative claim are explicit; no Eval matrix is inferred.
• Analyze: all four artifacts have complete component mapping (372/373 mapped, zero unmapped), 15 op types, and 24 explicit static commands across six rule-backed targets. QNN GPU/NPU Gather is partial; Where is unknown on rule-backed targets; rule-less CPU/CUDA/MIGraphX/DML groups remain unknown. These are static classifications, not runtime accelerator claims.
• Tester lineage: retry root is revision-specific; all 124 sealed manifest files re-hashed successfully. The first-root Analyze summarization/inventory incident is retained; the fresh retry uses explicit targets and before/after provider/package snapshots show no retry package-state change.
• Knowledge/body: learner directly mined configs, HTP (117,653,760 parameters; 88/228 modules traced), L0-L3, and Analyze evidence and declared no_friction:true. Lane A PR #216 is OPEN/DRAFT, labeled model-scale-by-skill, exact head aae6bd6d8fda03325efe7fbd295cdcdce800c6e9, and changes only model_knowledge/bert.json. Canonical local body SHA is 48bf951882583b9da2f211fdd4e9d835205e378c5231788c175db01f4caebf76; normalized live SHA is 5ec1ccdf490f39c7a126ea112e2ff019bc07d7cdea9734ae19425483fe2b1a3f; required hierarchy/order and all seven validation subsections are present with no scratch paths/hashes.
• Independent quality reruns: focused compatibility 13 passed; uv run --no-sync ruff check src/ tests/ passed; uv run --no-sync mypy -p winml.modelkit passed for 435 files. Exact workflow partitions passed: analyze 1526 passed, 45 skipped; models 1527 passed, 6 skipped, 2 xfailed; optim 711 passed, 16 skipped, 1 xfailed; commands 3563 passed, 9 skipped; remaining 869 passed, 2 skipped, 1 deselected. --no-sync used the existing locked environment because the first exact uv run dependency sync was blocked fetching wheel metadata by TLS HandshakeFailure; no packages were installed.
• Discussion: 0 line comments, 1 ordinary conversation comment, 0 submitted reviews, 0 review threads, 0 open threads. The sole ordinary comment is a stale APPROVE for old head 59530534 and was not trusted as evidence.
• GitHub gate: current head has nine PR checks, all COMPLETED/SUCCESS; PR remains OPEN/DRAFT and labeled model-scale-by-skill.

No implementation, recipe, L0, L2, L3, Analyze, thread, status-check, body, or Lane A blocker was found beyond the explicit fp16 L1 printed-line requirement above. The orchestrator should route this fix to tester and explainer, then return the refreshed exact head and sealed handoffs for re-review.

[ssss141414]

APPROVE

Independent re-review of microsoft/winml-cli PR #1182 after my prior REQUEST_CHANGES conversation comment.

Reviewed identity and scope

• Head: 8e7c621a2312a3ef45b08c43e7347301e992f842
• Base/current main: e564a6375d6cd2b596fb3d21d918f07824b349e0
• Merge base: e564a6375d6cd2b596fb3d21d918f07824b349e0; current origin/main is unchanged, so no moved-main invalidation applies.
• State: OPEN and DRAFT; label model-scale-by-skill is retained.
• Exact six-file scope:
  • examples/recipes/intfloat_multilingual-e5-small/cpu/cpu/feature-extraction_fp16_config.json
  • examples/recipes/intfloat_multilingual-e5-small/cpu/cpu/feature-extraction_fp32_config.json
  • examples/recipes/intfloat_multilingual-e5-small/cpu/cpu/sentence-similarity_fp16_config.json
  • examples/recipes/intfloat_multilingual-e5-small/cpu/cpu/sentence-similarity_fp32_config.json
  • src/winml/modelkit/models/winml/feature_extraction.py
  • tests/unit/models/auto/test_feature_extraction.py
• The dedicated reviewer worktree is clean at the reviewed head. The shared source change remains metadata-driven: it synthesizes only graph-declared missing token_type_ids, enforces declared dtype/static dimensions, preserves provided IDs, and leaves strict validation authoritative for other missing inputs. No model-ID branch, task-registry change, production README change, or scope regression was introduced.

Prior blocker: resolved

The previous blocker was an evidence-contract/CLI-output drift issue, not an fp16 artifact failure. Under the current reviewer contract and Lane A _meta-108, authoritative schema-v2 model_info.precision == "fp16" is preferred; benchmark_info.precision == "fp16" corroborates requested policy, while the legacy console literal is required only on versions that render it.

I independently parsed the two raw repaired winml perf --format json outputs and confirmed:

Artifact	Schema	Resolved precision	Policy precision	Provider/device	Mean	p50	Throughput	RSS delta
feature-extraction fp16	2	model_info.precision=fp16	benchmark_info.precision=fp16	CPUExecutionProvider/cpu	179.643 ms	178.686 ms	5.57 samples/s	+88.73 MB
sentence-similarity fp16	2	model_info.precision=fp16	benchmark_info.precision=fp16	CPUExecutionProvider/cpu	180.017 ms	179.447 ms	5.56 samples/s	+88.71 MB

Both records identify the exact candidate-owned artifacts, use 100 iterations plus 10 warmups, and exited 0. Captured stdout is text-identical to each JSON report; stderr contains zero Model Precision: fp16 literals, so no console echo was fabricated. The repair command log contains exactly two functional perf reruns, one per fp16 artifact, with no artifact rebuild, no Eval rerun, no other tier rerun, no recipe-owned override, and no candidate edit.

The other two parts of the fail-closed precision proof also hold for both artifacts:

• Build command includes --precision fp16 and does not include --no-quant.
• Resolved quant config is mode=fp16, task/model-specific, with fp16_keep_io_types=true.
• Each artifact has 200 FLOAT16 initializers and 234,931,456 external-data bytes versus 200 FLOAT initializers and 470,027,776 bytes for fp32.
• Exact fp16/fp32 external-data ratio: 0.4998246231303573.

The original blocker is therefore fully resolved by authoritative schema-v2 precision plus unchanged structural proof.

Independent validation retained on unchanged code

I revalidated the sealed prior reviewer evidence against the unchanged head/base: all 97 entries match their recorded hashes. I also revalidated all 160 original tester evidence entries and all 44 repair entries; no missing or mismatched file was found. Reuse is valid because candidate, base/current main, recipes, built artifacts, and relevant dependencies are unchanged. I did not rerun Eval.

Prior independent reruns remain:

• Focused wrapper tests: 13 passed.
• Ruff: PASS; mypy: no issues in 435 source files.
• CI partitions: analyze 1,526 passed; models/loader/datasets/export 1,527 passed; optim 711 passed; commands/config/build/compiler/session/eval 3,563 passed; remaining 869 passed.
• Non-overlapping partition total: 8,196 passed, with only the previously recorded skips/xfails/deselection.

Goal evidence remains complete and unchanged except for the two refreshed fp16 L1 rows:

• L0: 4/4 clean CPU builds. ONNX checker passes; IR 8/opset 17; three named int32 [1,512] inputs and float [1,512,384] output; 372 fp32 nodes and 373 fp16 nodes. Runtime smoke produces finite float32 output, and synthesized token_type_ids is int32, [1,512], and all-zero only when omitted by the tokenizer.
• L1: 4/4 CPU perf PASS. The refreshed fp16 metrics are listed above; unaffected fp32 rows remain feature-extraction 116.4/115.92 ms at 8.59 samples/s and sentence-similarity 118.74/118.04 ms at 8.42 samples/s.
• L2: 4/4 PyTorch parity PASS. fp32 max-abs error is 1.7583370208740234e-06; fp16 max-abs error is 0.003210783004760742; minimum embedding cosines are 0.9999999403953552 and 0.9999998211860657; all retrieval rankings are identical.
• L3: exactly one bounded final-head FP32 CPU functional smoke Eval, not an inferred per-precision matrix: 2 bilingual source rows x 2 passages, 4 processed pairs, 8 inference calls, explicit sequence/fan-out bounds, and cosine_spearman=89.4427. It is correctly labeled operability-only, not representative accuracy.

Complete Analyze evidence is retained for all four artifacts. Component mapping covers embeddings plus 12 encoder attention/feed-forward blocks, with 372/373 mapped nodes and zero partial or unmapped nodes; runtime pooling/normalization is explicitly excluded from the ONNX mapping. Op analysis reports 372/373 operators across 15 types, with dominant Reshape 121, Gemm 72, Transpose 48, Add 38, and LayerNormalization 25. Across the six rule-backed targets, no unsupported type is reported; Where is unknown and QNN GPU/NPU additionally reports Gather partial. Rule-less CPU/CUDA/MIGraphX/DML groups are correctly classified as unknown rather than inferred supported.

Body and methodology fidelity

The live body is byte-equal to the repaired canonical body after CRLF/LF normalization, hash 83bf778ffca05a084a738d385b23eda97eb4ff0950c3b4d52f1bc0f7819ee47c; the canonical file hash is 801c79145770fb66e8f1ba564c149559ac1543da2becf3c1d01544591278fa94. It retains the required hierarchy and all source-owned metadata, L0/L2/L3, delta, model identity, commands, and component/op analysis. The canonical before/after diff changes only the two superseded fp16 perf rows, the schema-v2 precision explanation, and the methodology declaration. It contains no local scratch paths or uncommitted evidence hashes and has exactly one functional smoke Eval path.

The methodology declaration now accurately cites ModelKitArtifacts Lane A PR #216, exact head 8c67d9cb69ac1ca464beabe82bac1771c4edd912, as no_friction:false with reviewer-discovered doc-code drift _meta-108. I reviewed that exact four-file Lane A diff from base 9b5fa7fe55a8351af2049e93a640b788d80cc397: BERT findings bert-011 through bert-016 remain retained; commit 9ec5236e35e84a8a2c23f1303e7d4cc6f74a63a6 pairs the tester/reviewer precision-contract edits; tip commit 8c67d9cb69ac1ca464beabe82bac1771c4edd912 atomically records _meta-108. Lane A PR #216 is OPEN/DRAFT, retains model-scale-by-skill, and its visible check is COMPLETED/SUCCESS.

GitHub gates

• Discussion inventory before scoring: 2 ordinary conversation comments, 0 line comments, 0 submitted reviews, 0 review threads, 0 open threads.
• The old APPROVE conversation comment is stale and ignored. My prior REQUEST_CHANGES is preserved as repair lineage; ordinary comments are not review threads.
• Exact-head statusCheckRollup: 9/9 COMPLETED/SUCCESS: Analyze (Python), lint, five test partitions, CodeQL, and license/cla.
• Final publication is conditioned on a repeated paginated discussion/thread enumeration and an immediate head/base/Draft/label/check recheck. Any head movement invalidates this opinion before publication.

No blockers remain. This approval is a normal conversation-comment verdict only; the PR correctly remains DRAFT, and this comment does not authorize readiness or any other PR mutation.

[xieofxie]

One correctness issue remains in the shared token-type defaulting behavior.

[xieofxie]

This wrapper cannot infer from tensor shape or ONNX metadata that the request contains only one segment. If a caller passes tokenized paired BERT input without token_type_ids, segment-B positions require 1, but this path silently supplies all zeros and returns incorrect embeddings instead of preserving the missing-input error. Could zero synthesis be gated by explicit preprocessing metadata that establishes a single-text request (or a declarative safe-default policy), rather than applied universally whenever the tensor is omitted?