Skip to content

Commit 4fb7d5e

Browse files
ToolboxAidToolboxAid
committed
BUILD_PR_LEVEL_10_6B_STANDALONE_SAMPLE_GENERIC_FAILURE_CLOSEOUT
- Expanded standalone sample contract harness to cover all generic failures - Added tool-specific contracts for remaining failing tools - Fixed remaining standalone sample payload/binding issues PR Details: - Closes Level 10.6 generic failure set - No silent fallback data - No hardcoded asset paths - No start_of_day changes
1 parent 175074b commit 4fb7d5e

16 files changed

Lines changed: 826 additions & 75 deletions

File tree

‎docs/dev/reports/expected_delta_template.md‎

Lines changed: 17 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,35 +1,30 @@
11
# Expected Codex Return / Delta Template
22

33
## Expected Changed Files
4-
- affected `samples/phase-*/####/**`
5-
- affected standalone tool data-loading/binding files
6-
- `tests/runtime/SampleStandaloneToolDataFlow.test.mjs` or equivalent
7-
- `docs/dev/reports/level_10_6_sample_schema_validation_report.md`
8-
- `docs/dev/reports/level_10_6_standalone_tool_data_flow_report.md`
4+
- affected `samples/phase-*/####/sample.*.json`
5+
- affected tool standalone binding files
6+
- `tests/runtime/SampleStandaloneToolDataFlow.test.mjs`
7+
- optional test helper updates
8+
- `docs/dev/reports/level_10_6b_standalone_generic_failure_closeout_report.md`
9+
- `docs/dev/reports/level_10_6b_tool_contract_matrix.md`
910
- `docs/dev/roadmaps/MASTER_ROADMAP_ENGINE.md` if status update needed
1011

11-
## Required Targeted Regression Results
12-
- `sample_0204_asset_browser_pipeline_input_loaded=true`
13-
- `sample_1413_asset_browser_or_pipeline_input_loaded=true`
14-
- `sample_1505_asset_browser_pipeline_input_loaded=true`
15-
- `sample_0510_asset_pipeline_input_loaded=true`
16-
- `sample_1417_asset_pipeline_input_loaded=true`
17-
- `sample_0213_palette_displayed=true`
18-
- `sample_0308_palette_displayed=true`
19-
- `sample_0313_palette_displayed=true`
20-
2112
## Expected Validation Summary
22-
- `sample_payload_files_scanned=<count>`
23-
- `schema_resolution_failures=0`
24-
- `tool_payload_contract_failures=0`
25-
- `standalone_data_flow_failures=0`
26-
- `palette_display_failures=0`
27-
- `pipeline_input_failures=0`
13+
- `generic_failure_signals_before=25`
14+
- `generic_failure_signals_after=0`
15+
- `three_d_camera_path_editor_failures=0`
16+
- `three_d_asset_viewer_failures=0`
17+
- `physics_sandbox_failures=0`
18+
- `tile_model_converter_failures=0`
19+
- `json_payload_normalizer_failures=0`
20+
- `parallax_editor_failures=0`
21+
- `performance_profiler_failures=0`
22+
- `replay_visualizer_failures=0`
2823
- `silent_fallbacks_used=0`
2924
- `hardcoded_asset_paths_added=0`
3025
- `start_of_day_changes=0`
3126

3227
## Expected Delta ZIP
3328
Codex must create:
3429

35-
`tmp/BUILD_PR_LEVEL_10_6_SAMPLE_SCHEMA_AND_STANDALONE_TOOL_DATA_FLOW_VALIDATION_delta.zip`
30+
`tmp/BUILD_PR_LEVEL_10_6B_STANDALONE_SAMPLE_GENERIC_FAILURE_CLOSEOUT_delta.zip`

‎docs/dev/reports/level_10_6b_standalone_generic_failure_closeout_report.md‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
# Level 10.6B Standalone Generic Failure Closeout Report
2+
3+
- Generated: 2026-04-27
4+
- BUILD: `BUILD_PR_LEVEL_10_6B_STANDALONE_SAMPLE_GENERIC_FAILURE_CLOSEOUT`
5+
- Command run: `npm run test:sample-standalone:data-flow`
6+
- Raw validation artifact: `tmp/sample-standalone-tool-data-flow-results.json`
7+
8+
## Result
9+
10+
- Previous generic failure signals: `25`
11+
- Generic failure signals after fixes: `0`
12+
- Acceptance status: `PASS`
13+
14+
## Root Cause Summary
15+
16+
All 25 generic failures were valid sample preset files with data stored under `config.*`, while affected tool standalone preset extractors were only reading top-level and/or `payload.*` keys.
17+
18+
Fixed root cause classes:
19+
20+
- tool binding to wrong input slot (`config.*` not consumed)
21+
- payload shape mismatch (`config` wrapper not recognized)
22+
23+
No fallback/demo data was added. No hardcoded asset paths were added.
24+
25+
## Files Changed
26+
27+
- `tools/3D Camera Path Editor/main.js`
28+
- `tools/3D Asset Viewer/main.js`
29+
- `tools/Physics Sandbox/main.js`
30+
- `tools/Tile Model Converter/main.js`
31+
- `tools/3D JSON Payload Normalizer/main.js`
32+
- `tools/Parallax Scene Studio/main.js`
33+
- `tools/Performance Profiler/main.js`
34+
- `tools/Replay Visualizer/main.js`
35+
- `tests/runtime/SampleStandaloneToolDataFlow.test.mjs`
36+
37+
## 25 Failure Closeout Coverage
38+
39+
Closed sample/tool failures:
40+
41+
- `0201:3d-camera-path-editor`
42+
- `0202:3d-camera-path-editor`
43+
- `0220:3d-camera-path-editor`
44+
- `0204:3d-asset-viewer`
45+
- `1208:3d-asset-viewer`
46+
- `1413:3d-asset-viewer`
47+
- `0210:physics-sandbox`
48+
- `0303:physics-sandbox`
49+
- `1606:physics-sandbox`
50+
- `0221:tile-model-converter`
51+
- `0305:tile-model-converter`
52+
- `1209:tile-model-converter`
53+
- `0221:3d-json-payload-normalizer`
54+
- `0305:3d-json-payload-normalizer`
55+
- `1208:3d-json-payload-normalizer`
56+
- `0306:parallax-editor`
57+
- `1204:parallax-editor`
58+
- `1205:parallax-editor`
59+
- `1208:parallax-editor`
60+
- `0512:performance-profiler`
61+
- `1319:performance-profiler`
62+
- `1407:performance-profiler`
63+
- `0708:replay-visualizer`
64+
- `1315:replay-visualizer`
65+
- `1406:replay-visualizer`
66+
67+
## Contract Assertions Added (Test)
68+
69+
`tests/runtime/SampleStandaloneToolDataFlow.test.mjs` now enforces tool-specific contracts for all 25 entries:
70+
71+
- 3D Camera Path Editor: waypoint payload loaded, count matches preset
72+
- 3D Asset Viewer: vertex payload loaded, count matches preset
73+
- Physics Sandbox: body input loaded and exactly matches preset body
74+
- Tile Model Converter: candidate/conversion payload loaded and matches preset
75+
- 3D JSON Payload Normalizer: points/segments loaded, counts match preset
76+
- Parallax Editor: parallax layers loaded, count matches preset
77+
- Performance Profiler: profile settings inputs bound from preset
78+
- Replay Visualizer: replay events loaded, count matches preset
79+
80+
## Blockers
81+
82+
- None.
83+
84+
## Roadmap Update
85+
86+
- No Level 10.6B status marker exists in `docs/dev/roadmaps/MASTER_ROADMAP_ENGINE.md`.
87+
- No roadmap prose/status edits were applied.

‎docs/dev/reports/level_10_6b_tool_contract_matrix.md‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# Level 10.6B Tool Contract Matrix
2+
3+
- Generated: 2026-04-27
4+
- Source test: `tests/runtime/SampleStandaloneToolDataFlow.test.mjs`
5+
- Source results: `tmp/sample-standalone-tool-data-flow-results.json`
6+
7+
| Tool | Samples | Required Contract | Verification Signal | Status |
8+
| --- | --- | --- | --- | --- |
9+
| `3d-camera-path-editor` | `0201`, `0202`, `0220` | `cameraPath` payload present, `waypoints.length > 0` and equals preset | `#cameraPathInput` parsed JSON waypoints count equals preset; no preset-failure status | PASS |
10+
| `3d-asset-viewer` | `0204`, `1208`, `1413` | 3D asset payload present, `vertices.length > 0` and equals preset | `#asset3dInput` parsed JSON vertex count equals preset; no preset-failure status | PASS |
11+
| `physics-sandbox` | `0210`, `0303`, `1606` | physics body payload present and valid | `#physicsBodyInput` parsed JSON deep-equals preset `config.physicsBody`; no preset-failure status | PASS |
12+
| `tile-model-converter` | `0221`, `0305`, `1209` | conversion payload present (`candidate`, `conversion`) | `#converterInput` parsed JSON candidate/conversion deep-equals preset; no preset-failure status | PASS |
13+
| `3d-json-payload-normalizer` | `0221`, `0305`, `1208` | input map payload present; points/segments valid | `#map3dInput` parsed JSON points/segments counts equal preset; no preset-failure status | PASS |
14+
| `parallax-editor` | `0306`, `1204`, `1205`, `1208` | parallax scene/layers payload present; layer count > 0 | `#layerList li` count equals preset `parallaxDocument.layers.length`; no preset-failure status | PASS |
15+
| `performance-profiler` | `0512`, `1319`, `1407` | profile settings payload present and bound to UI inputs | `#workloadIterationsInput`, `#workSizeInput`, `#frameSamplesInput` equal preset `profileSettings` | PASS |
16+
| `replay-visualizer` | `0708`, `1315`, `1406` | replay events payload present; events count > 0 | `#replayEventList [data-replay-index]` count equals preset events count; no preset-failure status | PASS |
17+
18+
## Implementation Boundary Notes
19+
20+
- No runtime validator modules were introduced.
21+
- No fallback/default demo data was introduced for failing tools.
22+
- Preset extraction was normalized to consume explicit `config.*` payload shape where the sample contract stores tool input.

‎docs/operations/dev/codex_commands.md‎

Lines changed: 38 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -2,53 +2,50 @@ MODEL: GPT-5.3-codex
22
REASONING: high
33

44
TASK:
5-
Apply BUILD_PR_LEVEL_10_6_SAMPLE_SCHEMA_AND_STANDALONE_TOOL_DATA_FLOW_VALIDATION.
5+
Apply BUILD_PR_LEVEL_10_6B_STANDALONE_SAMPLE_GENERIC_FAILURE_CLOSEOUT.
6+
7+
INPUT CONTEXT:
8+
Use the Level 10.6 delta result. It fixed targeted flows but reported 25 generic standalone data-flow failures.
69

710
NON-NEGOTIABLE RULES:
811
- Do not use silent fallback data.
912
- Do not add hardcoded asset paths.
10-
- Do not hide invalid sample payloads by loading defaults.
11-
- Valid sample data must be explicitly received and displayed by the target tool.
13+
- Do not mask invalid samples with defaults.
14+
- Valid sample payload must explicitly reach the tool state/UI slot.
1215

1316
STEPS:
14-
1. Read docs/pr/PLAN_PR_LEVEL_10_6_SAMPLE_SCHEMA_AND_STANDALONE_TOOL_DATA_FLOW_VALIDATION.md.
15-
2. Read docs/pr/BUILD_PR_LEVEL_10_6_SAMPLE_SCHEMA_AND_STANDALONE_TOOL_DATA_FLOW_VALIDATION.md.
16-
3. Audit all sample payload files:
17-
- samples/**/sample.*.*.json
18-
- samples/**/sample.*.palette.json
19-
4. Validate schema references and tool payload contracts.
20-
5. Add/update an automated standalone sample tool data-flow test:
21-
- discover samples
22-
- open/route to target tool
23-
- verify data is received
24-
- verify UI/state displays expected data
25-
6. Specifically fix/test:
26-
- sample 0204 -> Asset Browser / Import Hub Pipeline input
27-
- sample 1413 -> Asset Browser / Import Hub and/or Asset Pipeline Tool Pipeline input
28-
- sample 1505 -> Asset Browser / Import Hub Pipeline input
29-
- sample 0510 -> Asset Pipeline Tool Pipeline input
30-
- sample 1417 -> Asset Pipeline Tool Pipeline input
31-
- sample 0213 -> palette displays
32-
- sample 0308 -> palette displays
33-
- sample 0313 -> palette displays
34-
7. Update tool standalone binding only where needed:
35-
- Asset Browser / Import Hub Pipeline input
36-
- Asset Pipeline Tool Pipeline input
37-
- Palette Browser palette display
38-
8. Ensure no fallback/demo data is loaded.
39-
9. Write reports:
40-
- docs/dev/reports/level_10_6_sample_schema_validation_report.md
41-
- docs/dev/reports/level_10_6_standalone_tool_data_flow_report.md
42-
10. Update docs/dev/roadmaps/MASTER_ROADMAP_ENGINE.md status only if needed:
43-
- [ ] -> [.]
44-
- [.] -> [x]
45-
- no prose rewrite/delete
46-
11. Do not add runtime validators.
47-
12. Do not modify start_of_day.
48-
13. Create Codex delta ZIP:
49-
tmp/BUILD_PR_LEVEL_10_6_SAMPLE_SCHEMA_AND_STANDALONE_TOOL_DATA_FLOW_VALIDATION_delta.zip
17+
1. Read docs/pr/PLAN_PR_LEVEL_10_6B_STANDALONE_SAMPLE_GENERIC_FAILURE_CLOSEOUT.md.
18+
2. Read docs/pr/BUILD_PR_LEVEL_10_6B_STANDALONE_SAMPLE_GENERIC_FAILURE_CLOSEOUT.md.
19+
3. Run or inspect `npm run test:sample-standalone:data-flow`.
20+
4. Expand `tests/runtime/SampleStandaloneToolDataFlow.test.mjs` to include tool-specific contracts for all 25 generic failures.
21+
5. Fix each listed sample/tool binding:
22+
- 3d-camera-path-editor: 0201,0202,0220
23+
- 3d-asset-viewer: 0204,1208,1413
24+
- physics-sandbox: 0210,0303,1606
25+
- tile-model-converter: 0221,0305,1209
26+
- 3d-json-payload-normalizer: 0221,0305,1208
27+
- parallax-editor: 0306,1204,1205,1208
28+
- performance-profiler: 0512,1319,1407
29+
- replay-visualizer: 0708,1315,1406
30+
6. Fix root causes:
31+
- stale sample paths
32+
- wrong payload shape
33+
- missing required payload data
34+
- tool binding to wrong input slot
35+
- UI state not reflecting loaded payload
36+
7. Re-run the standalone data-flow test until generic failure signals are zero, or document exact remaining blockers with file names.
37+
8. Write reports:
38+
- docs/dev/reports/level_10_6b_standalone_generic_failure_closeout_report.md
39+
- docs/dev/reports/level_10_6b_tool_contract_matrix.md
40+
9. Update docs/dev/roadmaps/MASTER_ROADMAP_ENGINE.md status only if needed:
41+
- [ ] -> [.]
42+
- [.] -> [x]
43+
- no prose rewrite/delete
44+
10. Do not add runtime validators.
45+
11. Do not modify start_of_day.
46+
12. Create Codex delta ZIP:
47+
tmp/BUILD_PR_LEVEL_10_6B_STANDALONE_SAMPLE_GENERIC_FAILURE_CLOSEOUT_delta.zip
5048

5149
ACCEPTANCE:
52-
- all targeted sample failures fixed or reported with exact blocker
53-
- automated test catches standalone tool data-flow failures
50+
- generic failure signals after = 0, or exact remaining blocker report
5451
- delta ZIP exists

‎docs/operations/dev/commit_comment.txt‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
1-
BUILD_PR_LEVEL_10_6_SAMPLE_SCHEMA_AND_STANDALONE_TOOL_DATA_FLOW_VALIDATION
1+
BUILD_PR_LEVEL_10_6B_STANDALONE_SAMPLE_GENERIC_FAILURE_CLOSEOUT
22

3-
- Added standalone sample schema and tool data-flow validation
4-
- Fixed targeted Asset Browser, Asset Pipeline, and Palette sample loading/display failures
5-
- Prevented fallback/demo data from masking invalid payloads
3+
- Expanded standalone sample contract harness to cover all generic failures
4+
- Added tool-specific contracts for remaining failing tools
5+
- Fixed remaining standalone sample payload/binding issues
66

77
PR Details:
8-
- Validates standalone tool mode receives/displays sample data
8+
- Closes Level 10.6 generic failure set
9+
- No silent fallback data
910
- No hardcoded asset paths
10-
- No runtime validators
1111
- No start_of_day changes

0 commit comments

Comments
 (0)