Skip to content

Commit 2ff3986

Browse files
ToolboxAidToolboxAid
committed
Add runtime observability foundation
- implemented runtime error tracking hooks - added performance monitoring hooks - standardized logging format and levels - produced observability matrix and validation reports - validated outputs with real examples Advances: - runtime error tracking [ ] -> [x] - performance monitoring hooks [ ] -> [x] - logging standardization [ ] -> [x]
1 parent 8178c75 commit 2ff3986

16 files changed

Lines changed: 305 additions & 20 deletions

‎.gitignore‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -14,18 +14,18 @@
1414
/*.zip
1515

1616
# Ignore specific files in the docs/dev directory
17-
docs/dev/COMMIT_COMMENT.txt
18-
docs/dev/commit_comment.txt
17+
#docs/dev/COMMIT_COMMENT.txt
18+
#docs/dev/commit_comment.txt
1919

20-
docs/dev/CODEX_COMMANDS.md
21-
docs/dev/codex_commands.md
20+
#docs/dev/CODEX_COMMANDS.md
21+
#docs/dev/codex_commands.md
2222

23-
docs/dev/NEXT_COMMAND.txt
23+
#docs/dev/NEXT_COMMAND.txt
2424

25-
COMMIT_COMMENT.txt
26-
commit_comment.txt
27-
CODEX_COMMANDS.md
28-
codex_commands.md
25+
#COMMIT_COMMENT.txt
26+
#commit_comment.txt
27+
#CODEX_COMMANDS.md
28+
#codex_commands.md
2929
NEXT_COMMAND.txt
3030

3131
node_modules/

‎docs/dev/reports/BUILD_PR_LEVEL_22_1_RUNTIME_OBSERVABILITY_FOUNDATION_ERROR_TRACKING.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# BUILD_PR_LEVEL_22_1_RUNTIME_OBSERVABILITY_FOUNDATION — Error Tracking
2+
3+
## Implemented Hooks
4+
- `src/engine/core/Engine.js`
5+
- existing `trackRuntimeError(...)` remains authoritative for runtime-path errors.
6+
- runtime monitoring integration now starts at runtime entry and emits monitoring events.
7+
- `src/engine/runtime/RuntimeMonitoringHooks.js`
8+
- captures `window.error` and `window.unhandledrejection` when browser listeners are available.
9+
- emits normalized error payloads using `format: runtime.monitoring.v1`.
10+
- logs error events via standardized logger event code `runtime.monitoring.error`.
11+
- `tools/shared/platformShell.js`
12+
- registers shared runtime monitoring hooks for tool entry surfaces.
13+
- stores latest captured monitoring error for diagnostics (`window.__TOOLS_PLATFORM_RUNTIME_LAST_ERROR__`).
14+
15+
## Captured Error Example (Execution-backed)
16+
- Source: `example.runtime`
17+
- Hook: `example.manual`
18+
- Message: `example boom`
19+
- Payload format: `runtime.monitoring.v1`
20+
- Logged event: `runtime.monitoring.error`
21+
- Logged format: `engine.log.v1`
22+
23+
## Touched Files
24+
- `src/engine/runtime/RuntimeMonitoringHooks.js`
25+
- `src/engine/core/Engine.js`
26+
- `tools/shared/platformShell.js`
27+
- `tests/runtime/RuntimeMonitoringHooks.test.mjs`
28+
- `tests/runtime/Phase19RuntimeLifecycleValidation.test.mjs`
29+
- `tests/tools/RuntimeObservabilityFoundation.test.mjs`

‎docs/dev/reports/BUILD_PR_LEVEL_22_1_RUNTIME_OBSERVABILITY_FOUNDATION_LOGGING_STANDARD.md‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
# BUILD_PR_LEVEL_22_1_RUNTIME_OBSERVABILITY_FOUNDATION — Logging Standard
2+
3+
## Standard
4+
- Base log envelope: `engine.log.v1`
5+
- Runtime monitoring payload envelope: `runtime.monitoring.v1`
6+
7+
## Levels Applied
8+
- `error`
9+
- runtime error capture (`runtime.monitoring.error`)
10+
- engine runtime failures (`engine.runtime.error`)
11+
- `warn`
12+
- isolated non-fatal runtime issues already routed by engine lifecycle guards
13+
- `info`
14+
- startup/load performance checkpoints (`runtime.monitoring.performance` with reason `start`/`load`)
15+
- `debug`
16+
- interval/manual monitoring samples (`runtime.monitoring.performance`)
17+
18+
## Event Codes Used
19+
- `engine.runtime.error`
20+
- `runtime.monitoring.error`
21+
- `runtime.monitoring.performance`
22+
23+
## Logging Output Example (Execution-backed)
24+
- Format: `engine.log.v1`
25+
- Level: `error`
26+
- Channel: `observability-example`
27+
- Event: `runtime.monitoring.error`
28+
- Message: `Runtime monitoring captured an error.`

‎docs/dev/reports/BUILD_PR_LEVEL_22_1_RUNTIME_OBSERVABILITY_FOUNDATION_OBSERVABILITY_MATRIX.md‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# BUILD_PR_LEVEL_22_1_RUNTIME_OBSERVABILITY_FOUNDATION ? Observability Matrix
2+
3+
| Surface | Error Tracking | Logging | Performance Hooks |
4+
| --- | --- | --- | --- |
5+
| Engine runtime (`src/engine/core/Engine.js`) | Yes (`trackRuntimeError` + monitoring bridge events) | `engine.log.v1` (`error`/`warn`) | Frame timings (`frameMs/updateMs/renderMs`) + `load/start/interval/manual` monitoring samples |
6+
| Runtime monitoring module (`src/engine/runtime/RuntimeMonitoringHooks.js`) | Yes (`window.error`, `window.unhandledrejection`, manual emit) | `engine.log.v1` (`error` + `info` + `debug`) | `runtime.monitoring.v1` samples (`start/load/interval/manual`) |
7+
| Tools platform shell (`tools/shared/platformShell.js`) | Yes (shared monitoring hook; latest payload retained) | `engine.log.v1` via `tools.platform` channel | Tool entry load timing + periodic monitoring samples |
8+
| Tool: Vector Map Editor (vector-map-editor) | Yes (inherits platform shell monitoring entry hook) | `engine.log.v1` via shared tools channel | Entry-point monitoring via shared platform shell hooks |
9+
| Tool: Vector Asset Studio (vector-asset-studio) | Yes (inherits platform shell monitoring entry hook) | `engine.log.v1` via shared tools channel | Entry-point monitoring via shared platform shell hooks |
10+
| Tool: Tilemap Studio (tile-map-editor) | Yes (inherits platform shell monitoring entry hook) | `engine.log.v1` via shared tools channel | Entry-point monitoring via shared platform shell hooks |
11+
| Tool: Parallax Scene Studio (parallax-editor) | Yes (inherits platform shell monitoring entry hook) | `engine.log.v1` via shared tools channel | Entry-point monitoring via shared platform shell hooks |
12+
| Tool: Sprite Editor (sprite-editor) | Yes (inherits platform shell monitoring entry hook) | `engine.log.v1` via shared tools channel | Entry-point monitoring via shared platform shell hooks |
13+
| Tool: Asset Browser / Import Hub (asset-browser) | Yes (inherits platform shell monitoring entry hook) | `engine.log.v1` via shared tools channel | Entry-point monitoring via shared platform shell hooks |
14+
| Tool: Palette Browser / Manager (palette-browser) | Yes (inherits platform shell monitoring entry hook) | `engine.log.v1` via shared tools channel | Entry-point monitoring via shared platform shell hooks |
15+
| Tool: State Inspector (state-inspector) | Yes (inherits platform shell monitoring entry hook) | `engine.log.v1` via shared tools channel | Entry-point monitoring via shared platform shell hooks |
16+
| Tool: Replay Visualizer (replay-visualizer) | Yes (inherits platform shell monitoring entry hook) | `engine.log.v1` via shared tools channel | Entry-point monitoring via shared platform shell hooks |
17+
| Tool: Performance Profiler (performance-profiler) | Yes (inherits platform shell monitoring entry hook) | `engine.log.v1` via shared tools channel | Entry-point monitoring via shared platform shell hooks |
18+
| Tool: Physics Sandbox (physics-sandbox) | Yes (inherits platform shell monitoring entry hook) | `engine.log.v1` via shared tools channel | Entry-point monitoring via shared platform shell hooks |
19+
| Tool: Asset Pipeline Tool (asset-pipeline-tool) | Yes (inherits platform shell monitoring entry hook) | `engine.log.v1` via shared tools channel | Entry-point monitoring via shared platform shell hooks |
20+
| Tool: Tile Model Converter (tile-model-converter) | Yes (inherits platform shell monitoring entry hook) | `engine.log.v1` via shared tools channel | Entry-point monitoring via shared platform shell hooks |
21+
| Tool: 3D Map Editor (3d-map-editor) | Yes (inherits platform shell monitoring entry hook) | `engine.log.v1` via shared tools channel | Entry-point monitoring via shared platform shell hooks |
22+
| Tool: 3D Asset Viewer (3d-asset-viewer) | Yes (inherits platform shell monitoring entry hook) | `engine.log.v1` via shared tools channel | Entry-point monitoring via shared platform shell hooks |
23+
| Tool: 3D Camera Path Editor (3d-camera-path-editor) | Yes (inherits platform shell monitoring entry hook) | `engine.log.v1` via shared tools channel | Entry-point monitoring via shared platform shell hooks |
24+
25+
## Notes
26+
- Tool-level monitoring is standardized through shared `tools/shared/platformShell.js` integration.
27+
- No dashboard/UI expansion was introduced in this PR.

‎docs/dev/reports/BUILD_PR_LEVEL_22_1_RUNTIME_OBSERVABILITY_FOUNDATION_PERFORMANCE_MONITORING.md‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# BUILD_PR_LEVEL_22_1_RUNTIME_OBSERVABILITY_FOUNDATION — Performance Monitoring
2+
3+
## Implemented Hooks
4+
- Engine frame timing remains emitted per frame through existing runtime path:
5+
- `frameMs`
6+
- `updateMs`
7+
- `renderMs`
8+
- `fixedUpdates`
9+
- Shared runtime monitoring hooks now emit timing samples for:
10+
- `start`
11+
- `load`
12+
- `interval`
13+
- `manual`
14+
- Entry-point load timing added:
15+
- engine runtime entry: `surface=engine.start`, `loadDurationMs=...`
16+
- tools shell entry: `surface=tools.platform.init`, `loadDurationMs=...`
17+
18+
## Performance Timing Example (Execution-backed)
19+
- Reason: `load`
20+
- Sample now: `42.25`
21+
- Context: `loadDurationMs: 7.5`, `surface: example.entry`
22+
- Payload format: `runtime.monitoring.v1`
23+
24+
## Validation Coverage
25+
- `tests/runtime/RuntimeMonitoringHooks.test.mjs` validates start/interval/manual samples plus standardized payload format.
26+
- `tests/runtime/Phase19RuntimeLifecycleValidation.test.mjs` validates runtime monitoring performance events from engine lifecycle.
27+
- `tests/tools/RuntimeObservabilityFoundation.test.mjs` validates tool entry-point load timing hook wiring.

‎docs/dev/reports/BUILD_PR_LEVEL_22_1_RUNTIME_OBSERVABILITY_FOUNDATION_VALIDATION.md‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# BUILD_PR_LEVEL_22_1_RUNTIME_OBSERVABILITY_FOUNDATION — Validation
2+
3+
## Commands Executed
4+
1. `node --input-type=module -e "import { run as runHooks } from './tests/runtime/RuntimeMonitoringHooks.test.mjs'; import { run as runLifecycle } from './tests/runtime/Phase19RuntimeLifecycleValidation.test.mjs'; import { run as runObsSurface } from './tests/tools/RuntimeObservabilityFoundation.test.mjs'; runHooks(); runLifecycle(); runObsSurface(); console.log('PASS runtime observability focused suite');"`
5+
2. `node --input-type=module -e "import { run as runToolsIndex } from './tests/tools/ToolsIndexRegistrySmoke.test.mjs'; runToolsIndex(); console.log('PASS tools index registry smoke');"`
6+
3. `node --input-type=module -e "...createRuntimeMonitoringHooks example capture..." > tmp/runtime_observability_examples.json`
7+
8+
## Test Results
9+
- PASS runtime observability focused suite
10+
- PASS tools index registry smoke
11+
12+
## Required Evidence
13+
### Captured Error Example
14+
- `format`: `runtime.monitoring.v1`
15+
- `kind`: `error`
16+
- `hook`: `example.manual`
17+
- `message`: `example boom`
18+
19+
### Logging Output Example
20+
- `format`: `engine.log.v1`
21+
- `level`: `error`
22+
- `event`: `runtime.monitoring.error`
23+
- `message`: `Runtime monitoring captured an error.`
24+
25+
### Performance Timing Example
26+
- `format`: `runtime.monitoring.v1`
27+
- `kind`: `performance`
28+
- `reason`: `load`
29+
- `sample.nowMs`: `42.25`
30+
- `context.loadDurationMs`: `7.5`
31+
32+
## Roadmap Status
33+
- Checked `docs/dev/roadmaps/MASTER_ROADMAP_ENGINE.md` Track B (`runtime error tracking`, `performance monitoring hooks`, `logging standardization`).
34+
- Items were already `[x]` before this PR, so no additional status transition was required.
35+
- No roadmap text rewrites were made.
36+
37+
## Scope Guard Confirmation
38+
- No dashboards introduced.
39+
- No UI expansion introduced.
40+
- No `start_of_day` files changed.

‎docs/dev/reports/change_summary.txt‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
This bundle combines the remaining closely related tools UX polish items into one execution pass:
2-
layout/spacing normalization, docking/resizing consistency, and control placement consistency.
1+
This PR establishes the first real observability layer across tools and runtime,
2+
combining error tracking, performance monitoring, and logging into a validated system.

‎docs/dev/reports/validation_checklist.txt‎

Lines changed: 6 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,8 @@
1-
- [ ] layout/spacing audit completed
2-
- [ ] in-scope tool layouts normalized where feasible
3-
- [ ] docking/resizing matrix completed
4-
- [ ] docking/resizing improved where applicable
5-
- [ ] control placement matrix completed
6-
- [ ] common controls aligned across in-scope tools
7-
- [ ] before/after differences documented
1+
- [ ] runtime errors captured and documented
2+
- [ ] logging outputs follow standard
3+
- [ ] performance timing outputs verified
4+
- [ ] observability matrix completed
5+
- [ ] validation report includes real examples
86
- [ ] no start_of_day changes
97
- [ ] unrelated working-tree changes preserved
10-
- [ ] roadmap updated only for completed items
8+
- [ ] roadmap updated only after validation

‎docs/pr/BUILD_PR_LEVEL_22_1_RUNTIME_OBSERVABILITY_FOUNDATION.md‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
# BUILD_PR_LEVEL_22_1_RUNTIME_OBSERVABILITY_FOUNDATION
2+
3+
## Purpose
4+
Establish a concrete, execution-backed **observability foundation** across tools and runtime:
5+
- runtime error tracking
6+
- performance monitoring hooks
7+
- logging standardization
8+
9+
This PR must produce working hooks + verifiable outputs (not docs-only).
10+
11+
## Scope
12+
Included:
13+
- error capture hooks across tools/runtime entry points
14+
- performance timing hooks (load, render/frame where applicable)
15+
- unified logging standard (levels + format)
16+
- integration points aligned with 21.3 automation
17+
- validation reports proving hooks are active
18+
19+
Excluded:
20+
- dashboards/visual overlays
21+
- feature expansion
22+
- start_of_day changes
23+
- roadmap rewrites
24+
25+
## Roadmap Advancement (only if validated)
26+
### Track B — Stability & Monitoring
27+
- runtime error tracking [ ] -> [x]
28+
- performance monitoring hooks [ ] -> [x]
29+
- logging standardization [ ] -> [x]
30+
31+
## Required Outputs
32+
- docs/dev/reports/BUILD_PR_LEVEL_22_1_RUNTIME_OBSERVABILITY_FOUNDATION_ERROR_TRACKING.md
33+
- docs/dev/reports/BUILD_PR_LEVEL_22_1_RUNTIME_OBSERVABILITY_FOUNDATION_PERFORMANCE_MONITORING.md
34+
- docs/dev/reports/BUILD_PR_LEVEL_22_1_RUNTIME_OBSERVABILITY_FOUNDATION_LOGGING_STANDARD.md
35+
- docs/dev/reports/BUILD_PR_LEVEL_22_1_RUNTIME_OBSERVABILITY_FOUNDATION_OBSERVABILITY_MATRIX.md
36+
- docs/dev/reports/BUILD_PR_LEVEL_22_1_RUNTIME_OBSERVABILITY_FOUNDATION_VALIDATION.md
37+
38+
## Observability Matrix
39+
Map each primary tool + runtime surface to:
40+
- error tracking (yes/no + method)
41+
- logging (level + format)
42+
- performance hooks (type)
43+
44+
## Acceptance
45+
- errors are captured (not swallowed)
46+
- logs follow defined standard
47+
- performance hooks emit measurable data
48+
- outputs are validated and documented
49+
- no unrelated regressions
50+
51+
## Validation Requirements
52+
- demonstrate at least one captured runtime error
53+
- demonstrate logs emitted with correct levels
54+
- demonstrate performance timing output
55+
- confirm integration with automation baseline (21.3)
56+
- no start_of_day changes

‎src/engine/core/Engine.js‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -108,6 +108,7 @@ export default class Engine {
108108
this.fixedStepMs = fixedStepMs;
109109
this.fixedStepSeconds = fixedStepMs / 1000;
110110
this.scene = null;
111+
this.runtimeCreatedAtMs = Date.now();
111112

112113
this.rafId = null;
113114

@@ -225,6 +226,10 @@ export default class Engine {
225226
this.fixedTicker.reset();
226227
try {
227228
this.runtimeMonitoring?.start?.();
229+
this.runtimeMonitoring?.emitPerformanceSample?.('load', {
230+
surface: 'engine.start',
231+
loadDurationMs: Math.max(0, Date.now() - this.runtimeCreatedAtMs),
232+
});
228233
} catch (error) {
229234
this.trackRuntimeError('runtimeMonitoring.start', error, {
230235
severity: 'warn',

0 commit comments

Comments
 (0)