Skip to content

Instrument skill handlers and dialog rendering - #509

Open
goldyfruit wants to merge 4 commits into
OpenVoiceOS:devfrom
goldyfruit:perf/runtime-handler-metrics
Open

Instrument skill handlers and dialog rendering#509
goldyfruit wants to merge 4 commits into
OpenVoiceOS:devfrom
goldyfruit:perf/runtime-handler-metrics

Conversation

@goldyfruit

@goldyfruit goldyfruit commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

What changed

  • measure registered skill-handler execution without counting internal bus housekeeping callbacks
  • measure Mustache dialog rendering in speak_dialog and get_response retry paths
  • publish the fixed-cardinality collectors through ovos.performance.metrics
  • document that handler, service, and rendering timings are nested rather than additive

Architecture and scope

Workshop owns these skill-level observations; ovos-core remains the sole HTTP exporter owner. No session, skill, utterance, request, or other unbounded value is exported as a label.

The earlier reply-tracing experiment was removed from the final diff. Direct-emitting skills can bypass a skill-mixin hook, and generic request IDs are not an OVOS message contract. This PR is metrics-only.

Jarbas's shared-helper review is tracked explicitly: OpenVoiceOS/ovos-utils#416 adds the dependency-free monotonic histogram primitive. The intended release order is to review and release #416, then replace this PR's local helper and bump the minimum ovos-utils version. Workshop will not import Core and create a dependency cycle, and no fallback or getattr will hide package skew.

Why

The HiveMind bridge metrics stop before OVOS skill execution. Handler and dialog boundaries expose the remaining user-visible work while preserving message, handler, and rendering semantics.

Validation

Current head 9efd827ee6b3f29d31407f950f74019a46ca66a8:

  • focused metrics tests: passed
  • broad unit run: 550 passed; two intent-layer setup errors reproduce unchanged on untouched dev in the same reused environment
  • Ruff on touched Python files: passed
  • git diff --check: passed
  • GitHub Python 3.10–3.14, lint, audit, release-preview, and CodeRabbit checks: green
  • no CI workflow changed

@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Adds process-local latency histograms for skill-handler execution and dialog rendering. Instruments eligible handlers and rendering paths, registers the metrics provider, adds runtime tests, and documents histogram semantics and exposure.

Changes

Runtime performance metrics

Layer / File(s) Summary
Histogram provider and metric exposure
ovos_workshop/_metrics.py, pyproject.toml, docs/performance-metrics.md, docs/index.md
Adds thread-safe fixed-bucket histograms, JSON-friendly snapshots, predefined metrics, provider registration, and metric documentation.
Runtime measurement and validation
ovos_workshop/skills/ovos.py, test/unittests/test_runtime_metrics.py
Measures eligible skill handlers and dialog rendering, preserves handler metadata, excludes internal events, and tests the new measurements. Related debug and trailing-content cleanup is included.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant EventSystem
  participant OVOSSkill
  participant SKILL_HANDLER
  participant DIALOG_RENDER
  EventSystem->>OVOSSkill: dispatch handler with handler_info
  OVOSSkill->>SKILL_HANDLER: measure handler execution
  SKILL_HANDLER-->>OVOSSkill: record duration
  OVOSSkill->>DIALOG_RENDER: measure dialog rendering
  DIALOG_RENDER-->>OVOSSkill: record duration
Loading
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 43.48% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main changes: instrumenting skill-handler execution and dialog rendering.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@goldyfruit

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Comment thread ovos_workshop/_metrics.py
)


class LatencyHistogram:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have some doubts on this one, maybe we need metrics helpers in ovos-utils to use across all other packages? we have the StopWatch class in there already.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed—the exporter is already provider/entry-point driven, but the low-level monotonic histogram storage should be shared instead of copied. I extracted that dependency-free primitive to OpenVoiceOS/ovos-utils#416. Workshop cannot import Core without reversing the dependency direction, and I do not want to hide the coordination behind a fallback/getattr. The intended order is: review and release #416, then replace this PR’s local helper and bump the explicit ovos-utils minimum. Metric ownership stays in Workshop; scrape/export policy stays in the host runtime. I am leaving this thread unresolved until that prerequisite and consumer rebase are complete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants