feat(cli): find experiments by metric across roles#47
Conversation
Add --metric <id|name> and --metric-role to `abs experiments list` to find every experiment using a given metric as primary, secondary, guardrail, or exploratory (or a chosen subset of roles). The API has no server-side metric filter, so --metric scans all pages (200/page) honoring the other filters, matches against primary_metric_id and secondary_metrics[].type (a typeless entry counts as secondary), tags each match with a metric_role column, paginates the matched set client-side, and prints a metric-specific footer. Without --metric, `list` behaves exactly as before.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (10)
WalkthroughThis PR introduces metric-based filtering to the experiments list command. A new Estimated code review effort🎯 3 (Moderate) | ⏱️ ~22 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
ESLint install timed out. The project may have too many dependencies for the sandbox. Comment |
What
Adds two flags to
abs experiments listto find every experiment using a given metric, in any role or a chosen subset:--metric <id|name>— numeric → metric ID (validated; name fetched for the footer); otherwise resolved by exact, case-insensitive name. Ambiguous/missing names error with guidance (archived/draft metrics must be referenced by ID).--metric-role <roles>— comma-separated subset ofprimary,secondary,guardrail,exploratory; default all four; invalid values error with the allowed list.How
The API exposes no server-side metric filter (and
/metrics/usagesreturns only aggregate counts), so--metric:listfilters.primary_metric_idandsecondary_metrics[].type(asecondary_metricsentry with notypecounts assecondary, mirroring existing summary logic).metric_rolecolumn (e.g.guardrail, secondary) — added as a real field so it respects--show/--exclude/--show-only/-o json.12 experiments use metric "Revenue" (#42) — page 1/3.Without
--metric,listbehaves exactly as before (single page, untouched path).Tests
metric-filter.test.ts(new): role matching incl. typeless→secondary, role narrowing, multi-page fetch, name resolution (exact / ambiguous / not-found),parseMetricRolesvalidation.list.test.ts: metric path returns tagged matches, scans at 200/page, client-side pagination, true match total + resolved metric in pagination, appendsmetric_roleto shown fields.pagination.test.ts:printMetricFootersingular/plural, page indicator, json/yaml silence.Includes the
1.12.0minor bump so the publish workflow ships it.Summary by CodeRabbit
Release Notes
New Features
--metricoption, supporting metric ID or name lookup.--metric-roleoption to further refine results by metric role (primary, secondary, guardrail, exploratory).Documentation