Skip to content
This repository was archived by the owner on Jul 8, 2026. It is now read-only.

feat(cli): find experiments by metric across roles#47

Merged
joalves merged 2 commits into
mainfrom
feat/find-experiments-by-metric
Jun 11, 2026
Merged

feat(cli): find experiments by metric across roles#47
joalves merged 2 commits into
mainfrom
feat/find-experiments-by-metric

Conversation

@joalves

@joalves joalves commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

What

Adds two flags to abs experiments list to find every experiment using a given metric, in any role or a chosen subset:

abs experiments list --metric "Revenue"                       # any role
abs experiments list --metric 42 --metric-role guardrail,primary
abs experiments list --metric 42 --type test,feature --state running -o ids
  • --metric <id|name> — numeric → metric ID (validated; name fetched for the footer); otherwise resolved by exact, case-insensitive name. Ambiguous/missing names error with guidance (archived/draft metrics must be referenced by ID).
  • --metric-role <roles> — comma-separated subset of primary,secondary,guardrail,exploratory; default all four; invalid values error with the allowed list.

How

The API exposes no server-side metric filter (and /metrics/usages returns only aggregate counts), so --metric:

  1. Resolves the metric to an ID/name.
  2. Scans all pages (200/page) honoring the other list filters.
  3. Matches against primary_metric_id and secondary_metrics[].type (a secondary_metrics entry with no type counts as secondary, mirroring existing summary logic).
  4. Tags each match with a metric_role column (e.g. guardrail, secondary) — added as a real field so it respects --show/--exclude/--show-only/-o json.
  5. Paginates the matched set client-side and prints a footer like 12 experiments use metric "Revenue" (#42) — page 1/3.

Without --metric, list behaves exactly as before (single page, untouched path).

Tests

  • metric-filter.test.ts (new): role matching incl. typeless→secondary, role narrowing, multi-page fetch, name resolution (exact / ambiguous / not-found), parseMetricRoles validation.
  • list.test.ts: metric path returns tagged matches, scans at 200/page, client-side pagination, true match total + resolved metric in pagination, appends metric_role to shown fields.
  • pagination.test.ts: printMetricFooter singular/plural, page indicator, json/yaml silence.
  • Full suite green (2615 passed); typecheck, ESLint, Prettier clean.

Includes the 1.12.0 minor bump so the publish workflow ships it.

Summary by CodeRabbit

Release Notes

  • New Features

    • Added metric-based filtering to the experiments list command with --metric option, supporting metric ID or name lookup.
    • Added --metric-role option to further refine results by metric role (primary, secondary, guardrail, exploratory).
    • Enhanced output formatting with metric-specific pagination information.
  • Documentation

    • Added example demonstrating how to filter experiments by metric.

joalves added 2 commits June 11, 2026 22:36
Add --metric <id|name> and --metric-role to `abs experiments list` to
find every experiment using a given metric as primary, secondary,
guardrail, or exploratory (or a chosen subset of roles).

The API has no server-side metric filter, so --metric scans all pages
(200/page) honoring the other filters, matches against primary_metric_id
and secondary_metrics[].type (a typeless entry counts as secondary),
tags each match with a metric_role column, paginates the matched set
client-side, and prints a metric-specific footer. Without --metric,
`list` behaves exactly as before.
@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 45ba4c32-8854-4c0e-a0ca-5cca2e00be47

📥 Commits

Reviewing files that changed from the base of the PR and between aa29093 and fbf866a.

📒 Files selected for processing (10)
  • README.md
  • package.json
  • src/commands/experiments/list.ts
  • src/core/experiments/list.test.ts
  • src/core/experiments/list.ts
  • src/core/experiments/metric-filter.test.ts
  • src/core/experiments/metric-filter.ts
  • src/core/types.ts
  • src/lib/utils/pagination.test.ts
  • src/lib/utils/pagination.ts

Walkthrough

This PR introduces metric-based filtering to the experiments list command. A new metric-filter.ts module provides utilities for parsing metric roles, filtering experiments by specific metrics, and aggregating paginated API results. The listExperiments function is extended to accept metric and metricRole parameters, implementing a client-side filtering path that resolves metric identifiers, fetches all matching experiments across pages, applies role-based filtering, and returns paginated results. The CLI now exposes --metric and --metric-role options, and metric-specific pagination footers are rendered when appropriate. Type definitions for pagination responses are expanded to carry metric and total counts, with corresponding test coverage across filtering logic and API interactions.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Poem

🐰 Hops through metrics with a measuring tape,
Filters experiments in every shape,
Roles parsed and roles arranged,
Experiments matched, pagination staged,
Showing metrics the rabbit's way! 🥕

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 71.43% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'feat(cli): find experiments by metric across roles' accurately and concisely summarizes the main change—adding metric-based filtering to the experiments list command with role support.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/find-experiments-by-metric

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

ESLint install timed out. The project may have too many dependencies for the sandbox.


Comment @coderabbitai help to get the list of available commands and usage tips.

@joalves joalves added this pull request to the merge queue Jun 11, 2026
Merged via the queue into main with commit 3e6dce3 Jun 11, 2026
5 checks passed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant