WIP: proposal: add info-metric label discovery APIs for info() autocomplete#85
WIP: proposal: add info-metric label discovery APIs for info() autocomplete#85aknuds1 wants to merge 3 commits into
Conversation
ab1b9ec to
6bf4a84
Compare
| | `start`, `end` | rfc3339 / unix_timestamp | No | last 1h | As per PROM-74. | | ||
| | `limit` | int ≥ 0 | No | 100, capped by `--web.search.max-limit` | Maximum number of label NAMES returned. Same `has_more` semantics as PROM-74. | | ||
| | `values_limit` | int ≥ 0 | No | 0 (no cap) | Maximum number of values returned per label. New to this endpoint. | | ||
| | `batch_size` | int > 0 | No | 100 | As per PROM-74. | |
There was a problem hiding this comment.
What does a batch size limit in this API? Label names or label values?
|
|
||
| Constructing the same response with the endpoints that exist today requires either fetching far more than needed or multiple round-trips per autocomplete suggestion. | ||
|
|
||
| * **`/api/v1/labels` + per-name `/api/v1/label/{name}/values`** returns *all* labels matching a selector, including labels carried by the base metric, not just data labels on info metrics. Filtering client-side requires downloading the full universe of labels and then making one `/label/{name}/values` call per label of interest (N+1). |
There was a problem hiding this comment.
This works for the existing autocomplete implementations for PromQL, why doesn't it work for the info function?
| * The response is NDJSON (`application/x-ndjson`) with the same batch + trailer contract as PROM-74. | ||
| * The endpoint is dual-gated: `--enable-feature=search-api` covers the NDJSON + parsing infrastructure it reuses; `--enable-feature=promql-experimental-functions` covers the only consumer (`info()`). Either missing flag returns the standard Prometheus JSON error with `errorType: unavailable` and a flag-specific message. | ||
|
|
||
| ### `GET|POST /api/v1/info_labels` |
There was a problem hiding this comment.
Given that this new endpoint leverages params and NDJSON from the search endpoint, perhaps this should be /api/v1/search/info_labels
There was a problem hiding this comment.
I thought about this. The (subjective) reason for not going that route was that info function autocomplete endpoint is not in the same logical family. I'm open to rethinking this, however.
| | `start`, `end` | rfc3339 / unix_timestamp | No | last 1h | As per PROM-74. | | ||
| | `limit` | int ≥ 0 | No | 100, capped by `--web.search.max-limit` | Maximum number of label NAMES returned. Same `has_more` semantics as PROM-74. | | ||
| | `values_limit` | int ≥ 0 | No | 0 (no cap) | Maximum number of values returned per label. New to this endpoint. | | ||
| | `batch_size` | int > 0 | No | 100 | As per PROM-74. | |
There was a problem hiding this comment.
Note that I believe we added a max batch size ceiling. 10,000?
|
|
||
| ### 1. Extend `/api/v1/search/label_names` to optionally return values per name | ||
|
|
||
| Would collapse two endpoints into one. Rejected on cohesion grounds: PROM-74 keeps names and values in separate endpoints precisely so that each endpoint's response shape stays simple. The values payload is only useful when the names are scoped to info metrics, so the coupling does not belong on the general search endpoint. |
There was a problem hiding this comment.
I would not dismiss this option so quickly. Part of the reason for the search results format being objects/maps per record was so that additional data could be augmented with metric names, label names or label values.
The idea of extending the response format is noted https://github.com/prometheus/proposals/blob/main/proposals/0074-new-labels-values-api.md#extensibility-for-mimir-thanos-cortex.
This endpoint could be extended to include a "include_values=true&values_limit=10" and the values collections could be decorated into each label record.
We already have the "only useful in a certain context" problem on the metric_names endpoint. It supports the include_metadata=true to decorate in metric metadata records for each metric name. But this only can decorate metric names where we have metadata. It's acceptable that this part of the response may not be there.
There was a problem hiding this comment.
Thanks for the feedback @tcp13equals2, will consider.
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
Document the request profiles and client requirements learned from the Grafana info() autocomplete proof of concept. Clarify expression interpolation, response bounds, and terminal NDJSON handling. Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
|
I've implemented a PoC in From PoC stand point it's working nicely. But we should challenge the return value data type (instead of returning all potential labels/values we might return labels and then values for the selected label. In simple terms I'd like to follow current |
1cb8122 to
8a7dc22
Compare
aknuds1
changed the title
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
8a7dc22 to
ebb9e87
Compare
4 participants
Add a proposal for two top-level info-metric label discovery endpoints that support autocomplete for the second argument of the PromQL
info()function:GET|POST /api/v1/info_labelssearches info data-label names.GET|POST /api/v1/info_label_valuessearches values for one exact data-label name.Both endpoints share the same scope: an optional instant-vector
expr, repeated fullmetric_match[]matchers on__name__, and repeated fulldata_match[]matchers. Expression-derived storage selection follows the same lookback,offset, and@semantics as an instantinfo()evaluation.The proposal reuses the experimental search API storage and NDJSON contracts while keeping these function-specific operations at top-level paths. It specifies separate name/value limits, pre-search bounds on expression-derived matcher construction, strict stream completion, and the dual
search-api,promql-experimental-functionsfeature gate.The contract has been validated in both the Prometheus UI implementation and the Grafana Prometheus datasource integration.