Repository navigation
Enable gradient explanations in client - #34
JGagacheva wants to merge 31 commits into
Conversation
ef58795 to
87c751f
Compare
Allow the explanation output for kumo-tabular (one scored row), return it on result.explanation, and drop the old 1,000-row context rule for relational explanations, which the SDM NIM does not have. Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
The architecture table of what each model declares still marked `explanation` as absent for kumo-tabular, and a session test still said the endpoint only explains under the 'fast' run mode. Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
87c751f to
78b0a5e
Compare
Every scored row is explained, on both models, and result.explanations maps each row's key (id, (id, forecast_step), or row_index) to its entry. The docs, skills and changelog drop the one-row limit and result.explanation, describe the entry shape, related rows with keys and values, preprocessed columns and to_pandas() with hop, and name the server settings a NIM needs to explain (SDM_NUM_ESTIMATORS=1, and CUDA graph replay off on kumo-tabular), which otherwise answer 422. probe_tabular.py probes a multi-row explanation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
The NIM now explains every predict row, one explanation per record, so the client no longer refuses an explanation over more than one row: explanation_predict_rows leaves TABULAR_LIMITS and RELATIONAL_LIMITS, and the tabular row check and the relational one-entity check go with it. Tests that asserted those refusals are removed or now assert the rows are sent, and the skills limits table no longer maps a row to the removed limit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
result.explanation kept the first record's explanation and dropped the
rest, and a forecast gathered its steps under {'by_forecast_step': ...}.
Both are replaced by result.explanations, a mapping from the key that
names a row in to_pandas() -- its id, (id, forecast_step) for a
forecast, or row_index when the request has no entity key -- to that
row's explanation, kept out of the frame and empty when none was asked
for.
Each entry is read beside the payload of the call that produced it, so
a split request's positions, which restart at 0 in every call, are
translated before the calls are merged. The instance block drops its
row_indices and its gradients become [output][column]. A related row
is given back as its primary key, when the table has one of its own,
and always as its values as sent; kumo-tabular entries have no related
key. Explanations.to_pandas() flattens every gradient to one line, with
each related row's hop from the entity worked out from the
relationships and key values sent. Explanations is exported, since
result.explanations hands it out.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
The client refused an explanation whose format was not sdm_gradient_v1 with INVALID_RESPONSE. Both models answer in that one format, so the check guarded nothing; the details are now read directly. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
kumo-relational sends the entity as two tables: the per-prediction instance row (entity key, anchor time, engine bookkeeping) and the entity's own feature row as a related table. The docs put the entity's features under instance; they are under related['users'], and to_pandas() shows the entity on an instance line and a users line, both hop 0. The example tables now name each output by its result column (probability_yes), and say what key and hop hold on an instance line and for an unreached row. The relational probe for an explanation over several entities expected a refusal before the wire; it now checks one entry per scored entity, keyed by id, with related tables. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Tabular: every scored row is explained under its id, or its row_index when the request has no key; the instance gradients are the row's own, [output][column]; an entry has no related key; outputs and columns are kept as returned. Relational, on payloads shaped as the engine gathers them and answers shaped as the NIM user guide gives them: related positions are read as that call's rows, a keyed table's keys come back as their declared dtype (a key past 2**53 included) while rows keep the values as sent, a keyless (__node_id) table has no keys, and gradients line up with rows. A request split into calls that each number from 0 keeps every entity's own rows, and row_index keys move with their rows. A forecast is keyed (id, forecast_step). Explanations.to_pandas(): its columns, one line per (row, table, related row, output, column), hop 0/1/2 through a two-hop graph and empty for an unlinked row, and outputs named as the frame's columns. The NIM explanation builder is a shared conftest fixture, and tabular_request can build a request with no key column. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
The parity case expected a kumo-relational NIM to refuse an explanation with UNSUPPORTED_OUTPUT, which a NIM that explains every row no longer does. It is removed; the in-process suite now checks that a model run in this process refuses 'explanation' for both models. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
The explanation caps they were named after are gone; the tests still check that a gathered payload is sent in one call. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
A kumo-relational entry is only outputs and related; the entity's own table sits under related at hop 0, and the flat table has no instance line for it. The relational probe checks for that. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
The relational engine sends each entity as a question row in instance_table and the entity's own row in related_tables. The NIM explains both, but the question row carries no caller columns, so a kumo-relational entry now has only 'outputs' and 'related', where the entity's own table sits at hop 0. kumo-tabular entries keep 'instance' and have no 'related'. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
kumo-tabular: several rows in one call are each explained under their id, or their row_index without a key column, with only 'outputs' and 'instance', gradients [output][column], and a flat table of the row's own block at hop 0. kumo-relational: several entities, in one call and split across calls that each number from 0, are each explained under their id with only 'outputs' and 'related'. The entity's own table is among the related ones at hop 0, each keyed row carries its key and the caller's values, and gradients line up with the rows. The skills test that expected the relational endpoint to refuse an explanation now expects one per entity in that shape. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
The live fixture graph had only keyed tables, so the keyless case of the explanation contract (rows without keys, no __node_id, null key in the flat table) was never checked against a NIM. Add events, linked to users.user_id with no primary key, and check its blocks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Rename the result class Explanations to GradientExplanations, and move explanations_of and the relational adapter's related-row helpers into one GradientDecoder in wire/explanations.py, built once per call with the call's payload (kumo-relational) or none (kumo-tabular). No behaviour change. The public API snapshot is regenerated for the rename. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
The kumo-relational NIM sends a related table's primary key only when a relationship joins on it, so orders blocks carry rows without keys, and it now serves explanation for every relational target. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Both were public in 1.0.0. result.explanation returns the first record's raw explanation, or None, and PredictionResult(explanation=...) is accepted again; each raises DeprecationWarning naming result.explanations and removal in 2.0.0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
P6 was stale (in-process supports explanation since #34), P3 named only the entry points, P5 was silent on forecasts, D1 named a file that does not exist. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
P6 was stale (in-process supports explanation since #34), P3 named only the entry points, P5 was silent on forecasts, D1 named a file that does not exist. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Jana Gagacheva <jgagacheva@nvidia.com>
| return GradientExplanations() | ||
| ids = [item.id for item in explained] | ||
| unique = len(set(ids)) == len(ids) | ||
| by_id = unique and any(key is not None for key in ids) |
There was a problem hiding this comment.
P2: Missing IDs omit row positions from flattened explanations
Question: Should explanations fall back to row-index keys when any scored row lacks an ID? With accepted IDs [None, 90], one explanation is keyed by None and the flattened frame has no row_index column, so callers cannot join that row by position directly. The documentation does not specify mixed null IDs; using row-index keys would preserve that join.
| entries: dict[Any, dict[str, Any]] = {} | ||
| hops: dict[Any, dict[str, list[int | None]]] = {} | ||
| for item in explained: | ||
| key = item.id if by_id else item.row_index |
There was a problem hiding this comment.
P2: Native forecasts overwrite earlier normalized explanations
When a materialized forecast receives several steps for one row, result.explanations keeps only the last step. Predictions and raw item explanations survive, but normalized access cannot retrieve earlier steps. Include forecast_step in the explanation keys and position metadata, and preserve those pairs when renumbering and merging batches.
Manushpm8
left a comment
There was a problem hiding this comment.
All three earlier comments look fixed, thanks. One blocker: 9 commits lack a Signed-off-by, and AGENTS.md says unsigned PRs can't merge; git rebase --signoff main fixes it.
Summary
Enable gradient explanations for kumo-tabular and kumo-relational, one per scored row.
explanationinoutputsis accepted on both models with no row limit. The one-row/one-entity checks and the old 1,000-row context rule for relational explanations are removed; the SDM NIM(structured-data-nims
0a8230d) explains every predict row.result.explanationsmaps each row's key (id;(id, forecast_step)for forecasts;row_indexwhen the request has no id column) to its explanation, kept out ofto_pandas().Replaces
result.explanationand{'by_forecast_step': ...}.sdm_gradient_v1record with row positions resolved: kumo-tabular gives{'outputs', 'instance'}; kumo-relational gives{'outputs', 'related'}with one block pertable (the entity's own included), each row carrying its primary key when the table has one and
its values as sent. The engine's question-row block is dropped. Positions are resolved per call,
so split requests are correct.
result.explanations.to_pandas()flattens to one line per (row, table, hop, key, row, output,column, gradient);
hopis computed from the relationships and keys sent.(
-m live_nim) for both models pass.API
Relational
As one table
result.explanations.to_pandas()gives one line per (row, output, column), across all entities:Tabular