feat(appkit): mlflow tracing for agents (stack 1/5)#477
Open
MarioCadenas wants to merge 1 commit into
Open
Conversation
Trace agent turns and tool calls to MLflow via the mlflow-tracing SDK. Adds an optional 'experiment' resource to the agents plugin; when bound (MLFLOW_EXPERIMENT_ID), each turn opens an AGENT span and each tool call a nested TOOL span, with auth resolved from the app's Databricks credentials. A turn's trace can be linked to an evaluation run via mlflow.sourceRun. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack 1/5 · targets
main.Adds MLflow tracing to the agents plugin. Agent turns and tool calls are traced to a bound MLflow experiment via the
mlflow-tracingSDK (OpenTelemetry under the hood). Tracing is a no-op unless the plugin's optionalexperimentresource is set (MLFLOW_EXPERIMENT_ID); auth is resolved from the app's own Databricks credentials, so no tokens or OTLP headers are wired by hand.withAgentSpanwraps each turn (AGENT span) and tool dispatch (TOOL span, auto-nested).mlflow.traceNametag.experimentoptional resource in the agents manifest.This is the base of a 5-PR stack that builds out an agent evaluation framework. Reviewable on its own — touches only the agents plugin.