🔴 Required Information
Is your feature request related to a specific problem?
adk conformance test --mode live has remained unimplemented for about a year since its introduction in v1.15.0 (September 2025).
Every discovered test case fails with “Live mode is not implemented yet.”
https://adk.dev/evaluate/#evaluate-with-conformance-testing
Live Mode: Runs evaluation-based verification against active environments (Note: This mode is a work in progress).
It is also unclear when to use adk conformance test --mode replay instead of adk test (since v2.0.0).
Describe the Solution You'd Like
Implement live execution and evaluation for conformance test cases, with pass/fail results and report generation.
Please also clarify the intended use of replay mode versus adk test in the documentation.
Impact on your work
I cannot use conformance tests to check behavior against live model responses, and the distinction between the two replay-style commands makes it hard to choose a testing workflow.
Willingness to contribute
No
🔴 Required Information
Is your feature request related to a specific problem?
adk conformance test --mode livehas remained unimplemented for about a year since its introduction in v1.15.0 (September 2025).Every discovered test case fails with “Live mode is not implemented yet.”
https://adk.dev/evaluate/#evaluate-with-conformance-testing
It is also unclear when to use
adk conformance test --mode replayinstead ofadk test(since v2.0.0).Describe the Solution You'd Like
Implement live execution and evaluation for conformance test cases, with pass/fail results and report generation.
Please also clarify the intended use of replay mode versus
adk testin the documentation.Impact on your work
I cannot use conformance tests to check behavior against live model responses, and the distinction between the two replay-style commands makes it hard to choose a testing workflow.
Willingness to contribute
No