Repository navigation
Conversation
00b2441 to
0f82bda
Compare
kabir
left a comment
There was a problem hiding this comment.
Hi @Shay-Deng - thank you for noticing this, and the PR!
However, it needs a bit more work. Please adjust this parsing so it ignores unrecognized field names without silently dropping unrecognized values for known enum fields. ignoringUnknownFields() does both.
For example, a JSON-RPC ListTasks request with "status": "NOT_A_TASK_STATE" currently has its status value discarded. Because status is optional, the request can then succeed as if no filter was supplied and return tasks of any status. Before this change, the invalid enum value was rejected as invalid params. The spec’s guidance to ignore unrecognized fields doesn’t appear to require accepting invalid values for recognized fields.
One approach is to remove only unknown JSON properties using the target protobuf message’s field descriptors, then parse with the strict protobuf parser. Another is to validate known enum fields against their descriptors before the lenient parse. Please cover this with tests: unknown extra fields are still accepted, while an unrecognized value for JSON-RPC ListTasks’s status field is rejected. Since this helper is also used to parse responses, please preserve enum validation there too.
|
Thanks @kabir, good catch. I went with your first suggestion. Tests added:
|
|
@Shay-Deng Needs a rebase :-) |
Spec Section 5.7 says implementations SHOULD ignore unrecognized fields for forward compatibility, but JSON-RPC and HTTP+JSON parsed protobuf JSON strictly, so a request or response with an extra field failed with "Invalid request content" or a parse error. The TCK reports this as DM-SERIAL-005 (an expected failure, since the level is SHOULD). Parse with JsonFormat.parser().ignoringUnknownFields() in the JSON-RPC request and response path, the REST server handler, and the REST client and its SSE listener. The agent card resolver already did this. The v0.3 compatibility layer is unchanged. The REST SSE listener used to detect error events because they failed to parse; it now routes an event with no payload and a top-level "error" object to RestErrorMapper.
ignoringUnknownFields() also drops enum values it does not recognize, so a JSON-RPC ListTasks with "status": "NOT_A_TASK_STATE" listed tasks of any status instead of failing with invalid params. Remove only the keys that name no field of the target message, using its descriptors, and parse the rest with the strict parser. JSON-RPC request and response parsing, the REST handler, and the REST client (responses and SSE events) all go through ProtoJsonUtils.mergeIgnoringUnknownFields or removeUnknownFields, so values of known fields are checked again.
a2aproject#1344 accepted extra top-level SendMessage parameters through ProtoJsonUtils.retainKnownFields, but kept nested objects and other methods strict. Section 5.7 and TCK DM-SERIAL-005 cover unrecognized fields inside the message as well, so SendMessage and SendStreamingMessage now use the same descriptor-based removal as every other method, and retainKnownFields is removed. A SendMessage without a message is still rejected. The a2aproject#1344 tests that expected nested extras and an extra GetTask parameter to be rejected now expect them to be ignored. The enum tests no longer check the error text, only that the value is rejected, so they don't depend on how the message is worded.
0f587ff to
4c500d1
Compare
|
Rebased on main. #1344 landed in the meantime with
@ehsavoie if nested objects were kept strict on purpose, I can narrow this to top-level fields instead. |
|
@Shay-Deng I didn't want to open the constraints too much, but there was no real reason apart some personal preferences |
|
Thanks for explaining, @ehsavoie, and for taking a look. Being careful there makes sense. Since Section 5.7 doesn't distinguish nested fields from top-level ones, I'd like to keep them handled the same way, but I'm happy to change it if you'd prefer otherwise. |
Description
Section 5.7 says implementations SHOULD ignore unrecognized fields, so that peers on different protocol revisions can still talk to each other. The JSON-RPC and HTTP+JSON transports parse protobuf JSON strictly instead, so a
SendMessagewhose message carries one extra field is rejected:That is TCK requirement
DM-SERIAL-005. It shows up in this repo'sRun TCKjob onmainasFAILED REQUIREMENTS: DM-SERIAL-005 (jsonrpc, http_json); the job stays green only because SHOULD-level failures are reported as xfail.Changes:
ProtoJsonUtils.mergeIgnoringUnknownFields/removeUnknownFieldsremove the keys that name no field of the target message, using its descriptors, and then parse with the strict parser. So unknown field names are ignored, but an unrecognized value for a known field, such as an enum value, is still rejected.JSONRPCUtils.parseRequestBody(JsonElement, ...)uses it. It backs both server request parsing and client response parsing for JSON-RPC.RestHandlerparses request bodies with it.SendMessageandSendStreamingMessagego through the same handling, replacingProtoJsonUtils.retainKnownFieldsfrom fix(itk): Fixing ITK issues #1344, which removed only top-level fields.RestTransportand the RESTSSEEventListenerparse responses and stream events the same way, so a client keeps working when a newer server adds a field.SSEEventListenerused to recognize an error event because{"error": {...}}failed to parse as aStreamResponse. With unknown fields ignored it parses to an empty response, so an event with no payload and a top-levelerrorobject now goes toRestErrorMapper, as before. An event with neither keeps the existing "Invalid stream response" error.RestTransportmaps non-2xx responses before parsing, so it needed no such change.A2ACardResolveralready parsed agent cards this way. gRPC already skips unknown fields. The v0.3 compatibility layer is left as it is.Enum values: protobuf's
ignoringUnknownFields()would also drop unrecognized enum values, which for an optional field such as JSON-RPCListTasks'sstatusturns an invalid filter into no filter. Removing only unknown keys and parsing strictly keeps those values rejected, on requests and on responses.Tests:
JSONRPCUtilsTest.testParseUnknownField_ThrowsJsonMappingExceptionasserted the old rejection. It came in with fix: Sending an invalid message to the server raises an InternalServerError #536, which was about mapping parse errors to the right error code; it now asserts that the unknown fields are ignored.testParseInvalidProtoStructure_ThrowsInvalidParamsJsonMappingExceptionused an unknown field as its invalid structure; it now uses a wrongly typed field, so it still checks the error and the request id.RestHandlerTest.testPushNotificationConfigParseError_DoesNotLogSensitiveDataused an unknown field to trigger a parse error. It now uses a wrongly typed field, so it still covers log redaction.SendMessagerequest, a REST client response, a REST SSE event with unknown fields, and a REST SSE error event.ListTasksstatus(invalid params), a JSON-RPC response and stream event, a REST client response and a REST SSE event; plus unit tests forProtoJsonUtils.SendMessage, and an extraGetTaskparam. They now expect the fields to be ignored; aSendMessagewithoutmessageis still rejected.Testing:
mvn clean installwith tests across the whole project (JDK 21): all modules pass. The first full run caught the SSE error-event regression inQuarkusA2ARest*Test(testSubscribeNoExistingTaskError,testSubscribeToTerminalTaskError,testMainQueueClosesForFinalizedTasks); after theSSEEventListenerchange,client/transport/restand every module that depends on it pass.The new tests fail on
mainand pass with this change.a2a-tck
main(104a7b4) againsttck/(mvn quarkus:dev), before and after: the only requirement that changes isDM-SERIAL-005, from FAIL to PASS on bothjsonrpcandhttp_json. The other failures are the same on both runs.After the review follow-up:
mvn installforspec-grpc,transport/rest,client/transport/rest,http-clientand the modules they depend on (JDK 21): 1510 tests, all pass. The new JSON-RPC enum tests fail on the first commit and pass now.After rebasing on fix(itk): Fixing ITK issues #1344:
mvn installforspec-grpc,transport/rest,client/transport/rest,http-clientand their dependencies (1578 tests), and forreference/jsonrpc,reference/restand their dependencies (2440 tests): all pass.Follow the
CONTRIBUTINGGuide.Make your Pull Request title in the https://www.conventionalcommits.org/ specification.
Ensure the tests pass
Appropriate READMEs were updated (if necessary)
AI assistance: developed with help from Claude Code. I reviewed the change, and the checks listed under Testing were run locally.