Skip to content

fix: ignore unrecognized JSON fields in 1.0 JSON-RPC and REST - #1317

Open
Shay-Deng wants to merge 3 commits into
a2aproject:mainfrom
Shay-Deng:fix/ignore-unknown-json-fields
Open

Shay-Deng wants to merge 3 commits into
a2aproject:mainfrom
Shay-Deng:fix/ignore-unknown-json-fields

Conversation

@Shay-Deng

@Shay-Deng Shay-Deng commented Nov 11, 2026 •

Copy link
Copy Markdown

Description

Section 5.7 says implementations SHOULD ignore unrecognized fields, so that peers on different protocol revisions can still talk to each other. The JSON-RPC and HTTP+JSON transports parse protobuf JSON strictly instead, so a SendMessage whose message carries one extra field is rejected:

Invalid request content: tckUnknownField in message lf.a2a.v1.Message. Please verify the request matches the expected schema for this method.

That is TCK requirement DM-SERIAL-005. It shows up in this repo's Run TCK job on main as FAILED REQUIREMENTS: DM-SERIAL-005 (jsonrpc, http_json); the job stays green only because SHOULD-level failures are reported as xfail.

Changes:

  • ProtoJsonUtils.mergeIgnoringUnknownFields / removeUnknownFields remove the keys that name no field of the target message, using its descriptors, and then parse with the strict parser. So unknown field names are ignored, but an unrecognized value for a known field, such as an enum value, is still rejected.
  • JSONRPCUtils.parseRequestBody(JsonElement, ...) uses it. It backs both server request parsing and client response parsing for JSON-RPC.
  • RestHandler parses request bodies with it.
  • SendMessage and SendStreamingMessage go through the same handling, replacing ProtoJsonUtils.retainKnownFields from fix(itk): Fixing ITK issues #1344, which removed only top-level fields.
  • RestTransport and the REST SSEEventListener parse responses and stream events the same way, so a client keeps working when a newer server adds a field.
  • The REST SSEEventListener used to recognize an error event because {"error": {...}} failed to parse as a StreamResponse. With unknown fields ignored it parses to an empty response, so an event with no payload and a top-level error object now goes to RestErrorMapper, as before. An event with neither keeps the existing "Invalid stream response" error. RestTransport maps non-2xx responses before parsing, so it needed no such change.
  • A2ACardResolver already parsed agent cards this way. gRPC already skips unknown fields. The v0.3 compatibility layer is left as it is.

Enum values: protobuf's ignoringUnknownFields() would also drop unrecognized enum values, which for an optional field such as JSON-RPC ListTasks's status turns an invalid filter into no filter. Removing only unknown keys and parsing strictly keeps those values rejected, on requests and on responses.

Tests:

  • JSONRPCUtilsTest.testParseUnknownField_ThrowsJsonMappingException asserted the old rejection. It came in with fix: Sending an invalid message to the server raises an InternalServerError #536, which was about mapping parse errors to the right error code; it now asserts that the unknown fields are ignored. testParseInvalidProtoStructure_ThrowsInvalidParamsJsonMappingException used an unknown field as its invalid structure; it now uses a wrongly typed field, so it still checks the error and the request id.
  • RestHandlerTest.testPushNotificationConfigParseError_DoesNotLogSensitiveData used an unknown field to trigger a parse error. It now uses a wrongly typed field, so it still covers log redaction.
  • New tests for a JSON-RPC request and response, a REST SendMessage request, a REST client response, a REST SSE event with unknown fields, and a REST SSE error event.
  • New tests that an unrecognized enum value is still rejected: JSON-RPC ListTasks status (invalid params), a JSON-RPC response and stream event, a REST client response and a REST SSE event; plus unit tests for ProtoJsonUtils.
  • Three tests from fix(itk): Fixing ITK issues #1344 expected unknown fields to be rejected: a nested field in a JSON-RPC and a REST SendMessage, and an extra GetTask param. They now expect the fields to be ignored; a SendMessage without message is still rejected.

Testing:

  • mvn clean install with tests across the whole project (JDK 21): all modules pass. The first full run caught the SSE error-event regression in QuarkusA2ARest*Test (testSubscribeNoExistingTaskError, testSubscribeToTerminalTaskError, testMainQueueClosesForFinalizedTasks); after the SSEEventListener change, client/transport/rest and every module that depends on it pass.

  • The new tests fail on main and pass with this change.

  • a2a-tck main (104a7b4) against tck/ (mvn quarkus:dev), before and after: the only requirement that changes is DM-SERIAL-005, from FAIL to PASS on both jsonrpc and http_json. The other failures are the same on both runs.

  • After the review follow-up: mvn install for spec-grpc, transport/rest, client/transport/rest, http-client and the modules they depend on (JDK 21): 1510 tests, all pass. The new JSON-RPC enum tests fail on the first commit and pass now.

  • After rebasing on fix(itk): Fixing ITK issues #1344: mvn install for spec-grpc, transport/rest, client/transport/rest, http-client and their dependencies (1578 tests), and for reference/jsonrpc, reference/rest and their dependencies (2440 tests): all pass.

  • Follow the CONTRIBUTING Guide.

  • Make your Pull Request title in the https://www.conventionalcommits.org/ specification.

  • Ensure the tests pass

  • Appropriate READMEs were updated (if necessary)

AI assistance: developed with help from Claude Code. I reviewed the change, and the checks listed under Testing were run locally.

@Shay-Deng
Shay-Deng force-pushed the fix/ignore-unknown-json-fields branch from 00b2441 to 0f82bda Compare November 16, 2026 16:47
@kabir
kabir requested review from jmesnil and removed request for jmesnil November 18, 2026 10:53

@kabir kabir left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Shay-Deng - thank you for noticing this, and the PR!

However, it needs a bit more work. Please adjust this parsing so it ignores unrecognized field names without silently dropping unrecognized values for known enum fields. ignoringUnknownFields() does both.

For example, a JSON-RPC ListTasks request with "status": "NOT_A_TASK_STATE" currently has its status value discarded. Because status is optional, the request can then succeed as if no filter was supplied and return tasks of any status. Before this change, the invalid enum value was rejected as invalid params. The spec’s guidance to ignore unrecognized fields doesn’t appear to require accepting invalid values for recognized fields.

One approach is to remove only unknown JSON properties using the target protobuf message’s field descriptors, then parse with the strict protobuf parser. Another is to validate known enum fields against their descriptors before the lenient parse. Please cover this with tests: unknown extra fields are still accepted, while an unrecognized value for JSON-RPC ListTasks’s status field is rejected. Since this helper is also used to parse responses, please preserve enum validation there too.

@kabir kabir left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See previous comment

@Shay-Deng

Copy link
Copy Markdown
Author

Thanks @kabir, good catch. I went with your first suggestion. ProtoJsonUtils.removeUnknownFields drops only the keys that name no field of the target message (by JSON name or proto name), recursing into nested, repeated and map-valued messages, and the result is then parsed with the strict parser. Well-known types with a special JSON mapping (Struct, Value, Timestamp, ...) are passed through as they are. JSON-RPC request and response parsing, RestHandler, and the REST client (responses and SSE events) all use it, so enum values are validated again on both the request and the response side.

Tests added:

  • JSON-RPC ListTasks with "status": "NOT_A_TASK_STATE" is rejected as invalid params again; with a valid status plus an unknown field, the status filter is kept.
  • An unrecognized enum value is rejected in a JSON-RPC response and stream event, a REST response and a REST SSE event.
  • Unit tests for the helper: nested, repeated and map-valued messages, Struct contents left alone, input not modified, unknown enum rejected.

testSendMessageWrongValueBody is back to its original assertion. The two JSON-RPC enum tests fail on the previous commit and pass with this one. I've updated the description to match.

@kabir

kabir commented Nov 20, 2026

Copy link
Copy Markdown
Collaborator

@Shay-Deng Needs a rebase :-)

Spec Section 5.7 says implementations SHOULD ignore unrecognized fields
for forward compatibility, but JSON-RPC and HTTP+JSON parsed protobuf
JSON strictly, so a request or response with an extra field failed with
"Invalid request content" or a parse error. The TCK reports this as
DM-SERIAL-005 (an expected failure, since the level is SHOULD).

Parse with JsonFormat.parser().ignoringUnknownFields() in the JSON-RPC
request and response path, the REST server handler, and the REST client
and its SSE listener. The agent card resolver already did this. The
v0.3 compatibility layer is unchanged.

The REST SSE listener used to detect error events because they failed
to parse; it now routes an event with no payload and a top-level
"error" object to RestErrorMapper.
ignoringUnknownFields() also drops enum values it does not recognize, so
a JSON-RPC ListTasks with "status": "NOT_A_TASK_STATE" listed tasks of
any status instead of failing with invalid params.

Remove only the keys that name no field of the target message, using its
descriptors, and parse the rest with the strict parser. JSON-RPC request
and response parsing, the REST handler, and the REST client (responses
and SSE events) all go through ProtoJsonUtils.mergeIgnoringUnknownFields
or removeUnknownFields, so values of known fields are checked again.
a2aproject#1344 accepted extra top-level SendMessage parameters through
ProtoJsonUtils.retainKnownFields, but kept nested objects and other
methods strict. Section 5.7 and TCK DM-SERIAL-005 cover unrecognized
fields inside the message as well, so SendMessage and
SendStreamingMessage now use the same descriptor-based removal as every
other method, and retainKnownFields is removed.

A SendMessage without a message is still rejected. The a2aproject#1344 tests that
expected nested extras and an extra GetTask parameter to be rejected now
expect them to be ignored.

The enum tests no longer check the error text, only that the value is
rejected, so they don't depend on how the message is worded.
@Shay-Deng
Shay-Deng force-pushed the fix/ignore-unknown-json-fields branch from 0f587ff to 4c500d1 Compare November 20, 2026 14:39
@Shay-Deng

Copy link
Copy Markdown
Author

Rebased on main. #1344 landed in the meantime with ProtoJsonUtils.retainKnownFields, which accepts extra top-level SendMessage params but keeps nested objects and other methods strict. That overlaps with this PR, so I resolved it in a separate commit (4c500d1) to keep it easy to review:

  • SendMessage / SendStreamingMessage now use the same descriptor-based removal as the other methods, and retainKnownFields is removed. TCK DM-SERIAL-005 puts its unknown field inside message (tckUnknownField), and Section 5.7 doesn't limit this to top-level params, so keeping nested objects strict would leave that requirement failing. The ACTS DM-EXTRA-001 case (top-level x_custom_field) is still accepted.
  • Three assertions from fix(itk): Fixing ITK issues #1344 changed: a nested extra field is now ignored in JSONRPCUtilsTest and RestHandlerTest (both tests renamed), and so is the extra GetTask param in ignoresExtraParametersForOtherMethods. A SendMessage with no message is still rejected on both transports.
  • The enum tests no longer check the error text, so they don't depend on fix(spec-grpc): clarify invalid enum errors #1349's wording. I merged fix(spec-grpc): clarify invalid enum errors #1349 on top locally and spec-grpc passes.

@ehsavoie if nested objects were kept strict on purpose, I can narrow this to top-level fields instead.

@ehsavoie

Copy link
Copy Markdown
Collaborator

@Shay-Deng I didn't want to open the constraints too much, but there was no real reason apart some personal preferences

@Shay-Deng

Copy link
Copy Markdown
Author

Thanks for explaining, @ehsavoie, and for taking a look. Being careful there makes sense. Since Section 5.7 doesn't distinguish nested fields from top-level ones, I'd like to keep them handled the same way, but I'm happy to change it if you'd prefer otherwise.

@ehsavoie
ehsavoie requested a review from kabir November 20, 2026 17:43

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants