KAFKA-20850: Test suppress changelog upgrade and downgrade

[lh0156]

What

Add system-test coverage for the headers-aware suppress() changelog across the 4.3/4.4 boundary.

• Allow Streams smoke job runners to receive additional Streams properties.
• Configure dsl.store.format=HEADERS for the new coverage.
• Exercise both LATEST_4_3 -> DEV_VERSION and DEV_VERSION -> LATEST_4_3 full restarts.
• Extract the shared application transition setup so the existing upgrade matrix keeps the same behavior.

The test keeps the existing suppress() topology and continuously generates the smoke workload while all three processors are restarted. Each processor must rejoin, process records after the restart, and close cleanly; a changelog restore failure surfaces as a startup, processing, or shutdown failure. This test intentionally does not claim a separate final output-topic comparison because the driver runs in disableAutoTerminate mode during the transition.

Testing

• RED: a pre-change contract check failed because the HEADERS property and bidirectional test were absent.
• GREEN: python3 -m py_compile tests/kafkatest/services/streams.py tests/kafkatest/tests/streams/streams_application_upgrade_test.py.
• GREEN: service property rendering confirmed dsl.store.format=HEADERS.
• ./gradlew :streams:test --tests org.apache.kafka.streams.DslStoreFormatTest :streams:compileTestFixturesJava :streams:spotlessCheck passed.
• git diff --check passed.

The Ducktape system test itself was not executed locally because this environment has no Kafka system-test cluster. No API, protocol, or KIP changes.

Reviewers: Alieh Saeedi asaeedi@confluent.io

[github-actions]

A label of 'needs-attention' was automatically added to this PR in order to raise the
attention of the committers. Once this issue has been triaged, the triage label
should be removed to prevent this automation from happening again.

[aliehsaeedii]

Thanks for picking this up — I'm closing this in favour of #23075.
Note that this ticket was already assigned to me and actively in progress, so please check the assignee and comment before opening a PR next time.

The substantive issue: 4.3 has the dsl.store.format config but not the headers-aware suppress buffer, so both sides tag records V3 while writing different layouts — meaning a downgrade mismatch is silently misread, not rejected, and "rejoins, processes, closes cleanly" won't catch it. That plus the Ducktape test never having been run is why I'm taking it from here; #23075 adds an explicit restore-rejection check and is green on both directions.