Experimental Avro/Protobuf/Arrow reference encoders + PubSub adapters

[marcschier]

Summary

Adds two experimental class libraries to the stack that implement the draft OPC UA DataEncoding extensions (Avro binary, Protobuf, Apache Arrow) as hand-rolled reference encoder/decoders, plus PubSub NetworkMessage adapters, with NUnit round-trip tests. These are the C# reference implementation of the spec drafts in the marcschier/opcua-drafts repo.

New projects

• Libraries/Opc.Ua.Core.Experimental — IEncoder/IDecoder implementations:
  • Avro — streaming Avro binary (zig-zag varint, little-endian floats, nullable unions, NodeId-by-parts), full Variant/Matrix support, plus Action invoke/response + Discovery. SchemaId = CRC-64-AVRO (Rabin) fingerprint with Avro single-object prefix.
  • Protobuf — hand-rolled protobuf wire format; scoped to gRPC service messages only.
  • Arrow — genuinely typed Apache Arrow (Apache.Arrow 18.1.0): typed arrays, StructArray, ListArray, dense UnionArray, FixedSizeBinaryArray. No JSON/base64/blob carrier (asserted by tests).
• Libraries/Opc.Ua.PubSub.Experimental — INetworkMessageEncoder/Decoder:
  • Avro adapter drives the real AvroEncoder per field (self-describing Avro-in-Avro bytes).
  • Arrow adapter emits a real columnar RecordBatch (rows = DataSetMessages, typed per-field + leading header columns; NetworkMessage headers in schema metadata) per the Part 14 Arrow batch mapping (ADBC/historian oriented).
• Test projects for both, Apache.Arrow 18.1.0 added to central package management, four projects added to UA.slnx.

Target frameworks / CI

• The experimental libraries target net8.0;net9.0;net10.0 only (they rely on Apache.Arrow and net5+ BCL APIs such as BitConverter.SingleToUInt32Bits, Convert.ToHexString, Stream.ReadExactly). They opt into RestrictForLegacyTfm so the per-TFM CI matrix entries pinned to legacy TFMs (net472/net48/netstandard2.x) no-op them (verified locally).
• Verified: both libraries build 0 warnings / 0 errors on net8.0/net9.0/net10.0; Opc.Ua.Core.Experimental.Tests 17/17 and Opc.Ua.PubSub.Experimental.Tests 3/3 pass on net8.0 and net10.0; legacy CustomTestTarget builds no-op cleanly.

Conformance / scope

• Round-trip-only conformance for this first reference (encode → decode reproduces the input), including null vs empty vs present-zero, nullable scalars, arrays, matrices (dimensions + values), all four NodeId kinds, Variant/ExtensionObject/DataValue/DiagnosticInfo, and float NaN/±0/±Inf bit preservation.
• Out-of-scope / unsupported decode paths fail loud (NotSupportedException) rather than corrupt.

Code-review fixes included

• Avro (critical): a present-empty array previously emitted a spurious Avro block terminator that desynced the following field — and, because every 1-D array Variant carries a present-empty dimensions array, such Variants decoded as Null. Fixed in AvroEncoder.WriteArray + a dedicated multi-field regression test.
• Arrow: dense-union declared type-ids now include the written code so external Arrow readers (pyarrow/ADBC) can map Variant/DataValue union slots; XmlElement now round-trips instead of being silently dropped.
• Removed a dead/misleading tautology guard in the Arrow PubSub adapter.

Known limitation

• Protobuf optional presence: the reference decoder reconstructs optional-field presence by probing field names against the wire; combined with positional field numbering this is unreliable for structures with two or more optional fields where an earlier optional is absent. The in-scope gRPC service messages do not use such structures. A general fix requires an explicit on-wire encoding mask (documented in ProtobufDecoder).

Companion to the encoding spec drafts in marcschier/opcua-drafts (PR #4).