Skip to content

fix(core)!: MapLiteral cannot represent a map with repeated keys #1081

Description

@nielspardon

Substrait's Expression.Literal.Map is a repeated KeyValue key_values with no uniqueness constraint, so a literal map that repeats a key is a valid plan. Expression.MapLiteral holds its pairs in a Map<Literal, Literal>, which cannot represent one.

Proto import is worse than lossy. ProtoExpressionConverter builds the map with Collectors.toMap, which throws IllegalStateException: Duplicate key when a plan repeats a key, so a valid plan fails to import rather than round-tripping imperfectly. The HashMap that toMap collects into also discards the order the producer emitted the pairs in, so key_values comes back reordered even when the keys are distinct. EmptyMapLiteral is unaffected — it carries only the key and value types.

Expression.NestedMap was given an ordered List<NestedMap.KeyValue> for exactly this reason (#1062), and SqlMapValueConstructorCallConverter in isthmus now refuses to collapse a repeated-key MAP[...] into a MapLiteral for the same reason (#1063). MapLiteral is the remaining place where the limitation lives.

Two ways out:

  • Replace values() with an ordered list of literal key/value pairs, mirroring NestedMap. This is the representation that matches the proto, but it is a breaking change to a public API that predates this repo's current shape, and it touches every consumer of MapLiteral — hence keeping it out of feat(core)!: support nested map expressions and import nested structs #1062.
  • Keep the Map and make the limitation explicit: import fails with a message naming the duplicate key instead of a bare IllegalStateException, and the map is collected into a LinkedHashMap so distinct-key plans at least keep their pair order.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions