Skip to content

Query on a single-attribute key schema applies a condition on a non-key attribute to the sort key #674

Open

Description

@LeeroyHannigan

On a key schema with one HASH and one RANGE attribute, the engine checks only that the key condition has an equality on the HASH attribute. A second clause on any other attribute reaches storage as the sort key condition, and the SQLite and PostgreSQL backends apply a sort key condition that names no RANGE attribute to the first RANGE column (crates/storage-sqlite/src/data/query_scan.rs:79 and crates/storage-postgres/src/data/query_scan.rs:162 on main; lines 90 and 190 on #429, which keeps the fallback). The query returns items whose sort key matches the value, and a value whose type differs from the sort key's answers 500.

Reproduction

main at 3051e50, dev builds with the SQLite and the PostgreSQL backend; both answer the same. Table key pk (S) and sk (N). GSI gsi with HASH h (S) and RANGE a (S), projection ALL. Two items:

  • pk "p1", sk 4, h "o1", a "x", zz 9
  • pk "p1", sk 5, h "o1", a "p1", zz 4
request answer
table, pk = :p AND zz = :v, :p "p1", :v N 4 200, one item: sk 4, zz 9
table, pk = :p AND zz = :v, :p "p1", :v S "x" 500 InternalServerError
gsi, h = :h AND pk = :v, :h "o1", :v S "p1" 200, one item: a "p1", sk 5
gsi, h = :h AND sk = :v, :h "o1", :v N 4 500 InternalServerError
gsi, h = :h AND sk = :v, :h "o1", :v S "x" 200, one item: a "x", sk 4

The first request filters on sk: it returns the item whose sk is 4, and the item whose zz is 4 has sk 5. The third filters on a, the index's RANGE attribute, and returns one of the two items with pk "p1". The last applies sk = "x" to a.

#429 adds a value type check on this path that reads the table's attribute definitions. On the #429 branch the last request answers 400 One or more parameter values were invalid: Condition parameter type does not match schema type, because the table defines sk as N, and the other four answer as on main. A clause on an attribute the table does not define, or one whose value has the defined type, still reaches storage. Once this path refuses non-key conditions, its type check can read the queried key schema's definitions only, as #429 does on a multi-attribute key schema.

Expected

ValidationException with Query condition missed key schema element: <RANGE attribute>, before the request reaches storage, whatever the value's type. Measured on the service on 2026-10-30 on a GSI with HASH GSI2PK (S) and RANGE GSI2SK (S) of a table keyed on PK (S): GSI2PK = :h AND PK = :k answers Query condition missed key schema element: GSI2SK with :k an S and with :k an N, two samples each. #489 has the base table form with a comparison, pk = :x AND z > :x, measured on 2026-10-23: the service answers Query condition missed key schema element: sk. On #429's branch the multi-attribute path refuses such a clause this way, naming the first RANGE attribute without a condition, and the service answers the same on multi-attribute key schemas (three shapes, 2026-10-30). The base table form with an equality (pk = :p AND zz = :v), a table with no RANGE attribute, and conditions with three or more clauses on a single-attribute key schema, where the service's condition count rule may answer first, were not measured and should be before the fix.

Activity

  1. LeeroyHannigan commented on Nov 1, 2026

    @LeeroyHannigan
    CollaboratorAuthor

    Measured on the service on 1 November 2026 at about 13:18 UTC, on a table keyed on pk (S) with no RANGE attribute, two samples each:

    request answer
    pk = :p AND ssn = :s 400 ValidationException: The number of query conditions (2) exceeds the number of key attributes defined in the schema (1)
    pk = :p AND #s = :s, #s = ssn the same

    So on a table with no RANGE attribute the condition count rule answers. A SQLite build with main's Query code answered the first request with 200 and the item whose ssn matched.

  2. LeeroyHannigan commented on Nov 10, 2026

    @LeeroyHannigan
    CollaboratorAuthor

    Measured on the service on 10 November 2026 in us-east-1, two sends each. The service answers every shape below with 400 ValidationException.

    main at e681b8b, SQLite backend, dev build, replayed with the same expressions and values. Each shape got two sends, plus one send whose values make the second clause false. Table A has the key pk (S), a GSI with the key g (S), and the items p1 (x v1, y w1, g v1), p2 (x v2, y w2, g v1) and p3 (g v3). Table B has the key pk (S), sk (S) and the items (p1, s1, x v1), (p1, s2, x v2) and (p1, v1, x zz).

    table key KeyConditionExpression service main at e681b8b, SQLite
    pk pk = :p AND #n = :v AND #m = :w, #n = x, #m = y, :p p1, :v v1, :w w1 The number of query conditions (3) exceeds the number of key attributes defined in the schema (1) 200, no item, while p1 has x v1 and y w1
    pk pk = :p AND begins_with(#n, :v), #n = x, :p p1, :v v The number of query conditions (2) exceeds the number of key attributes defined in the schema (1) 200, item p1; with :p p3, item p3, which has no x
    pk, GSI on g pk = :p AND #g = :v, #g = g, :p p1, :v v1, sent to the table the same message with (2) and (1) 200, item p1; with :p p3, item p3, whose g is v3
    pk, sk pk = :p AND sk = :s AND #n = :v, #n = x, :p p1, :s s1, :v v1 The number of query conditions (3) exceeds the number of key attributes defined in the schema (2) 200, no item, while (p1, s1) has x v1
    pk, sk pk = :p AND #n = :v, #n = x, :p p1, :v v1 Query condition missed key schema element: sk 200, the item whose sk is v1 (x zz); with :v s2, the item whose sk is s2

    Rows 1 and 4 are conditions with three clauses on a single-attribute key schema. The last row is the base table form with an equality. The description of this issue lists these shapes as not measured on the service.

    The code path on main. On a key schema with one HASH and at most one RANGE attribute, handle_query makes one check on the parsed condition. The clause the parser took as the partition key must name the HASH attribute (crates/engine/src/query.rs lines 329 to 336). Nothing checks the other clauses. With three or more clauses, classify_multi takes the first equality as the partition key clause and the last clause as the sort key condition. The equalities between them go to extra_pk_conditions (crates/core/src/expression/key_condition.rs lines 716 to 765). resolve_key_schema_with reclassifies those on a multi-attribute key schema only (lines 399 to 424). On these tables they reach storage. The SQLite backend encodes the partition key value and every extra_pk_conditions value as one composite key text (crates/storage-sqlite/src/data/query_scan.rs lines 55 to 63). No row of a table with one HASH attribute has that key. That is why the two three-clause shapes return no item. A sort key condition that names no RANGE attribute applies to the first RANGE column (lines 90 to 92). A table with no RANGE attribute has no such column, so the two-clause shapes on A return the partition's item whatever the second clause says. The PostgreSQL backend builds the key and the sort key condition the same way (crates/storage-postgres/src/data/query_scan.rs lines 147 to 158 and 190 to 192). This replay used SQLite only.

    Expected: the condition count message. On a table with one key attribute, a KeyConditionExpression with two or more clauses answers it with the clause count and (1). Three clauses on a table with two key attributes answer it with (3) and (2). pk = :p AND #n = :v on a table with a RANGE attribute answers Query condition missed key schema element: sk. Each answer comes before the request reaches storage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions