shared: dimensions as types - #27
Open
havogt wants to merge 1 commit into
Open
Conversation
havogt
added a commit
that referenced
this pull request
Aug 5, 2026
Part II of `dimension-generic-fields` spelled a staggering shift as `a(I
+ 1/2)`
for `a: Field[Dims[I]]` — naming the dimension the field is *on*. gt4py
as
shipped (#2667, ADR 0026) does the opposite, and the proposal's spelling
is not
a different result there but a hard error:
```
I + 0.5 -> domain(result)=I codomain(field)=_StaggeredI offset=1
IH + 0.5 -> domain(result)=_StaggeredI codomain(field)=I offset=0
a on I: a(IH + 0.5) -> Domain(_StaggeredI=(0:10)) OK
a(I + 0.5) -> ValueError: Incompatible dimensions in the connectivity codomain(s)
```
The implementation is right and the proposal moves.
## Why: a shift is precomposition
A field is a map `f: I → value`, a connectivity a map `φ: D → I`;
applying one
is `f ∘ φ`, a field on `D`. The connectivity's domain is therefore the
*result's* dimension and its codomain the field's — forced by
composition, not
chosen. The index expression is written in the result's index space and
must
evaluate to an index of the field's own dimension, which is exactly what
the
in-tree test comment states: `a(IHalfDim + 0.5) # always pass an I-index
to an
IField`.
The unstructured connectivities already encode this in their names —
`V2E` is
"for each Vertex, its Edges", domain `Vertex`, codomain `Edge`, so it
consumes
an edge field and produces a vertex field. Staggering is merely the
first place
the convention becomes observable, because integer Cartesian shifts have
domain == codomain.
§4.2 records the discarded alternative (source-naming) with its one real
advantage and the reason it loses: it gives no rule for `a(E2C[0])`, so
unstructured shifts would have to become the exception.
## What changed
- **§1, §4.1–§4.3**: the spelling flips (`a(Staggered[I] + 1/2)`), the
position
convention becomes the shipped one (staggered point `i` at `i - 1/2`),
and the
index rule is restated as `ceil` when the shift names an unstaggered
dimension
/ `floor` when it names a staggered one — keyed on the result's
dimension,
which is what `connectivity_for_cartesian_shift` already implements.
- **Prototype** (`typed_dimensions.py`, `static_checks.py`,
`static_errors.py`,
`test_typed_dimensions.py`): overloads, runtime rule and every
expectation
flipped. **19/19 pass, mypy clean, the error oracle fires on exactly the
marked lines.**
- **`dual` is now a typed function.** This is the part that cost more
than
swapping two type arguments: because the shift names the result grid, a
dual-generic body cannot shift on its own dimension and needs
`dual(dim)` —
the type-level counterpart of gt4py's value-level `flip_staggered`. It
gets
the same overload pair, so it stays precise under a dimension variable
(`to_staggered` in `static_checks.py`).
## Three corrections found while verifying
- **§4.4 "backends are unaffected" was wrong.** gtfn needs
`_add_staggered_aliases` (`itir_to_gtfn_ir.py`), emitting the staggered
tag as
a C++ `using` alias of its base tag, because a gtfn shift can offset a
SID but
cannot *rename* its axis. The FOAST bullet also had `source`/`target`
inverted; it now matches the shipped `source=conn.codomain,
target=(conn.domain_dim,)`.
- **The `I + 0.7` gap is mypy-only**, not runtime-only: the DSL already
rejects
it at FOAST time with a `DSLError` and a hint.
- **`check_dims` is not statically expressible** — a dimension and its
staggered
counterpart may not co-occur, but `Field[Dims[I, Staggered[I]]]` stays
writable for mypy. Added as a known gap.
## Incidental
The prototype's `mypy.ini` pinned `python_version = 3.10`, which numpy's
stubs
now reject outright, so both mypy-oracle tests were failing on `main`
for
reasons unrelated to any of this. Bumped to 3.12 (gt4py has dropped
3.10/3.11).
With that fixed, **mypy 2.3 reproduces the metaclass quirk and its
suppression
exactly as documented** — §10 risk 1's "could break on a mypy upgrade"
has not
materialised; pyright remains untested. `.gitignore` gained
`__pycache__/`,
since the vendored prototypes are runnable.
## Note for merge order
Touches `dimension-generic-fields.md` alongside #27 (which stubs out
§3). The
edits are in disjoint sections; the only adjacency is the `>
**Prototypes**:`
block near the top, where #27 inserts a note and this branch edits the
mypy
version on the following line. Whichever lands second may need a trivial
rebase.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Moves the base of the dimension redesign — a concrete dimension is a type, not
an instance — out of the personal investigation and into
shared/, scoped tothat alone.
Why now
Field[Dims[I], float64]is not a valid annotation today (GridTools/gt4py#2503),and the shipped answer is a compatibility shim:
mypy_plugin.py, wired in viapyproject.toml. It caps tracking at four distinct dimensions per run globally(keyed by argument name, so same-named dims in different modules collide), makes
TypeVars over dimensions impossible, serves only mypy, and blurs dtypes in away that erases exactly what dtype generics need. The success criterion for this
proposal is that the dimension half of that plugin gets deleted rather than
maintained.
Scope
In: dimension is a class; value-level behaviour preserved on the class object via
DimensionMeta; old- and new-style dimensions coexist by name-based equality;dimension types usable as
TypeVarbounds.Out, each with its own proposal: staggering syntax and index conventions,
dimension variables, dependent local dimensions,
Dimsordering semantics.The extension point
The two-level split — root
DimensionBase(anything usable inDims[...]) aboveDimension(user-declarable) — exists so parameterized dimension types canoccupy the first without the second. Both known follow-ups need precisely that
slot, and both already have mypy-verified prototypes:
Staggered[D](parameterized by a dimension) and
Local[C](parameterized by a connectivity).Neither is decided here. That is the whole forward-compatibility claim, and it is
narrow enough to check.
On the shipped
_Staggeredname prefixADR 0026's rationale holds under today's model: a dimension is its name, so a
flag would have to be threaded through
common.Dimension,itir.AxisLiteral,and backend tag strings. Dimensions-as-types removes that premise — identity
becomes nominal and the name demotes to what crosses the IR boundary.
Step 1 keeps the prefix and costs nothing:
Staggered[I]materializes withvalue == "_StaggeredI", and every staggering helper reads only.value/.kind(
is_staggered,flip_staggered,as_non_staggered,check_dims,order_dimensions), while gtfn's_add_staggered_aliasesrebuilds aDimensionfrom a string — so none of them change. Established by reading those call sites,
not by running a migration.
Step 2 — dropping the string sniffing — is a separate change behind this surface.
The document is explicit that distinct axes still need distinct identifiers at the
IR boundary, so some mangling survives there; what goes away is
startswith("_Staggered")and the fact that the prefix is user-observable.Also in this PR
personal/havogt/dimension-generic-fieldsloses §3 (replaced by a stub keepingonly what Parts II and III refer back to) so the two documents do not duplicate,
and its suggested ADR numbers move
0024/0025→0027/0028— the originalsare taken by Compilation Runners and Crash Consistent Build Caches.
Open before the group commits
pyright is untested. The metaclass operator overloads rely on a mypy quirk
(def-site rejection, call-site correctness) suppressed with
# type: ignore[misc]. The fallback is checker-agnostic free functions, so thisis a notation risk rather than a design risk — but it should be settled before
adopting the
I + 1/2spelling.