Add validated API 3 world warps and cross-engine contracts

.github/workflows/api3-contract.yml

@@ -0,0 +1,56 @@
+name: API 3 contracts
+on:
+  pull_request:
+    paths:
+      - '.github/workflows/api3-contract.yml'
+      - 'src/**'
+      - 'tests/api3_contract/**'
+      - 'tests/engine/api3_*.lua'
+      - 'tools/api3-contract-check/**'
+  push:
+    branches: [main, dev]
+    paths:
+      - '.github/workflows/api3-contract.yml'
+      - 'src/**'
+      - 'tests/api3_contract/**'
+      - 'tests/engine/api3_*.lua'
+      - 'tools/api3-contract-check/**'
+permissions:
+  contents: read
+concurrency:
+  group: api3-contract-${{ github.ref }}
+  cancel-in-progress: true
+jobs:
+  compare:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v7
+        with:
+          path: gen2
+          sparse-checkout: |
+            src
+            data
+            tests
+            tools/api3-contract-check
+      - uses: actions/checkout@v7
+        with:
+          repository: bryanthaboi/gen1recomp
+          ref: 79ea62f45fb2ea8f65d081a109431e4eb1bbdb36
+          path: gen1-reference
+          sparse-checkout: |
+            src
+            data
+            tests
+            assets/fonts
+      - name: install LuaJIT
+        run: sudo apt-get update && sudo apt-get install -y luajit
+      - name: identical mod contracts on both engines
+        run: >-
+          python3 gen2/tools/api3-contract-check/check.py
+          --gen1 gen1-reference --gen2 gen2
+          --report "${{ runner.temp }}/api3-report.json"
+      - uses: actions/upload-artifact@v7
+        with:
+          name: api3-contract-report
+          path: ${{ runner.temp }}/api3-report.json
+          if-no-files-found: error

docs/api3-portability.md

@@ -0,0 +1,150 @@
+# Shared API 3 entry
+
+Use the same manifest and entry file on Gen1Recomp and Gen2Recomped:
+
+```json
+{
+  "id": "portable_example",
+  "name": "Portable Example",
+  "version": "1.0.0",
+  "api": 3,
+  "games": ["gen1", "gen2", "gen3"],
+  "entry": "main.lua"
+}
+```
+
+```lua
+local mod = ...
+if mod.content.moves:has("TACKLE") then
+  local ok, reason = mod.content.moves:patch("TACKLE", {power = 45})
+  if not ok then mod.log:warn("%s", reason) end
+end
+```
+
+Create a portable starter with:
+
+```sh
+python3 tools/modkit.py scaffold portable_example --api 3
+```
+
+It generates the shared games list and a guarded canonical move patch. It
+omits an engine-release range because Gen1 and Gen2 have independent engine
+version numbers. The default scaffold remains API 2 with its existing range
+and profile templates. The `--api` value is now honored instead of silently
+emitting API 2 for every value.
+
+API 3 automatically selects unified content, callbacks and portable scripts.
+No `content_format`, `callback_format` or `script_format` fields are needed.
+An explicit `"native"` for one of those fields is an error under API 3;
+`"unified"` is accepted. API 1/2 retain native defaults, and API 2 retains
+its existing individual opt-ins. The engine advertises `Version.modApi = 3`.
+A higher unknown API remains a load error.
+
+This is a common entry and a tested compatibility subset, not a claim that
+all of Gen1Recomp's services have been ported. It builds on PRs #90–#99.
+The existing unified-content limitations still apply: unsupported writes
+return `nil, reason` before recording an operation, native diagnostics are
+read-only, and ordinary schema errors raise an attributed load error.
+Entry failure rolls back the mod's content and subscriptions. Content is
+frozen after loading.
+
+## Public contracts
+
+Core `pokemon`, `moves`, `items`, `trainers` and `encounters` handles use the
+existing unified adapters. Other registries retain their native record
+shapes, rather than inventing canonical fields. Every content handle exposes
+`has(id)`, `supports()` and `available`. API-3 passthrough reads and iteration
+return detached values too; change data through the write methods.
+
+The API-3 Pokemon boundary uses short statuses (`SLP`, `PSN`, `BRN`, `FRZ`,
+`PAR`, `TOX`), nickname-first display names and a common Special alias.
+Writing `stats.special` in a writable callback sets both later Special stats;
+contradictory split values are rejected before commit. API-2 unified status
+names and callback behavior are unchanged. DVs/shiny presentation is read-only
+in the callback boundary. Party and battle reads remain detached snapshots.
+
+Capture `mod.world` and `mod.battle` at entry time. World methods can be called
+before Game exists and return `nil, reason`; reading the facade never boots
+Game. The same world handle follows the loader's current Game/save instead of
+retaining an earlier one. Battle snapshots and intents retain their existing
+revision/replay validation.
+
+The common world calls currently enabled for generations 1–3 are:
+
+- `overworld`, `current`, `party`, `activeBlockAt`, `liveMaps`, `mapOverview`,
+  `effectiveEncounters`;
+- `canReorderParty`, `reorderParty`, `availableFieldActions`, `useFieldAction`,
+  `canFly`, `flyTo`;
+- `getFlag`/`setFlag` for private `mod:<key>` boolean flags;
+- `queueScript` for the existing bounded portable script language;
+- [`warpTo`](api3-warp.md) for validated native map transitions.
+
+Check `mod:supports("world", name)` before relying on a call. The other
+Gen1 world method names remain callable but explicitly return
+`nil, "unsupported: ..."`: object/block mutations, runtime NPC operations,
+`startWildBattle` and `invalidateMap` have not yet been verified against the
+Gen1 API-3 contract. Their native/API-2 counterparts are unaffected. Native
+symbolic flag mappings and `mod.datasets:open` are also explicitly unavailable.
+Gen4/5 do not claim the shared world surface in this first entry PR.
+
+Private flags use `save.modData[modId].worldFlags`, independently of native
+flags and other mods. A read does not create a bucket. Clearing removes the
+stored key. A change emits `flag.changed` with
+`name = "mod.<modId>.<key>"` and the boolean value; setting the same value
+emits nothing. Invalid save stores are refused without overwriting them.
+
+Gen1 API-3 callback names from the pinned reference are recognized even when
+this engine has no call site for them. Registration is allowed; availability
+still requires a native call site in the selected version. For example,
+`render.output` can be subscribed to but reports unavailable here. Unknown
+names remain errors. An accepted subscription is not proof that a feature
+runs. Source-specific engine handles in lifecycle/UI payloads remain native.
+
+For scripts, use `mod:supports("scripts", verb)` to test Gen2Recomped's current
+bounded vocabulary. A `queueScript` method does not imply support for every
+Gen1 script verb. See [portable world scripts](portable-world-scripts.md).
+
+## Reproducible contract checks
+
+```sh
+luajit tests/engine/api3_entry.lua
+luajit tests/engine/api3_compat.lua
+python3 tools/api3-contract-check/check.py \
+  --gen1 /path/to/clean/pinned/gen1recomp --lua luajit \
+  --report /tmp/api3-contract-report.json
+```
+
+The external reference must be a clean checkout at
+`bryanthaboi/gen1recomp@79ea62f45fb2ea8f65d081a109431e4eb1bbdb36`.
+Use a separate checkout; do not reset a working installation. No Gen1 engine
+implementation is vendored into this repository. Only public callback names
+are recorded as a compatibility catalog.
+
+The exact same manifest and mod entry run on both engines for Red, Blue,
+Yellow, Gold, Silver, Crystal, FireRed and Emerald, in both success and
+intentional-rollback cases. Each engine/version/case has its own LuaJIT
+process. The shared harness constructs native-shaped synthetic datasets;
+the mod fixtures contain no engine imports or engine-specific branches.
+The runner also generates an API-3 mod through the real scaffold CLI and
+loads those same generated files on both engines for each version. This adds
+eight comparisons to the sixteen normal/rollback cases. Eight additional warp
+cases use the same public mod on both engines and native-shaped timing seams:
+32 comparisons and 64 isolated loader runs. It compares JSON projections of public results and
+records hashes of both shared and generated fixture files. It refuses a different or modified reference.
+
+Coverage includes canonical content reads/writes/iteration and native merge
+results, detached core and passthrough reads, callable early handles, later
+Game replacement, party fields, private flags, Gen1 battle snapshot/invalid-intent boundaries, callback payloads and native
+writeback, custom events, unsupported-name handling, frozen content and failed
+owner cleanup. Warp cases also compare validation errors, native dispatch
+arguments, destination events, boundary coordinates, arrival options, callback
+argument holes/replay, stale sessions/owners, synchronous completion and error
+attribution. Separate Gen2 tests check API-1/2 behavior, contradictory
+manifest fields, honest availability and malformed flag stores. CI pins the
+same external commit and uploads the report.
+
+These are ROM-free loader/runtime-boundary tests, not retail-ROM gameplay
+or a proof of every exposed native callback's payload. Native action details
+remain covered by the prerequisite PR suites. Extend these shared fixtures
+whenever another API-3 service becomes portable; do not make a failing
+comparison pass by changing only one engine's mod fixture.

docs/api3-warp.md

@@ -0,0 +1,130 @@
+# Portable API-3 world warps
+
+API-3 mods can use the same public call on Gen1Recomp and Gen2Recomped:
+
+```lua
+local mod = ...
+local world = mod.world -- Safe to retain at entry; does not create Game.
+
+-- Call later, from an idle overworld interaction/update.
+local function goToDestination(mapId, cellX, cellY)
+  if not mod:supports("world", "warpTo") then return end
+  local accepted, reason = world:warpTo(mapId, cellX, cellY, "down", {
+    keepMusic = true,
+    onDone = function()
+      mod.log:info("Arrived")
+    end,
+  })
+  if not accepted then mod.log:warn("Warp declined: %s", reason) end
+end
+```
+
+`warpTo(mapId, x, y, facing, options)` returns `true` after native admission,
+or `nil, reason`. Admission does not mean arrival. It uses the existing native
+`OverworldState:startWarpTo` transition, map-entry events, spawn bookkeeping
+and arrival-tile guard. No direct player/save relocation bypasses that path.
+This API-3 adapter does not change API-1/2 facade defaults or validation.
+Generations 1–3 advertise support; API 3 still refuses this world operation on
+Gen4/5. Availability is a capability, not a promise that the current world is
+idle or the named map exists. Reading the facade or querying support does not
+bootstrap Game.
+
+## Admission and coordinates
+
+The current Game must have a save and a live, topmost overworld. Screens,
+native/portable scripts, queued/parallel scripts, movement, transitions,
+field/controller locks, link sessions and another pending API-3 warp decline.
+Gen2Recomped reuses its existing world-action admission guard and checks its
+additional native foreground owners. Idle-state details differ between the
+engines; both keep their own native readiness checks.
+
+Map IDs are nonempty strings of at most 256 bytes, without control characters.
+The map must exist in the current merged `game.data.maps`; custom merged maps
+are supported. Gen3 also accepts unprefixed names when the current version's
+prefixed map exists (`FR_`, `EM_`, `RU_`, `SA_`). An exact ID wins over an alias.
+There is no cross-game map-name translation: the mod selects a real map in
+the active dataset.
+
+`x` and `y` are finite, nonnegative whole numbers in zero-based **16-pixel
+walk cells**. Gen1/2 dimensions are twice the map's block dimensions; Gen3
+uses the map's metatile dimensions directly. The final valid cell is
+`widthInCells - 1, heightInCells - 1`. Invalid/missing map dimensions refuse
+before dispatch. Coordinates are bounded, not clamped; terrain passability is
+not part of this contract.
+
+Facing is `up`, `down`, `left` or `right`. Omitted facing follows Gen1's common
+contract: `down` on generation 1, current player facing on generations 2/3,
+with `down` as fallback. Explicit facing is recommended for portable scenes.
+
+Options must be a plain table. Only these fields are accepted:
+
+| Field | Meaning |
+| --- | --- |
+| `arrive` | `"fly"` or `"teleport"`, using native arrival presentation |
+| `keepMusic` | Boolean; forwarded to native map loading |
+| `onDone` | Function, called at most once after accepted native completion |
+| `fade` | Boolean, generation 3 only; false uses the same native midpoint/completion without pushing a fade screen |
+
+Unknown options, invalid types and unsupported `fade` use decline the whole
+request before native dispatch. Options are copied; later caller edits cannot
+change an accepted request. Invalid admission does not set arrival effects,
+change maps/coordinates/saves, raise map-entry events or call `onDone`.
+Representative reasons include `no game`, `no overworld`, `no save`,
+`world is busy`, `invalid map id`, `unknown map: <id>`, `invalid coordinates`,
+`coordinates out of bounds`, `invalid facing`, `invalid options`,
+`invalid arrival`, `invalid callback`, `unsupported fade` and
+`unknown warp option: <key>`. These are admission failures, not exceptions.
+Native failures after dispatch can already have native effects; there is no
+transactional rollback of an entered transition.
+
+## Callback ownership
+
+`onDone` runs only for an accepted request whose Game, save, data, stack,
+overworld, player, version, loader buses and mod lease are still current, at
+the requested destination. It preserves the native callback arguments,
+including nil holes. Synchronous native completion is held until the native
+return confirms admission. A rejected/throwing native dispatch cannot invoke
+the mod callback, and duplicate native completions are ignored.
+
+A replaced session/owner observed at input-step boundaries, another map entry
+or leaving the destination retires the completion permanently. Returning to an
+older session cannot revive a completion already retired. Loader rollback,
+reload and teardown retire the facade and its pending callback; a retained
+facade returns `nil, "warp facade is retired"`. Retirement does not pop screens
+or undo work belonging to the native controller.
+
+The callback is attributed to its mod, errors enter the loader's runtime error
+list once, and the previous runtime owner is restored. An error never retries
+a warp or the callback. A completion may start a new request after the native
+world becomes idle. The adapter adds no synthetic map-entry event; normal
+native map lifecycle remains in control.
+
+The portable `queueScript` language is unchanged: adding this direct method
+does not enable a `warp` script row.
+
+## Verification
+
+```sh
+luajit tests/engine/api3_warp.lua
+python3 tools/api3-contract-check/check.py \
+  --gen1 /path/to/clean/pinned/gen1recomp --lua luajit \
+  --report /tmp/api3-contract-report.json
+```
+
+The existing Gen1 pin remains
+`79ea62f45fb2ea8f65d081a109431e4eb1bbdb36`. The same manifest/entry is loaded by
+both real loaders for Red, Blue, Yellow, Gold, Silver, Crystal, FireRed and
+Emerald. The shared driver builds each engine's native-shaped Game/world data
+and replaces only native transition timing seams. It compares validation
+reasons, dispatch arguments, public destination events and callback results.
+The mod has no engine imports or backend-specific branches. Together with the
+existing normal, rollback and generated-starter cases, CI now performs 32
+comparisons / 64 isolated loader runs.
+
+The local suite additionally drives Gen2Recomped's real native
+`startWarpTo` and `Transition` using synthetic map materialization, including
+fade/no-fade, music options, native bookkeeping, lock release and one-shot
+completion. Further cases cover host-specific locks, malformed maps, callback
+reentry, native refusal/exception, lease retirement and API-2 behavior. These
+are ROM-free runtime-boundary tests; no retail-ROM playthrough or rendered
+arrival-effect equivalence is claimed.