compat(M7): align Gen2 registry field contracts and native validation

.github/workflows/api3-contract.yml

@@ -0,0 +1,60 @@
+name: API 3 contracts
+on:
+  pull_request:
+    paths:
+      - '.github/workflows/api3-contract.yml'
+      - 'src/**'
+      - 'tests/api3_contract/**'
+      - 'tests/engine/api3_*.lua'
+      - 'tools/api3-contract-check/**'
+  push:
+    branches: [main, dev]
+    paths:
+      - '.github/workflows/api3-contract.yml'
+      - 'src/**'
+      - 'tests/api3_contract/**'
+      - 'tests/engine/api3_*.lua'
+      - 'tools/api3-contract-check/**'
+permissions:
+  contents: read
+concurrency:
+  group: api3-contract-${{ github.ref }}
+  cancel-in-progress: true
+jobs:
+  compare:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        reference: [79ea62f45fb2ea8f65d081a109431e4eb1bbdb36, 535a5ec088c1ae1bf4d91dd3e35a56f64cf46d91]
+    steps:
+      - uses: actions/checkout@v7
+        with:
+          path: gen2
+          sparse-checkout: |
+            src
+            data
+            tests
+            tools/api3-contract-check
+      - uses: actions/checkout@v7
+        with:
+          repository: bryanthaboi/gen1recomp
+          ref: ${{ matrix.reference }}
+          path: gen1-reference
+          sparse-checkout: |
+            src
+            data
+            tests
+            assets/fonts
+      - name: install LuaJIT
+        run: sudo apt-get update && sudo apt-get install -y luajit
+      - name: identical mod contracts on both engines
+        run: >-
+          python3 gen2/tools/api3-contract-check/check.py
+          --gen1 gen1-reference --gen2 gen2
+          --report "${{ runner.temp }}/api3-report.json"
+      - uses: actions/upload-artifact@v7
+        with:
+          name: api3-contract-report-${{ matrix.reference }}
+          path: ${{ runner.temp }}/api3-report.json
+          if-no-files-found: error

.gitignore

@@ -178,3 +178,10 @@ mobile/ios/bundle_id.local
 # secrets as public -- but GitHub's secret scanning flags it, so it is kept out
 # of the repository and injected into release builds instead.
 /cloud_clients.lua
+
+# Owned compatibility fixtures; external source remains outside this checkout.
+!/tests/fixtures/
+/tests/fixtures/*
+!/tests/fixtures/upstream_mods/
+!/tests/fixtures/legacy_npc_controls/
+!/tests/engine/

docs/api3-npc.md

@@ -0,0 +1,142 @@
+# API-3 public NPC actions
+
+Generations 1–3 support the same public creation, lookup and removal calls as
+Gen1Recomp. Check availability and retain the world facade at mod entry:
+
+```lua
+local mod = ...
+local world = mod.world
+
+local function greet(mapId, spriteId)
+  if not mod:supports("world", "spawnNpc") then return end
+  local id, why = world:spawnNpc(mapId, {
+    name = "GREETER", sprite = spriteId, x = 3, y = 4, facing = "down",
+  })
+  if not id then return nil, why end
+  local npc, reason = world:npc(mapId, id)
+  if not npc then return nil, reason end
+  return npc:scriptMove("down", 2, function()
+    npc:face("left")
+  end)
+end
+```
+
+Invoke this later from an idle world. Reading the facade never bootstraps Game.
+Gen4/5 report these three calls unavailable. API-1/2 retain their native behavior;
+this does not add portable `queueScript` verbs.
+
+## Creation, lookup and ownership
+
+- `spawnNpc(mapId, definition)` returns the native runtime ID or `nil, reason`.
+  It can add a runtime definition to a known inactive map; an actor appears when
+  the native map lifecycle instantiates it. The return value promises a recorded
+  definition, not immediate live lookup or a common lifecycle event order.
+- `npc(mapId, reference)` returns an opaque handle or `nil, reason`. References
+  are the live actor's string ID, its definition's numeric index or its name.
+  Only the active map and currently live neighboring map actors can be queried.
+- `removeNpc(id)` returns `true` or `nil, reason`. Only runtime objects owned by
+  the calling mod may be removed. Imported objects and other owners are refused.
+  A mod may still look up and control an imported live actor.
+
+Map, sprite and object identifiers remain dataset-specific. Gen3 recognizes the
+current version's `FR_`, `EM_`, `RU_` or `SA_` map prefix when an exact map ID is
+absent. IDs/names are nonempty strings of at most 256 bytes without control
+characters; numeric references are finite nonnegative integers. Use the ID
+returned by `spawnNpc` for removal.
+
+Definitions are copied before native construction. Required `x` and `y` are
+whole nonnegative walk-cell coordinates, at most 1,000,000. Optional `facing` is
+`up`, `down`, `left` or `right`; optional `name` and `sprite` are identifiers.
+A supplied sprite must exist in the active sprite registry. Native generation
+fields remain supported as data; this API does not invent a canonical sprite,
+movement or NPC definition schema. The host assigns runtime ownership and
+identity, ignoring caller-supplied ownership/index markers.
+
+The copy accepts plain acyclic tables, strings, finite numbers and booleans.
+Functions, userdata, metatables and cycles are refused before native writes.
+Limits are depth 16, 4096 visited values/keys, 256 entries per table, 4096 bytes
+per string and 64 KiB total string bytes. Refusal returns `nil, reason`.
+
+## Handles
+
+Handles expose `id` and these methods; they never expose the native actor:
+
+| Method | Result and behavior |
+| --- | --- |
+| `position()` | `x, y` in zero-based 16-pixel walk cells, in the actor's own map frame. |
+| `isMoving()` | Boolean native movement state. |
+| `canStep(direction)` | Boolean native collision permission, including the host's current seam cast for connected maps. It is a query, not a reservation. |
+| `face(direction)` | Set facing. |
+| `stepNow(direction)` | Begin one authored step; refuse an already moving actor. |
+| `placeAt(x, y, facing?)` | Place at whole nonnegative coordinates up to 1,000,000, clear in-flight movement and synchronize pixel position. |
+| `setPassable(boolean)` | Set the live actor's passability. |
+| `setAppearance(spriteId)` | Refresh a live actor using an existing sprite registry entry. |
+| `scriptMove(direction, tiles?, onDone?)` | Queue native movement, default one cell; `tiles` must be an integer from 1 to 1024. |
+| `marchInPlace(onDone?)` | Queue one native stationary march. |
+
+Directions are the four lowercase names above. Mutations return `true` or
+`nil, reason`; a stale handle returns `nil, "NPC handle is stale"`. Handles are
+bound to the original actor, map instance, Game, save, data and version. Reused
+native IDs or editing the public `id` field cannot redirect a retained handle.
+The facade follows the current Game, while old actor handles do not.
+
+Spawn, removal and handle mutations use the shared API-3 idle-world guard:
+a save and topmost overworld are required; menus, native/portable/parallel
+scripts, player movement, controller locks, transitions and link sessions refuse
+admission. Reads remain usable during overlays. Synchronous reentry into NPC or
+object/block mutations is refused. Movement tracks require the active map;
+other handle calls also work for live neighboring actors.
+
+`stepNow` and `placeAt` are authored operations, matching the native distinction
+from `canStep`: they do not automatically perform collision validation or clamp
+to map bounds. Choose coordinates and query collision as appropriate for the
+scene. Appearance changes construct a renderer before committing the actor
+change, retain its cell position and leave the native template sprite intact;
+they are not persistent content edits.
+
+## Movement and mod lifetime
+
+Completion callbacks are optional, one-shot and attributed to the initiating
+mod. Native synchronous completion is deferred until admission finishes and
+the shared mutation guard is released. Native callback arguments, including
+nil holes, are preserved. Callback errors are contained and reported once.
+
+Pending completions are permanently retired on observed Game/save/data/world,
+map, stack, version, actor or runtime-bus replacement, and on native map lifecycle
+events. The input-step boundary observes replacements before and after native
+updates. Retiring removes only the movement entries owned by that operation;
+it does not pop screens or cancel another mod's tracks.
+
+Rollback, reload and teardown retire the mod's facade and pending tracks and
+remove its recorded runtime definitions/actors, including definitions spawned
+on inactive maps and instantiated later as neighbors. Foreign runtime objects
+and imported actors remain. Controls already applied to imported actors are
+ordinary native scene changes, not a transactional patch with automatic undo.
+Retained handles/facades return `nil, "world facade is retired"` after retirement.
+
+## Verification
+
+```sh
+luajit tests/engine/api3_npc.lua
+python3 tools/api3-contract-check/check.py \
+  --gen1 /path/to/clean/pinned/gen1recomp --lua luajit \
+  --report /tmp/api3-contract-report.json
+```
+
+The clean Gen1 reference stays pinned to
+`79ea62f45fb2ea8f65d081a109431e4eb1bbdb36`. The same public-only NPC mod runs
+through both production loaders for Red, Blue, Yellow, Gold, Silver, Crystal,
+FireRed and Emerald. Its eight cases extend the full checker to 48 comparisons
+and 96 isolated runs. The shared fixture compares creation/lookup, detached
+definitions, collision/passability, appearance, authored/queued movement,
+callback replay/arguments, optional callbacks and ownership-aware removal.
+
+Local tests additionally cover bounded validation, busy worlds, reentry, stale
+sessions/handles, callback errors, lifecycle cancellation and scoped cleanup.
+They exercise the host's actual NPC constructor, native runtime add/remove,
+scriptMove/marchInPlace/updateScriptMoves, NPC updates and seam collision casts.
+Synthetic datasets and a deterministic SpriteRenderer construction seam keep
+these ROM-free. The tests verify the documented shared subset, not retail-ROM
+playthroughs, GPU artwork or identical native internals/event timing. Host
+validation and lifecycle guards are intentionally stricter than some reference
+native paths.

docs/api3-portability.md

@@ -0,0 +1,158 @@
+# Shared API 3 entry
+
+Use the same manifest and entry file on Gen1Recomp and Gen2Recomped:
+
+```json
+{
+  "id": "portable_example",
+  "name": "Portable Example",
+  "version": "1.0.0",
+  "api": 3,
+  "games": ["gen1", "gen2", "gen3"],
+  "entry": "main.lua"
+}
+```
+
+```lua
+local mod = ...
+if mod.content.moves:has("TACKLE") then
+  local ok, reason = mod.content.moves:patch("TACKLE", {power = 45})
+  if not ok then mod.log:warn("%s", reason) end
+end
+```
+
+Create a portable starter with:
+
+```sh
+python3 tools/modkit.py scaffold portable_example --api 3
+```
+
+It generates the shared games list and a guarded canonical move patch. It
+omits an engine-release range because Gen1 and Gen2 have independent engine
+version numbers. The default scaffold remains API 2 with its existing range
+and profile templates. The `--api` value is now honored instead of silently
+emitting API 2 for every value.
+
+API 3 automatically selects unified content, callbacks and portable scripts.
+No `content_format`, `callback_format` or `script_format` fields are needed.
+An explicit `"native"` for one of those fields is an error under API 3;
+`"unified"` is accepted. API 1/2 retain native defaults, and API 2 retains
+its existing individual opt-ins. The engine advertises `Version.modApi = 3`.
+A higher unknown API remains a load error.
+
+This is a common entry and a tested compatibility subset, not a claim that
+all of Gen1Recomp's services have been ported. It builds on PRs #90–#99.
+The existing unified-content limitations still apply: unsupported writes
+return `nil, reason` before recording an operation, native diagnostics are
+read-only, and ordinary schema errors raise an attributed load error.
+Entry failure rolls back the mod's content and subscriptions. Content is
+frozen after loading.
+
+## Public contracts
+
+Core `pokemon`, `moves`, `items`, `trainers` and `encounters` handles use the
+existing unified adapters. Other registries retain their native record
+shapes, rather than inventing canonical fields. Every content handle exposes
+`has(id)`, `supports()` and `available`. API-3 passthrough reads and iteration
+return detached values too; change data through the write methods.
+
+The API-3 Pokemon boundary uses short statuses (`SLP`, `PSN`, `BRN`, `FRZ`,
+`PAR`, `TOX`), nickname-first display names and a common Special alias.
+Writing `stats.special` in a writable callback sets both later Special stats;
+contradictory split values are rejected before commit. API-2 unified status
+names and callback behavior are unchanged. DVs/shiny presentation is read-only
+in the callback boundary. Party and battle reads remain detached snapshots.
+
+Capture `mod.world` and `mod.battle` at entry time. World methods can be called
+before Game exists and return `nil, reason`; reading the facade never boots
+Game. The same world handle follows the loader's current Game/save instead of
+retaining an earlier one. Battle snapshots and intents retain their existing
+revision/replay validation.
+
+The common world calls currently enabled for generations 1–3 are:
+
+- `overworld`, `current`, `party`, `activeBlockAt`, `liveMaps`, `mapOverview`,
+  `effectiveEncounters`;
+- `canReorderParty`, `reorderParty`, `availableFieldActions`, `useFieldAction`,
+  `canFly`, `flyTo`;
+- `getFlag`/`setFlag` for private `mod:<key>` boolean flags;
+- `queueScript` for the existing bounded portable script language;
+- [`warpTo`](api3-warp.md) for validated native map transitions;
+- [`toggleObject` and `replaceBlock`](api3-world-edits.md) for validated native
+  visibility and active block/metatile edits;
+- [`spawnNpc`, `removeNpc` and `npc`](api3-npc.md) for runtime NPC creation,
+  ownership-aware removal and opaque live actor controls.
+
+Check `mod:supports("world", name)` before relying on a call. The other
+Gen1 world method names remain callable but explicitly return
+`nil, "unsupported: ..."`: `startWildBattle` and `invalidateMap` have not yet been verified against the
+Gen1 API-3 contract. Their native/API-2 counterparts are unaffected. Native
+symbolic flag mappings and `mod.datasets:open` are also explicitly unavailable.
+Gen4/5 do not claim the shared world surface in this first entry PR.
+
+Private flags use `save.modData[modId].worldFlags`, independently of native
+flags and other mods. A read does not create a bucket. Clearing removes the
+stored key. A change emits `flag.changed` with
+`name = "mod.<modId>.<key>"` and the boolean value; setting the same value
+emits nothing. Invalid save stores are refused without overwriting them.
+
+Gen1 API-3 callback names from the pinned reference are recognized even when
+this engine has no call site for them. Registration is allowed; availability
+still requires a native call site in the selected version. For example,
+`render.output` can be subscribed to but reports unavailable here. Unknown
+names remain errors. An accepted subscription is not proof that a feature
+runs. Source-specific engine handles in lifecycle/UI payloads remain native.
+
+For scripts, use `mod:supports("scripts", verb)` to test Gen2Recomped's current
+bounded vocabulary. A `queueScript` method does not imply support for every
+Gen1 script verb. See [portable world scripts](portable-world-scripts.md).
+
+## Reproducible contract checks
+
+```sh
+luajit tests/engine/api3_entry.lua
+luajit tests/engine/api3_compat.lua
+python3 tools/api3-contract-check/check.py \
+  --gen1 /path/to/clean/pinned/gen1recomp --lua luajit \
+  --report /tmp/api3-contract-report.json
+```
+
+The external reference must be a clean checkout at
+`bryanthaboi/gen1recomp@79ea62f45fb2ea8f65d081a109431e4eb1bbdb36`.
+Use a separate checkout; do not reset a working installation. No Gen1 engine
+implementation is vendored into this repository. Only public callback names
+are recorded as a compatibility catalog.
+
+The exact same manifest and mod entry run on both engines for Red, Blue,
+Yellow, Gold, Silver, Crystal, FireRed and Emerald, in both success and
+intentional-rollback cases. Each engine/version/case has its own LuaJIT
+process. The shared harness constructs native-shaped synthetic datasets;
+the mod fixtures contain no engine imports or engine-specific branches.
+The runner also generates an API-3 mod through the real scaffold CLI and
+loads those same generated files on both engines for each version. This adds
+eight comparisons to the sixteen normal/rollback cases. Eight additional warp
+cases use the same public mod on both engines and native-shaped timing seams.
+Eight object/block cases and eight NPC cases extend this to 48 comparisons
+and 96 isolated loader runs. It compares JSON projections of public results and
+records hashes of both shared and generated fixture files. It refuses a different or modified reference.
+
+Coverage includes canonical content reads/writes/iteration and native merge
+results, detached core and passthrough reads, callable early handles, later
+Game replacement, party fields, private flags, Gen1 battle snapshot/invalid-intent boundaries, callback payloads and native
+writeback, custom events, unsupported-name handling, frozen content and failed
+owner cleanup. Warp cases also compare validation errors, native dispatch
+arguments, destination events, boundary coordinates, arrival options, callback
+argument holes/replay, stale sessions/owners, synchronous completion and error
+attribution. Object/block cases compare native reference routing, visibility/flag
+changes, block geometry/bounds, native reads and one event per successful edit.
+NPC cases compare runtime creation/lookup, movement/appearance, optional
+completion callbacks and ownership-aware removal using the same public handles.
+Separate Gen2 tests check API-1/2 behavior, contradictory
+manifest fields, honest availability and malformed flag stores. CI pins the
+same external commit and uploads the report.
+
+These are ROM-free loader/runtime-boundary tests, not retail-ROM gameplay
+or a proof of every exposed native callback's payload. Native action details
+remain covered by the prerequisite PR suites. Extend these shared fixtures
+whenever another API-3 service becomes portable; do not make a failing
+comparison pass by changing only one engine's mod fixture.