feat: generate Swift and Kotlin client models and a sync-aware schema IR

[dawsontoth]

Generates typed Swift and Kotlin client models, and a versioned schema IR — an intermediate representation: one language-neutral JSON description of each table's client contract (attributes, types, nullability, read/write projections, replica layout), the sync profiles and the contract hashes — from the live Harper schema — the codegen the client SDK needs (HarperFast/harper-client-sdk-spike#8). The spike's iOS demo now builds and syncs on the generated package instead of hand-written models (companion: HarperFast/harper-client-sdk-spike#19). Existing TypeScript/JSDoc output is unchanged, except that its paths now resolve against the application directory.

The IR builder reads live tables, the GraphQL type registry and sync.yaml; the Swift and Kotlin emitters read only the IR; contract hashes and a read/write/storage compatibility diff tell a team when a schema change would break clients already in the field.

Product and architecture tour

One IR feeds every client artifact

What does the generator read, and what does it produce?

From schema to client packages

Live tables, the GraphQL type registry and sync.yaml become one IR; the emitters and the compatibility diff read only that IR, never Harper.

flowchart LR
  T[Live tables] --> B[buildSchemaIR]
  G[GraphQL type registry] --> B
  Y[sync.yaml] --> B
  P[Previous IR] -. type names .-> B
  B --> IR[(Schema IR JSON)]
  IR --> SW[Swift package]
  IR --> KT[Kotlin module]
  IR --> D[diffSchemaIR]
  IR -. future .-> GW[Gateway and Studio]

Loading

Nested object types come from the GraphQL type registry because live Table.attributes keep only a type name (address: Address becomes { type: 'Address' }, verified on 5.3.0-beta.2). The IR is plain JSON (irVersion: 1), byte-for-byte deterministic for any input order, and published after the sources, so it is always the baseline the published sources were generated from.

What an app gets for each table

Which reads and writes does a generated model allow?

One property per stored attribute, optional unless it is the primary key, server-managed or declared non-null; computed attributes are read-only and optional, and a Blob is always optional because sync does not deliver blob content.
_version carries the record version; on tables that are not @sealed, _extra keeps undeclared attributes for round trips, and declared names are filtered out of it on decode and again on encode.
New (insert, primary key optional), Patch (each attribute unset, set or cleared) and encodeUpsertRow() (full replacement, refused for tables holding a Blob) follow core's defineTable projections.

Example: a patch that clears one attribute

An app marks an order closed and clears its amount with Order.Patch(status: "closed", clear: [.amount]).encodeRow().

Outcome: the body is { status: "closed", amount: null }; attributes left unset are absent, so the server leaves them unchanged.

Sync profiles fail closed

What happens to a profile with a mistake in it?

Any error in a profile — an unknown table or attribute, a mistyped key, an invalid value — drops the whole profile and logs an error, so a typo can never widen a scope to every row.
An imported IR is held to the same shapes: validation refuses a scope that normalization would not have produced.

Example: a misspelled scope attribute

A profile scopes Order with scope: { custmerId: $token.sub }.

Outcome: the component logs profile "broken" scopes "Order" by "custmerId", which is not a stored attribute, and the profile is absent from the IR and the generated code.

Hashes and schema drift

How does a team learn that a schema change breaks clients already in the field?

A table's hash covers its wire and storage contract, including every nested type and embedded table it carries, each listed once by name; a profile's schemaHash covers its tables and is the value a device sends in its hello frame.
Equal hashes mean an identical contract, not identical source: descriptions, attribute order and generated type names are excluded.
An IR file is refused when a hash does not match its contents or a derived field (readOnly, projections, column affinity) does not follow from them, so a hand-edited IR cannot slip a change past the CI diff or emit a model its hash does not describe.

The diff classifies each change by what it breaks for clients generated from the older IR: read (their decoding can fail), write (the server rejects their writes, or a PUT drops data) and storage (their replica cannot be extended in place). Additive changes come with a delta an older client can apply without regenerating: new tables' storage, added columns, and computed expressions to recompute.

Example: changing Product.price from Float to String

The dev-mode component compares the new IR with the previous IR file on regeneration.

Outcome: it warns data.Product.price type changed from Float to String (breaks read, write) and data.Product.price is now stored as TEXT (breaks storage), and schema-codegen diff exits 1.

Regeneration in dev mode

What runs, in what order, when the schema changes?

One regeneration

1. Coalesce — schema events and sync.yaml edits collapse into one trailing run, and runs never overlap.
2. Collect once — one filtered table list feeds both the TypeScript output and the client outputs.
3. Build — the IR is built from that list, the type registry as it stands for this run, sync.yaml and the previous IR.
4. Warn — a diff against the previous IR logs every change that breaks reads, writes or storage.
5. Render — every output is rendered and validated in memory; an invalid option writes nothing.
6. Publish — every changed file is staged beside its target, then all are renamed into place with the IR last; a failed rename restores what it had replaced, and a run whose scope closed stops before this step.

Generated code samples

The spike's Order table:

type Order @table(audit: true) @export {
	id: ID @primaryKey
	customerId: ID @indexed
	status: String @indexed
	amount: Float
	points: Float @computed(from: "Math.round((amount || 0) * (status == 'open' ? 10 : 25))")
}

Generated Swift (abridged):

public struct Order: HarperRecord, Identifiable {
	public static let schema = HarperTableSchema(
		database: "data", name: "Order", primaryKey: "id", sealed: false,
		hash: "379d4b048bb7c07979b6c89302af72a913aabf70628a6c84cef1a0d970b636be",
		versionColumn: "_version", extraColumn: "_extra", replaceable: true,
		columns: [
			HarperColumn(name: "id", type: .id, affinity: .text, nullable: false, primaryKey: true),
			HarperColumn(name: "customerId", type: .id, affinity: .text, nullable: true, indexed: true),
			…
			HarperColumn(name: "points", type: .float, affinity: .real, nullable: true, readOnly: true, computedFrom: "Math.round((amount || 0) * (status == 'open' ? 10 : 25))"),
		]
	)

	public let id: String
	public var customerId: String?
	public var status: String?
	public var amount: Double?
	/// Computed on the client from `Math.round((amount || 0) * (status == 'open' ? 10 : 25))`.
	public let points: Double?
	public let _version: Double?
	public var _extra: [String: HarperValue]

	public init(row: HarperRow, version: Double?) throws {
		self.id = try row.harperRequired("id", of: "Order")
		self.customerId = try row.harperOptional("customerId", of: "Order")
		…
		self._version = version
		self._extra = row.harperExtra(excluding: Self.declaredAttributes)
	}

	public func encodeUpsertRow() throws -> HarperRow {
		var row = self._extra.harperExtra(excluding: Self.declaredAttributes)
		row["id"] = self.id.harperValue
		if let value = self.customerId { row["customerId"] = value.harperValue }
		…   // no points: computed attributes are never written
		return row
	}

	public struct New: HarperWritable { … }   // insert body: id optional
	public struct Patch: HarperWritable {     // patch body: unset, value, or cleared
		public enum Field: String, Sendable, Hashable {
			case customerId = "customerId"
			case status = "status"
			case amount = "amount"
		}
		…
	}
}

Generated Kotlin (abridged):

private val stringListConverter = HarperConverters.list(HarperConverters.string)   // built once per file, not per decode

public data class Order(
	public val id: String,
	public val customerId: String? = null,
	public val status: String? = null,
	public val amount: Double? = null,
	/** Computed on the client from `Math.round((amount || 0) * (status == 'open' ? 10 : 25))`. */
	public val points: Double? = null,
	override val _version: Double? = null,
	public val _extra: Map<String, HarperValue> = emptyMap(),
) : HarperRecord {
	override fun encodeUpsertRow(): HarperRow = buildMap {
		putAll(this@Order._extra.harperExtra(Order.declaredAttributes))
		put("id", HarperConverters.string.encode(this@Order.id))
		this@Order.customerId?.let { put("customerId", HarperConverters.string.encode(it)) }
		…
	}

	public data class New(…) : HarperWritable
	public data class Patch(…, public val clear: Set<Field> = emptySet()) : HarperWritable

	public companion object : HarperTable<Order> {
		override val schema: HarperTableSchema = HarperTableSchema(…)
		override fun decode(row: HarperRow, version: Double?): Order = Order(
			id = row.harperRequired("id", "Order", HarperConverters.string),
			customerId = row.harperOptional("customerId", "Order", HarperConverters.string),
			…
		)
	}
}

Using the models:

import HarperModels

let row = json.mapValues { HarperValue(any: $0) }   // JSONSerialization, SQLite, or JSONDecoder → HarperRow
var order = try Order(row: row, version: version)   // HarperDecodingError names type + attribute on a mismatch
order.status = "shipped"
let putBody = try order.encodeUpsertRow()           // id + writable attributes + _extra; never `points`
let postBody = Order.New(customerId: "alice", status: "open", amount: 12.5).encodeRow()
let patchBody = Order.Patch(status: "closed", clear: [.amount]).encodeRow()   // ["status": "closed", "amount": null]

HarperSyncProfile.myOrders.schemaHash               // what a device sends in its hello frame
try attachment.encodeUpsertRow()                    // ✗ HarperEncodingError.replacementUnsupported(type: "coverage_Attachment", attribute: "file")

val order = Order.decode(row, version)
val putBody = order.copy(status = "shipped").encodeUpsertRow()
val patchBody = Order.Patch(status = "closed", clear = setOf(Order.Patch.Field.amount)).encodeRow()
val profile = HarperProfiles.myOrders

An excerpt of the IR (ios/harper-schema.json in the spike):

{
  irVersion: 1,
  schemaHash: '09d59bb0…',                 // every table's wire/storage contract
  tables: [{
    database: 'data', name: 'Order', typeName: 'Order', primaryKey: 'id', sealed: false,
    hash: '379d4b04…', profiles: ['my-orders'],
    attributes: [
      { name: 'points', type: { kind: 'scalar', scalar: 'Float' }, nullable: true, readOnly: true,
        computed: { from: "Math.round((amount || 0) * (status == 'open' ? 10 : 25))", version: … }, … },
      …
    ],
    projections: { record: […], insert: […], upsert: […], patch: […], clearable: ['customerId', 'status', 'amount'], unwritable: [], query: […] },
    storage: { versionColumn: '_version', extraColumn: '_extra', columns: [{ name: 'id', affinity: 'TEXT' }, …] },
  }],
  profiles: [{
    name: 'my-orders', direction: 'pull', retention: null, retentionMs: null,
    tables: [{ database: 'data', table: 'Order',
      scope: { type: 'match', conditions: [{ attribute: 'customerId', operator: 'equals', source: 'user', field: 'username' }] } }],
    hash: 'cac036c2…',                      // the profile definition (its version)
    schemaHash: '947763dc…',                // the tables it covers: the hello-frame value
  }],
  diagnostics: [],
}

Configuration, in the application's config.yaml and sync.yaml:

'@harperfast/schema-codegen':
  package: '@harperfast/schema-codegen'
  schemaIR: 'client/harper-schema.json'
  syncProfiles: 'sync.yaml'
  swift: 'ios/HarperModels'
  kotlin: 'android/harper-models'
  kotlinPackage: 'com.example.harper'

profiles:
  storefront:
    retention: 14d
    tables: [Product]
  my-orders:
    retention: 30d
    scope:
      customerId: $user.username # $token.<claim> also works; a list means "any of"
    tables: [Order]

The same compatibility check runs in CI without Harper:

npx @harperfast/schema-codegen diff before.json after.json --fail-on read,write   # exit 1 on a breaking change
npx @harperfast/schema-codegen generate-client --ir client/harper-schema.json --swift ios/HarperModels --kotlin android/harper-models

Changes

• Schema IR. In utils/client/buildSchemaIR.js, describeTable applies the read-nullability rules ("undefined means nullable"; primary key and server-managed required; computed and derived optional) and lays out the replica columns, with metadata column names that step aside for attributes named _version/_extra; toType resolves nested types through the registry, embedded tables as records (warning when an embedded table's database is excluded) and anything unresolvable to Any with a diagnostic; relationships are described but never stored; projectionsOf ports the defineTable projections with the Blob rules; requiredOnInsert is the shared insert-requiredness rule; and allocateTypeNames assigns identifier-safe type names in one namespace.
• Contract hashes. hashContracts and hashProfile in utils/client/contractHash.js hash each table contract, whose nested shapes are listed once by name so shared and recursive types hash in linear time; utils/client/canonicalHash.js supplies canonicalJSON (sorted keys at every depth) and hashCanonical.
• Sync profiles. utils/client/syncProfiles.js holds parseDuration (retention rounds to a positive safe integer), loadSyncProfilesFile (YAML or JSON), normalizeProfile (returns nothing once any part fails) and normalizeCondition (literals, $token.<claim>, $user.<field>, $$ escapes, membership lists).
• IR validation. assertSchemaIR in utils/client/validateSchemaIR.js checks shapes, unique identifier-safe type names, references, projections, metadata columns, a single-line generator, profile scopes that derived fields follow from the contract, and, last, recomputes every hash.
• Shared definitions. utils/client/irTypes.js declares the IR shapes; utils/client/irConstants.js the IR version and scalar set; utils/client/generator.js the generator stamp; and utils/client/naming.js the reserved type names, toCodeIdentifier, the collision-suffixing allocator, member allocation that claims verbatim names first against the runtime types and the owning type, which member bodies name (profile constants also avoid every generated type), and keyword escaping.
• Compatibility diff. In utils/client/diffSchemaIR.js, diffSchemaIR validates both IRs and summarizes each axis; attribute removals and additions carry the sealed, overflow-echo and relationship rules; relationships are diffed one by one; a table that stops supporting full replacement breaks writes; compareTypes compares each nested type pair once per table; a table that embeds a broken table inherits its read and write breaks (found by embeddedTableKeys), and one whose contract changed only through what it embeds is named as such; and diffProfiles lists profile changes.
• Swift emitter. In utils/client/emitSwift.js, tableModel emits each record struct (decode, encode, upsert refusal, New, Patch with Field/clear); objectModel emits nested types; schemaSource emits HarperSchema and the profile constants, allocated by raw profile name; and packageManifest is the Package.swift scaffold, written once.
• Kotlin emitter. In utils/client/emitKotlin.js, tableModel emits each data class with its companion HarperTable; converterOf hoists every non-scalar converter into a file-level value named against every name that could shadow it, so list converters are built once and member bodies never name another generated type; contentEquality gives classes holding byte arrays, in lists too, content equality; and schemaSource emits HarperSchema and HarperProfiles.
• Emitter support. utils/client/emitSupport.js holds the generated-file header, runtime templates read with LF endings, comment sanitizing, Swift and Kotlin string escaping, and recursiveValueEdges, which finds the by-value cycles Swift must box.
• Runtimes. utils/client/runtime/HarperRuntime.swift defines HarperValue, HarperBigInt, the row decoding helpers, dates within JavaScript's range, HarperTableSchema, HarperRecord and HarperSyncProfile; utils/client/runtime/HarperRuntime.kt defines the Kotlin HarperValue, HarperConverters (decoding a Long from a double only within its exact range and a date only within JavaScript's range), HarperContent (byte-array content equality), HarperTable and HarperSyncProfile.
• Publication. In utils/client/clientOutputs.js, renderClientOutputs renders everything in memory with the IR last, publishClientOutputs stages every changed file before renaming any and restores replaced files if a rename fails, and findStaleGeneratedFiles reports sources a renamed module left behind, skipping the package's build folders and unreadable files; utils/writeIfChanged.js adds stageFile (symlinks followed, dangling ones included, cycles refused) and commitStaged (in-place fallback when Windows holds the target open), which writeIfChanged now uses; utils/generateTablesDTS.js writes through writeIfChanged.
• Regeneration. regenerateClient in utils/client/regenerateClient.js reads the previous IR, ignoring an invalid one, stops when its scope closed, logs diagnostics and breaking changes, publishes, and warns about the manifest and stale files; utils/regenerateAll.js loads the client modules only when an output is configured; createRegenerationScheduler in utils/createRegenerationScheduler.js coalesces and serializes runs, and keeps serving queued requests when the error reporter throws.
• Component. extensionModule.js reads the client options, passes the registry and a close signal per run, starts after the initial delay, watches sync.yaml when a client output is configured, and aborts and detaches on close.
• CLI and package. utils/client/cli.js implements generate-client and diff with --fail-on and --json; bin/schema-codegen.js is the bin entry; index.js is the public API; package.json adds bin, main and files and the yaml dependency, locked in package-lock.json here.
• Docs and CI. README.md gains the Client models section; docs/design/client-codegen.md is the design note and decision record; .github/workflows/verify-pr.yaml adds a macOS job that compiles and runs the generated Swift and Kotlin and skips the native tests in the Node matrix, as does the release workflow.

For the human reviewer

1. Scope. Covers HarperFast/harper-client-sdk-spike#8 as asked, plus a read/write/storage diff and CLI that nothing consumes until the gateway (HarperFast/harper-client-sdk-spike#10); those can be dropped without touching the emitters. Emitting only the IR and leaving models to the SDKs was rejected because the demo needs models now.
2. One compatibility rule to confirm: adding a computed attribute or relationship to a table that is not @sealed counts as write-breaking, because older models echo the new key from _extra and core's validate() rejects it.
3. Hard to change once shipped: the type mapping (Long → Int64/Long, BigInt → decimal string/BigInteger, Date → epoch ms, Any → HarperValue), the sync.yaml profile format (which fails closed), and the public index.js API, which has no consumer until the gateway and Studio arrive. There is no exports map, so existing deep imports keep working.
4. Blob is read-only and optional on read; a table with a writable Blob refuses full replacement (patches work) until blob sync exists, and a table that gains one breaks writes in the diff.
5. Behavior change: existing schemaTypes/globalTypes/jsdoc paths now resolve against the app directory rather than Harper's working directory.
6. Conventions: new modules are JavaScript with JSDoc, since Node won't strip types under node_modules; the one new dependency, yaml, loads only when syncProfiles is set.
7. Smaller calls, cheap to revisit: sealing breaks writes but not storage; generated type names stay stable through the previous IR file; the runtime ships in each package until the SDKs exist; decoding is strict; dev mode publishes a breaking change with a warning, leaving CI's diff as the gate.

Verification

• npm test (vitest): 30 files, 325 tests; with KOTLINC/JAVA_HOME set and SCHEMA_CODEGEN_NATIVE_TESTS=require, all pass locally, and the PR's CI is green on Ubuntu and Windows (Node 20/22/24) and on the macOS native job. npm run lint:check and npm run format:check pass.
• Native contract: native.test.js type-checks the generated Swift in Swift 5 and compiles it warning-free in Swift 6 strict concurrency, then runs SwiftHarness.swift (JSON and Foundation decoding, extras, null states, patch clears, Blob refusal, a Long at 2^63, a NaN date, raw-name round trips, records through a reopened SQLite replica built from the descriptors); it compiles the generated Kotlin with -Werror and runs KotlinHarness.kt for the same contract on the JVM, byte arrays inside lists compared by content included. The adversarial fixtures include attributes named like a runtime type and like a generated type, and a table named ModelsKt, which the previous emitter could not compile.
• IR and hashes: buildSchemaIR.test.js covers nullability, projections (incl. a required Blob and its warning, Blobs at any depth read as optional and a computed Blob), nested types, relations, an excluded embedded table, naming and contract hashes (determinism, embedded tables, a 40-level doubly shared type chain in linear time); canonicalHash.test.js and naming.test.js cover their helpers; fixtures live in clientSchema.js.
• Profiles: syncProfiles.test.js covers every fail-closed case, duration bounds and a YAML alias bomb.
• Compatibility: diffSchemaIR.test.js covers every rule, including echoed computed and relationship additions, attribute-to-relationship conversion, Blob requiredness, a table losing full replacement, itemized contract changes, including breaks inherited from an embedded table, dotted table names and shared nested types compared once.
• Validation: validateSchemaIR.test.js rejects each corruption, including header-injecting metadata, stale hashes, malformed scopes, reserved type names and derived fields that disagree with the contract.
• Emitters: emitSwift.test.js, emitKotlin.test.js (incl. hoisted converters and colliding profile names) and emitSupport.test.js (escaping, CRLF templates, recursion edges).
• Publication and lifecycle: clientOutputs.test.js (staging failures, rename rollback including a half-written target, build folders at the root and below, unreadable files), writeIfChanged.test.js (symlinks, dangling links, cycles, locked targets), regenerateClient.test.js, createRegenerationScheduler.test.js (synchronous throws, re-entry, a throwing reporter), regenerateAll.test.js and extensionModule.test.js; each of the new lifecycle and publication tests fails against the previous implementation. The CLI is covered by cli.test.js and the packed file list and entry point by package.test.js.
• Live: a hermetic harper dev (5.3.0-beta.2) started from a different working directory wrote the IR, Swift package, Kotlin module and TS types into the application directory for the spike schema plus a coverage database (nested types, GraphQL and defineTable relationships, sealed and Blob tables); swift build and kotlinc built both; editing sync.yaml regenerated without a restart; the Product.price change produced the warning above, and schema-codegen diff exited 1. The README's npx commands ran verbatim from a packed tarball.
• Spike acceptance (HarperFast/harper-client-sdk-spike#19): the demo's hand-written models are gone; after regenerating with this head, xcodebuild succeeds, and in the iOS simulator against the spike gateway the demo checked out 500 products and alice's orders through the generated models, with a live REST update arriving with on-device computed points and an undeclared attribute preserved in _extra.

Related PRs: #57 independent (dependency updates, already on main; only package.json and package-lock.json are shared)
Complexity: complicated

_{Review-Coverage: authored=claude; ran=gemini,codex,cursor-composer,cursor-muse; adjudicated=domain; blocked=cursor-grok(failed); declined=cursor-kimi; rounds=4; full=2 @ 8dde785}

_{Review-Attention: deep ~160m (critical: schema-codegen.js, buildSchemaIR.js +2; sensitive: verify-pr.yaml, package-lock.json; decisions: do-less-alternative, file-level-converters-for-name-stability, rollback-by-rewrite, blob-read-optional-scope, date-range-js-bound, rethrow-reporter-failure, js-over-ts, base-directory-for-existing-outputs, public-root-api-now) @ 8dde785}

[kriszyp]

Cool, this will be nice!

_{🤖 Reviewed with Codex}

[github-actions]

🎉 This PR is included in version 2.1.0 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀