feat(platform-objects,cli): record which source revision a generated translation leaf was filled from (#12557)

* feat(platform-objects,cli): record which source revision a generated translation leaf was filled from

Extends the #8765 Option B source-hash mechanism to the generated bundles, per
maintainer ruling #12069 Option A, and corrects the false "this hole cannot
occur there" note that kept them out of it.

`os i18n extract --source-hashes` emits <locale>.source-hashes.generated.ts —
per leaf, the digest of the source revision that leaf is still a byte copy of.
withSourceFallback takes it as a fourth argument and judges objects /
metadataForms alongside the hand-authored sections.

Records are written only where a leaf is currently a copy of the CURRENT
source, so day-one stale count is 0 by construction: 9030 translated leaves
measured, 1543 recorded, 7487 left legacy-trusted. No committed bundle moved a
byte.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W6HFzyH98W1YaQXhJUJt6o

* test(platform-objects): pin single-locale stale-fill detection — the property the ruling turns on

The suite pinned per-locale RECOVERY for the hand-authored half but had no
analogue for the generated half's per-locale DETECTION, which is the whole of
what maintainer ruling #12069 Option A buys over the gate that shipped first.

`check:i18n-stale-fill` infers provenance from two locales holding byte-identical
text, so a leaf stranded in exactly one locale has no second witness and is
invisible to it by construction. A recorded hash needs no witness. Measured on
this tree: 18 generated leaves are recorded in exactly one locale (zh-CN 2,
ja-JP 3, es-ES 13) — English-looking terms one locale left as a fill while the
others translated them.

Verified falsifiable: deleting the third conjunct of findStaleFills turns these
3 tests into 1 failure among 3 total, predicted in writing before the run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W6HFzyH98W1YaQXhJUJt6o

---------

Co-authored-by: Claude <noreply@anthropic.com>

Author: os-warren

.changeset/i18n-generated-leaf-source-provenance.md

@@ -0,0 +1,55 @@
+---
+"@objectstack/platform-objects": minor
+"@objectstack/cli": minor
+---
+
+feat(platform-objects,cli): record which source revision a generated translation leaf was filled from (#11671)
+
+Closes the half of the sticky-translation-drift class that no value comparison
+could reach, under maintainer ruling #12069 Option A — by extending the existing
+#8765 Option B source-hash mechanism to the generated bundles rather than
+building a second one.
+
+**The hole.** `os i18n extract --fill=default` fills gaps only: any non-empty
+value in a translated locale wins forever. So the ordinary sequence — extract,
+revise the source string, extract again — rewrites `en` and strands the previous
+source text in every other locale. The bundle is still in sync by key, so
+`check:i18n` reports OK; the leaf is still present, so `check:i18n-coverage`
+counts it translated. Measured on #11659 at `bbe0b17`: three locales serving a
+602-char superseded draft of a 411-char help string under 31 green checks. Once
+the source has moved, that stale fill is indistinguishable **by value** from a
+real translation — 2648 of 3010 leaves differ from `en`, so "untranslated AND
+differing from the source" describes an empty set, not a noisy one.
+
+**What is new.** `os i18n extract --source-hashes` writes
+`<locale>.source-hashes.generated.ts` beside each generated bundle: per leaf,
+the digest of the source revision that leaf is **still a byte copy of**.
+`withSourceFallback` takes that table as a fourth argument and now judges the
+`objects` / `metadataForms` sections as well as the hand-authored ones, so a
+leaf whose source has moved underneath it serves the current source string
+instead of a superseded draft — the same degradation an untranslated key already
+produces, which is the invariance the #8765 ruling turned on.
+
+The generated half needs one conjunct the hand-authored half does not: the leaf
+must still hold the recorded bytes. Its hash table is itself generated, so a
+translator cannot be asked to refresh a digest by hand the way
+`<locale>.source-hashes.ts` asks; without that conjunct, re-translating a stale
+leaf would leave the old record standing and report the fresh translation as
+stale forever. With it, editing the value clears the flag by itself.
+
+**Behaviour on the day it lands: unchanged for every leaf.** Records are
+written only where a leaf is currently a byte copy of the **current** source, so
+every record equals the current digest and nothing is stale. Measured across the
+nine bundle sets: 9030 translated leaves, 1543 byte-equal to `en` (records
+written), 7487 differing (left with no record — legacy-trusted, per the ruling's
+property 1, since nothing in the tree says which revision they were made from).
+No committed bundle changed a byte.
+
+**Scope.** `--source-hashes` is off by default and `@objectstack/platform-objects`
+is the one bundle set that opts in, by documenting the flag in its extract
+config. The other eight sets keep exactly today's behaviour and can be enabled
+file-by-file later; a set with no companion is entirely legacy-trusted.
+
+The false "this hole cannot occur there" note that kept the generated sections
+out of the mechanism is corrected in `source-hash.ts`, with the measurement that
+falsifies it.

package.json

@@ -29,7 +29,7 @@
     "objectui:refresh": "bash scripts/bump-objectui.sh && bash scripts/build-console.sh",
     "objectui:clean": "rm -rf packages/console/dist .cache/objectui-*",
     "lint": "node --stack-size=4000 node_modules/eslint/bin/eslint.js . --no-inline-config",
-    "i18n:extract": "tsx packages/cli/bin/run-dev.js i18n extract packages/platform-objects/scripts/i18n-extract.config.ts --locales=zh-CN,ja-JP,es-ES --fill=default --out=packages/platform-objects/src/apps/translations",
+    "i18n:extract": "tsx packages/cli/bin/run-dev.js i18n extract packages/platform-objects/scripts/i18n-extract.config.ts --locales=zh-CN,ja-JP,es-ES --fill=default --source-hashes --out=packages/platform-objects/src/apps/translations",
     "check:i18n": "node scripts/check-i18n-bundles.mjs --self-test && node scripts/check-i18n-bundles.mjs",
     "check:i18n-coverage": "node scripts/check-i18n-coverage.mjs --self-test && node scripts/check-i18n-coverage.mjs",
     "check:i18n-stale-fill": "node scripts/check-i18n-stale-fill.mjs --self-test && node scripts/check-i18n-stale-fill.mjs",

packages/cli/src/commands/i18n/extract.ts

@@ -16,7 +16,13 @@ import {
   emitJson,
   isExitSignal,
 } from '../../utils/format.js';
-import { extractTranslations, renderTranslationModule, type FillStrategy } from '../../utils/i18n-extract.js';
+import {
+  extractTranslations,
+  renderTranslationModule,
+  renderSourceHashModule,
+  parseSourceHashModule,
+  type FillStrategy,
+} from '../../utils/i18n-extract.js';
 
 const FILL_STRATEGIES: FillStrategy[] = ['empty', 'default', 'todo'];
 
@@ -89,6 +95,12 @@ export default class I18nExtract extends Command {
       default: true,
       allowNo: true,
     }),
+    'source-hashes': Flags.boolean({
+      description:
+        'Also write <locale>.source-hashes.generated.ts — the provenance companion that lets a stale fill be told from a translation (#11671). Off by default: it is a format addition, so a bundle set opts in by documenting the flag in its extract config.',
+      default: false,
+      allowNo: true,
+    }),
     'dry-run': Flags.boolean({
       description: 'Print to stdout instead of writing to --out',
       default: false,
@@ -131,9 +143,25 @@ export default class I18nExtract extends Command {
           ? declared.defaultLocale
           : 'en');
 
+      // Resolved before the extract because the previously committed provenance
+      // records are an INPUT to it: they are the mechanism's only memory, and a
+      // run that could not read them would silently re-derive every record from
+      // the current tree and forget the drift it is supposed to be holding on to.
+      const outDir = flags.out ? path.resolve(process.cwd(), flags.out) : undefined;
+      const previousSourceHashes: Record<string, Record<string, string>> = {};
+      if (flags['source-hashes'] && outDir) {
+        for (const locale of locales ?? []) {
+          const file = path.join(outDir, `${locale}.source-hashes.generated.ts`);
+          if (!fs.existsSync(file)) continue;
+          const table = parseSourceHashModule(fs.readFileSync(file, 'utf8'));
+          if (table) previousSourceHashes[locale] = table;
+        }
+      }
+
       const result = extractTranslations(normalized, {
         defaultLocale,
         locales,
+        previousSourceHashes,
         fill: flags.fill as FillStrategy,
         filter,
         // Merge (the default) never overwrites an existing non-default-locale
@@ -222,7 +250,8 @@ export default class I18nExtract extends Command {
         return;
       }
 
-      const outDir = path.resolve(process.cwd(), flags.out);
+      // `flags.out` is non-empty here — the two branches above return otherwise.
+      const resolvedOutDir = outDir as string;
 
       // Every file a normal run would emit, paired with its rendered content.
       // Both branches below iterate this, so `--check` can never diverge from
@@ -231,18 +260,29 @@ export default class I18nExtract extends Command {
       for (const locale of localesEmitted) {
         if (result.counts[locale] > 0) {
           emitted.push({
-            file: path.join(outDir, `${locale}.objects.generated.ts`),
+            file: path.join(resolvedOutDir, `${locale}.objects.generated.ts`),
             content: renderTranslationModule(result.bundles[locale], { locale, objectsOnly }),
             keys: result.counts[locale],
           });
         }
         if (emitsMetadataForms(locale)) {
           emitted.push({
-            file: path.join(outDir, `${locale}.metadata-forms.generated.ts`),
+            file: path.join(resolvedOutDir, `${locale}.metadata-forms.generated.ts`),
             content: renderTranslationModule(result.bundles[locale], { locale, kind: 'metadataForms' }),
             keys: metadataFormsCounts[locale],
           });
         }
+        // The provenance companion rides in the SAME list, so `--check` compares
+        // it by the same byte-for-byte rule as the bundles it belongs to and can
+        // never diverge from what a real extract writes.
+        const table = result.sourceHashes[locale];
+        if (flags['source-hashes'] && table) {
+          emitted.push({
+            file: path.join(resolvedOutDir, `${locale}.source-hashes.generated.ts`),
+            content: renderSourceHashModule(table, { locale }),
+            keys: Object.keys(table).length,
+          });
+        }
       }
 
       if (flags.check) {
@@ -269,7 +309,7 @@ export default class I18nExtract extends Command {
         process.exit(1);
       }
 
-      fs.mkdirSync(outDir, { recursive: true });
+      fs.mkdirSync(resolvedOutDir, { recursive: true });
       let written = 0;
       for (const { file, content, keys } of emitted) {
         fs.writeFileSync(file, content, 'utf8');

packages/cli/src/utils/i18n-extract.ts

@@ -84,6 +84,7 @@ import { DEFAULT_METADATA_TYPE_REGISTRY } from '@objectstack/spec/kernel';
 import { deriveFieldGroupLayout } from '@objectstack/spec/data';
 import { expandViewContainer } from '@objectstack/spec/ui';
 import { authorWarnedProperties, walkPageComponents } from '@objectstack/lint';
+import { collectFilledFromHashes } from '@objectstack/platform-objects/apps';
 
 // ─── Public types ──────────────────────────────────────────────────────
 
@@ -163,6 +164,18 @@ export interface ExtractOptions extends ExpectedEntryOptions {
    * This makes extract idempotent — re-running only fills the gaps.
    */
   mergeExisting?: boolean;
+  /**
+   * The `<locale>.source-hashes.generated.ts` tables already committed beside
+   * the bundles, keyed by locale.
+   *
+   * This is the mechanism's ONLY memory (#11671 / #12069 Option A): a leaf that
+   * is a byte copy of a source revision keeps its record across runs, which is
+   * what makes the drift detectable after the source moves. Passing nothing
+   * makes the run behave like a first extract — every record is re-derived from
+   * the tree, so leaves that already drifted stay legacy-trusted rather than
+   * being reported.
+   */
+  previousSourceHashes?: Record<string, Record<string, string>>;
 }
 
 export interface ExtractResult {
@@ -172,6 +185,18 @@ export interface ExtractResult {
   counts: Record<string, number>;
   /** Total expected entries before per-locale merge filtering. */
   totalExpected: number;
+  /**
+   * Per translated locale, the digest of the source revision each GENERATED
+   * leaf is still a byte copy of — the content of
+   * `<locale>.source-hashes.generated.ts`.
+   *
+   * Computed by `collectFilledFromHashes` in
+   * `@objectstack/platform-objects/apps`, the module maintainer ruling #8765
+   * Option B put the mechanism in; the extractor supplies the tree and the
+   * previous records and owns none of the rule. The default locale gets no
+   * entry: it is the source, not a copy of one.
+   */
+  sourceHashes: Record<string, Record<string, string>>;
 }
 
 // ─── Walk helpers ──────────────────────────────────────────────────────
@@ -1313,7 +1338,18 @@ export function extractTranslations(config: any, opts: ExtractOptions = {}): Ext
     counts[locale] = count;
   }
 
-  return { bundles, counts, totalExpected: entries.length };
+  const sourceHashes: Record<string, Record<string, string>> = {};
+  const sourceBundle = bundles[defaultLocale];
+  for (const locale of locales) {
+    if (locale === defaultLocale) continue;
+    sourceHashes[locale] = collectFilledFromHashes(
+      bundles[locale],
+      sourceBundle,
+      opts.previousSourceHashes?.[locale],
+    );
+  }
+
+  return { bundles, counts, totalExpected: entries.length, sourceHashes };
 }
 
 // ─── Serialization ─────────────────────────────────────────────────────
@@ -1390,6 +1426,80 @@ export function renderTranslationModule(
   return lines.join('\n');
 }
 
+/**
+ * Render one locale's generated source-hash table as a TypeScript module body —
+ * the `<locale>.source-hashes.generated.ts` companion.
+ *
+ * Deliberately types the export STRUCTURALLY (`Readonly<Record<string,
+ * string>>`) instead of importing `SourceHashes`. The companion is written into
+ * whichever package owns the bundles, and only one of those packages can spell
+ * the type with a relative import; an import path guessed per package is a
+ * portability bug waiting for the second package to use this. The structural
+ * type is what `SourceHashes` is defined as, so nothing is lost.
+ *
+ * Keys are emitted sorted, and every key is quoted (they are dotted paths, so
+ * `formatKey` would quote them anyway). Both are load-bearing for `--check`:
+ * the comparison is byte-for-byte, so a table that reordered with the walk
+ * would fail on a tree that is in fact in sync.
+ */
+export function renderSourceHashModule(
+  hashes: Record<string, string>,
+  options: { locale: string; exportName?: string },
+): string {
+  const exportName = options.exportName ?? `${camelize(options.locale)}GeneratedSourceHashes`;
+  const keys = Object.keys(hashes).sort();
+  const lines: string[] = [];
+  lines.push('// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.');
+  lines.push('');
+  lines.push('/**');
+  lines.push(` * Auto-generated by 'os i18n extract' for locale '${options.locale}'. Do not hand-edit.`);
+  lines.push(' *');
+  lines.push(" * Each entry is the digest of the SOURCE REVISION that this locale's leaf at");
+  lines.push(' * that path is still a byte copy of — provenance for the generated half of the');
+  lines.push(' * bundles (#11671, maintainer ruling #12069 Option A, extending #8765 Option B).');
+  lines.push(' *');
+  lines.push(' * An entry exists only while the leaf IS such a copy. Re-translate the leaf in');
+  lines.push(' * `<locale>.objects.generated.ts` and the next extract drops its entry by');
+  lines.push(' * itself — the table makes no claim about text a translator wrote. A path with');
+  lines.push(' * no entry is LEGACY-TRUSTED and never reported stale.');
+  lines.push(' *');
+  lines.push(' * ⚠️ Do not "fix" a staleness report by editing this file. Refreshing a digest');
+  lines.push(' * records that the current text was copied from the current source, which is');
+  lines.push(' * the false claim the mechanism exists to detect. Fix the TRANSLATION.');
+  lines.push(' */');
+  lines.push('');
+  lines.push(`export const ${exportName}: Readonly<Record<string, string>> = {`);
+  for (const key of keys) lines.push(`  ${JSON.stringify(key)}: ${JSON.stringify(hashes[key])},`);
+  lines.push('};');
+  lines.push('');
+  return lines.join('\n');
+}
+
+/**
+ * Read a committed `<locale>.source-hashes.generated.ts` back into a table.
+ *
+ * The module body is written by {@link renderSourceHashModule}, which quotes
+ * every key and every value, so the object literal is already valid JSON — the
+ * parse needs no TypeScript and no evaluation. A file that does not parse is a
+ * hard `undefined` (treated as "no previous records", i.e. everything
+ * legacy-trusted) rather than a guess: inventing records from a file we cannot
+ * read is how a mechanism starts asserting provenance it does not have.
+ */
+export function parseSourceHashModule(source: string): Record<string, string> | undefined {
+  const marker = source.indexOf('export const');
+  const open = marker < 0 ? -1 : source.indexOf('= {', marker);
+  if (open < 0) return undefined;
+  const literal = source.slice(open + 2).replace(/;\s*$/, '');
+  try {
+    const parsed = JSON.parse(literal.replace(/,(\s*})/g, '$1'));
+    if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) return undefined;
+    for (const value of Object.values(parsed)) if (typeof value !== 'string') return undefined;
+    return parsed as Record<string, string>;
+  } catch {
+    return undefined;
+  }
+}
+
 function camelize(locale: string): string {
   // 'zh-CN' → 'zhCN', 'ja-JP' → 'jaJP', 'es-ES' → 'esES'
   return locale.replace(/-(.)/g, (_m, c) => c.toUpperCase());