Skip to content

fix(2c): separate policy gain from normalized reward credit - #2372

Open
chiefmojo wants to merge 9 commits into
MemTensor:mainfrom
chiefmojo:pr/wp272-policy-gain-repair
Open

chiefmojo wants to merge 9 commits into
MemTensor:mainfrom
chiefmojo:pr/wp272-policy-gain-repair

Conversation

@chiefmojo

@chiefmojo chiefmojo commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Separates policy gain (used for repair-eligibility/promotion decisions) from normalized reward credit V, and fixes a real promotion stall: with the current scheme, gain is a bare running average with no contributor-count scaling, so once enough negative-evidence traces accumulate, a policy's gain floor-locks below the promotion threshold even when recent evidence is strongly positive — candidates never get a fair look and the pool grows unbounded.

This is a deeper fix than #2364/#2367 (which only retunes MIN_ADAPTIVE_BASELINE, a constant, without changing the underlying scaling). It:

  • Persists a contribution-adjusted gainValue per trace, separate from V, with versioned historical provenance (live_normalized / inferred_normalized / legacy_unscaled / unresolved).
  • Ships an idempotent, bounded historical-inference pass (core/reward/gain-inference.ts) that runs once per boot, before the repair timer/consumers start, so partially-converted state is never observable. Bound is config-tunable (gainInferenceBootMaxGroups / gainInferenceBootTimeBudgetMs) for hosts with a large backlog.
  • Adds a budgeted, timer-driven repair engine (core/memory/l2/gain-repair.ts) that drains the candidate/active backlog on a configurable cadence (gainRepairBatchSize, gainRepairIntervalMs, gainRepairMaxTotal) instead of requiring a bulk one-shot migration.
  • Adds a read-only preview RPC (policies.gainPreview) and a policy-field CAS rollback RPC (policies.gainRollback) for safe operability.
  • Schema-only migration (019-policy-gain-value.sql): adds columns + the repair queue/journal tables. No data conversion, no queue seeding, no policy/trace writes at migration time — all of that happens in the idempotent TS passes above.

All of this is gated behind algorithm.l2Induction.gainV2Enabled (default false) — a blank/upgraded config is a no-op until explicitly turned on.

Burned in on three internal hosts at increasing corpus scale (the largest ~1.8k policy groups / ~28k traces) with zero errors/conflicts/failed across every observed repair tick, and real candidate→active promotions where the fleet had produced none in 16–69 days prior.

Test plan

  • npm run lint (tsc --noEmit) — clean, no new errors
  • npx vitest run — all green across the suites that cover this change (memory/l2, reward, storage, pipeline, config, bridge, agent-contract): 65 files / 611 tests passing. The only files that cannot execute in this environment are adapters/deepseek-harness-*, which need private @deepseek-ai/* packages; they are excluded from the counts rather than reported as failures.
  • Targeted suites for the new code: tests/unit/memory/l2/, tests/unit/reward/, tests/unit/storage/gain-repair-migration.test.ts, tests/unit/pipeline/gain-repair-*.test.ts, tests/unit/bridge/methods.test.ts, tests/unit/agent-contract/gain-maintenance-contract.test.ts, tests/unit/storage/owner-exact.test.ts, tests/unit/config/load.test.ts — all pass

@Memtensor-AI Memtensor-AI added area:plugin OpenClaw & Hermes status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Sep 16, 2026
@Memtensor-AI

Memtensor-AI commented Sep 16, 2026 •

Copy link
Copy Markdown
Collaborator

🤖 Open Code Review

Target: PR #2372
Task: 3d3589b921ab9726
Base: main
Head: pr/wp272-policy-gain-repair
Head SHA: 9ea3084176a79e24c62cb74ea44035624dcffe1f

🔍 OpenCodeReview found 52 issue(s) in this PR.

⚠️ Review warnings (14)

  • semantic groups split by file count and complete diff byte size; all selected files remain in coverage
  • tool output exceeded the byte/line budget; use narrower file ranges or paginated searches; omitted output is not reviewed
  • bounded analysis finalized with explicit task_done after reviewing all group files
  • bounded analysis finalized with explicit task_done after reviewing all group files
  • bounded analysis finalized with explicit task_done after reviewing all group files
  • bounded analysis finalized with explicit task_done after reviewing all group files
  • bounded analysis finalized with explicit task_done after reviewing all group files
  • bounded analysis finalized with explicit task_done after reviewing all group files
  • bounded analysis finalized with explicit task_done after reviewing all group files
  • bounded analysis finalized with explicit task_done after reviewing all group files
  • … 4 additional warning(s) omitted.

1. apps/memos-local-plugin/core/pipeline/memory-core.ts (L321-L324)

This union reconcile runs synchronously during bootstrap and is completely unbounded: reconcileGainRepairQueueFromEvidenceUnion lists up to 100_000 candidate + 100_000 active policies and performs getWithTraceIds + getGainRowsByIds + getByPolicy + upsert per policy inside one transaction (see recompute-gain.ts:523-602 — no maxGroups/time-budget parameter is accepted). Only the preceding inference pass is bounded (maxGroups/timeBudgetMs); a large DB can therefore stall startup indefinitely even though the inference budget was added exactly to avoid that. Either give the reconcile a per-policy/time budget (maxPolicies + deadline, resumable via a watermark) or defer it past startup (bootstrap returns, run it in a background task).


2. apps/memos-local-plugin/bridge/methods.ts (L380-L384)

For a destructive rollback this silently drops invalid entries: journalIds: ["a", 123] rolls back only a, and the caller believes the whole set was processed. The same filter is repeated inside rollbackGainRepair, so the truncation is invisible at both layers and produces a partial CAS batch instead of a rejected request. Validate the array element-by-element and reject (invalid_argument) on any non-string/empty entry.

💡 Suggested Change

Before:

        const journalIds = Array.isArray(p.journalIds)
          ? (p.journalIds as unknown[]).filter(
              (id): id is string => typeof id === "string" && id.length > 0,
            )
          : undefined;

After:

        let journalIds: string[] | undefined;
        if (Array.isArray(p.journalIds)) {
          journalIds = p.journalIds.map((id) => {
            if (typeof id !== "string" || id.length === 0) {
              throw new MemosError(
                "invalid_argument",
                `${method}: 'journalIds' must contain only non-empty strings`,
              );
            }
            return id;
          });
        }

3. apps/memos-local-plugin/core/pipeline/memory-core.ts (L3604-L3607)

This "exact namespace" check only tests truthiness, so namespace: {} (or { agentKind: 123 }) passes. namespaceParam in bridge/methods.ts is an unchecked cast, and ownerFromNamespace then silently normalizes missing fields to agentKind: "unknown" / profileId: "default" (core/runtime/namespace.ts:77-84). The RPC therefore does guess an owner instead of requiring one, contradicting the guarantee in the comment — a caller can preview/rollback the unknown/default owner's rows without ever naming it. Validate that agentKind and profileId are non-empty strings before building the owner (same for rollbackGainRepair).


4. apps/memos-local-plugin/bridge/methods.ts (L365-L366)

typeof p.limit === "number" accepts NaN. Downstream Math.max(1, Math.min(500, Math.floor(NaN))) yields NaN (gain-maintenance.ts preview), so slice(offset, offset + limit) silently returns an empty page and the response echoes limit: null after JSON serialization. Reject non-finite values here (or clamp with Number.isFinite / Number.isInteger) so a bad client gets invalid_argument rather than a confusing empty page.


5. apps/memos-local-plugin/core/pipeline/deps.ts (L139-L141)

The comment is inaccurate and reads as an unfinished editing draft: it says "all SIX v2 scoring/repair keys" but eight keys are listed, and "never read past the typed slice slice" has a duplicated word/broken sentence. Since this block is the reference projection for the same keys duplicated in l2ConfigSlice(), please fix the wording and, ideally, derive both projections from a single builder/pick so the count cannot drift.


6. apps/memos-local-plugin/core/pipeline/memory-core.ts (L4837-L4839)

The queue seeding failure is swallowed with a bare catch {}, so a broken upsertPending (schema drift, locked table) leaves imported policies unqueued with zero diagnostics while the surrounding import reports success. Preserve the "never fail the import" behaviour but log at warn with the policy id and error message.

💡 Suggested Change

Before:

            } catch {
              // Queue bookkeeping must never fail the policy import.
            }

After:

            } catch (err) {
              // Queue bookkeeping must never fail the policy import, but it
              // must stay diagnosable.
              log.warn("import.gain_queue_seed_failed", {
                policyId: dto.id,
                err: err instanceof Error ? err.message : String(err),
              });
            }

7. apps/memos-local-plugin/core/memory/l2/gain-maintenance.ts (L120-L127)

limit: 100_000 is silently clamped to 500 by the repos layer (clampLimit in core/storage/repos/_helpers.ts caps at 500), so this does not hydrate "the whole universe" as the comment claims — but total comes from an unbounded count(). Two consequences: (1) pagination is broken — for total > 500, pages past the 500 newest-by-updated_at rows are always empty even though total/offset say otherwise; (2) every call still fully recomputes up to 500 policies (each recomputePolicyGain issues several queries: with-links, per-linked-episode trace pages, getGainRowsByIds), then sorts them, even when the caller asked for limit: 50. Consider clamping total to the derivable set, or better, deferring recomputation to the requested page only (compute offset..offset+limit after ranking on cheap SQL-side columns).


8. apps/memos-local-plugin/core/memory/l2/gain-maintenance.ts (L486-L491)

The five-field CAS (status/support/gain/gain_version/updated_at) is implemented verbatim twice: once in the compare phase and once here in the apply transaction. The two copies must be kept in sync; if a future field is added to only one, the drift re-check silently stops guarding it. Extract a single helper, e.g. firstFieldMismatch(policy, recorded) => GainRollbackConflict["field"] | null, and use it in both phases.


9. apps/memos-local-plugin/core/memory/l2/gain-repair.ts (L544-L552)

runGainInference is invoked without maxGroups/timeBudgetMs, so it screens every episode group in the namespace (it opens its own db.tx per group, which becomes nested savepoints here) inside this single enclosing transaction. That means the entire historical screening runs synchronously while holding the write transaction open on the repair timer tick — no interleaving with request-path writes, and any late failure rolls back the whole re-screen (including the un-stamping performed just above). Consider bounding it (maxGroups/timeBudgetMs) and only recording the consumed marker for the groups actually completed, or running the screening outside the enclosing transaction with per-step durability, so a long pass does not block the DB.


10. apps/memos-local-plugin/core/memory/l2/gain-repair.ts (L186-L192)

{ kind: "failed" } is never produced by applyGainRepairAttempt (failures surface as thrown errors handled in the tick's catch), yet runGainRepairTick keeps a case "failed" branch for it. Dead union member + dead switch branch mislead readers into thinking the apply path can report failure. Remove the member and the corresponding case, or actually return it in place of throwing.


11. apps/memos-local-plugin/core/memory/l2/gain-repair.ts (L126-L129)

Two issues here: (1) nested ternary (maxTotal === null ? null : parsed === null ? 0 : ...) is hard to read and against the project's style rule — split it into explicit if branches; (2) when the persisted counter is corrupt, this reports remaining: 0 but attempted: 0, so policies.gainPreview shows "0 attempts" while runGainRepairTick silently returns early at cap <= 0. The only warning (gain_repair.budget_corrupt) is emitted inside reserveGainRepairAttempt, which is never reached in that case — the operator sees stalled repair with no diagnostic log. Return an explicit corrupt flag and log it from both the tick and the preview path.


12. apps/memos-local-plugin/core/memory/l2/l2.ts (L399-L406)

This continue drops the deferred link intents for the policy without persisting them: the association recorded in Step 1 for this matched trace is discarded, and the comment at the flush site documents that as intended. But the link is the only durable record that the trace belongs to this policy — recomputePolicyGain rediscovers evidence exclusively through persisted with-links ∪ sourceTraceIds ∪ the current run's traces. A policy whose only matching traces are currently unresolved (gainValue null/invalid, or unknown_owner) accumulates no links at all, so when those traces later become resolved via inference, nothing re-associates them and the policy stays permanently starved of support/gain. Consider persisting links regardless of the gain skip (the pre-change behaviour), or at least persisting them when the skip reason is evidence-transient (no_resolved_with) rather than unknown_owner.


13. apps/memos-local-plugin/core/memory/l2/recompute-gain.ts (L539-L540)

limit: 100_000 does not do what the comment above it claims. policies.list builds its page via buildPageClauses → clampLimit (core/storage/repos/_helpers.ts), which caps the limit at 500 and orders by updated_at DESC. So this "complete evidence union rebuild" only re-derives the 500 most-recently-updated candidate policies plus 500 active ones; every older policy is silently skipped, contradicting the function contract ("every candidate/active policy of the owner is re-derived"). This is not benign: the kv seed watermark is then written for the whole version, so the later reconcileGainRepairQueue path skips seeding too, and the skipped policies never enter the repair queue. Use statusIn: ["candidate", "active"] with an explicit keyset/offset pagination loop (or add a repo method that pages the owner's policies) instead of a magic limit.


14. apps/memos-local-plugin/core/memory/l2/recompute-gain.ts (L579-L582)

This rebuild runs on every boot (pipeline/memory-core.ts calls it unconditionally) and never reads GAIN_REPAIR_QUEUE_SEED_KEY — it only writes it. Consequently every candidate/active policy with ≥ 1 resolved with-trace is forced back to pending on each startup, including policies that were already repaired and removed from the queue. upsertQueue preserves attempt_count, but the timer reserves a budget unit before isNaturallyRepaired short-circuits the recompute, and that unit is deliberately never refunded (see applyGainRepairAttempt), so repeated boots keep consuming the durable per-policy/gainRepairMaxTotal ceiling for no-op attempts. Gate the rebuild on the persisted watermark (skip when the stored version/owner already matches and no stamp invalidated it), or skip policies that already carry gainVersion === 2 with unchanged inferenceVersion unless an explicit inference_refresh entry exists.


15. apps/memos-local-plugin/core/memory/l2/recompute-gain.ts (L155-L156)

input.namespace is required by every caller (l2.ts, gain-repair.ts, gain-maintenance.ts pass it) and documented as used for owner resolution ("falling back to the caller's exact namespace" in the field comment and the file header), but recomputePolicyGain never reads it — only the policy's own owner fields are used. Either implement the documented fallback (e.g. resolve the policy owner against input.namespace when the policy carries unknown/default) or drop the field so the public contract does not advertise behaviour that does not exist.


16. apps/memos-local-plugin/core/memory/l2/recompute-gain.ts (L395-L398)

The with-evidence union (tracePolicyLinks.getWithTraceIds(policy.id) ∪ policy.sourceTraceIds, deduped) is duplicated here and again inside reconcileGainRepairQueueFromEvidenceUnion. The reconcile comment explicitly requires the two paths to stay identical, so extract a shared helper (e.g. collectWithEvidenceIds(policy, tracePolicyLinks)) and use it in both places to prevent drift.


17. apps/memos-local-plugin/core/memory/l2/recompute-gain.ts (L706)

traces.ts already exports a named TraceGainRow interface for exactly this projection. Importing it makes the parameter type readable and keeps the owner/provenance field contract explicit instead of deriving it through two nested ReturnType lookups.

💡 Suggested Change

Before:

function toEvidenceFromGainRow(r: ReturnType<ReturnType<typeof makeTracesRepo>["getGainRowsByIds"]>[number]): GainEvidenceTrace {

After:

function toEvidenceFromGainRow(r: TraceGainRow): GainEvidenceTrace {

18. apps/memos-local-plugin/core/memory/l2/recompute-gain.ts (L193-L194)

isResolvedGainValue(gain) already guarantees non-null, so the extra gain != null only exists to satisfy TypeScript narrowing (the predicate is not a type guard). Declaring it as gainValue is number removes the redundant check here and also lets the reconcile loop drop its implicit null handling, keeping the shared predicate the single source of truth.

💡 Suggested Change

Before:

    const gain = trace.gainValue;
    return isResolvedGainValue(gain) && gain != null && gain >= config.minGainValue;

After:

    return isResolvedGainValue(trace.gainValue) && trace.gainValue >= config.minGainValue;

19. apps/memos-local-plugin/core/memory/l2/subscriber.ts (L31-L32)

episodes and gainRepair were added to the repos pick but are never referenced in this file (only traces, policies, tracePolicyLinks, candidatePool, embeddingRetryQueue are used). This needlessly widens the required dependency surface for every attachL2Subscriber caller — drop the unused entries.


20. apps/memos-local-plugin/core/pipeline/memory-core.ts (L3612-L3615)

previewGainRepair is documented/advertised as a paginated read-only sanity check, but the call it delegates to recomputes EVERY candidate/active policy of the namespace (it hydrates up to 100_000 rows and runs recomputePolicyGain per policy, each doing its own trace/link queries) before the slice(offset, offset + limit) is applied. So a limit: 10 request still pays the full-namespace cost, synchronously on the RPC/event-loop thread. On a large namespace this blocks the dispatcher and looks like a hang to adapters. Consider paginating/recomputing after slicing (or shipping a lower server-side cap) so preview cost is bounded by limit.


21. apps/memos-local-plugin/agent-contract/jsonrpc.ts (L110-L111)

Every existing method name in this registry is all-lowercase after the dot (skill.archive, memory.get_trace, config.patch). policies.gainPreview / policies.gainRollback are the only camelCase entries. Since names here are a public, non-removable contract (per the file header), an inconsistent casing now is hard to fix later — prefer policies.gain_preview / policies.gain_rollback (or policies.preview / policies.rollback) to match the rest of the registry before the names ship.


22. apps/memos-local-plugin/agent-contract/memory-core.ts (L450-L451)

The contract only models failures as the { ok: false; conflicts } variant, with not_found_or_forbidden for rows that are missing or foreign. The implementation instead THROWS invalid_argument when a batchId has no rows in the caller's namespace (and for missing/invalid arguments). Adapters written against this type therefore cannot distinguish "batch unknown" from a transport/protocol error, and an unknown batch never surfaces as a conflict. Either make the unknown-batch path return { ok: false, conflicts: [{ reason: "not_found_or_forbidden", policyId: null }] } or document the thrown invalid_argument case here.


23. apps/memos-local-plugin/core/memory/l2/gain-repair.ts (L400-L413)

gainVersion: 2 is hardcoded even though recomputed.gainVersion already carries the certified version of the calculation that was just performed (selectAndComputeGain returns 1 when scoreMode === "value", e.g. a legacy/unscored run or an inference_refresh performed while v2 is off). Today the tick gate (gainV2Enabled) makes the two agree, but this function is exported and re-used, so a future change of the flag/mode path would silently stamp an uncertified legacy gain as v2 — in the policy row AND in the journal (newGainVersion: 2 a few lines below). Use the recompute result so the stored certification always matches the calculation that produced it.

💡 Suggested Change

Before:

    const support = policy.support;
    let status: "candidate" | "active";
    if (policy.status === "active") {
      status = "active";
    } else {
      status =
        support >= deps.thresholds.minSupport && recomputed.persistedGain >= deps.thresholds.minGain
          ? "active"
          : "candidate";
    }
    deps.repos.policies.updateStats(policyId, {
      support,
      gain: recomputed.persistedGain,
      gainVersion: 2,

After:

    deps.repos.policies.updateStats(policyId, {
      support,
      gain: recomputed.persistedGain,
      gainVersion: recomputed.gainVersion,

24. apps/memos-local-plugin/core/memory/l2/l2.ts (L579-L581)

Using inline import("../../types.js").RuntimeNamespace twice in the signature/return type is hard to read and bypasses the file's existing type-import block (which already imports PolicyId, TraceId, etc. from ../../types.js). Add RuntimeNamespace to that import and reference it directly — this also keeps the cast consistent with namespaceFromOwner in gain-repair.ts, which uses a plain named type.

💡 Suggested Change

Before:

function namespaceFromPolicy(policy: PolicyRow): import("../../types.js").RuntimeNamespace {
  return {
    agentKind: (policy.ownerAgentKind ?? "unknown") as import("../../types.js").RuntimeNamespace["agentKind"],

After:

function namespaceFromPolicy(policy: PolicyRow): RuntimeNamespace {
  return {
    agentKind: (policy.ownerAgentKind ?? "unknown") as RuntimeNamespace["agentKind"],

25. apps/memos-local-plugin/core/memory/l2/recompute-gain.ts (L437)

This orphan-diagnostic block issues one extra COUNT(*) per linked episode on EVERY recompute, and recomputePolicyGain is called per touched policy inside the ordinary L2 hot path (l2.ts Step 4) and per repair attempt. For a policy linked to dozens of episodes this multiplies into hundreds of extra queries per episode settle, only to emit a debug log. Consider computing the signal once per episode (cached across policies in a single pass) or moving it behind an explicit debug/config flag instead of paying it unconditionally.


26. apps/memos-local-plugin/core/memory/l2/recompute-gain.ts (L595-L599)

This watermark is written without ownerWorkspaceId, which breaks the exact-namespace watermark contract documented on GAIN_REPAIR_QUEUE_SEED_KEY in reward/gain-inference.ts (its reader compares (stored.ownerWorkspaceId ?? null) === owner.ownerWorkspaceId). For any workspace-scoped owner the marker written here is therefore never recognized as "already seeded", so the two seed paths disagree about ownership of the watermark. Add ownerWorkspaceId to the payload (and type the value) so the shape matches the reader.


27. apps/memos-local-plugin/core/memory/l2/recompute-gain.ts (L263-L264)

selectAndComputeGain only iterates poolIds, so any id present in withIds but absent from poolIds is silently ignored (it is neither counted as dangling nor as with-evidence), while the returned withIds/result still reports it. That makes the exported helper easy to mis-call. Union withIds into the pool internally (const allIds = Array.from(new Set([...input.poolIds, ...input.withIds]))) so the two inputs can never contradict each other.


28. apps/memos-local-plugin/core/memory/l2/recompute-gain.ts (L476)

resetEma is derived only from the explicit flag and mode === "inference_refresh". A mode: "preview" call cannot express a queue-driven refresh reset, so once gain-maintenance.ts previews a policy whose queue reason is inference_refresh, isFirst stays false and the reported persistedGain (newGain) is EMA-blended — while the real repair (gain-repair.ts passes mode: "inference_refresh") resets the EMA and writes a different value. Either document that preview callers must pass resetEma from the queue row, or let the helper derive it (e.g. accept the queue reason) so preview and repair stay consistent.


29. apps/memos-local-plugin/core/reward/gain-inference.ts (L370-L371)

The owner predicate is missing parentheses, so AND binding tighter than OR reduces it to ((kind=@kind AND profile=@profile) OR kind='unknown'): every unknown-kind episode in the workspace is selected regardless of profile. screenGainGroup then only compares members against the episode owner (never against deps.owner), so a kind='unknown' episode of profile B is fully screened and stamped (gain_value, gain_inference_version) by the profile-A pass. Because the stamp is per-trace and gain_inference_version < @version is the only re-selection gate, profile B's own pass will never reconsider those rows — the stamp permanently preempts the other tenant's derivation. Please group the OR terms and scope the unknown bypass to the caller's profile (and ideally assert the episode owner equals deps.owner before stamping).

💡 Suggested Change

Before:

       WHERE (e.owner_agent_kind = @kind AND e.owner_profile_id = @profile
             OR e.owner_agent_kind = 'unknown')

After:

      WHERE ((e.owner_agent_kind = @kind AND e.owner_profile_id = @profile)
             OR (e.owner_agent_kind = 'unknown' AND e.owner_profile_id = @profile))

30. apps/memos-local-plugin/core/reward/gain-inference.ts (L411-L412)

The safety defaults are inverted: omitting maxGroups/timeBudgetMs yields a completely unbounded synchronous pass, which is exactly the initWatchdogMs crash-loop scenario the file header warns about. There is already a production caller that omits both bounds (core/memory/l2/gain-repair.ts:545 calls runGainInference with only inferenceVersion), so a large backlog will block that path unbounded. Default to the boot bounds (or make the fields required) so only an explicit Infinity can disable the cap.

💡 Suggested Change

Before:

  const maxGroups =
    deps.maxGroups === undefined ? Number.POSITIVE_INFINITY : Math.max(1, Math.floor(deps.maxGroups));

After:

  const maxGroups = Math.max(
    1,
    Math.floor(deps.maxGroups ?? GAIN_INFERENCE_BOOT_MAX_GROUPS),
  );

31. apps/memos-local-plugin/core/reward/gain-inference.ts (L611-L614)

Spreading the whole report into a single log.info line includes report.affectedTraceIds, which accumulates every stamped trace id (up to thousands per run) — one very large log record plus the array being retained for the whole pass. No production consumer reads affectedTraceIds (the queue seeding derives its set from listTraceIdsStampedAt(version) in reconcileGainRepairQueueFromEvidenceUnion), so it is effectively log-only. Log the count and keep the id list out of the report/log.

💡 Suggested Change

Before:

  log.info("gain_inference.done", {
    version,
    ...report,
  });

After:

  const { affectedTraceIds, ...summary } = report;
  log.info("gain_inference.done", {
    version,
    ...summary,
    affectedTraceIds: affectedTraceIds.length,
  });

32. apps/memos-local-plugin/core/reward/gain-inference.ts (L626)

reconcileGainRepairQueue has no production caller — bootstrap (core/pipeline/memory-core.ts:321) uses reconcileGainRepairQueueFromEvidenceUnion, and only tests reference this one. That leaves ~80 lines of exported-but-unused startup logic whose docstring claims to be the durable seeding path. In addition, the docstring/implementation require per-owner namespacing (kind+profile+workspace are compared on read) but the watermark is a single global key, so in a multi-owner DB each owner's write invalidates the others and kv.del(GAIN_REPAIR_QUEUE_SEED_KEY) in this file clears it for every owner. Please either delete this function (and its tests) in favour of the evidence-union rebuild, or make the key owner-scoped (e.g. suffix with kind/profile/workspace).


33. apps/memos-local-plugin/core/experience/feedback-builder.ts (L411-L412)

mergePolicy unconditionally downgrades gainVersion to 1, even when the stored gain is not overwritten: gain: Math.max(existing.gain, draft.salience, 0.02) keeps existing.gain whenever it is already >= draft.salience. In that case a v2-certified policy (gainVersion 2, gain unchanged) is silently demoted by an unrelated merge, which drops it out of the v2 evidence/skill gates ((policy.gainVersion ?? 1) !== 2) until some later recompute re-certifies it. Only invalidate certification when the gain actually changes.

💡 Suggested Change

Before:

    gainVersion: 1,
    status: existing.status === "archived" ? existing.status : "active",

After:

    gainVersion: draft.salience > existing.gain ? 1 : (existing.gainVersion ?? 1),
    status: existing.status === "archived" ? existing.status : "active",

34. apps/memos-local-plugin/core/storage/repos/traces.ts (L397)

This lookup filters on gain_inference_version, but migration 019 only adds the column (ALTER TABLE traces ADD COLUMN gain_inference_version ...) and creates no index on it (the indexes it creates are for gain_repair_queue / gain_repair_journal). The query therefore performs a full scan of traces — the widest table in the DB (text, tool_calls_json, vec_* blobs) — every time reconcileGainRepairQueue runs and the watermark is stale (startup / version bump), and it materializes an unbounded id array in memory. Consider adding CREATE INDEX IF NOT EXISTS idx_traces_gain_inference_version ON traces(gain_inference_version) in migration 019 (or a covering (gain_inference_version, id) index) and bounding the result set. Separately, AND gain_inference_version > 0 is redundant for any real version (GAIN_INFERENCE_VERSION = 1) and makes version = 0 silently return nothing; if that is the intended "0 means never screened" semantics, an explicit early return or a comment would be clearer than a tautological predicate in SQL.


35. apps/memos-local-plugin/core/storage/repos/traces.ts (L348)

The "Preserves input order" claim does not hold: the SELECT has no ORDER BY (ordering an IN list by caller order isn't possible in SQLite) and rows are pushed per chunk in database-return order, so a multi-chunk call returns chunk 1's rows first, and within a chunk the order is whatever the index/PK scan yields. Today's callers (gain-inference.ts, recompute-gain.ts, gain-repair.ts) all key the result by id, so there is no live bug, but the doc invites positional alignment. Please correct the comment (or add an explicit ordering guarantee if one is needed).

💡 Suggested Change

Before:

     * payloads). Preserves input order and de-duplicates.

After:

     * payloads). Duplicate ids are de-duplicated; row order is NOT the
     * input order (rows come back per chunk in database order), so callers
     * must key results by `id`.

36. apps/memos-local-plugin/core/storage/migrator.ts (L217-L219)

Skipping the whole file means the two brand-new tables (gain_repair_queue, gain_repair_journal) are never created whenever traces or policies is absent — yet runMigrations still records version 19 in schema_migrations (the insert at line 119 is unconditional inside the same db.tx), so the DB can never acquire them later. Since makeRepos constructs makeGainRepairRepo unconditionally and StorageDb.prepare compiles eagerly (raw.prepare in connection.ts), any such database that later gets policies/traces will throw no such table: gain_repair_queue at repo construction. Only the ALTER TABLE ... ADD COLUMN statements are schema-dependent; the table DDL is valid even when the FK targets do not exist yet (SQLite resolves FKs at DML time). Suggest moving the queue/journal DDL into a separate unguarded migration (or an ensureGainRepairTables(db) helper called before the guard) so the guard only gates the column additions.


37. apps/memos-local-plugin/core/storage/migrations/019-policy-gain-value.sql (L108-L109)

This column belongs to a table that is created by this very file, so it should be declared inline in the CREATE TABLE gain_repair_journal body rather than appended with a trailing ALTER TABLE. As written, the file is not self-idempotent: running it twice (or re-running it against a DB whose journal table was created by an earlier revision of this file) fails with duplicate column name: new_updated_at, and JOURNAL_COLUMNS in repos/gain-repair.ts already assumes the column exists, so an out-of-sync table breaks every journal insert.

💡 Suggested Change

Before:

ALTER TABLE gain_repair_journal
  ADD COLUMN new_updated_at INTEGER;

After:

-- declare `new_updated_at INTEGER` inside the CREATE TABLE gain_repair_journal(...) body
-- and drop this trailing ALTER TABLE.

38. apps/memos-local-plugin/core/storage/repos/gain-repair.ts (L344)

The returned methods insertJournal, updateJournalOutcome and setJournalResult have the same identifiers as the prepared-statement consts declared above them. It happens to work today only because shorthand method names are not local bindings, so insertJournal.run(...) still resolves to the outer const. This is fragile (any refactor that adds a local binding or reorders the object breaks the shadowing silently) and confusing to read. Rename one side, e.g. insertJournalStmt / updateJournalOutcomeStmt / setJournalResultStmt.


39. apps/memos-local-plugin/core/storage/repos/gain-repair.ts (L158-L159)

The DO UPDATE SET list refreshes state/reason/inference_version/blocked_reason/updated_at but never the three owner columns, so the row keeps whatever namespace it was first seeded under. If the same policy id is ever re-seeded under a different owner, all owner-scoped reads (listPendingForRepair, reconcileArchivedOrMissing, countByState) keep using the stale namespace, and the row becomes unreachable for the correct namespace. Refreshing the owner columns in the conflict clause keeps the queue row's owner consistent with the policy it points at, at no cost.

💡 Suggested Change

Before:

     ON CONFLICT(policy_id) DO UPDATE SET
       state = excluded.state,

After:

     ON CONFLICT(policy_id) DO UPDATE SET
       owner_agent_kind = excluded.owner_agent_kind,
       owner_profile_id = excluded.owner_profile_id,
       owner_workspace_id = excluded.owner_workspace_id,
       state = excluded.state,

40. apps/memos-local-plugin/core/storage/repos/gain-repair.ts (L141-L142)

newUpdatedAt is optional and updateJournalOutcome silently coerces a missing value to NULL, which per migration 19 permanently makes the row NOT rollback-eligible (the five-field CAS cannot prove the post-write state). Current callers in memory/l2/gain-repair.ts do pass it on the completing path, but nothing in the type prevents a future result: "completed" write from omitting it and silently losing the ability to undo that policy write. Encode the invariant in the type, e.g. a discriminated union { result: "completed"; newUpdatedAt: number; ... } | { result: Exclude<..., "completed">; newUpdatedAt?: number | null }, so the compiler enforces it instead of a silent NULL.


41. apps/memos-local-plugin/core/storage/repos/policies.ts (L134-L137)

This ~24-line filter block (statusIn placeholders + minSupport + the three owner conditions) is now duplicated verbatim in list() and count(). Keeping two copies of the same WHERE builder is how page/count results silently drift apart (e.g. a future fix to the NULL-exact owner_workspace_id IS @owner_workspace_id handling applied to only one of them). Extract a small shared buildPolicyFilter(filter, params): string[] helper and call it from both, so count() can never disagree with list().


42. apps/memos-local-plugin/core/reward/gain-inference.ts (L318)

report.invalidJsonGroups is the only query in the pass that is not owner-scoped: SELECT COUNT(*) ... FROM episodes WHERE json_valid(trace_ids_json) = 0 counts invalid-JSON episodes of every owner in the shared DB (and json_valid(NULL) evaluates to NULL, not 1/0, so NULL trace_ids_json rows are silently excluded). The startup audit log then reports a workspace-wide number labelled as if it belonged to the owner being inferred, which is misleading when triaging a single namespace. Scope it with the same owner predicate used by selectCandidates (and treat NULL explicitly).

💡 Suggested Change

Before:

      `SELECT COUNT(*) AS n FROM episodes WHERE json_valid(trace_ids_json) = 0`,

After:

      `SELECT COUNT(*) AS n FROM episodes
        WHERE json_valid(COALESCE(trace_ids_json, 'null')) = 0
          AND owner_agent_kind = @kind AND owner_profile_id = @profile
          AND owner_workspace_id IS @workspace_id`,

43. apps/memos-local-plugin/core/reward/gain-inference.ts (L528)

A trace id listed in episodes.trace_ids_json but missing from traces is synthesized here as a member of this episode with value: NaN and episodeId: id. Because members.length === S.length always holds for this call site, the dedicated missing_listed_member outcome in screenGainGroup is unreachable in production, and a genuine data-integrity problem (S references a non-existent trace) gets reported as invalid_value/mixed_ownership instead — which sends operators debugging the wrong cause. Let the synthetic member be dropped so the length mismatch produces missing_listed_member.

💡 Suggested Change

Before:

            episodeId: tr ? String(tr.episodeId) : id,

After:

        members: S.flatMap((tid) => {
          const tr = byId.get(String(tid));
          // A listed id with no trace row must surface as missing_listed_member
          // (length mismatch), never as a synthetic NaN member.
          if (!tr) return [];
          return [{
            id: String(tid),
            episodeId: String(tr.episodeId),
            value: tr.value,
            rHuman: tr.rHuman,
            ts: tr.ts,
            ownerAgentKind: tr.ownerAgentKind,
            ownerProfileId: tr.ownerProfileId,
            ownerWorkspaceId: tr.ownerWorkspaceId,
          }];
        }),

44. apps/memos-local-plugin/core/storage/repos/traces.ts (L294-L295)

stampGain only guards against overwriting live_normalized, but not against a stale pass overwriting newer provenance. Two entry points call the inference pass (memory-core.ts boot pass at GAIN_INFERENCE_VERSION and gain-repair.ts generation-bump re-screen at a possibly newer version); if a boot pass is still in flight when the repair path un-stamps/screens at a newer version (or vice versa), the older pass can stampGain last and downgrade gain_inference_version back to its own version while clobbering the newer gain. That downgrade also makes listTraceIdsStampedAt(newVersion) miss the row, so the durable queue reconciliation drops the affected policy. Adding a monotonic guard makes the stamp order-independent.

💡 Suggested Change

Before:

            WHERE id = @id
              AND (gain_value_source IS NULL OR gain_value_source != ${SQL_LIVE_GAIN_SOURCE})`,

After:

            WHERE id = @id
              AND (gain_value_source IS NULL OR gain_value_source != ${SQL_LIVE_GAIN_SOURCE})
              AND gain_inference_version <= @version`,

45. apps/memos-local-plugin/core/storage/repos/traces.ts (L1203-L1204)

The "gainValue/gainValueSource are an atomic pair" invariant is enforced only in updateScore. insert/upsert go through rowToParams, which silently writes half-pairs (gainValue set with gainValueSource null, or a source with a null value). Such a row has NULL provenance, so the inference candidate selection treats it as unscored and later overwrites it, and the write looks successful — a silent data-integrity hole for the same invariant. Consider validating/normalizing here (throw or coerce both to NULL) so all write paths share one contract.


46. apps/memos-local-plugin/core/storage/repos/traces.ts (L382-L384)

The keyset paging here keys on id, but no index covers (episode_id, id) — the only episode-scoped index is idx_traces_episode_ts(episode_id, ts). For the malformed/wrong-shaped episode groups this method exists for, SQLite must therefore either scan the PK index filtering on episode_id (cost proportional to total trace count per page) or materialize + sort all of the episode's members on every page, making the loop in gain-inference.ts quadratic on large episodes. Either add an (episode_id, id) index in migration 019 or keyset on the existing (episode_id, ts) order (WHERE episode_id = ? AND (ts, id) > (?, ?)).


47. apps/memos-local-plugin/core/storage/migrations/019-policy-gain-value.sql (L77)

policies is written with INSERT OR REPLACE (repos/policies.ts upsert, used by feedback.ts:350, experience/feedback-builder.ts:105, memory/l2/l2.ts:279). With foreign_keys = ON (connection.ts), REPLACE deletes the conflicting parent row first, so this ON DELETE SET NULL silently nulls out policy_id on every journal row of that policy — and gain-maintenance.ts rollback explicitly requires row.policyId != null to consider a row eligible (not_rollback_eligible otherwise). Likewise gain_repair_queue's ON DELETE CASCADE deletes pending repair entries whenever a policy is upserted via feedback, so the repair is silently lost until the next full re-seed. Since the queue/journal tables are new, consider dropping the FK (reconciling explicitly, as reconcileArchivedOrMissing already does) or switching policies.upsert to ON CONFLICT(id) DO UPDATE so parent rows are never deleted.


48. apps/memos-local-plugin/core/storage/repos/gain-repair.ts (L234-L235)

setQueueState declares lastAttemptAt?: number | null / lastAttemptBatchId?: string | null, which implies null means "clear", but COALESCE makes null behave identically to "leave unchanged". A caller attempting to reset these fields (e.g. when a queue entry is deliberately re-seeded to pending) will silently keep stale attempt metadata that points at an old batch, which reconcileInterruptedGainRepairClaims then uses to mark the wrong journal rows failed. Either narrow the patch types to number / string (no null) or use a CASE like the blocked_reason branch to distinguish "omitted" from "set to null".


49. apps/memos-local-plugin/core/storage/repos/gain-repair.ts (L547)

removeAllForOwner (and countByState further down) are never referenced anywhere in the repo, including tests — the only owner-wide/state-count readers use reconcileArchivedOrMissing and listByOwnerAndStates. Unused prepared statements (deleteByOwner) are also created eagerly at repo construction. Please drop them or add the callers/tests that justify them.


50. apps/memos-local-plugin/core/storage/repos/gain-repair.ts (L452-L453)

listByOwnerAndStates re-prepares a statement on every invocation (it is on the startup inference/maintenance path), even though states can only be drawn from the 3-value GainRepairQueueState enum. Since SQLite already caches compiled statements, consider a fixed state IN ('pending','blocked','claimed')-style statement with an enum-encoded filter, or memoize the prepared statements by the sorted state signature, instead of db.prepare(...) per call.


51. apps/memos-local-plugin/core/config/schema.ts (L365)

The doc comment above claims this is "capped well under bridge.initWatchdogMs", but maximum: 300_000 (5 min) contradicts that: core/reward/gain-inference.ts explicitly designs the pass against "the 120s watchdog default", and core/pipeline/memory-core.ts passes the configured value straight through (timeBudgetMs: config.algorithm.l2Induction.gainInferenceBootTimeBudgetMs ?? GAIN_INFERENCE_BOOT_TIME_BUDGET_MS) with no clamp. An operator setting 300_000 will therefore let the synchronous boot pass outlive the watchdog, which is exactly the restart crash-loop the comment says must be prevented. Either lower the max to stay under the watchdog (e.g. 60_000) or validate it against the configured watchdog value at load time.


52. apps/memos-local-plugin/templates/config.hermes.yaml (L48)

gainInferenceBootMaxGroups and gainInferenceBootTimeBudgetMs are declared in core/config/defaults.ts / schema.ts (and their schema comments tell operators to raise them when a backlog does not converge in one restart), but they are missing here, in config.openclaw.yaml, and in config.demo.yaml. templates/README.md requires every config field to be present in the templates ("These templates are the source of truth for the JSON Schema ... If you add a field, add it in both"), so these two knobs are currently undiscoverable for operators. Please add them to all three templates.

Generated by cloud-assistant via Open Code Review.

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

⚠️ Automated Test Results: ENV ISSUE

The test environment encountered an issue that requires manual attention.

Details: Environment preparation failed before any gating tests executed. Failed scopes: memos_local_plugin
Branch: pr/wp272-policy-gain-repair

L2 gain was a bare running average with no contributor-count scaling, so a
policy's gain floor-locks below the promotion threshold once enough
negative-evidence traces accumulate: recent strongly-positive evidence can
never lift it back over the line, the candidate is never promoted, and the
candidate pool grows without bound.

Score gain as clamp(N·V, -1, 1) — N the number of contributing episodes, V
the normalized reward credit — and persist it alongside V:

- Persist a contribution-adjusted `gainValue` per trace with versioned
  provenance (`live_normalized` / `inferred_normalized` / `legacy_unscaled`
  / unresolved), so partially-converted state is never observable.
- Add an idempotent, bounded historical-inference pass that runs once per
  boot, before the repair timer and its consumers start. Both bounds are
  config-tunable for hosts with a large backlog.
- Add a budgeted, timer-driven repair engine that drains the candidate and
  active backlog on a configurable cadence, instead of requiring a bulk
  one-shot migration.
- Add a read-only preview RPC (`policies.gainPreview`) and a policy-field
  CAS rollback RPC (`policies.gainRollback`) for safe operability.
- Migration 019 is schema-only: new columns plus the repair queue and
  journal tables. No data conversion, queue seeding, or policy/trace writes
  at migration time — all of that happens in the idempotent passes above.

Everything is gated behind `algorithm.l2Induction.gainV2Enabled` (default
false): an upgraded config is a no-op until an operator turns it on.
A large corpus can fail to converge its historical gain-value inference in
a single bridge restart under the built-in 30s / 2000-group bound. Expose
`gainInferenceBootMaxGroups` and `gainInferenceBootTimeBudgetMs` under
`algorithm.l2Induction` so a host can raise the budget for one restart
instead of requiring many manual restarts to converge.
@chiefmojo
chiefmojo force-pushed the pr/wp272-policy-gain-repair branch from b657e28 to d6a0a5a Compare September 16, 2026 03:05
`applyMigration` wraps migrations that touch `traces` in a `tableExists` check
(010/012/018): some harnesses build partial schemas where those tables are
absent, and migration 018's comment states the rule - the statement is
meaningless there and must not fail boot.

Migration 019 issued bare `ALTER TABLE traces` / `ALTER TABLE policies`, so it
aborted the migration pass on any partial schema. That is the regression behind
tests/unit/storage/migrator.test.ts ("treats embedding retry lease migration as
satisfied when columns already exist"), which passes on main and failed on this
branch.
@chiefmojo

chiefmojo commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor Author

Open Code Review findings — dispositions

Fixed

Rebutted

Deferred

None.

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

⚠️ Automated Test Results: ENV ISSUE

The test environment encountered an issue that requires manual attention.

Details: Environment preparation failed before any gating tests executed. Failed scopes: memos_local_plugin
Branch: pr/wp272-policy-gain-repair

@github-actions

Copy link
Copy Markdown
Contributor

This PR has been automatically marked as stale due to inactivity.

@github-actions github-actions Bot added the status:stale Auto-marked stale after 30d inactivity; closes 7d later unless exempt. label Oct 17, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This PR has been automatically closed due to inactivity.

@github-actions github-actions Bot closed this Oct 26, 2026
@syzsunshine219 syzsunshine219 reopened this Nov 9, 2026
@Memtensor-AI Memtensor-AI removed the status:stale Auto-marked stale after 30d inactivity; closes 7d later unless exempt. label Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:plugin OpenClaw & Hermes status:in-progress Someone or AI is working on it | 人工或 AI 正在处理

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants