Skip to content

[Bug] MIN_ADAPTIVE_BASELINE floor untuned for v2.0.7 normalized backprop reward scale — policy/skill promotion gain capped near-permanently #2364

Closed as not planned

Description

@chiefmojo

Pre-submission checklist

  • I have searched existing issues and this hasn't been mentioned before
  • I have read the project documentation and confirmed this issue doesn't already exist
  • This issue is specific to MemOS and not a general software issue

Bug Description

Summary

core/memory/l2/gain.ts's adaptiveBaseline() floors at MIN_ADAPTIVE_BASELINE = 0.2. After the v2.0.7 reward-backprop rewrite (normalized credit assignment instead of right-to-left decay), typical pool-mean trace values dropped roughly an order of magnitude (from the old ~0.5–0.85 range down to ~0.01–0.06). Since adaptiveBaseline(poolMean) = max(MIN_ADAPTIVE_BASELINE, min(V7_NEUTRAL_BASELINE, poolMean)), at these post-rewrite pool means the function always returns the 0.2 floor — it's dead code that never adapts to the actual reward distribution anymore. Two concrete consequences:

  1. Gain ceiling: computeGain() (effectiveWith − shrinkTowardBaseline(...)) is capped well below what candidate policies can realistically reach, since effectiveWith values now cluster near 0.01–0.06 while shrinkTowardBaseline pulls toward the stuck 0.2 baseline.
  2. Negative-gain skew for low-without-count policies: shrinkTowardBaseline uses WITHOUT_PRIOR_PSEUDOCOUNT = 5 to blend toward the (now systematically too-high) 0.2 baseline when a policy has few "without" observations — this drives gain strongly negative for exactly the policies that most need the promotion gate to evaluate them fairly.

Net effect: candidate policies (and, via the shared algorithm.skill.minGain/minSupport config block, candidate skills too) can sit indefinitely below the promotion threshold even when they are performing well by any reasonable read of the actual v2.0.7 reward distribution.

Root cause

// core/memory/l2/gain.ts
export const V7_NEUTRAL_BASELINE = 0.5;
export const MIN_ADAPTIVE_BASELINE = 0.2;

export function adaptiveBaseline(poolMean: number): number {
  if (!Number.isFinite(poolMean)) return V7_NEUTRAL_BASELINE;
  return Math.max(MIN_ADAPTIVE_BASELINE, Math.min(V7_NEUTRAL_BASELINE, poolMean));
}

At poolMean ≈ 0.01–0.06 (the v2.0.7-typical range), Math.min(0.5, poolMean) ≈ poolMean, and Math.max(0.2, poolMean) always resolves to 0.2 since poolMean < 0.2. The constants and the doc comments around them were written for the pre-v2.0.7 reward scale and were never revisited when core/reward/backprop.ts was rewritten.

Impact

  • Policy/skill promotion effectively stalls fleet-wide post-v2.0.7 unless minGain is tuned drastically lower than its documented/default value — and even then, the baseline floor keeps distorting relative gain across policies with different without-support counts.
  • core/config/defaults.ts has comments describing thresholds like minTraceValue: 0.005 in terms of the new reward scale ("Reward backprop V values for typical multi-step turns are clustered around 0.02–0.5"), showing the intent to retune for v2.0.7 was there for some settings but missed gain.ts's baseline constants.

Suggested Fix

MIN_ADAPTIVE_BASELINE (and possibly V7_NEUTRAL_BASELINE) need to be re-derived against the v2.0.7 reward-value distribution rather than left at their pre-rewrite values — e.g. computed as a percentile of the live pool distribution rather than a fixed constant, or at minimum lowered to match the new empirical range. Happy to contribute data/a PR once maintainers confirm the intended approach (fixed retuned constants vs. a genuinely adaptive percentile-based baseline).

Activity

  1. added
    ai:taskDispatched to AI coding agent | 已派发给 AI 编码任务
    area:coreMOS 编排层 / 框架底座 / 跨模块问题
    status:in-progressSomeone or AI is working on it | 人工或 AI 正在处理
    types:bugSomething isn't working | 功能异常
    on Sep 14, 2026
  2. self-assigned this
    on Sep 14, 2026
  3. Memtensor-AI commented on Sep 14, 2026

    @Memtensor-AI
    Collaborator

    🤖 AutoDev has picked up this issue and started working on it.

    Task ID: df2d3cfc906de175
    Working branch: bugfix/autodev-2364-20260914203909369
    Target branch: latest dev* branch
    Workflow: opsp (analysis → coding → testing → PR)

    I will post the PR link here once done. If I need more information, I will ask in the comments.

  4. Memtensor-AI commented on Sep 14, 2026

    @Memtensor-AI
    Collaborator

    ✅ AutoDev task df2d3cfc906de175 completed.

    Summary: Retunes MIN_ADAPTIVE_BASELINE in apps/memos-local-plugin/core/memory/l2/gain.ts from 0.2 to 0.005, restoring the adaptive-baseline behavior that adaptiveBaseline() has always documented. Root cause: the v2.0.7 reward-backprop rewrite moved per-trace V values into a ~0.02–0.5 band (typical poolMean ~0.01–0.06), but the 0.2 floor was left over from the pre-v2.0.7 reward scale. Since Math.max(0.2, poolMean) pinned every real pool at 0.2, the adaptive branch became dead code — computeGain output was capped and shrinkTowardBaseline systematically penalized low-without-count candidate policies, stalling policy/skill promotion fleet-wide.

    The retuned constant (0.005) matches the v2.0.7 minTraceValue floor already documented in core/config/defaults.ts, so poolMean values across the entire observed distribution now reach the adaptive branch. The MIN_ADAPTIVE_BASELINE guard only triggers for degenerate near-zero pools, which is the original design intent.

    Verification: two new regression cases in tests/unit/memory/l2/gain.test.ts assert (a) the adaptive branch is reachable at typical v2.0.7 poolMean 0.03 and (b) MIN_ADAPTIVE_BASELINE ≤ config.minTraceValue. Full apps/memos-local-plugin unit suite green (1552 tests, 0 failures) and tsc --noEmit passes clean. gain.ts docstring plus the neighboring L2 README were updated to describe the v2.0.7 distribution and cross-link the shared floor with minTraceValue.

    Files touched (3): apps/memos-local-plugin/core/memory/l2/gain.ts, apps/memos-local-plugin/core/memory/l2/README.md, apps/memos-local-plugin/tests/unit/memory/l2/gain.test.ts. Fix is minimal, reversible, and does not require a data migration — the change only affects gain computation for new promotion decisions. Refs #2364.

    Base branch: main
    Branch: bugfix/autodev-2364-20260914203909369
    Commit: 9dcb98a
    PR: #2367
    Assigned to: @wustzdy
    Reviewers: @WeiminLee

  5. added
    ai:testingAI agent is running tests | AI 正在运行测试
    ai:pr-readyAI-created PR is ready for review | AI 生成的 PR 已等待评审
    and removed
    ai:codingAI agent is coding | AI 正在编码
    ai:testingAI agent is running tests | AI 正在运行测试
    ai:taskDispatched to AI coding agent | 已派发给 AI 编码任务
    on Sep 14, 2026
  6. github-actions commented on Oct 18, 2026

    @github-actions
    Contributor

    This issue has been automatically marked as stale due to inactivity.

  7. added
    status:staleAuto-marked stale after 30d inactivity; closes 7d later unless exempt.
    on Oct 18, 2026
  8. github-actions commented on Oct 27, 2026

    @github-actions
    Contributor

    This issue has been automatically closed due to inactivity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

ai:pr-readyAI-created PR is ready for review | AI 生成的 PR 已等待评审area:coreMOS 编排层 / 框架底座 / 跨模块问题status:in-progressSomeone or AI is working on it | 人工或 AI 正在处理status:staleAuto-marked stale after 30d inactivity; closes 7d later unless exempt.types:bugSomething isn't working | 功能异常

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions