Skip to content

turn.start injection collapses to empty when top hits are dominated by near-duplicate question traces #1913

Closed

Description

@fancyboi999

Summary

turn.start returns injectedContext: "" for a query whose top retrieval hits are dominated by near-duplicate question traces from previous sessions — even though memory.search for the same query returns strong hits including the answer-bearing trace. Net effect: the more often a user asks the same question, the deafer the memory becomes for exactly that question.

Environment

  • main @ b60616d, compiled dist/bridge.cjs (real core + SQLite), Hermes adapter
  • embedding: local; reproduced with llm.provider: host and with no usable LLM (A/B tested — LLM availability is not the variable)

Reproduction (verified on a live install)

  1. Session 1: Remember this fact: HERMES_REAL_E2E_1910 means the bridge leak fix verification. → trace persisted ✓
  2. Session 2 (immediately after): ask What does HERMES_REAL_E2E_1910 mean? → recall works (clean store: the fact is the only hit)
  3. Ask the same question in 3 more separate sessions (each one stores its own question-trace)
  4. Now drive turn.start directly over JSON-RPC with the same question:
memory.search  → 4 hits (tier 2), including the answer-bearing trace
turn.start     → injectedContext: "" (0 chars)

Control test at the same moment, same config, against the same store: a clean marker (HERMES_HOST_LLM_0612, stored once, never re-asked) → turn.start injects 826 chars. So the collapse is driven purely by the near-duplicate question traces crowding the hit set — the injection filter drops them (reasonably, they're self-similar to the live query) but the answer-bearing trace is lost with them, and nothing is injected at all.

Why it matters

Repeated questions are precisely the signal that a memory matters to the user, yet that repetition poisons future recall for it. The failure is silent: capture keeps succeeding, memory.search keeps hitting, the user just stops getting injections and has no error to act on.

Suggested directions

  • After de-duplicating near-identical question traces from the candidate set, re-fill the injection budget from remaining hits (the answer trace was ranked right behind them) instead of returning empty
  • Consider down-weighting pure-question traces (no informative assistant text) at injection time, or merging Q-repeat traces with their answered ancestor
  • Emit a debug-level reason when injection collapses to empty despite non-empty retrieval (filteredAll=true) so this is diagnosable in the viewer

Found while validating #1910 / PR #1911 on a real hermes-agent v0.16.0 host; unrelated to the process-leak fix (reproduces on unpatched core paths — the injection pipeline is untouched by that PR).

Activity

  1. added
    ai:taskDispatched to AI coding agent | 已派发给 AI 编码任务
    types:bugSomething isn't working | 功能异常
    contrib:help wantedExtra attention is needed | 需要社区帮助
    area:coreMOS 编排层 / 框架底座 / 跨模块问题
    ai:codingAI agent is coding | AI 正在编码
    on Jun 12, 2026
  2. self-assigned this
    on Jun 15, 2026
  3. Memtensor-AI commented on Jun 15, 2026

    @Memtensor-AI
    Collaborator

    🤖 AutoDev has picked up this issue and started working on it.

    Task ID: ea3f7cba69f4a5b6
    Working branch: bugfix/autodev-1913
    Target branch: latest dev* branch
    Workflow: opsp (analysis → coding → testing → PR)

    I will post the PR link here once done. If I need more information, I will ask in the comments.

  4. Memtensor-AI commented on Jun 15, 2026

    @Memtensor-AI
    Collaborator

    ✅ AutoDev task ea3f7cba69f4a5b6 completed.

    Summary: Fixed issue #1913: turn.start injection collapsing to empty when the LLM relevance filter returned selected: [] over a non-empty ranked candidate list. The reproduction described in the issue happens when the top retrieval hits are dominated by near-duplicate question traces from previous sessions — the filter prompt's "drop scaffolding chatter / surface-similar wrong sub-problem" rubric applies to every candidate and the answer-bearing trace gets swept along, leaving injectedContext: "" even though memory.search for the same query still returns the same 4 hits.

    The fix adds a rescue path in apps/memos-local-plugin/core/retrieval/llm-filter.ts (rescueFromEmptySelection + isInformativeCandidate). When the LLM empties the kept set, the rescue partitions ranked candidates into informative (skill / episode / experience / world-model, or a trace with a non-trivial summary/reflection/agentText) and chatter (acknowledgement-only / near-duplicate question traces), then keeps the top-K ordered informative-first up to llmFilterMaxKeep. A new outcome label "llm_filtered_refilled" and a debug log llm_filter.collapsed_refill { filteredAll: true, ranked, rescued, informative, chatter } make the safety net diagnosable from the Logs viewer. sufficient is forced to false so downstream callers know the injection is weak. llmFilterMaxKeep: 0 is preserved as the explicit "drop everything" escape hatch.

    Implementation touched 5 files: core/retrieval/llm-filter.ts (rescue + new outcome), core/retrieval/types.ts (widen the RetrievalStats.llmFilterOutcome union), viewer/src/views/LogsView.tsx (count the new variant as "LLM ran" in the Logs viewer heuristic), tests/unit/retrieval/llm-filter.test.ts (4 new tests + replace the prior "honour empty" test), tests/unit/retrieval/integration.test.ts (turn_start end-to-end regression). All retrieval / bridge / pipeline unit tests pass (146 / 146); tsc --noEmit clean. Two pre-existing unrelated storage unit failures reproduced on the base branch and are outside this PR's boundary.

    Base branch: main
    Branch: bugfix/autodev-1913
    Commit: 8075271
    PR: #1923
    Assigned to: @MatthewZhuang, @CarltonXiang, @syzsunshine219
    Reviewers: @MatthewZhuang, @CarltonXiang, @syzsunshine219

  5. added
    ai:testingAI agent is running tests | AI 正在运行测试
    ai:pr-readyAI-created PR is ready for review | AI 生成的 PR 已等待评审
    and removed
    ai:codingAI agent is coding | AI 正在编码
    ai:testingAI agent is running tests | AI 正在运行测试
    ai:taskDispatched to AI coding agent | 已派发给 AI 编码任务
    on Jun 15, 2026
  6. added a commit that references this issue on Jul 1, 2026
    56644ba
  7. added and removed
    ai:pr-readyAI-created PR is ready for review | AI 生成的 PR 已等待评审
    on Jul 1, 2026
  8. added a commit that references this issue on Sep 18, 2026
    547a4ab
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:coreMOS 编排层 / 框架底座 / 跨模块问题contrib:help wantedExtra attention is needed | 需要社区帮助types:bugSomething isn't working | 功能异常

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions