Skip to content

fix(plugin): report filter budget timeouts as llm_timeout and make the budget configurable - #2641

Open
syzsunshine219 wants to merge 2 commits into
mainfrom
codex/maintainer-edit-pr-2594-20261119
Open

syzsunshine219 wants to merge 2 commits into
mainfrom
codex/maintainer-edit-pr-2594-20261119

Conversation

@syzsunshine219

Copy link
Copy Markdown
Collaborator

The source branch for the change proposed in #2594 now lives in MemTensor/MemOS, so repository writers can contribute fixes through the normal upstream branch permissions.

The code and commit authors are unchanged. This PR retains target branch main and exact source head 9e0b217f79c26ae1384b34d2bc70e8b6be8f67bd from #2594.

Previous discussion and review remain in #2594. Review and checks must be re-evaluated for this replacement PR; no previous approval or new test result is asserted by this migration.

Previous PR description and implementation details

Restores the change proposed in #2506 for review against main after its former base branch, dev-v2.0.36, was deleted. The source commits and authors are preserved.

Source head: 9e0b217f79c26ae1384b34d2bc70e8b6be8f67bd

This PR is open for review. Review and CI against the current base are pending; validation claims in the original description below refer to the earlier snapshot. The original discussion and evidence remain in #2506.


Original change description

Description

Two stacked defects made local OpenAI-compatible endpoints unusable for the retrieval LLM filter (issue #2499): on every turn_start the filter's local call was doomed to abort at its 2-second cap, the abort was misreported as llm_unavailable, and the plugin silently replayed the filtering on the fallback (host/paid) provider.

Root cause:

  1. Budget timeout misclassified as provider-unavailable. mergeSignals (apps/memos-local-plugin/core/llm/fetcher.ts:382) called ctrl.abort() with no reason, so whenever the caller passed its own signal — the retrieval filter always does — AbortSignal.timeout()'s TimeoutError reason was discarded and fetch rejected with a bare AbortError. isTimeout (apps/memos-local-plugin/core/llm/fetcher.ts:338) could not recognize it either: the legacy check compares code === "ABORT_ERR", but DOMException.code is the numeric 20, so the comparison never matched. A budget timeout was therefore classified llm_unavailable and replayed on the host provider.
  2. Hard-coded filter budget. filterTimeoutMs (apps/memos-local-plugin/core/retrieval/retrieve.ts:691) clamped the filter call to a hard-coded 2_000 ms — too short for self-hosted models on consumer hardware.

Fix:

  • apps/memos-local-plugin/core/llm/fetcher.ts — mergeSignals now forwards the abort reason, so AbortSignal.timeout()'s TimeoutError survives when the caller passes its own signal; isTimeout additionally classifies AbortError / DOMException.code === 20 as a timeout. Caller cancellation keeps its distinct path (it is detected in httpPostJson via opts.signal?.aborted before this classification runs), so user-interrupted requests behave exactly as before.
  • apps/memos-local-plugin/core/retrieval/retrieve.ts (call sites at lines 432 and 493, filterTimeoutMs at line 691) plus the config plumbing (core/config/schema.ts:599, core/config/defaults.ts:322, core/retrieval/types.ts:331, core/pipeline/deps.ts:170) — new algorithm.retrieval.llmFilterTimeoutMs option (range 100–30 000 ms, default 2_000 preserved, documented in the schema) caps a single filter call. It is still clamped by the remaining retrieval deadline, so a larger value cannot extend the overall retrieval window.

Reporter expectation ① (force-disabling the filter via algorithm.lightweightMemory) is intentionally left untouched — the lightweight-mode override of llmFilterEnabled is the documented lightweight-mode contract (issue #2063).

Regression tests cover: budget-timeout classification (red: llm_unavailable, green: llm_timeout), configurable budget, legacy default, deadline clamping, and the caller-cancellation distinction.

Related Issue (Required): Fixes #2499

Type of change

  • Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

  • Unit Test
  • Test Script Or Test Steps (please provide)

Red first, on unpatched main @ a7367d0 with only the new tests applied: the timeout-classification test fails with exactly the reported symptom (llm_unavailable, "This operation was aborted"); the budget-config test fails with 2000 ≠ 4321. All green after the fix — run from apps/memos-local-plugin/ on macOS arm64, Node 24.21.0:

  • npx vitest run tests/unit/llm/fetcher.test.ts tests/unit/retrieval/integration.test.ts → 41 passed
  • npm run test:unit → 1594 passed / 2 skipped / 0 failed
  • npm test (release gate: unit + integration + e2e) → 186 files, 1602 passed / 3 skipped / 0 failed
  • npm run lint (tsc --noEmit) → clean, exit 0

Reproduce from apps/memos-local-plugin/:

npx vitest run tests/unit/llm/fetcher.test.ts tests/unit/retrieval/integration.test.ts   # includes the #2499 regression tests
npm run test:unit
npm test
npm run lint

Checklist

  • I have performed a self-review of my own code | 我已自行检查了自己的代码
  • I have commented my code in hard-to-understand areas | 我已在难以理解的地方对代码进行了注释
  • I have added tests that prove my fix is effective or that my feature works | 我已添加测试以证明我的修复有效或功能正常
  • I have linked the issue to this PR (if applicable) | 我已将 issue 链接到此 PR(如果适用)

(The MemOS-Docs checkbox is not applicable: the new option is documented in-repo via the algorithm.retrieval schema description in core/config/schema.ts. The "mentioned the person who will review" checkbox is left unchecked as no reviewer has been assigned yet.)

AI disclosure

This fix was developed with AI assistance (an AI coding agent in an automated contributor pipeline): code change, regression tests, and this PR description. All checks listed above were executed locally; the result was reviewed before submission.

…e budget configurable

Two stacked defects made local OpenAI-compatible endpoints unusable for
the retrieval LLM filter (issue #2499): every turn_start fired a doomed
local call that was misreported as llm_unavailable and replayed on the
fallback (host) provider.

- core/llm/fetcher.ts: mergeSignals forwarded aborts without a reason,
  flattening AbortSignal.timeout()'s TimeoutError into a bare AbortError
  whenever the caller passed its own signal (the retrieval filter always
  does); isTimeout could not recognize it either — DOMException.code is
  the numeric 20, so the legacy "ABORT_ERR" string comparison never
  matched. Forward the abort reason and classify AbortError / code 20 as
  a timeout (caller cancellation is handled before this classification).
- core/retrieval/retrieve.ts: the filter wall-clock budget was hard-coded
  at 2000 ms. Add algorithm.retrieval.llmFilterTimeoutMs (schema +
  defaults, default 2000 preserves current behavior) and clamp the filter
  budget with it, so self-hosted endpoints can opt into a larger window.

Regression tests: budget-timeout classification (red: llm_unavailable,
green: llm_timeout), configurable budget, legacy default, deadline
clamping, and caller-cancellation distinction.

Fixes #2499
…n fetcher

Tighten isTimeout so a bare AbortError is only treated as a timeout when
its reason is a TimeoutError (i.e. from AbortSignal.timeout()); a caller
ctrl.abort() racing past the opts.signal?.aborted guard is no longer
misclassified. In mergeSignals, prefer the TimeoutError reason when both
signals are already aborted so downstream classification stays accurate.

Addresses review feedback on PR #2506.
@syzsunshine219 syzsunshine219 added the area:plugin OpenClaw & Hermes label Nov 19, 2026
@Memtensor-AI Memtensor-AI added the status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 label Nov 19, 2026
@Memtensor-AI

Copy link
Copy Markdown
Collaborator

🤖 Open Code Review

Target: PR #2641
Task: 2e0f3fbb3e55506a
Base: main
Head: codex/maintainer-edit-pr-2594-20261119
Head SHA: 9e0b217f79c26ae1384b34d2bc70e8b6be8f67bd

🔍 OpenCodeReview found 7 issue(s) in this PR.


1. apps/memos-local-plugin/core/retrieval/retrieve.ts (L695-L698)

The configured cap is silently ignored whenever there is no retrieval deadline: filterTimeoutMs returns undefined as soon as deadlineAt is undefined, so llmFilterTimeoutMs never reaches completeJson on those paths. This is a real gap, not just a test-only case — retrieveToolDriven / retrieveSkillInvoke / retrieveSubAgent / retrieveRepair (orchestrator.ts:1148-1182) call the retrieval entry points without any opts.deadlineAt, so tool-driven memos_search and sub-agent retrievals keep using the LLM client's own (much longer) default timeout even after the operator sets llmFilterTimeoutMs. It also means a low cap (e.g. 500ms to bound latency) cannot be enforced on those paths, whereas the new JSDoc in types.ts describes the setting as an unconditional per-call wall-clock budget. Also guard against a non-positive capMs (a partial config literal with 0 would collapse to a 1ms timeout via Math.max(1, ...)).

💡 Suggested Change

Before:

function filterTimeoutMs(deadlineAt: number | undefined, capMs?: number): number | undefined {
  if (deadlineAt === undefined) return undefined;
  return Math.max(1, Math.min(capMs ?? DEFAULT_FILTER_TIMEOUT_MS, deadlineAt - Date.now()));
}

After:

function filterTimeoutMs(deadlineAt: number | undefined, capMs?: number): number | undefined {
  // `capMs` is the operator-configured per-call budget; fall back to the
  // legacy default when it is unset/non-positive so the cap still applies
  // on retrieval paths that carry no deadline.
  const cap = capMs !== undefined && capMs > 0 ? capMs : DEFAULT_FILTER_TIMEOUT_MS;
  if (deadlineAt === undefined) return cap;
  return Math.max(1, Math.min(cap, deadlineAt - Date.now()));
}

2. apps/memos-local-plugin/core/retrieval/retrieve.ts (L432)

The new cap is only plumbed into the retrieval-local filter calls. The other LLM filter invocation still hard-codes its budget: finalFilterMergedHits in core/pipeline/memory-core.ts (lines ~923-931) passes FINAL_HUB_LLM_FILTER_TIMEOUT_MS = 3_000 when there is no deadline and Math.min(DEADLINE_FILTER_SAFE_CUTOFF_MS = 2_000, remaining) otherwise. That path runs on turn_start/memos_search whenever hub hits are merged (skipLlmFilter is only set when hubHits.length > 0), so operators running self-hosted endpoints will still see the 2 s abort described in #2499 on hub-enabled installs despite setting llmFilterTimeoutMs. Consider replacing those constants with config.llmFilterTimeoutMs ?? <default> (clamped by the deadline) so all filter call sites share one budget.


3. apps/memos-local-plugin/core/llm/fetcher.ts (L349)

This fallback contradicts the comment three lines above. code === 20 is the WebIDL legacy code of DOMException named "AbortError" (and "ABORT_ERR" is undici's abort code), so every caller ctrl.abort() rejection matches here and is re-classified as a timeout — exactly the flattening #2499 is trying to fix. It also makes the new reason.name === "TimeoutError" branch redundant in standard DOM runtimes. Today this is only masked because the caller checks opts.signal?.aborted before calling isTimeout (line 164); any future call site loses the distinction. Either remove the code fallback, or restrict it to runtimes that drop signal.reason and document that precondition plus the magic 20.

💡 Suggested Change

Before:

    if (code === "ABORT_ERR" || code === 20) return true;

After:

    // Legacy abort codes: DOMException("AbortError").code === 20 and undici
    // uses "ABORT_ERR". Note these also match a caller ctrl.abort(), so this
    // fallback is only safe because callers check `signal.aborted` first
    // (see the cancellation branch in httpPostJson); it must not be relied
    // on to distinguish a budget timeout (#2499).
    const code: unknown = (err as { code?: unknown }).code;
    if (code === "ABORT_ERR" || code === 20) return true;

4. apps/memos-local-plugin/core/llm/fetcher.ts (L392-L395)

Nested ternary (prohibited by the repo style rules) and the comment's intent is only half-implemented: the TimeoutError reason is preferred only when it comes from a. If b (the AbortSignal.timeout()) already fired with a TimeoutError while a also aborted with a caller reason, a.reason wins and the timeout is lost. Extracting a small helper makes both signals symmetric and removes the nesting.

💡 Suggested Change

Before:

  const immediateReason =
    (a.aborted && a.reason instanceof Error && a.reason.name === "TimeoutError")
      ? a.reason
      : a.aborted ? a.reason : b.reason;

After:

  const timeoutReason = (s: AbortSignal): unknown =>
    s.reason instanceof Error && s.reason.name === "TimeoutError" ? s.reason : undefined;
  // Prefer a TimeoutError reason from either signal so downstream
  // classification stays accurate even when both fire simultaneously.
  const immediateReason =
    timeoutReason(a) ?? timeoutReason(b) ?? (a.aborted ? a.reason : b.reason);

5. apps/memos-local-plugin/core/llm/fetcher.ts (L389)

This is a near-verbatim copy of mergeSignals in core/embedding/fetcher.ts (lines 246-254), which still does ctrl.abort() without forwarding from.reason (and whose isTimeout still classifies on code === "ABORT_ERR"). The same timeout-vs-cancellation flattening therefore persists for embedding calls, and the two copies are already drifting. Consider extracting one shared util (e.g. core/util/abort.ts) and fixing the embedding fetcher in the same change.


6. apps/memos-local-plugin/core/llm/fetcher.ts (L397-L398)

mergeSignals attaches two abort listeners to the caller-supplied signal a on every call and never removes them: { once: true } only detaches on abort, so on the success path (and on each retry attempt, plus httpPostStream) the caller's signal keeps growing a listener list pointing at throw-away ctrl/closure pairs. With a request-scoped signal (createRequestDeadline) shared by capture/reward/L2/filter LLM calls this easily passes Node's 10-listener EventTarget threshold, producing MaxListenersExceededWarning noise and retaining the controllers for the whole request. Please detach both listeners once either signal fires (a detach() helper called inside each forwarder) so the merged controller does not outlive the call.

💡 Suggested Change

Before:

  a.addEventListener("abort", forwardFrom(a), { once: true });
  b.addEventListener("abort", forwardFrom(b), { once: true });

After:

  const detach = () => {
    a.removeEventListener("abort", onA);
    b.removeEventListener("abort", onB);
  };
  const onA = () => {
    detach();
    ctrl.abort(a.reason);
  };
  const onB = () => {
    detach();
    ctrl.abort(b.reason);
  };
  a.addEventListener("abort", onA, { once: true });
  b.addEventListener("abort", onB, { once: true });

7. apps/memos-local-plugin/core/llm/fetcher.ts (L344-L347)

This branch looks unreachable in practice: a DOMException/AbortError has no own reason property, and neither undici nor this repo's abort helpers (retry-after.ts, semaphore.ts, request-deadline.ts, llm-filter.ts all rethrow signal.reason directly) attach the abort reason to the rejected error. In the timeout case the rejection is the TimeoutError (already caught by the first check), while caller cancellations are handled earlier via opts.signal?.aborted in httpPostJson. So this code adds no discrimination beyond the code === 20 fallback below it — consider dropping it (or, if a wrapped-AbortError source really exists, add a comment naming it) so the classifier does not suggest a guarantee it cannot make.

Generated by cloud-assistant via Open Code Review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:plugin OpenClaw & Hermes status:in-progress Someone or AI is working on it | 人工或 AI 正在处理

Projects

None yet

Development

Successfully merging this pull request may close these issues.

轻量模式强制打开检索用的 LLM 过滤,它自己的 2 秒上限让本地模型每轮都被中止,然后静默改用宿主模型

5 participants