You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Feedback classifier leaks raw LLM completion (think tags, special tokens) into persisted feedback rationale #2336
Feedback classification responses can leak the raw LLM completion — including closing <think> tags and DeepSeek special tokens — into the persisted feedback rationale. The existing thinking-disable only covers capture.summarize; every other op (e.g. feedback.classify) can receive thinking-enabled model output, and nothing strips think-tags from message.content before it is stored.
This is related to #2308 (the op parameter was dropped at the LLM boundary): even with op correctly propagated, thinking is only disabled for capture.summarize.
Reproduction / evidence
Two feedback rows stored on 2026-09-02 in our deployment contain a rationale that is clearly a raw model completion, not classified output:
For thinking-enabled models served through gateways that keep <think> blocks inside message.content, the raw completion (with </think>, <|end▁of▁sentence|>, …) flows back unmodified.
dist/core/experience/feedback-builder.js then persists feedback.rationale (and raw verifier text) verbatim into the store — polluted text becomes retrieval-visible memory.
Suggested fix
Either (ideally both):
Sanitize at the provider choke point: strip <think>…</think> blocks, orphan closing </think> fragments, and DeepSeek special tokens (<|begin▁of▁sentence|>, <|end▁of▁sentence|>, <|end▁of▁session|>) from choice.message.content before returning from the provider — one place, fixes all ops.
Extend thinking-disable to deterministic/analytical ops (feedback.classify, l3.abstraction, verifier, retrieval filter), not just capture.summarize.
Summary: Fixes issue #2336 — feedback classifier and other ops were leaking raw LLM completion tokens (<think>...</think> blocks and DeepSeek gateway tokens like <|end▁of▁sentence|>) into persisted FeedbackRow.rationale, which then polluted retrieval-visible memory on subsequent turns.
Root cause: apps/memos-local-plugin/core/llm/providers/openai.ts::complete returned choice.message.content verbatim, and no downstream sanitizer knew about <think> tags or DeepSeek special tokens. Thinking-enabled DeepSeek-family models served through gateways that keep the reasoning block inside content (rather than a separate reasoning_content field) leaked it end-to-end into storage. Adopts the reporter's preferred fix (option 1: sanitize at the provider choke point) — one transform covers all current and future ops (feedback.classify, feedback.refine, l3.abstraction, retrieval.filter, …) without requiring per-op thinking-disable coverage.
Changes: new core/llm/sanitize.ts module (strips matched <think>...</think> blocks, orphan closing </think>, orphan opening <think> + trailing text, DeepSeek special tokens, and collapses excess blank lines), applied to openai_compatible, anthropic, gemini, bedrockcomplete() return values. Streaming path is deliberately out of scope (chunk boundaries make safe chunk-level sanitization stateful — deferred; the reported leak is on the storage path which uses complete()).
Tests: added tests/unit/llm/sanitize.test.ts (17 unit tests including the exact artifact soup captured in the #2336 report, an ASCII-pipe safety guard, and idempotency) plus 2 provider-level regression tests in tests/unit/llm/providers.test.ts pinning end-to-end behavior. Verification: npm run test — 1576 passed / 2 skipped across 184 files; npm run lint (tsc --noEmit) — clean.
Base branch:main Branch:bugfix/autodev-2336-20260903031435996 Commit:f334037 PR:#2339 Assigned to:@whipser030 Reviewers:@whipser030, @hijzy
Activity
Memtensor-AI commentedon Sep 3, 2026
🤖 AutoDev has picked up this issue and started working on it.
Task ID:
372e4cf70b86ec48Working branch:
bugfix/autodev-2336-20260903031435996Target branch:
latest dev* branchWorkflow: opsp (analysis → coding → testing → PR)
I will post the PR link here once done. If I need more information, I will ask in the comments.
fix(plugin): sanitize <think> tags from LLM completions (MemTensor#2336)
Memtensor-AI commentedon Sep 3, 2026
✅ AutoDev task
372e4cf70b86ec48completed.Summary: Fixes issue #2336 — feedback classifier and other ops were leaking raw LLM completion tokens (
<think>...</think>blocks and DeepSeek gateway tokens like<|end▁of▁sentence|>) into persistedFeedbackRow.rationale, which then polluted retrieval-visible memory on subsequent turns.Root cause:
apps/memos-local-plugin/core/llm/providers/openai.ts::completereturnedchoice.message.contentverbatim, and no downstream sanitizer knew about<think>tags or DeepSeek special tokens. Thinking-enabled DeepSeek-family models served through gateways that keep the reasoning block insidecontent(rather than a separatereasoning_contentfield) leaked it end-to-end into storage. Adopts the reporter's preferred fix (option 1: sanitize at the provider choke point) — one transform covers all current and future ops (feedback.classify,feedback.refine,l3.abstraction,retrieval.filter, …) without requiring per-op thinking-disable coverage.Changes: new
core/llm/sanitize.tsmodule (strips matched<think>...</think>blocks, orphan closing</think>, orphan opening<think>+ trailing text, DeepSeek special tokens, and collapses excess blank lines), applied toopenai_compatible,anthropic,gemini,bedrockcomplete()return values. Streaming path is deliberately out of scope (chunk boundaries make safe chunk-level sanitization stateful — deferred; the reported leak is on the storage path which usescomplete()).Tests: added
tests/unit/llm/sanitize.test.ts(17 unit tests including the exact artifact soup captured in the #2336 report, an ASCII-pipe safety guard, and idempotency) plus 2 provider-level regression tests intests/unit/llm/providers.test.tspinning end-to-end behavior. Verification:npm run test— 1576 passed / 2 skipped across 184 files;npm run lint(tsc --noEmit) — clean.Base branch:
mainBranch:
bugfix/autodev-2336-20260903031435996Commit:
f334037PR: #2339
Assigned to: @whipser030
Reviewers: @whipser030, @hijzy
fix(plugin): sanitize <think> tags in openai/gemini streaming path (M…
github-actions commentedon Oct 4, 2026
This issue has been automatically marked as stale due to inactivity.
github-actions commentedon Oct 14, 2026
This issue has been automatically closed due to inactivity.