Repository navigation
Description
Pre-submission checklist | 提交前检查
- I have searched existing issues and this hasn't been mentioned before
- I have read the project documentation and confirmed this issue doesn't already exist
- This issue is specific to MemOS and not a general software issue
Bug Description | 问题描述
memos-local-plugin v2.0.10 feeding untruncated/un-chunked trace text to the embedding provider causes HTTP 400 errors. When using 智谱 embedding-3 (single-request cap 3072 tokens), any traces.summary or agent_text longer than ~6KB triggers code:1210 API 调用参数有误, and because the batch is aborted on the first failure (failed = batch.length), the entire batch — including short, valid entries — is marked failed. The "修复向量 / rebuild vectors" button in the Memory Viewer returns 400 immediately, and the periodic embedding heartbeat probe shows ~60% error rate.
Environment | 环境信息
- Plugin:
@memtensor/memos-local-pluginv2.0.10 (also confirmed in v2.0.6) - Embedding provider:
openai_compatible→https://open.bigmodel.cn/api/paas/v4/embeddings, modelembedding-3 - 智谱 embedding-3 official limits: 8K context window, 3072 tokens per single input, ≤64 inputs per batch (docs)
- Storage: SQLite (
memos.db), 19,114 traces - Host: macOS arm64 (OpenClaw runtime)
Evidence
1. DB-side: source-text length distribution
-- field count max bytes
-- agent_text > 6k chars: 19 59644
-- summary > 6k chars: 40 59644
-- user_text > 6k chars: 21 22430So ~40 traces exceed the 6KB (~3000 Chinese-char) mark. The longest summary and agent_text are 59,644 bytes each — roughly 30k Chinese characters, an order of magnitude over the 3072-token cap.
2. API-side: reproducing the 400
# short input → 200 OK
curl -X POST https://open.bigmodel.cn/api/paas/v4/embeddings \
-H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
-d '{"model":"embedding-3","input":"hello world"}' # HTTP 200
# 50,000 'a' → 400
LONG=$(python3 -c "print('a'*50000)")
curl ... -d "{\"model\":\"embedding-3\",\"input\":\"$LONG\"}"
# → {"error":{"code":"1210","message":"API 调用参数有误,请检查文档。"}}
# HTTP 4003. Code path: no truncation / no chunking anywhere
Three places feed source text straight into embedMany with no length guard:
core/pipeline/memory-core.ts:3998 (inside rebuildEmbeddings — the "rebuild vectors" button):
const vecs = await handle.embedder.embedMany(
batch.map((slot) => ({ text: slot.sourceText || "(empty)", role: "document" as const })),
);core/pipeline/memory-core.ts:4126 (slot source for vec_summary):
sourceText: row.summary?.trim() || row.userText.trim() || "(empty)",…and the vec_action slot is built from traceActionEmbeddingText(row), which includes the full agent_text. Neither is truncated.
core/capture/embedder.ts:93 (live capture path):
return [step.agentText.trim(), toolSig].filter((s) => s.length > 0).join("\n---\n");grep for chunk|truncat|slice|max.*token|token.*limit in embedder.ts returns only one hit — an unrelated .slice(0, 300) on tool-call inputs. The main text path is unguarded.
4. Blast radius: a single long entry nukes the whole batch
In rebuildEmbeddings:
} catch (err) {
failed = batch.length; // ← whole batch marked failed
error = err instanceof Error ? err.message : String(err);
}A 30k-char trace anywhere in the batch → 400 → the whole batch (every other short, valid trace included) is counted as failed. This is why the Memory Viewer's heartbeat shows ~94 errors out of 160 probes.
Impact
- "Rebuild vectors" button is unusable when the DB contains any long trace — returns 400 within milliseconds.
- Live capture silently degrades:
embedStepscatches the same exception and writesnullvectors for the entire step batch (seeembedder.tslines 53-58), so vector search becomes increasingly incomplete as long traces accumulate. - Heartbeat probe error rate ~60%, polluting the system-model-status log.
embedding_retry_queueaccumulates: failed jobs keep retrying with the same oversized payload and never succeed.
Suggested Fixes
In rough order of effort:
-
Truncate source text before embedding (smallest change). Add a per-input character cap (e.g. 6000 chars ≈ 2700 tokens with safety margin) in
embedder.embedManyor at thecollectEmbeddingSlotsboundary. Log a warning when truncation fires. This won't perfectly represent long traces but keeps retrieval working. -
Chunked embedding for long inputs (best quality). When
sourceText.length > cap, split into chunks ofcap, embed each chunk separately, and average (or L2-normalize-pool) the vectors into a single embedding. TheEmbedderinterface already returns aFloat32Array, so the contract stays the same; the batch loop just needs to coalesce per-slot. -
Don't abort the whole batch on one failure (orthogonal but important). Either (a) send inputs one-per-request when batch mode throws, or (b) split the batch in half on exception and retry each half — so a single oversized input only fails itself, not its neighbors.
-
Persist the provider's response body in
http.non_okwarnings (also raised in LLM provider 400 errors from SiliconFlow + missing response body logging (memos-local-plugin) #1775, still relevant). Right nowgateway.logshowsembedding_unavailable: HTTP 400 from openai_compatiblewith no response body, which is why this bug took so long to pin down — thecode:1210message that points at the real cause is swallowed.
(1)+(3)+(4) together would resolve this end-to-end; (2) is the "correct" long-term fix.
How to Reproduce | 如何重现
- Configure plugin with embedding provider = 智谱
embedding-3. - Insert (or naturally accumulate) any trace whose
summaryoragent_textexceeds ~6KB. The easiest way is a long multi-tool OpenClaw run — those routinely produce 30-60KBagent_text. - Open the Memory Viewer → Embedding Maintenance → click "修复向量 / Rebuild".
- Observe: HTTP 400 returned within milliseconds,
gateway.logshows[嵌入模型] · error · embedding_unavailable: HTTP 400 from openai_compatiblewith no body.
Additional Context
- MemOS v2.0.13 still has the same problem (per a user in fix: Extracting too many missing_documents embeddings cause embedder error #1482), though that issue is about count of missing_documents, not length. Mentioning here because the two have overlapping symptoms.
- MemOS issue local-plugin v2.0.8: gateway CPU 100% — synchronous full-table vector scan (scanAndTopK) + unbounded trace re-insertion from dedup pagination cap #2076 is adjacent (long traces causing performance issues) but is about CPU time, not embedding rejection.
Happy to open a PR for fix (1) + (3) + (4) if maintainers are open to it — I have the code paths mapped.
Activity
Memtensor-AI commentedon Jul 19, 2026
🤖 AutoDev has picked up this issue and started working on it.
Task ID:
c7a870ef01a28176Working branch:
bugfix/autodev-2121-20260719143800575Target branch:
latest dev* branchWorkflow: opsp (analysis → coding → testing → PR)
I will post the PR link here once done. If I need more information, I will ask in the comments.
Memtensor-AI commentedon Jul 19, 2026
🤖 AutoDev has picked up this issue and started working on it.
Task ID:
c7a870ef01a28176Working branch:
bugfix/autodev-2121-20260719143800575Target branch:
latest dev* branchWorkflow: opsp (analysis → coding → testing → PR)
I will post the PR link here once done. If I need more information, I will ask in the comments.
hitken commentedon Jul 19, 2026
Update: 智谱 embedding-3 还有一个未文档化的 batch 总 token 上限
继续调试发现,除了已知的"单条 3072 tokens"限制外,智谱 embedding-3 还有一个整个请求所有 input 累计 tokens 的上限——大约 5000-6000 tokens(官方文档没写)。
实测数据(每条 input 4512 字符 ≈ 2864 tokens 的中文):
把每条截到 1512 字符(≈ 750 tokens)后,10 条 batch(9799 tokens)能过。
对 plugin 的影响
这给 daemon 的
embedder.embedMany(默认batchSize = 32)带来问题——即使每条 sourceText 都在单条限制内,32 条累加也可能超 batch 总 token 上限。对中文场景尤其严重(2 字符/token)。建议(补充)
除了原 issue 的 4 个 fix 建议,再补一条:
embedder.embedMany里基于累计 token 数动态调整 batch 拆分,而不是固定 32 条。需要简单的 token 估算(中文按 2 字符/token,英文按 4 字符/token),或者用tiktoken/provider 自带的 token counter。或者更简单——把默认 batchSize 从 32 改成 4-8,对大多数 embedding provider 都安全。
@hijzy @whipser030 @Memtensor-AI
fix(memos-local-plugin): guard oversized embedding inputs and isolate…
Memtensor-AI commentedon Jul 19, 2026
✅ AutoDev task
c7a870ef01a28176completed.Summary: Fixed issue #2121: memos-local-plugin no longer crashes the whole rebuild-embedding batch when a single trace exceeds the provider's per-input token limit (e.g. 智谱 embedding-3's 3072-token cap → HTTP 400 code:1210).
Three surgical fixes in apps/memos-local-plugin:
Embedder facade truncation (
core/embedding/embedder.ts+types.ts): addedEmbeddingConfig.maxInputChars(default 6000;0disables). Truncation runs BEFORE cache-key hashing so every provider is protected and repeat calls with the same head text hit the LRU. Oneinput_truncatedwarn is emitted per embedMany call (batched, not per-input).rebuildEmbeddings divide-and-conquer (
core/pipeline/memory-core.ts): replaced the blanketfailed = batch.lengthcatch with a halving retry. When a sub-batch is size 1, only that slot is marked failed; short valid rows next to a poison input are no longer punished. ExistingEmbeddingMaintenanceRunResultcontract preserved.Response body in log (
core/embedding/fetcher.ts):http.non_okwarn now carries a truncated response body (first 512 chars) so operators can pinpoint provider error codes (e.g. code:1210) from gateway.log alone.Also exposed
embedding.maxInputCharson the user config schema (default 6000, min 0). Chunked embedding (issue's fix #2) is deferred as follow-up per proposal.md.Tests: 5 new embedder cases + 2 new fetcher cases + 1 new memory-core case (all strict TDD).
pnpm --filter @memtensor/memos-local-plugin vitest run tests/unit/embedding tests/unit/pipeline/memory-core.test.ts tests/unit/config= 144 passed / 1 skipped.tsc --noEmitclean. Integration tests green. Two pre-existing unit failures (startup-recovery, storage/migrator) reproduce on the base commit and are unrelated to embedding.opsp artifacts (task.md + proposal.md + design.md + verification-report.md) archived to memos-autodev-specs/2026-07-19-2121-bug-memos-local-plugin-embedding-source-text-exceeds-embeddi/ (pushed to main).
Base branch:
mainBranch:
bugfix/autodev-2121-20260719143800575Commit:
b740411PR: #2123
Assigned to: @NydiaChung
Reviewers: @whipser030, @hijzy, @WeiminLee
fix(memos-local-plugin): address OCR review findings for MemTensor#2121…
github-actions commentedon Aug 19, 2026
This issue has been automatically marked as stale due to inactivity.
fix(plugin): harden local plugin runtime and embeddings (#2266)