Skip to content

fix(config): resolve masked apiKey from env on load - #2235

Open
kiwipaulrob wants to merge 2 commits into
MemTensor:mainfrom
kiwipaulrob:fix/config-apikey-env-fallback
Open

fix(config): resolve masked apiKey from env on load#2235
kiwipaulrob wants to merge 2 commits into
MemTensor:mainfrom
kiwipaulrob:fix/config-apikey-env-fallback

Conversation

@kiwipaulrob

@kiwipaulrob kiwipaulrob commented Aug 11, 2026

Copy link
Copy Markdown

Summary

The bridge persists config.yaml with API keys masked to __memos_secret__ (maskSecrets() in core/pipeline/memory-core.ts) and strips empty secrets from patches (stripEmptySecrets()). But nothing ever re-reads the real value back: when the daemon restarts, loadConfig() parses the mask as the literal API key, every LLM call fails auth, and the bridge restart-loops with lastOkAt: null while skill crystallize stays stuck. This was observed live on 2026-08-11 with 290 candidate skills backlogged and skill.crystallize.failed ... openai_compatible timed out after 120000 ms spam in the journal.

This PR makes resolveConfig() (the single choke point for both disk-loaded and in-memory patched configs) resolve masked/placeholder secret values from the environment, read-side only — the on-disk write stays masked, so the security posture of maskSecrets() is preserved.

Change

apps/memos-local-plugin/core/config/index.tsresolveConfig(raw) now walks SECRET_FIELD_PATHS before pruneUnknown/deepMerge:

  • ${ENV_VAR} references in any secret field resolve from process.env[ENV_VAR]
  • __memos_secret__ / empty-string apiKey fields fall back to LLM_API_KEY, EMBEDDING_API_KEY, then OPENCODE_GO_API_KEY / OPENCODE_ZEN_API_KEY
  • Hub tokens (hub.teamToken, hub.userToken) have no env convention and are left untouched unless the user writes an explicit ${VAR}
  • Real (non-placeholder) values pass through unchanged

Tests

New tests/unit/config/resolve-secret-env.test.ts (6 tests):

  1. ${ENV_VAR} expansion
  2. __memos_secret__ mask resolution
  3. empty-string fallback
  4. all apiKey SECRET_FIELD_PATHS leaves resolve (hub tokens stay masked)
  5. real values untouched
  6. placeholder stays when no env var is set
Test Files  4 passed (4)
     Tests  54 passed (54)   # incl. full existing config suite (load/paths/writer)

tsc -p tsconfig.json --noEmit passes clean.

Related

Environment

  • Bridge: @memtensor/memos-local-plugin 2.0.12-beta.1 (source matches apps/memos-local-plugin)
  • Trigger: migration to the opencode-go provider (https://opencode.ai/zen/go/v1, deepseek-v4-flash). Config written via the UI/viewer masks the key, restart loops, curl with the env key works (GO_DEEPSEEK_OK), bridge with masked key fails.
  • Platform: Debian 12 LXC (Hermes CT100), node 22, systemd memos-bridge.service

Type of change

  • Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

  • Unit Test (vitest run tests/unit/config — 54 passed)
  • Test Script Or Test Steps (live bridge: after patching dist the same way, lastOkAt populates and crystallize drains; verified 2026-08-11 on Hermes CT100)

Checklist

  • My code follows the style guidelines of this project (no new deps, existing patterns)
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation (PR body + code comments; doc note pending maintainer preference)
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective
  • New and existing unit tests pass locally

@Memtensor-AI Memtensor-AI added area:plugin OpenClaw & Hermes status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Aug 11, 2026
@Memtensor-AI

Memtensor-AI commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

🤖 Open Code Review

Target: PR #2235
Task: 1f83be90d13816eb
Base: main
Head: fix/config-apikey-env-fallback

🔍 OpenCodeReview found 4 issue(s) in this PR.


1. apps/memos-local-plugin/core/config/index.ts (L187-L192)

When val === '__memos_secret__' (a masked sentinel from maskSecrets()), the code infers envName = 'LLM_API_KEY'. If LLM_API_KEY is not set but OPENCODE_GO_API_KEY or OPENCODE_ZEN_API_KEY happen to be set in the environment (e.g., from a CI pipeline or another agent's config), they will be silently injected as the API key for ALL LLM-class fields — including l3Llm.apiKey and skillEvolver.apiKey. This means requests could be billed to an entirely unintended account, and the user gets no warning that an unexpected fallback key was used.

Consider emitting a warning when a generic fallback is applied, and requiring __memos_secret__ to only resolve via the primary LLM_API_KEY (no silent cross-provider fallback), or at minimum log which env var was ultimately used:

const envVal = process.env[envName];
const fallbackVal = (!envVal && genericFallbacks)
  ? (process.env.OPENCODE_GO_API_KEY ?? process.env.OPENCODE_ZEN_API_KEY)
  : undefined;
const resolved = envVal ?? fallbackVal;
if (resolved) {
  if (fallbackVal && !envVal) {
    warnings?.push(
      `config: '${dotted}' resolved via generic fallback (LLM_API_KEY not set); ` +
      `using OPENCODE_GO_API_KEY / OPENCODE_ZEN_API_KEY instead`
    );
  }
  (cursor as Record<string, unknown>)[leaf] = resolved;
}

2. apps/memos-local-plugin/core/config/index.ts (L139)

The allowlist pattern /^[A-Z][A-Z0-9_]*_(API_KEY|TOKEN)$/ is far too broad. It accepts any env var ending in _API_KEY or _TOKEN, including well-known CI/CD or cloud-provider tokens such as GITHUB_TOKEN, AWS_SESSION_TOKEN, NPM_TOKEN, VERCEL_TOKEN, etc. A user-supplied (or attacker-controlled) YAML with apiKey: "${GITHUB_TOKEN}" would pass the allowlist and forward the token to an LLM API endpoint — a credential exfiltration path.

Consider restricting the allowlist to a known set of LLM-provider env var prefixes (e.g., OPENAI_API_KEY, ANTHROPIC_API_KEY, LLM_API_KEY, EMBEDDING_API_KEY, etc.) via an explicit set rather than a regex, or at minimum tightening the regex to exclude well-known non-LLM suffixes:

// Explicit allowlist is safer than a broad suffix regex:
const ENV_REF_ALLOWLIST = new Set([
  "OPENAI_API_KEY", "ANTHROPIC_API_KEY", "GEMINI_API_KEY",
  "LLM_API_KEY", "EMBEDDING_API_KEY",
  "OPENCODE_GO_API_KEY", "OPENCODE_ZEN_API_KEY",
  // Add others as needed
]);
// Usage: if (!ENV_REF_ALLOWLIST.has(name)) { ... }

3. apps/memos-local-plugin/core/config/index.ts (L192)

When envName is set (from either a ${VAR} reference or masked/empty sentinel) but the environment variable is not defined, the function silently leaves the placeholder value (e.g., __memos_secret__ or ${OPENAI_API_KEY}) intact with no diagnostic. This causes confusing downstream auth failures — the placeholder is sent as a literal API key. A mistyped env var name (e.g., ${OPANAI_API_KEY}) that passes the allowlist will produce zero feedback at config load time.

Consider emitting a warning when the resolved env var is missing:

if (envVal) {
  (cursor as Record<string, unknown>)[leaf] = envVal;
} else {
  warnings?.push(
    `config: '${dotted}' references env var '${envName}' which is not set; ` +
    `the field will remain unresolved and authentication may fail`
  );
}

4. apps/memos-local-plugin/core/config/index.ts (L179)

The isEmbedding detection relies on the second-to-last path segment being exactly "embedding". While it works correctly for the current SECRET_FIELD_PATHS (embedding.apiKey, llm.apiKey, l3Llm.apiKey, skillEvolver.apiKey), this is fragile: if a future path like providers.embedding.apiKey is added, keys[keys.length - 2] would be "embedding" — correct by accident — but a path like embedding.model.apiKey would yield "model", incorrectly treating it as an LLM key and applying generic fallbacks.

Consider checking via dotted.startsWith("embedding.") instead, which is explicit about the full parent section:

const isEmbedding = dotted.startsWith("embedding.");

Generated by cloud-assistant via Open Code Review.

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

⚠️ Automated Test Results: ENV ISSUE

The test environment encountered an issue that requires manual attention.

Details: Executor error: Command failed: git merge --no-edit base/main
Branch: fix/config-apikey-env-fallback

Problem:
The bridge persists config.yaml with apiKey masked to __memos_secret__
(maskSecrets in memory-core.ts) and strips empty secrets from patches,
but nothing re-reads the real value back. On restart, loadConfig parses
the mask as the literal API key, so every LLM call fails auth and the
bridge restart-loops with lastOkAt: null while crystallize stays stuck
(observed with 290 candidate skills backlogged on 2026-08-11).

Solution:
resolveConfig now walks SECRET_FIELD_PATHS before merging and expands
placeholder values from the environment, read-side only:
- ${ENV_VAR} references are resolved from process.env
- __memos_secret__ / empty apiKey fields fall back to LLM_API_KEY,
  EMBEDDING_API_KEY, then OPENCODE_GO_API_KEY / OPENCODE_ZEN_API_KEY
- hub tokens (teamToken/userToken) have no env convention and are
  left untouched unless an explicit ${VAR} reference is used
- real values pass through unchanged; on-disk masking is preserved

Tests: 6 new unit tests in tests/unit/config/resolve-secret-env.test.ts
(env expansion, mask resolution, empty-string, all apiKey paths,
real-value passthrough, unset-env fallback). Full config suite 54/54.
@kiwipaulrob
kiwipaulrob force-pushed the fix/config-apikey-env-fallback branch from cfeb5b7 to 3f33609 Compare August 11, 2026 09:02
@kiwipaulrob

Copy link
Copy Markdown
Author

Thanks for the review — all four findings are fair, and I've folded them into the revised branch (now rebased onto the current main, 8d310a7).

1. Provider fallbacks leaking across secret fields (L126–L130) — agreed, good catch. The OPENCODE_GO/ZEN fallbacks were meant to spare opencode-go/zen users from defining a second env var, but applying them to embedding.apiKey, and to explicit ${VAR} references whose named variable is unset, is wrong: it can hand an LLM key to an embedding provider's calls, or silently substitute a different key than the one the user named. Fix: an explicit ${VAR} now resolves exactly that variable or stays untouched, and the OPENCODE_* fallbacks apply only on the inferred sentinel/empty path, only for LLM-class fields (parent != embedding). Tests cover both cases: embedding.apiKey resolves from EMBEDDING_API_KEY and never from an LLM key, and an unset ${VAR} stays literal even when generic keys are present.

2. Unrestricted ${VAR} expansion (L110–L112) — agreed that expansion should be bounded, with one adjustment to the proposed allowlist: restricting to *_API_KEY names would break a designed use case of this PR, namely hub tokens (hub.teamToken / hub.userToken) set via explicit ${VAR} — they have no other env convention. The allowlist is now ^[A-Z][A-Z0-9_]*_(API_KEY|TOKEN)$; names outside it warn and are left untouched. I'd rate the practical severity here as modest for a local single-user plugin (user-owned config file, auth-gated viewer patch path), but the same change has a robustness payoff beyond the threat model: it stops accidental expansion of unrelated variables (a typo'd ${HOME} or ${PATH}) from becoming a bogus credential — so I'm happy to take it.

3. In-place mutation of raw (L130) — fair. Both current call sites are safe (loadConfig parses fresh YAML; writer.ts passes doc.toJS()), but the signature didn't promise non-mutation. Rather than just documenting the contract I made the pass non-mutating: pruneUnknown already returns a fresh copy, so resolution now runs on that cleaned object — identical semantics (user-provided leaves only), no writes to the caller's object. The JSDoc states the input is never mutated, and a test asserts the raw object is untouched after resolution.

4. Shadowed leaf declaration (L113–L118) — agreed, a leftover from an earlier edit. Removed the inner declaration; the outer one is in scope.

On the automated test run ("ENV ISSUE": git merge --no-edit base/main failed) — that was a genuine merge conflict rather than an environment problem: main had since changed this same file (the viewer-port migration, #2230, gives loadConfig/resolveConfig an agent parameter and effectiveViewerPort handling), and GitHub reported the branch as not mergeable (mergeable_state: dirty). I've merged the latest main into the branch, keeping both sets of changes (agent/viewer-port handling + env resolution), and pushed the rebased branch. The executor should pick the update up on the next run; if a manual re-run is needed, that would be appreciated.

Validation on the revised branch: config suite 71/71 (5 files — includes main's new hermes-migration tests) and tsc --noEmit clean.

Thanks again — the embedding-key leak in particular was a sharp catch.

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (10/10 executed). memos_local_plugin/unit: 10/10. Duration: 3s

Branch: fix/config-apikey-env-fallback

@Memtensor-AI Memtensor-AI added status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 and removed status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Aug 11, 2026
@kiwipaulrob

Copy link
Copy Markdown
Author

Deployment note (from the production deployment that hit this)

The fix resolves masked keys from the daemon's process environment — which only works if the daemon actually has them. For systemd users:

# /etc/systemd/system/memos-bridge.service.d/env.conf
[Service]
EnvironmentFile=/path/to/durable/bridge.env

where bridge.env is a filtered copy of your secrets file, e.g. grep -v '^CUSTOM:' .env > <plugin-home>/daemon/bridge.env && chmod 600 <plugin-home>/daemon/bridge.env.

⚠️ Do NOT place the EnvironmentFile in /tmp. It is wiped on reboot; the unit then crash-loops with Failed to load environment files: No such file or directory (result 'resources', NRestarts +1 per 5s), and because the daemon is down a session host's ensure_viewer_daemon() spawns a rogue bridge.mjs --daemon that binds the viewer port — so /api/v1/health keeps answering while the systemd service is dead (observed live 2026-08-12 after a container reboot; the daemon must be restarted so systemd binds :18800 first).

Env var contract implemented by this PR:

  • generic: LLM_API_KEY (llm/skillEvolver/l3Llm apiKey), EMBEDDING_API_KEY (embedding.apiKey only — never an LLM key)
  • OpenCode fallbacks (LLM-class only): OPENCODE_GO_API_KEYOPENCODE_ZEN_API_KEY
  • explicit ${VAR}: allowlisted names matching ^[A-Z][A-Z0-9_]*_(API_KEY|TOKEN)$ (hub tokens use this — they have no other convention)

@Memtensor-AI Memtensor-AI added status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 and removed status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 labels Aug 13, 2026
@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (10/10 executed). memos_local_plugin/unit: 10/10. Duration: 3s [advisory, non-gating] AI-generated tests on branch test/auto-gen-1f83be90d13816eb-20260814063936: 47/47 passed — these do NOT affect the PR verdict; review the branch manually.

Branch: fix/config-apikey-env-fallback

@Memtensor-AI Memtensor-AI added status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 and removed status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:plugin OpenClaw & Hermes status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: masked apiKey written to config.yaml is never resolved on daemon restart — LLM auth fails and bridge restart-loops

3 participants