[NVIDIA #3] Upgrade embedding backend: Ollama nomic-embed-text → nv-embed-v2 on GB10

[AugustChaoTW]

Summary

Replace the current Ollama nomic-embed-text (768-dim) embedding with NVIDIA's nv-embed-v2 (4096-dim) running locally via vLLM or Ollama, leveraging the GB10's 121.6 GB VRAM. Higher-dimensional embeddings improve hybrid search quality in memory_query, leading to better memory retrieval.

Current vs Proposed

	Current	Proposed
Model	nomic-embed-text	nvidia/nv-embed-v2
Dimensions	768	4096
Backend	Ollama	vLLM or Ollama
MTEB Score	~62	~72 (top-tier)
VRAM needed	~0.3 GB	~6 GB
GB10 fits?	✓	✓ (121.6 GB available)

Why nv-embed-v2

• NVIDIA's best open embedding model; tops MTEB retrieval leaderboard
• Instruction-based: supports passage: / query: prefixes (same as nomic)
• 4096-dim → better cosine similarity discrimination in memory_vec (sqlite-vec)
• Native support for longer sequences (up to 32K tokens)

Changes

1. EmbeddingService — configurable model + dimension

Files: src/index.ts:404, src/mcp-server.ts:146

const EMBED_MODEL = process.env.MEMORY_EMBED_MODEL ?? "nomic-embed-text";
const EMBED_DIM   = parseInt(process.env.MEMORY_EMBED_DIM ?? "768");
const EMBED_ENDPOINT = process.env.MEMORY_EMBED_ENDPOINT ?? "http://localhost:11434/api/embed";

class EmbeddingService {
  async embed(text: string): Promise<Float32Array | null> {
    const resp = await fetch(EMBED_ENDPOINT, {
      method: "POST",
      body: JSON.stringify({ model: EMBED_MODEL, input: `passage: ${text}` }),
      ...
    });
    // nv-embed-v2 via vLLM uses OpenAI-compatible /v1/embeddings endpoint
    // Ollama uses /api/embed — both supported via env switch
  }
}

2. Schema migration v12 — rebuild vec index at new dimension

if (current < 12) {
  try {
    // Drop old 768-dim vec table; rebuild at configured EMBED_DIM
    db.exec("DROP TABLE IF EXISTS memory_vec");
    db.exec("DROP TABLE IF EXISTS memory_vec_rowmap");
    db.exec(`CREATE VIRTUAL TABLE IF NOT EXISTS memory_vec
             USING vec0(embedding float[${EMBED_DIM}] distance_metric=cosine)`);
    // Re-embed all existing memories asynchronously
    scheduleReembedding();
  } catch (e) { ... }
}

3. Environment variables

Variable	Default	Description
MEMORY_EMBED_MODEL	nomic-embed-text	Model name
MEMORY_EMBED_DIM	768	Embedding dimension (must match model)
MEMORY_EMBED_ENDPOINT	http://localhost:11434/api/embed	Ollama or vLLM endpoint
MEMORY_EMBED_API_FORMAT	ollama	ollama or openai

4. Setup for nv-embed-v2 via vLLM

# Start embedding server (separate from inference server, port 8001)
python3 -m vllm.entrypoints.openai.api_server \
  --model nvidia/NV-Embed-v2 \
  --task embed \
  --port 8001 \
  --gpu-memory-utilization 0.1   # only needs ~6 GB of 121.6 GB

# Configure opencode-owl
export MEMORY_EMBED_MODEL="nvidia/NV-Embed-v2"
export MEMORY_EMBED_DIM="4096"
export MEMORY_EMBED_ENDPOINT="http://localhost:8001/v1/embeddings"
export MEMORY_EMBED_API_FORMAT="openai"

Acceptance Criteria

• EmbeddingService reads model/dim/endpoint from env vars
• Schema migration v12 rebuilds memory_vec at EMBED_DIM dimensions
• Works with both Ollama format and OpenAI-compatible format
• Existing memories re-embedded asynchronously on first startup after config change
• memory_status reports current embedding model name and dimension
• Falls back to FTS5-only if embedding server unavailable (existing behavior preserved)

Effort: 2h | Priority: P1

[AugustChaoTW]

GPU-agnostic requirement: This feature must degrade gracefully without an NVIDIA GPU. See #56 for the full compatibility contract, fallback implementation details, and CI requirements that must pass before this issue can be merged.