fix(vllm): window long deduplicated KV loads

[ketor]

Problem

For same-host replicated-MLA loads, native GPU dedup publishes fetched results only after the enclosing batch_get_auto_sg completes. A 1M-token TP8 load contains 15,624 objects and takes much longer than the default follower wait, so all eight ranks time out and independently fetch the same KV data.

Change

• add opt-in load_window_keys to bound each native GET publication
• add load_window_min_keys so short requests retain one native GET
• validate both settings at connector construction
• preserve key/result ordering and one outer logical-load metric
• document production sizing, rollout, and acceptance guidance

Defaults remain disabled and backward-compatible.

Configuration guidance

• enable windowing only when node-dedup logs show follower fallback or TP-scale fetch amplification
• treat both values as dfkv key counts, not token counts; retune after model or KV-layout changes
• size load_window_keys against both limits:
  • arena starting bound: floor(0.75 * arena_bytes / sizing_bytes_per_key) using p99 or geometry-maximum bytes per key
  • per-window p99 fetch-and-publish latency should stay within 50-70% of DFKV_NODE_DEDUP_WAIT_MS
• set load_window_min_keys above the p99 short-load key count and no higher than the smallest long load that must be windowed
• use separate engine pools when short/long key-count distributions overlap instead of inventing an unverified threshold
• accept only after byte-identical long and short hot tests prove zero fallback/near-one-copy fetches and no short-path regression

The documented GLM-5.3 values (128/4096) are an example, not portable defaults.

Validation

• ruff check integration/vllm/src/dfkv_vllm/worker.py integration/vllm/tests/test_worker_lifecycle.py
• git diff --check
• vLLM GLM-5.3 image: 13/13 lifecycle/window tests passed
• xb01-0064, GLM-5.3, TP8, actual 1,000,000 input + 500 output:
  • default: TTFT 39.35 s, 22,756.73 total tok/s, ~8 remote copies
  • 128-key windows, 4,096-key threshold: TTFT 22.92 s, 36,458.56 total tok/s, ~1 remote copy, zero dedup fallback
• 65,536/500, C10, 100 prompts with the same threshold: 35,622.47 total tok/s, 100/100 success