perf: improve knowledge base retrieval quality

[lxfight]

Depends on #9454. Keep this PR in draft until that correctness fix is merged; the diff will then contain only the two retrieval-quality commits.

Equal-weight RRF over-promotes mediocre candidates that appear in both dense and sparse result lists, and multiple chunks from one source document can consume the limited final result slots. This PR addresses both issues with relative-score fusion and source-document diversification.

Modifications / 改动点

• Normalize dense similarity and BM25 scores per knowledge base before combining them.
• Use a dense weight of 0.9 selected on the development split.
• Keep deterministic RRF ordering only as the tie breaker.
• Return at most one highest-scoring chunk per source document in the final fused results.
• This is NOT a breaking change. / 这不是一个破坏性变更。

Screenshots or Test Results / 运行截图或测试结果

Local benchmark:

• Public dataset: mteb/DuRetrieval
• Dataset revision: 8d9b3d6a6a62b7fd2d4df0821d612366dc52d14e
• Fixed split seed: astrbot-kb-v1
• Corpus: 100,001 documents / 119,965 chunks
• Queries: 2,000
• Embedding model: baai/bge-m3
• Reranking disabled to isolate retrieval changes

Strategy	Recall@5	HitRate@5	nDCG@5
Production equal-weight RRF	0.6118	0.9230	0.7087
Relative-score fusion	0.7070	0.9610	0.8038
Relative-score fusion + source deduplication	0.7181	0.9615	0.8142

The baseline also produced duplicate source documents in the top five for 249/2,000 queries. Source deduplication increased the mean number of unique documents without changing the stored chunks. Benchmark data and generated result files remain local and are not included in this PR.

Verification steps:

uv run pytest -q tests/unit/test_rank_fusion.py
uv run ruff check astrbot/core/knowledge_base/retrieval/rank_fusion.py tests/unit/test_rank_fusion.py

Results:

• 8 tests passed.
• Ruff checks passed.

---

Checklist / 检查清单

• 😊 If there are new features added in the PR, I have discussed it with the authors through issues/emails, etc.
  / 如果 PR 中有新加入的功能，已经通过 Issue / 邮件等方式和作者讨论过。
• 👀 My changes have been well-tested, and "Verification Steps" and "Screenshots" have been provided above.
  / 我的更改经过了良好的测试，并已在上方提供了“验证步骤”和“运行截图”。
• 🤓 I have ensured that no new dependencies are introduced, OR if new dependencies are introduced, they have been added to the appropriate locations in requirements.txt and pyproject.toml.
  / 我确保没有引入新依赖库，或者引入了新依赖库的同时将其添加到 requirements.txt 和 pyproject.toml 文件相应位置。
• 😮 My changes do not introduce malicious code.
  / 我的更改没有引入恶意代码。

Summary by Sourcery

Improve knowledge base retrieval quality by switching to normalized relative-score fusion with document-level diversification and strengthening the robustness of the vLLM rerank provider.

New Features:

• Introduce configurable relative-score fusion of dense and sparse retrieval signals with per-knowledge-base score normalization.
• Ensure fused retrieval results return at most one highest-scoring chunk per source document to increase document diversity.

Enhancements:

• Add input validation and edge-case handling for rank fusion parameters and top-k values to avoid invalid configurations.
• Refine vLLM rerank provider behavior to short-circuit on empty inputs, raise for HTTP errors, and validate response structure and contents.

Tests:

• Extend rank fusion unit tests to cover relative-score behavior, tie-breaking, document deduplication, and parameter validation.
• Add unit tests for the vLLM rerank provider to verify request payloads, error handling, empty-input behavior, and session termination.