Repository navigation
Conversation
BM25, potion-base-2M and potion-base-8M over one text per graph node, scored on routing-bench.yaml's five meaning questions plus fifteen extra ones. The stdlib query path (WordPiece from tokenizer.json, safetensors read with struct) is checked against model2vec 0.9.0. The report is in the umbrella's MIP-0085 appendix. Tested: ruff check + format --check spike/ and the repo; parity.py (worst 1-cos 7.9e-14 and 1.1e-13 on 50 strings, 0 token-id mismatches on 1766 node texts); bench.py over 20 questions Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Throwaway, not for merge. This is MIP-0085 task 7's meaning spike. The code under
spike/meaning/exists so the numbers in the report can be reproduced. It should be closed unmerged once the report lands.The report is in the umbrella: marola-dev/marola#890, the MIP-0085 appendix "Meaning spike (task 7)". Issue #96 is resolved by that umbrella PR, not by this one.
nodetext.py: one text per graph node (label words, docstring and leading comment, path words).static_embed.py: the stdlib query path (BertNormalizer, BertPreTokenizer and WordPiece fromtokenizer.json;model.safetensorsrows read withstruct; mean, L2 norm, cosine).build_vectors.py,parity.py: model2vec 0.9.0 side (CI only, needs numpy); parity is MIP-0085 §7'squery_vector_parity.bench.py,questions.json: right first, bytes, latency for plainroute query, BM25, potion-2M, potion-8M.Tested:
ruff checkandruff format --checkonspike/and the repo;parity.py(worst 1−cos 7.9e-14 / 1.1e-13 on 50 strings, 0 token-id mismatches on 1,766 node texts);bench.pyover 20 questions. No model files or vectors are committed.