Reorder KEGG compound resolution to check ChEBI/PubChem bulk sources first - #9
Draft
m-crown wants to merge 1 commit into
Draft
Conversation
…first
get_ec_information.py's KEGG-direct compound-structure lookup
(get_kegg_compound_record/get_kegg_compound_smiles) previously ran
unconditionally for every unique KEGG compound code seen across all EC
reactions, hitting rest.kegg.jp once per compound with no rate limiting
of its own (unlike get_kegg_enzymes/get_kegg_reactions, which chunk and
throttle). ChEBI and PubChem resolution ran independently afterward,
meaning every compound got a live KEGG call regardless of whether a
bulk/cheap source already had it.
Per the coverage analysis in docs/cognate_ligands_generation_plan.md
("Question 3": ~93% of KEGG compound codes seen in reactions already have
a structure via ChEBI's bulk file or PubChem's batched CID lookup), moved
the ChEBI and PubChem blocks to run first, then filter the KEGG-direct
lookup to only the compound codes neither of those confirmed resolved.
Deliberately conservative on PubChem: codes are only excluded from the
KEGG-direct fallback after PubChem's own live batched CID->SMILES call
confirms a structure, not just because a CID mapping exists in the bulk
file - avoids any coverage regression for the rare code that has a
mapping but doesn't actually resolve. Net effect on final cognate ligand
coverage: none, by construction - purely a reduction in unnecessary live
KEGG traffic.
Left a second, separate get_kegg_compound_record call site (inside the
PubChem block, fetching full KEGG records just for a display name) alone
- smaller, lower-priority optimization, out of scope here.
Verified via py_compile, a duplication check, and an isolated logic test
against synthetic data mimicking the real post-filter shapes. Not run
against the full live pipeline (requires complete data_dir setup and
hours of upstream KEGG enzyme/reaction fetching before reaching this
code) - see docs/SESSION_HANDOFF.md for what verification was and wasn't
possible in-session.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
get_kegg_compound_record's resolution logic to check ChEBI/PubChem's bulk cross-reference data before falling back to a live KEGG structure-fetch call.Stacked on #8 (feature/reference-data-download) — this PR's diff is scoped to just the KEGG reordering work.
Test plan
get_kegg_compound_recordreturns identical structures to before for compounds covered by ChEBI/PubChem, sourced without a live KEGG call🤖 Generated with Claude Code