Repository navigation
Cell Ontology definition check and the external single-cell split (negative result) - #38
Merged
Merged
Conversation
definitions.MarkerDefinitions reads the presence and absence marker axioms of each Cell Ontology term (own and inherited) and maps their proteins to HGNC genes; DefinitionGrounder sends a record to review (BEV026) when the claimed term's definition is contradicted by the cluster's measurements. Thresholds (present below 10%, absent at 50% or more and higher than elsewhere) were set on the first six datasets. The case adds per-cluster definition panels and six datasets from other studies as the external split (81 clusters). Protocol 3 is protocol 2 plus the definition check; it is committed before the external split is run.
486 episodes on six datasets from other studies, with protocol 3 frozen in c86d5d4. The definition check flagged wrong annotations no better than chance (47% against 42%): several protein definitions do not hold for transcripts (mast cell CCR3, neutrophil CEACAM8, NK cells lacking CD3 epsilon). Fed back in the loop, its findings turned 15 correct answers into wrong ones. Identifier errors (70 of 479) were all caught.
Its findings still go back to the proposer in the feedback loop; the docs record that, fed back on transcript data, they turned correct answers into wrong ones, and that the check needs markers validated at the mRNA level.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
definitions(Cell Ontology presence/absence marker axioms mapped to genes, checked against measurements, BEV026) and an external split of six new datasets (81 clusters). Protocol 3 was frozen in c86d5d4 before the run.Result: negative. On the external split the check flagged wrong annotations no better than chance (47% vs 42%). Several protein definitions do not hold for transcripts: mast cell CCR3 and neutrophil CEACAM8 are undetected, and NK/ILC3 cells express CD3E. Fed back in the loop, the findings turned 15 correct answers into wrong ones, and compatible answers fell from 278 (model alone) to 242. The identifier layer still caught all 70 identifier errors.
Lessons, written into the docs:
Checks: ruff, mypy, pytest (98.71%), mkdocs --strict, wheel smoke test. Byte-for-byte replays of all three single-cell splits.
Not for merge as-is without a decision: keep the module with this documented limitation, or keep it out until markers are validated.