Generic identifier, table and reference grounders, and the feedback loop

CHANGELOG.md

@@ -2,6 +2,17 @@
 
 ## Unreleased
 
+- Add generic grounders that check what a record cites against pinned reference releases, with
+  messages that say what to fix: `OntologyGrounder` (ontology terms: existence, obsoletion, label,
+  kind), `GeneGrounder` (HGNC symbols, previous symbols and aliases), `VariantGrounder` (HGVS form,
+  RefSeq accession, genome build, position), `TableGrounder` (rows and values of a pinned table) and
+  `ReferenceGrounder` (a curated reference resource that contradicts the claim or its evidence).
+  New codes BEV023 (obsolete or superseded identifier), BEV024 (malformed or wrong kind) and BEV025
+  (reference conflict, review only). `bioevidence validate` gains `--ontology`, `--term-root`,
+  `--genes` and `--assembly-report`.
+- Add `feedback`, the validate–feedback–revise loop around any proposer: fixable findings go back as
+  reasons, conflicts and policy decisions go to a person unchanged, and verified evidence is carried
+  through revisions.
 - Benchmark scenario 2 (`agent_loop.py`): an agent searches PubMed, reads papers and submits a
   decision with citations; bioevidence checks each submission and returns its reasons, and the agent
   may revise. Five of six models cited only what they had read; Claude Haiku 4.5 misquoted in 8 of

docs/ENGINEERING.md

@@ -44,6 +44,9 @@ evidence items), and duplicate source/item/line/adjudication IDs cannot be admit
 | BEV020 | A required type the profile lists under `verified_evidence_types` has no item a grounder verified | Review affected use |
 | BEV021 | The use requires independent review, and no independent non-human reviewer accepted it (or one deferred or rejected) | Review affected use |
 | BEV022 | Semantic cue: a quote supporting a positive claim is negated, hedged (optional), or only about animals or cells for a human-scoped item | Review |
+| BEV023 | Grounding: a cited identifier is obsolete, withdrawn, or not its current name (a previous gene symbol or alias) | Reject |
+| BEV024 | Grounding: a cited identifier is malformed, or of the wrong kind for its place (e.g. not a cell type) | Reject |
+| BEV025 | Cross-check: a pinned reference resource contradicts the claim or its supporting evidence | Review |
 
 A human acceptance does not erase contradictions, missing evidence, source mismatches,
 or other human rejection/deferral. Low-strength extraction permissions are explicit
@@ -126,6 +129,79 @@ it cannot tell whether that source is itself right, whether a verbatim quote rea
 the claim (a correctly quoted sentence read with the wrong polarity passes), or anything about
 sources without a pinned snapshot and a grounder.
 
+### Identifiers, tables and reference resources
+
+These grounders are generic: they check what a record cites against a pinned release of a reference,
+whatever the domain. Their messages say what to fix, so a model can act on them (see the feedback loop
+below). Each grounder's name carries the release it used, and so does the report's `grounding` key.
+
+Identifiers are read from the statement's subject and object (`id`, `label`, `entity_type`), its scope
+tokens, and evidence locators written as `key=value` pairs separated by `;` (`cluster=3;gene=CD8A`).
+
+- `OntologyGrounder` (`identifiers`) checks terms against pinned OBO releases (Cell Ontology, HPO,
+  MONDO, UBERON …):
+  - the term exists (BEV016);
+  - it is not obsolete (BEV023, naming its replacement);
+  - its label is the term's name or an exact synonym, ignoring case, punctuation and a plain plural (BEV017, naming the
+    term that label belongs to);
+  - it is of the right kind: `roots` maps an entity type or locator key to the terms it must descend
+    from, so a `cell_type` must be under CL:0000000 (BEV024).
+- `GeneGrounder` checks human gene symbols and HGNC identifiers against a pinned HGNC complete set:
+  - an unknown symbol is BEV016;
+  - a previous symbol, an alias or a wrong case is BEV023, naming the approved symbol;
+  - a record scoped to another species is not checked (BEV015).
+- `VariantGrounder` checks HGVS descriptions on RefSeq sequences against pinned NCBI assembly reports:
+  - the description must be well formed and versioned (BEV024);
+  - a chromosome accession must be a real sequence (BEV016), on the genome build of the record's scope
+    (`GRCh38`, `GRCh37`, `hg19`, `hg38`; BEV017), with the position inside the chromosome (BEV016);
+  - variants on more than one build in one record are BEV017.
+- `TableGrounder` (`tables`) is the data counterpart of the quote check.
+  - An evidence item of a configured type names one row of a pinned table (TSV or CSV) by its key
+    columns in the locator, and may claim values from it in `extracted_text` (`logfc=2.4; pct_in=0.91`).
+  - A missing row is BEV016, naming the closest key values. Several matching rows are BEV015.
+  - A claimed value that differs from the table is BEV017; a number matches at the precision it is
+    written with.
+  - An item whose row exists and whose values match counts as verified.
+- `ReferenceGrounder` (`crosscheck`) compares a record with a pinned reference resource it does not cite,
+  a table of curated `subject`, `predicate`, `object` assertions. A disagreement sends the record to review
+  (BEV025) and never rejects it: references are partial and can be out of date, and their silence is not
+  support. It raises BEV025 in two cases:
+  - the reference asserts the opposite predicate for the statement's subject and object;
+  - supporting evidence names an entity (e.g. `gene=CD19`) that the reference relates (e.g. `marker_of`)
+    only to objects that conflict with the statement's object. By default a different term conflicts unless
+    the ontology makes it an ancestor or descendant. With `conflict="disjoint"`, only terms the ontology
+    declares disjoint conflict (`disjoint_from` on them or their ancestors, e.g. T cell and B cell), so a
+    marker the reference lists for a sibling type is not reported. That needs an ontology release that keeps
+    its disjointness axioms: the Cell Ontology's full `cl.obo` does, `cl-basic.obo` does not.
+
+```bash
+bioevidence validate record.json --ontology cl-basic.obo --term-root cell_type=CL:0000000     --genes hgnc_complete_set.txt --assembly-report GRCh38_assembly_report.txt
+```
+
+`TableGrounder` and `ReferenceGrounder` need their configuration (evidence types, keys, relation) and are
+used from Python.
+
+### The feedback loop
+
+`feedback.revise(propose, validator)` runs the loop around any proposer: a model call, an agent or a person.
+
+1. The proposer returns a record, and bioevidence validates it.
+2. Every finding the proposer can fix goes back as a reason that points at the part of the record
+   concerned ("evidence 2 (cluster=3;gene=CD8B): No row of the pinned table has cluster=3, gene=CD8B.
+   Closest gene values for cluster=3: CD8A.").
+3. The proposer may then revise, up to a set number of rounds.
+
+Two kinds of finding are not fed back. The record goes to a person as it is:
+
+- **Judgment:** its own evidence disagrees (BEV004), a reference contradicts it (BEV025), a semantic cue
+  flags it (BEV022), or it lacks an independent review (BEV021).
+- **Policy:** the use needs a human decision whatever the proposer does (BEV008–BEV013).
+
+Evidence verified in one attempt is carried into the next while the claim stays the same, so a revision can fix or replace what failed
+but cannot withdraw verified evidence against its answer. The literature benchmark's agent loop
+(`evaluation/llm_benchmark/agent_loop.py`) showed why: before that rule, an agent whose record held a
+misquote and a verified quote against its decision dropped the latter and was admitted.
+
 ## Semantic checks
 
 Grounding proves that a quote is real; it cannot tell whether the quote supports the claim.

src/bioevidence_validator/cli.py

@@ -13,6 +13,7 @@
 from .draft import build_record, draft_json_schema, load_draft
 from .engine import default_schema_path, generate_json_schema, list_profiles, profile_path, validate_record
 from .grounding import SnapshotStore, SourceBytesGrounder
+from .identifiers import Assemblies, GeneGrounder, Genes, Ontology, OntologyGrounder, VariantGrounder
 from .literature import CATALOG, LiteratureGrounder, ground_record, http_fetch
 
 
@@ -27,6 +28,14 @@ def parser() -> argparse.ArgumentParser:
     validate.add_argument("--snapshot-dir", type=Path,
                           help="Directory of source snapshots named by SHA-256; recompute source hashes from them, "
                                "and check cited publications if it holds a literature.json from `ground`")
+    validate.add_argument("--ontology", type=Path, action="append", default=[],
+                          help="A pinned OBO release; check the ontology terms the record cites (repeatable)")
+    validate.add_argument("--term-root", action="append", default=[], metavar="KIND=CURIE",
+                          help="Terms for this entity type or locator key must descend from CURIE, e.g. "
+                               "cell_type=CL:0000000 (repeatable)")
+    validate.add_argument("--genes", type=Path, help="A pinned HGNC complete set (TSV); check human gene symbols")
+    validate.add_argument("--assembly-report", type=Path, action="append", default=[],
+                          help="A pinned NCBI assembly report; check HGVS variants and genome builds (repeatable)")
     ground = commands.add_parser("ground", help="Resolve cited publications and pin their metadata and open-access "
                                                 "full text (uses the network)")
     ground.add_argument("input", type=Path)
@@ -175,6 +184,19 @@ def _run(args) -> int:
         grounders.append(SourceBytesGrounder(SnapshotStore.from_directory(args.snapshot_dir)))
         if (args.snapshot_dir / CATALOG).exists():
             grounders.append(LiteratureGrounder.from_directory(args.snapshot_dir))
+    if args.ontology:
+        roots: dict[str, list[str]] = {}
+        for rule in args.term_root:
+            kind, _, curie = rule.partition("=")
+            if not kind or not curie:
+                raise ValueError(f"--term-root must be KIND=CURIE, not {rule!r}")
+            roots.setdefault(kind, []).append(curie)
+        grounders.append(OntologyGrounder([Ontology.from_obo(path.read_bytes()) for path in args.ontology], roots,
+                                          locator_keys=list(roots)))
+    if args.genes:
+        grounders.append(GeneGrounder(Genes.from_hgnc(args.genes.read_bytes(), args.genes.name)))
+    if args.assembly_report:
+        grounders.append(VariantGrounder(Assemblies.from_reports(path.read_bytes() for path in args.assembly_report)))
     report = validate_record(record, schema_path=args.schema, profile=args.profile, grounders=grounders)
     rendered = json.dumps(report, indent=2) + "\n"
     if args.output:

src/bioevidence_validator/crosscheck.py

@@ -0,0 +1,92 @@
+"""Cross-check against a pinned reference resource: what a curated reference contradicts goes to an expert.
+
+Grounding checks a record against the sources it cites. A reference resource is a second, independent
+source the record does not cite: a knowledge base of curated assertions. Where it disagrees, the record is
+not wrong on that ground alone (the reference can be out of date, and its coverage is partial), so the
+finding sends the record to review (BEV025) and never rejects it. Silence of the reference is not support.
+
+Assertions are rows with `subject`, `predicate` and `object` (optionally `subject_label`, `object_label`),
+e.g. a CIViC-style `BRAF V600E | does_not_support_sensitivity | vemurafenib in X`, or a marker table
+`CD19 | marker_of | CL:0000236`. Two checks:
+
+- statement: the reference asserts the opposite predicate for the same subject and object (`opposites`
+  maps each predicate to its opposite);
+- evidence: supporting evidence names an entity in its locator (`evidence_key`, e.g. `gene=CD19`) that the
+  reference relates (`relation`, e.g. `marker_of`) only to objects that conflict with the statement's object.
+  With `conflict="unrelated"` (the default), an object conflicts unless it is the same term or, with an
+  ontology, an ancestor or descendant of it (a B cell marker supports "naive B cell"). With
+  `conflict="disjoint"`, an object conflicts only when the ontology declares the two disjoint (`disjoint_from`
+  on them or their ancestors, e.g. T cell and B cell): a marker the reference lists for a sibling type is not a
+  conflict, and where the ontology declares nothing, nothing is reported.
+"""
+from __future__ import annotations
+
+import csv
+import io
+from collections.abc import Iterable
+from typing import Any
+
+from .engine import Finding
+from .grounding import finding
+from .identifiers import Ontology, pairs
+
+
+class ReferenceGrounder:
+    def __init__(self, assertions: Iterable[dict[str, str]], *, label: str, opposites: dict[str, str] | None = None,
+                 evidence_key: str | None = None, relation: str | None = None, ontology: Ontology | None = None,
+                 conflict: str = "unrelated"):
+        if conflict not in ("unrelated", "disjoint") or (conflict == "disjoint" and ontology is None):
+            raise ValueError("conflict must be 'unrelated', or 'disjoint' with an ontology")
+        self.conflict = conflict
+        self.assertions = list(assertions)
+        self.label, self.ontology = label, ontology
+        self.opposites = dict(opposites or {})
+        self.opposites.update({v: k for k, v in self.opposites.items()})
+        self.key, self.relation = evidence_key, relation
+        self.name = f"reference:{label}"
+        self._pairs: dict[tuple[str, str], list[dict[str, str]]] = {}
+        self._objects: dict[str, list[dict[str, str]]] = {}
+        for row in self.assertions:
+            self._pairs.setdefault((row["subject"], row["object"]), []).append(row)
+            if row["predicate"] == relation:
+                self._objects.setdefault(row["subject"], []).append(row)
+
+    @classmethod
+    def from_table(cls, data: bytes, **options: Any) -> ReferenceGrounder:
+        text = data.decode("utf-8")
+        delimiter = "\t" if "\t" in text.partition("\n")[0] else ","
+        return cls(csv.DictReader(io.StringIO(text), delimiter=delimiter), **options)
+
+    def _conflicts(self, a: str, b: str) -> bool:
+        if self.conflict == "disjoint":
+            return bool(self.ontology and self.ontology.disjoint(a, b))
+        return not (a == b or bool(self.ontology and self.ontology.related(a, b)))
+
+    def _name(self, row: dict[str, str]) -> str:
+        return f"{row['object']} ({row['object_label']})" if row.get("object_label") else row["object"]
+
+    def check(self, record: dict[str, Any]) -> list[Finding]:
+        statement = record["statement"]
+        subject, predicate, target = statement["subject"]["id"], statement["predicate"], statement["object"]["id"]
+        out = []
+        for row in self._pairs.get((subject, target), []):
+            if self.opposites.get(row["predicate"]) == predicate:
+                out.append(finding(record, "BEV025", f"The {self.label} reference records {subject} {row['predicate']} "
+                                                     f"{target}, the opposite of this claim.", "$.statement.predicate"))
+        if not self.key:
+            return out
+        supporting = {i for line in statement["evidence_lines"] if line["direction"] == "supports"
+                      for i in line["evidence_item_ids"]}
+        for index, item in enumerate(record["evidence_items"]):
+            entity = pairs(item["locator"]).get(self.key)
+            rows = self._objects.get(entity or "", [])
+            if item["id"] not in supporting or not rows or not all(self._conflicts(r["object"], target) for r in rows):
+                continue
+            names = sorted({self._name(r) for r in rows})
+            more = f" and {len(names) - 3} more" if len(names) > 3 else ""
+            out.append(finding(record, "BEV025", f"The {self.label} reference lists {entity} as a {(self.relation or '').replace('_', ' ')} "
+                                                 f"{'; '.join(names[:3])}{more}, "
+                                                 + (f"cell types disjoint from {target}." if self.conflict == "disjoint"
+                                                    else f"not of {target} or a related term."),
+                               f"$.evidence_items[{index}].locator"))
+        return out