Skip to content

Calibrated triage and audit of auto-admitted records #24

Description

@NingyuSUN

Part of the AI validation roadmap (docs/AI_VALIDATION_ROADMAP.md).

Goal. Make "AI does most of the work, experts review only high-risk samples" measurable: route records into auto-admit / expert review (risk-ranked) / reject, and bound the error rate of what is auto-admitted.

Done when

  • Routing uses only risk signals shown to predict errors (e.g. dissenting evidence), each with its evidence
  • bioevidence review audit-sample: a seeded random sample of auto-admitted records for expert audit
  • Reported upper bound on the auto-admitted error rate (e.g. 0 errors in 300 audited → below 1% at 95%)
  • Per-route statistics: share of records, audited error rate, share of expert time

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    ai-validationAI validation roadmap (0.8.0)

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions