Skip to content

M2: scoring profiles — the triage policy becomes a choice #93

Open

Description

@Jiahuic

Goal: make the triage score a choice of profile, not a fixed policy. Part of #87.

WEIGHTS in s2f/m2_triage/score.py encodes an antibacterial target-selection policy: human homology is a penalty, essentiality a bonus, membrane proteins deprioritised. That is right for bacteria and wrong for other questions — a viral protein, a single hypothetical, or the breast-cancer test case in #84 (where the target is human).

Scope

  • Named profiles, e.g. bacterial-target (today's weights, the default), evidence-only (no policy, just evidence), and later a viral profile if Decision: viral proteins — scope, and what changes per module #88 goes that way.
  • --profile NAME or --weights FILE; run.json records which profile produced the ranking, so two runs cannot be confused.
  • With one or a few sequences, skip ranking and report evidence only.
  • --top (default 50) stays the default selection size.

Definition of done

  • run.json names the profile and its weights
  • Changing the profile re-scores recorded proteins.tsv with no re-querying
  • Any change to the default profile still needs a dated row in the change log in docs/02a-m2-pdb-evidence.md (pitfall M2: triage scoring and selection #12)

Related: #61 (weight calibration), #84 Direction C


Unassigned on purpose: comment to claim this task, then assign yourself.

Activity

  1. Jiahuic commented on Oct 9, 2026

    @Jiahuic
    CollaboratorAuthor

    Status: #102 implements the profiles (open, ready to merge, last in the #98 → #102 chain).

    Done:

    • --profile bacterial-target (the default, exactly today's WEIGHTS), --profile evidence-only (provisional), and --weights FILE. run.json records the profile and its weights.
    • python -m s2f.m2_triage.rescore re-ranks recorded proteins.tsv under another profile with no re-querying. Under the default profile it reproduces M2 exactly, and a test pins this.
    • The default profile is still governed by the change log in docs/02a-m2-pdb-evidence.md.

    Left before closing:

    • Skip ranking for very small inputs. With a handful of sequences (a single query, a small virus, see Decision: viral proteins — scope, and what changes per module #88) a rank is meaningless, so M2 should report evidence without ranking below some size. The threshold is a decision for the team.
    • Calibrate evidence-only. Its weights (pdb_evidence 0.40, ligandable_homolog 0.10) are a placeholder; see M2: calibrate the triage weights on a genome that can test them #61.
    • Compare profiles on real genomes. The recorded G37 and HS11286 runs cannot be rescored: their proteins.tsv predates surface_bonus, ligandable_homolog and membrane_penalty, and the tool refuses them rather than guessing zeros. A fresh M2 run on each is needed first.
  2. Jiahuic commented on Oct 10, 2026

    @Jiahuic
    CollaboratorAuthor

    First real comparison of profiles (2026-10-10), from fresh M2 runs rescored with python -m s2f.m2_triage.rescore --profile evidence-only:

    Genome Proteins Uncharacterized Top 50 uncharacterized: bacterial-target evidence-only Overlap of the two top 50s
    G37 (--full-evidence) 542 190 39 7 12 of 50
    HS11286 (default flags) 5,338 1,338 33 4 10 of 50

    The two profiles select almost disjoint sets. evidence-only favours well-characterised proteins with good PDB hits; bacterial-target favours hypotheticals through annotation_gap (0.30). That is the expected direction, and it shows the choice of profile matters more than any single weight.

    These fresh runs also fix the earlier limitation that the recorded runs predated three components.

  3. added a commit that references this issue on Nov 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions