From 6241734fbbd416561328ec19dd97d9ce7a1adcdf Mon Sep 17 00:00:00 2001 From: Jiahui Chen Date: Tue, 29 Sep 2026 14:05:42 -0500 Subject: [PATCH 1/2] docs: post-workshop plan and the three directions for the restart The team agreed to continue from mid-October. This writes down where the project actually stands so the restart argues about something concrete. What it records: - M1-M3 run end to end (5.5 min on G37, 418 tests, validating report.json); M4 and M5 were never built, and M4 is the binding evidence the project is named for. Three PRs have been open since 18 September. - The pipeline has never run on a genome whose relatives rather than itself are in the PDB: 47 of 50 selected proteins rest on a same-genus hit, the weights are uncalibrated, and two circularity defects surfaced late. - About 2,500 lines have no BV-BRC coupling at all -- the structure search, AlphaFold lookup, quality gates, contract and renderers. Coupling is concentrated in M1 and three M2 files. So the project is a generic evidence factory plus an antibacterial selection policy, and generalizing means making the policy pluggable rather than rewriting the core. Three directions rather than two: finish what exists (sparse-time work), build M4 (needs sustained attention, and Ping's script is the starting point), and write the generic/domain seam down before refactoring toward it. Breast cancer is proposed as the test case for that seam because it inverts the policy -- human homology stops being a penalty, DepMap replaces flux balance, and Open Targets becomes native rather than awkward. Food-science peptides are flagged as a weaker fit: short peptides defeat the evidence modules specifically, so what transfers is the architecture, not the modules. CEPI is placed in the same bucket: a teammate's services are another instance of "can a third party's evidence plug in without touching the core", and the fit needs establishing since their public work is viral and ours is bacterial. Co-Authored-By: Claude Opus 5 (1M context) --- docs/README.md | 1 + docs/post-workshop-plan.md | 134 +++++++++++++++++++++++++++++++++++++ 2 files changed, 135 insertions(+) create mode 100644 docs/post-workshop-plan.md diff --git a/docs/README.md b/docs/README.md index 0a40966..315c270 100644 --- a/docs/README.md +++ b/docs/README.md @@ -14,6 +14,7 @@ NIAID-BRCs AI Codeathon 2.0 · Argonne National Laboratory · Sept 16–18, 2026 | [pitfalls.md](pitfalls.md) | Before trusting any result, and before designing a scoring or docking step. | | [common-ids.md](common-ids.md) | Mapping any identifier — feature_id, UniProt, PDB, ChEMBL. Never write your own mapping. | | [07-decisions-and-risks.md](07-decisions-and-risks.md) | Open decisions, thread dependencies, non-negotiables. | +| [post-workshop-plan.md](post-workshop-plan.md) | **Start here for the restart** — where the project stands after the codeathon, and the three directions proposed for what comes next. | ## Modules diff --git a/docs/post-workshop-plan.md b/docs/post-workshop-plan.md new file mode 100644 index 0000000..c25c5e4 --- /dev/null +++ b/docs/post-workshop-plan.md @@ -0,0 +1,134 @@ +# Post-workshop plan + +**29 September 2026.** The codeathon ran 16–18 September. The team agreed to continue, with most +people available from **mid-October**. + +This is a proposal, not a decision. It exists so the restart argues about a written thing. + +## Where the project actually is + +M1 (genome), M2 (triage) and M3 (structure collection) run end to end: a full pass over +*M. genitalium* G37 takes about 5½ minutes — 542 CDS in, 50 proteins selected, 50 structures +collected, `report.json` validating against the schema, and a self-contained HTML report out. +418 tests pass with no network. + +**M4 (ligands and docking) and M5 (disease evidence) were never built.** M4 is the +*binding evidence* in "Hypothetical Protein and Binding-Evidence Factory", so the project's +headline deliverable is half-built. Ping's `bacteria_structure_docking.py` does much of M4 and +M3's pocket work, but it sits outside the pipeline with its own ranking and its own output +directories. + +Three pull requests have been open since 18 September: **#62** (evaluation harness, ~7,900 +lines), **#76** (declares M6's `jinja2` dependency), **#77** (M1 species metadata). 36 issues are +open. + +### What we have not shown + +The pipeline has never run on a genome whose **relatives** rather than **itself** are in the PDB. +On G37, 47 of the 50 selected proteins rest on a same-genus structural hit and 30 on the same +species. The triage weights are uncalibrated (#61), and two circularity defects surfaced late: +the self-match artifact (#12) and an essentiality reference set that can include the query +genome itself (#68). + +Everything below assumes that gap is the most important thing to close, in any direction. + +## The shape of the code, and why generalizing is cheaper than it looks + +About 2,500 lines have **no BV-BRC coupling at all**: + +| Portable today | Lines | +| --- | --- | +| `m2_triage/foldseek.py`, `pdb_evidence.py`, `afdb.py` | 972 | +| `m3_fold/` — structure collection and quality gates | 923 | +| `common/io.py`, `common/schema.py`, `common/http.py` | ~500 | +| `m6_report/`, `full_report/` | ~460 | + +BV-BRC coupling is concentrated in **M1** (24 references in `collect.py` alone, plus +`bvbrc_api.py`, `parse.py`, `priority.py`, `taxon.py`) and in **three M2 files**: +`essentiality.py`, `bvbrc_input.py`, `taxonomy.py`. + +So the project is really two things stacked: + +1. a **generic protein-evidence factory** — given protein sequences, find structures, homologs, + ligandability and knowledge-graph links, and report all of it with provenance; and +2. an **antibacterial target-selection policy** — the weights that say human homology is a + penalty, essentiality is a bonus, and membrane proteins are deprioritized. + +Generalizing the pipeline means making (2) pluggable. It is not a rewrite of (1). + +## Direction A — finish what exists + +Sparse-time work, suitable for everyone. In rough order of value: + +- [ ] **Decide the three stale PRs.** #76 is a dependency declaration and should just land. #62 is + the only *measurement* work anyone did, and its findings stand on their own even if the + code does not merge — including that M1's `--seq-cap` drops 58% of hypothetical proteins + before M2 sees them, in a project named after hypothetical proteins. +- [ ] **The defect backlog** — #67, #68, #69, #72, #80, #81. Each is one sitting. #68 matters + most: essentiality transferred from a reference set that can contain the query genome is + circular, and it is in a number we have quoted. +- [ ] **#14, pockets and receptor preparation** (p0, unstarted). Ping's script already produces + prepared receptors, pocket boxes and Boltz-2 inputs. Adapting it is cheaper than starting. +- [ ] **#22, CI.** 418 tests exist and nothing runs them automatically. Everything merged so far + was verified by whoever remembered to run `pytest`. +- [ ] **Close #21** (the presentation) — it is finished and historical. + +## Direction B — the biggest missing piece + +**M4: ligand candidates, scoring and docking** (#15, #16, #17). This is the one gap that changes +what the pipeline *claims to be*, and it needs sustained attention rather than spare evenings. +Starting point is Ping's existing script rather than a blank file. + +M5 (#19, #20) is the same shape but one step further out; it can wait. + +## Direction C — generalize, starting with a document + +**Write the seam down before refactoring anything.** One document that answers: where does the +generic evidence factory end and the domain policy begin, and what does a domain have to supply? + +Use **breast cancer** as the test case, because it inverts the policy in ways that expose whether +the seam is real: + +| Component | Bacterial pathogen | Human cancer target | +| --- | --- | --- | +| `human_homolog_penalty` (−0.25) | avoid human similarity — toxicity risk | **the target is human**; becomes a paralog-selectivity question instead | +| `essential` | BV-BRC flux-balance prediction | **DepMap CRISPR screens** — stronger evidence than we have now | +| Open Targets | human-centric, reachable only through a homolog | **native**; this is what it is built for | +| `surface_bonus` | vaccine and antibody angle | antibody and ADC targets — still applies | +| ChEMBL ligandability | applies | applies, with far more data | +| Input | genome assembly → BV-BRC annotation | a gene or protein list from expression or mutation data | + +The parts that were *weakest* for bacteria are *strongest* here, which is the best evidence that +the split is genuine rather than wishful. + +**Food-science peptides are a poorer fit, and worth saying so early.** Short peptides (5–20 +residues) defeat the evidence modules specifically: PDB sequence search at that length returns +noise against our 1e-5 gate, AlphaFold and ESMFold are unreliable, and Foldseek needs a fold that +does not exist. What transfers is the *architecture* — the report contract, the provenance +discipline, the transparent weighted score, the rule that "not run" is not "zero" — not the +evidence modules. That is still worth something, but it is a different kind of reuse. + +**CEPI belongs here too.** A teammate noted that services their group is developing could feed +several pipeline steps. CEPI (Coalition for Epidemic Preparedness Innovations) funds vaccine and +countermeasure work, including epitope prediction and immunogen design, and was among the +collaborators on the AlphaFold Database's viral protein complex release of 24 September 2026. +Their public work is largely viral and vaccine-facing while ours has only ever run on bacteria, +so the fit needs establishing rather than assuming. It is the same question as the seam: can a +third party's evidence service be plugged in without touching the core? + +## Sequencing + +A and C are not in competition. The refactor that makes the policy layer pluggable is also what +lets us run a **null-policy control** on the bacterial case — which is exactly what calibration +(#61) needs. Doing a slice of C serves A. + +What should not happen is building B and C at once on an unvalidated core. + +## Open questions for the restart + +1. Do we finish the bacterial case first, or is a second domain more motivating for people + working evenings? +2. Is anyone willing to own M4 with sustained time, or should it stay out of scope? +3. What do we do with #62 — merge, or keep the findings and close it? +4. Which CEPI services, concretely, and do they have a callable API? +5. Do we need a genome that is **not** in the PDB before any of this means anything? From 1ff90fb41242e53f8b89a70ad230592b68b63656 Mon Sep 17 00:00:00 2001 From: Jiahui Chen Date: Tue, 29 Sep 2026 14:31:19 -0500 Subject: [PATCH 2/2] docs: size the plan for the five people actually continuing Carla Mann, Alexander Partin, Ping Hu, Ashita Jawali and Jiahui Chen are continuing; Moises Gualapuro, MD Shakhaowat Hossain, Heng Ma and Archit Vasan are not. There is no longer a co-lead, so sequencing decisions land in one place, and five people working evenings is the constraint the proposals are measured against. Two things change substantively: M4 is the best-staffed gap on the board, which was not visible until the team shrank. Ping's prototype already produces prepared receptors, pocket boxes and Boltz-2 inputs; Alex holds Boltz-2 co-folding and built the structure collection M4 consumes; Ashita holds ligand candidate collection. Three of the five are already adjacent to it, and the work is less "build M4" than "bring Ping's prototype inside the contract" -- it currently has its own ranking and output directories and reads none of report.json. #62 becomes a different decision now its author has left: merging ~7,900 lines means owning them. Its most important finding is filed separately as #85 -- M1's --seq-cap removes 58% of hypothetical proteins before M2 sees them, because m1_priority scores a hypothetical 0 while annotation_gap pays 0.30 for exactly that property -- so the finding survives whichever way the PR goes. Refs #85. Co-Authored-By: Claude Opus 5 (1M context) --- docs/post-workshop-plan.md | 50 +++++++++++++++++++++++++++++--------- 1 file changed, 39 insertions(+), 11 deletions(-) diff --git a/docs/post-workshop-plan.md b/docs/post-workshop-plan.md index c25c5e4..178ca2b 100644 --- a/docs/post-workshop-plan.md +++ b/docs/post-workshop-plan.md @@ -1,7 +1,20 @@ # Post-workshop plan -**29 September 2026.** The codeathon ran 16–18 September. The team agreed to continue, with most -people available from **mid-October**. +**29 September 2026.** The codeathon ran 16–18 September. Five of the eight participants are +continuing, most of them available from **mid-October**. + +| Continuing | GitHub | Worked on | +| --- | --- | --- | +| **Jiahui Chen** (lead) | `Jiahuic` | M2 triage, `common/` contract, M6 report | +| Carla Mann | `cmmann21` | M1, M2 functional annotation, most of the documentation | +| Alexander Partin | `adpartin` | M3 structure collection | +| Ping Hu | `PingPnG` | M1 scripts, the docking and pocket prototype | +| Ashita Jawali | `Ashita2619` | ESMC embeddings | + +Not continuing: Moises Gualapuro (co-lead), MD Shakhaowat Hossain, Heng Ma, Archit Vasan. +**There is no longer a co-lead**, so sequencing decisions land in one place. + +Five people working evenings is the constraint every proposal below is measured against. This is a proposal, not a decision. It exists so the restart argues about a written thing. @@ -60,10 +73,13 @@ Generalizing the pipeline means making (2) pluggable. It is not a rewrite of (1) Sparse-time work, suitable for everyone. In rough order of value: -- [ ] **Decide the three stale PRs.** #76 is a dependency declaration and should just land. #62 is - the only *measurement* work anyone did, and its findings stand on their own even if the - code does not merge — including that M1's `--seq-cap` drops 58% of hypothetical proteins - before M2 sees them, in a project named after hypothetical proteins. +- [ ] **Decide the three stale PRs.** #76 is a dependency declaration and should just land; #77 + is M1's, and Carla is continuing. **#62 is the awkward one**: it is the only *measurement* + work anyone did, it is ~7,900 lines, and its author is not continuing — so merging it + means owning it. Its most important finding is now filed separately as **#85** (M1's + `--seq-cap` removes 58% of hypothetical proteins before M2 sees them), so the finding + survives whichever way the PR goes. Closing it with thanks is a reasonable outcome; + leaving it open indefinitely is not. - [ ] **The defect backlog** — #67, #68, #69, #72, #80, #81. Each is one sitting. #68 matters most: essentiality transferred from a reference set that can contain the query genome is circular, and it is in a number we have quoted. @@ -77,7 +93,15 @@ Sparse-time work, suitable for everyone. In rough order of value: **M4: ligand candidates, scoring and docking** (#15, #16, #17). This is the one gap that changes what the pipeline *claims to be*, and it needs sustained attention rather than spare evenings. -Starting point is Ping's existing script rather than a blank file. + +**It is also the best-staffed gap we have**, which was not obvious until the team shrank: +Ping's `bacteria_structure_docking.py` already produces prepared receptors, pocket boxes and +Boltz-2 inputs; Alex holds #18 (Boltz-2 co-folding) and built M3's structure collection, which +is what M4 consumes; and Ashita holds #15 (ligand candidate collection). Three of the five +continuing people are already pointed at this. + +The work is less "build M4" than "bring Ping's prototype inside the contract" — it currently has +its own ranking and its own output directories, and reads none of `report.json`. M5 (#19, #20) is the same shape but one step further out; it can wait. @@ -127,8 +151,12 @@ What should not happen is building B and C at once on an unvalidated core. ## Open questions for the restart 1. Do we finish the bacterial case first, or is a second domain more motivating for people - working evenings? -2. Is anyone willing to own M4 with sustained time, or should it stay out of scope? -3. What do we do with #62 — merge, or keep the findings and close it? + working evenings? With five people this is a real trade, not a both-at-once. +2. **M4**: does Ping's prototype come inside the contract, and who drives it — Ping, Alex, or + the two together? This is the highest-value thing on the board and the only one with three + people already adjacent to it. +3. What do we do with #62? Merging ~7,900 lines whose author has left means owning them. + The key finding is preserved in #85 either way. 4. Which CEPI services, concretely, and do they have a callable API? -5. Do we need a genome that is **not** in the PDB before any of this means anything? +5. Do we need a genome that is **not** in the PDB before any of this means anything? Nothing + we have measured so far survives that question unchanged.