#221 + #223: fix access-segmentation over-credit + accessible_km parity roll-up#228
Merged
Merged
Conversation
Per-WSG accessible_km roll-up + parity compare. Coho proof validated live (MORR 0.09%, BULK 0.27% vs tunnel-free fresh.streams_vw_bcfp). Snapshot barrier columns are comma-joined text, so accessible predicate is = '', not = array[]::text[]. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add data-raw/accessible_km_proof_co.R: reproducible +/-5% assertion that link's per-WSG coho accessible_km (streams_access.access_co IN (1,2), joined to streams on full PK for length) matches the tunnel-free bcfp snapshot (fresh.streams_vw_bcfp, barriers_ch_cm_co_pk_sk_dnstr = ''). 19/20 local WSGs within +/-5% (most <1%); SETN allowlisted as the documented bcfp-side stale-subsurfaceflow divergence. Comparison universe is the coho-present lnk CTE (per-species vs salmon-group semantics), with a null_ref guard for the link-built/bcfp-empty case. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
New exported lnk_rollup_wsg: predicate-driven per-(WSG, species) length roll-up over persisted streams + streams_access + streams_habitat_<sp>, joined on full PK (#203). Aliases the species-varying columns to generic access/spawning/rearing so the metrics SQL stays species-agnostic -- mirrors fresh::frs_aggregate's metrics/where shape. Default metrics emit accessible_km + spawning_km + rearing_km. accessible_km sums streams_access.access_<sp> IN (1,2) (the Phase-1 proof-validated source), NOT the streams_habitat_<sp>.accessible bool, which carries different pre-gating semantics and diverges (MORR coho 3424 vs the validated 3330 km). 27 tests (arg validation + injection guards + offline SQL build via DBI::ANSI()). Live-verified MORR coho accessible_km = 3330.25. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add @Seealso [lnk_aggregate()] and a note that lnk_rollup_wsg is a flat per-WSG GROUP BY (whole-WSG length by species), distinct from lnk_aggregate() / fresh::frs_aggregate() which roll habitat up the network upstream of individual crossings. Makes the rollup-vs-aggregate grain split intentional rather than an accidental shared-token clash. Regenerated Rd family-index back-references so the compare family lists lnk_rollup_wsg on each member (missed in the prior commit). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
.lnk_compare_rollup_link's km block now delegates to lnk_rollup_wsg via a COALESCE'd 5-metric vector, preserving the historical CASE-WHEN measured-0 semantics (FILTER over an empty set yields NULL, not 0), then renames species->species_code and reselects columns to keep the list(km, lake_ha, wetland_ha) contract. lake_ha/wetland_ha (DISTINCT-waterbody polygon joins) stay as-is. lnk_rollup_wsg's streams_access join changed JOIN->LEFT JOIN: access is optional metadata for a length rollup, so habitat length is never dropped when access is unbuilt (absent access -> NULL -> accessible_km 0). Safe even for mapping_code=FALSE persists since lnk_pipeline_access runs unconditionally. Byte-identical vs the old CASE-WHEN SQL on MORR BT+CO across all 5 metrics; lnk_rollup_wsg accessible_km still 3330.25. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
.lnk_compare_rollup_link appends accessible_km to km_metrics: round(COALESCE(sum(length_metre) FILTER (WHERE access IN (1,2)),0)/1000,2). It sources link's per-species access model via lnk_rollup_wsg's LEFT-joined `access` alias (streams_access.access_<sp>), NOT the divergent streams_habitat.accessible bool. streams_access's PK (id_segment, watershed_group_code) matches the join key, so the LEFT JOIN is 1:1 and cannot inflate any length sum. .lnk_compare_wsg_assemble_rollup gains `accessible` (unit km) in its four parallel named vectors. The bcfp habitat reference has no accessible column, so accessible's ref_value/diff_pct are NA by design until the tunnel-free reference path lands (Phase 2 4/4). Row-count assertions bumped 7->8 / 14->16 in test-lnk_compare_wsg.R and 7->8 in test-lnk_compare_rollup.R, each with a new accessible-row check (link value populated, ref NA). Live MORR coho accessible_km 3330.25 (= Phase-1 proof). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
.lnk_compare_wsg_accessible_ref(conn, aoi, species) sums fresh.streams_vw_bcfp.length_metre where a species' bcfp barrier-group column = '' (empty = no barrier downstream = accessible). Sourced from the local province-wide snapshot (conn), NOT the dead M1 :63333 tunnel. Only the Phase-1-proven salmon group (CH/CM/CO/PK/SK -> barriers_ch_cm_co_pk_sk_dnstr) is wired; other species short-circuit to NA without a query (each needs its own proof, Phase 3). barrier_group values are a trusted constant whitelist; aoi goes through DBI::dbQuoteLiteral. .lnk_compare_wsg_rollup_reference gains a local conn param and merges accessible_km onto the tunnel-based habitat-ref data.frame. The two reference sources stay intentionally decoupled: habitat stays tunnel-based on conn_ref; accessible is tunnel-free on conn. lnk_compare_rollup passes conn = conn. Live: MORR CO ref 3327.38 vs link 3330.25 -> diff_pct 0.1% (= Phase-1 proof); BT ref NA. SK ref = CO ref (bcfp models the salmon group with one shared column; per-species vs per-group coarsening surfaces in the diff, documented for Phase 3). 4 new helper tests (mock DBI::dbGetQuery: salmon -> query with the barriers_ch_cm_co_pk_sk_dnstr predicate; BT/ST/WCT -> short-circuit NA, no query). m_ref mock signature gained conn. 65+20 tests green in the two touched files; lint no new indent notes; docs regenerated. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Visual proof of the accessible_km BT over-credit: link's segment [3391,7998] spans the 25% gradient barrier at 3835 and stays accessible whole, while bcfp breaks at 3835 and blocks [3835,7998] (4163 m). Referenced by research/accessible_km_divergence.md and the tracking issue. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The accessible_km parity gap for BT/ST is a segmentation defect, not an access-decision defect. The per-species gradient/falls break source is built by running frs_barriers_minimal() (a downstream-most reduction) on each barrier set, which strips the interior break points; a link segment then straddles the accessibility frontier and the whole reach — including the blocked part above the barrier — is labelled accessible. At co-located positions link and bcfp agree on BT accessibility at 99.99%. Documents the code path, the blk 359209845 segment-level proof (4163 m over-credit on one blue line; aggregate FINA +1438 km / PCEA +2021 km), why it stayed hidden at 99%+ parity, and the fix plan: break at the full per-species barrier positions and reclassify like bcfp — no length clipping, no barrier moved. Also captures the accessible_km parity findings from the #221 investigation. Filed as #223. Relates to #221, #203. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Segmentation-frontier fix: gradient_barriers_minimal is fed the frs_barriers_minimal downstream-most reduction as a segmentation source, stripping interior breaks so a segment straddles the accessibility frontier and the whole reach is credited accessible. Phases: reproduce (tests-first), fix prepare.R, validate (no-regression), ship + rename follow-up. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Confirmed on local fwapg: working_fina.gradient_barriers_minimal empty on blk 359209845; fresh.streams id_segment 4218 = [3391,7998] 4607 m one segment straddling 3835; BT accessible_km FINA link 7520.7 / bcfp 6085.2 = +23.59%. Enriched #223 with the full code trace, the exact fix, and this fingerprint. Handoff note added for the next session to implement Phases 1-4. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Read-only DB validator that reproduces the BT/ST access-segmentation-frontier
over-credit. Parity sweep over {FINA,PARS,PCEA,LKEL} x species (link
fresh.streams JOIN fresh.streams_access on the full PK) vs the tunnel-free
reference fresh.streams_vw_bcfp, plus 4 structural checks on the canonical
evidence segment (blk 359209845, BT frontier 3834.78). Exits non-zero while the
bug is present; passes once the Phase 2 fix + re-run land.
- Validation set adds LKEL (smallest persisted WSG with steelhead + salmon) as
the no-regression sentinel: the 3 Peace WSGs carry no salmon/ST, so the plan's
item (d) salmon check would be vacuous there.
- One-sided-presence gate FAILs a link collapse-to-zero on a bcfp-present
species, allowlisting the one known #189 residence case (LKEL:sk).
- Confirmed pre-fix: 7 checks fail (FINA/PARS/PCEA BT parity +23.6/+3.4/+40.4%;
all 4 FINA structural, incl. the 4163 m above-frontier over-credit).
- Code-check (fresh-eyes agents): hardened S3 to isTRUE(all(...)) and closed the
one-sided-presence exit-code blind spot.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014kSEJoad4Bc553qU8eFiE6
…ate) Fix the BT/ST accessible_km over-credit. gradient_barriers_minimal (the stream break source) was fed the frs_barriers_minimal() downstream-most reduction as a SEGMENTATION source, so a single segment straddled the accessibility frontier and its whole reach — including the blocked part above the barrier — was credited accessible. lnk_pipeline_prepare.R: stop minimal-reducing the per-model barrier set; union the RAW per-model (gradient class-filtered + falls) positions into gradient_barriers_minimal so streams break at every frontier — mirroring the orphan path already used for sub-threshold classes, and matching bcfishpass (which segments at every gradient barrier). frs_barriers_minimal is now unused in link; the per-model _min tables are no longer built. Table name kept pending a rename issue; prepare.R + break.R roxygen updated. Validation (FINA/PARS/PCEA/LKEL re-run, mapping_code = TRUE): - BT accessible_km converges to bcfp: FINA +23.59% -> -0.02%, PARS +3.43% -> -0.01%, PCEA +40.36% -> -0.01%, LKEL exact. data-raw/accessible_km_fix_validate.R ALL PASS. - No downstream regression: BT spawning/rearing within ~1% on all 4 WSGs; LKEL ST/CO (salmon + steelhead sentinel) accessible exact, spawning/rearing hold; mapping_code improves. Habitat km is byte-identical pre/post fix — the fix is neutral on habitat. - Cost: segments 2-3.5x (FINA 26k->70k, PCEA 34k->106k); runs still 2-3 min each. Inherent to bcfp-matching; intersects perf issue #205. Also: validator S2 respects integer break rounding (breaks land at round(3834.78) = 3835, as bcfp does); the 4 .lnk_pipeline_prep_minimal unit tests updated to the new contract (raw barriers_<sp> in the union, no frs_barriers_minimal call). devtools::test() green (FAIL 1 = pre-existing environmental public.wsg_outlet, unrelated). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014kSEJoad4Bc553qU8eFiE6
…llup' into 223-access-segmentation-frontier * origin/221-per-wsg-habitat-access-km-rollup: research: root-cause the BT/ST accessible_km over-credit (#223) research: add blk 359209845 BT accessible_km segmentation proof figure #221 Phase 2: tunnel-free accessible_km reference (salmon group) #221 Phase 2: emit accessible_km as 8th habitat_type in the rollup #221 Phase 2: fold habitat km rollup into lnk_rollup_wsg #221 Phase 2: doc-clarify lnk_rollup_wsg vs lnk_aggregate grain #221 Phase 2 (1/4): add lnk_rollup_wsg reusable per-WSG roll-up #221 Phase 1: coho accessible_km parity proof (tunnel-free bcfp) Initialize PWF baseline for #221 # Conflicts: # planning/active/findings.md # planning/active/progress.md # planning/active/task_plan.md
…WSGs, 8 species) Add data-raw/parity_crosssection.R — a tunnel-free harness that extends the habitat parity rollup with the accessible_km column (#221): link via lnk_rollup_wsg vs bcfp fresh.streams_vw_bcfp, IN(1,2) predicate (bcfp codes access/spawning/rearing 0/1/2/3; a bare = 1 under-counts). Ran across 11 WSGs re-run with the #223 fix (FINA/PARS/PCEA/LKEL + BULK/MORR/KISP/ LFRA/USKE + ELKR/KOTR — Peace/Fraser/Skeena/Columbia, all 8 species): - accessible_km: 44/44 within 1%, max |diff| 0.05% — the fix converges exactly across every species and WSG (FINA/PCEA were +23.6% / +40.4% pre-fix). - spawning 42/43, rearing 34/35 within 5% — the ONLY two exceptions are BULK SK (spawn +11.0%, rear +35.2%) = the documented parked fresh#190 dual-rearing-lake topology (research/bcfishpass_methodology.md:124). Pre-existing, not a #223 regression. - #189 residence species (LKEL SK, BULK/MORR/USKE CM, USKE PK) correctly excluded. research/parity_accessible_habitat_2026_07_03.md records the proof. Combined branch test() green (FAIL 1 = pre-existing environmental wsg_outlet). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014kSEJoad4Bc553qU8eFiE6
Reduce data-raw script sprawl: fold accessible_km_fix_validate.R (#223 fix gate + structural checks) and accessible_km_proof_co.R (#221 coho-only accessible proof) into the single parity_crosssection.R, which already did the broader accessible + spawn + rear × 8-species × N-WSG sweep. parity_crosssection.R now also runs the #223 structural checks (FINA blk 359209845 frontier: breaks at frontier, above-frontier BT-blocked, accessible reach tops out) and exits non-zero on any accessible-over-tol, structural failure, or non-parked habitat divergence. Habitat parked cases (fresh#190 BULK SK) and #189 residence species (link-absent) are allowlisted. Full 11-WSG run PASSes (exit 0). Repoint lnk_rollup_wsg roxygen + the PWF Validation section to the single script. Net: -344 lines, 3 scripts -> 1. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014kSEJoad4Bc553qU8eFiE6
This was referenced Jul 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Combined PR closing #223 (access-segmentation-frontier fix) and #221
(per-WSG
accessible_kmroll-up). They are inseparable: #221's accessible columnsurfaced the #223 over-credit, and #223's fix makes #221's accessible parity pass.
#223 — the fix
gradient_barriers_minimal(the stream break source) was fed thefresh::frs_barriers_minimal()downstream-most reduction as a segmentationsource, so a single stream segment straddled the accessibility frontier and its whole
reach — including the blocked part above the barrier — was credited accessible. BT/ST
accessible_kmover-credited +40.4% (PCEA), +23.6% (FINA).One-file fix (
R/lnk_pipeline_prepare.R): union the raw per-model gradient + fallspositions into
gradient_barriers_minimalso streams break at every frontier —mirroring the orphan path already in the code, and matching bcfishpass (which segments
at every gradient barrier).
frs_barriers_minimalis now unused in link.#221 — the roll-up
Adds the
accessible_kmcolumn tolnk_compare_rollup+ a reusablelnk_rollup_wsg()(tunnel-free vs
fresh.streams_vw_bcfp).Proof
data-raw/parity_crosssection.R+research/parity_accessible_habitat_2026_07_03.md— across 11 WSGs × 8 species (Peace / Fraser / Skeena / Columbia):
FINA/PCEA went from +23.6% / +40.4% pre-fix to −0.02% / −0.01%. The only 2 habitat
exceptions are BULK SK — the documented parked fresh#190 dual-rearing-lake topology
(
research/bcfishpass_methodology.md:124), not a regression. Suite green (FAIL 1 | PASS 1298; the 1 FAIL is the environmentalpublic.wsg_outletmissing table, #227).Cost / scope
Segment count grows 2–3.5× (FINA 26k→70k, PCEA 34k→106k) — inherent to bcfp-matching
(streams now break at every gradient barrier, as bcfp does); intersects perf issue #205.
Re-runs stay 2–3 min/WSG.
Follow-ups (their own issues)
dam_dnstr_indreservoir-inflow propagation (bcfp reference quirk; out of scope).gradient_barriers_minimal→gradient_barriers_break(no longer minimal-reduced).accessible_kmbcfp-equivalence.public.wsg_outletbuilder + single-WSG downstream-state guard.Closes #221. Closes #223. Relates to NewGraphEnvironment/sred#24.
🤖 Generated with Claude Code
https://claude.ai/code/session_014kSEJoad4Bc553qU8eFiE6