Adding HIT l1b Science CDF Attributes by daralynnrhode · Pull Request #3484 · IMAP-Science-Operations-Center/imap_processing

daralynnrhode · 2026-09-21T23:25:19Z

Added the L1b Science Attributes for HIT

Overview

Adds complete, ISTP-compliant CDF variable attributes for HIT L1B science data products (standard-rates, summed-rates, sectored-rates). Previously only housekeeping attributes existed in imap_hit_l1b_variable_attrs.yaml; science variables had no CDF metadata at all.

All attribute values are sourced directly from the HIT CDF Product Definitions spreadsheet .

The spreadsheet has no row for he_energy_delta_plus/he_energy_delta_minus (every other summed-rates particle has one). Flagged with a comment in the YAML; Will need to be confirmed/corrected.

closes #3070

File changes

imap_processing/cdf/config/imap_hit_l1b_variable_attrs.yaml — full science attribute section added/rebuilt from the spreadsheet, replacing the placeholder/HK-only content. Includes dynamic_threshold_state, all standard-rates rate/uncertainty variables, and all summed/sectored-rates particle rate/uncertainty/energy-coordinate variables.
imap_processing/hit/l1b/hit_l1b.py — wires the attribute dictionary into .attrs for every L1B science dataset (process_science_data()), for all data vars and for epoch.
imap_processing/hit/hit_utils.py — add_energy_variables() and add_summed_particle_data_to_dataset() now accept an optional attr_mgr so the per-particle energy coordinates created fresh during summed-rates processing (which the shared attrs loop can't reach, since it only iterates data_vars, not coords) get real CDF attributes instead of none. Kept optional so L1A's existing call site (which sets attrs separately, later) isn't broken.

Testing

NA

daralynnrhode · 2026-09-22T18:25:50Z

@lacoak21 Its this link https://lasp.colorado.edu/galaxy/spaces/IMAP/pages/207619340/HIT+CDF+Product+Designs

lacoak21

Excellent jon!! I had a few minor comments about attribute structure. Wow there were a lot more than I thought 😆

@bryan-harter since these are not l2 attrs and dont need to be fully ISTP compliant can Daralynn skip sending them to Andriy for review or should she do that anyway?

bryan-harter · 2026-09-23T21:02:57Z

Excellent jon!! I had a few minor comments about attribute structure. Wow there were a lot more than I thought 😆

@bryan-harter since these are not l2 attrs and dont need to be fully ISTP compliant can Daralynn skip sending them to Andriy for review or should she do that anyway?

Yup, we don't need to send them to Andriy if they are below L2

@lacoak21

Per @lacoak21's review: - Add a `double_fillval` anchor for the repeated -1e+31 FILLVAL instead of writing the literal on every variable. - Add a `default_rate_attrs` anchor (FILLVAL, FORMAT, SCALEMAX, SCALEMIN, SCALETYP, UNITS, VALIDMAX, VALIDMIN, VAR_TYPE) for the Standard Rates block, merged into each of the 12 rate variables and their 24 plus/minus uncertainty variables via `<<:`. - Give each standard-rate variable's LABLAXIS the rate name (e.g. "sngrates Count Rate" instead of the generic, repeated "Count Rate"), per Andriy's preference for these to be unique. - Add `default_energy_attrs`/`default_energy_delta_attrs` anchors for the same kind of repetition across the 17 *_energy_mean coordinates and 18 *_energy_delta_plus/minus variables. This also fixes he_energy_mean's SCALETYP, which was blank in the spreadsheet (a gap, like the missing he_energy_delta rows already called out in a NOTE comment) and so was previously emitted with no SCALETYP key at all -- every other energy variable uses "log" and there's no reason "he" should differ, so it's now filled in correctly via the shared default. No variable's resolved attribute values changed other than the two fixes above (unique LABLAXIS, he_energy_mean's SCALETYP) -- verified by loading old and new YAML with PyYAML and diffing every variable's fully-resolved (anchors expanded) attribute dict. Verified: `pytest imap_processing/tests/cdf/ imap_processing/tests/hit/` (86 passed). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EYCHuh2ycXurU83FKN6pLf

Copilot

🟡 Changes recommended

Several attributes have incorrect dependencies, data types, descriptions, or product semantics, and the new metadata path lacks validation coverage.

11 open findings

Remove incompatible int8 pseudo-attribute · New
Add product-specific metadata for sectored attributes · New
Fix he dependency on he_energy_mean · New
Fix uncertainty dependency on he_energy_mean · New
Fix uncertainty dependency on he_energy_mean · New
Correct energy coordinate mean descriptions or calculations · New
Add DELTA pointers to he_energy_mean · New
Correct mislabeled minus uncertainty · New
Add metadata and strict CDF validation coverage · New
Correct misspelled published description · New
Mark optional parameter type entry · New

What changed in this PR

Adds CDF metadata to HIT L1B science products and applies it during dataset construction.

Changes:

Defines standard, summed, and sectored rate attributes.
Applies attributes to science variables and summed-rate energy coordinates.
Extends HIT helpers to accept an optional attribute manager.

File	Description
`imap_processing/hit/l1b/hit_l1b.py`	Integrates attributes into L1B processing.
`imap_processing/hit/hit_utils.py`	Adds attribute support for generated energy coordinates.
`imap_processing/cdf/config/imap_hit_l1b_variable_attrs.yaml`	Defines HIT L1B science metadata.

🧠 Review effort: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+# <=== Summed & Sectored Rates - Particle Variable Attributes
+# Rate/uncertainty variables are shared by both products under the same
+# bare name; see the comment above emit_particle_sheet() in the
+# generator for why (and note DEPEND_2/DEPEND_3 are intentionally
+# omitted for those shared variables). ===>


…entry' Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

daralynnrhode · 2026-10-07T21:11:46Z

Okay, after fixing the suggestions there are 3 things that still need to be addressed:

The sectored variables are using the summed metadata. Two L1B products both contain variables named h, he4 and fe. They are the summed-rates product and the sectored-rates product. The YAML file has only one entry per variable name, and the code looks attributes up by name alone. So the sectored versions get the entry that was written for the summed versions. Dimensions, dependencies, and are bins differ between the two.

The new test test_hit_l1b_depend_attrs_match_dims is marked as an expected failure for the sectored product for this reason. I think L2 fixes something similar with the macropixel uncertainties. But the change would effect l1b.

The energy "mean" is described as geometric but computed as arithmetic in l1a and I followed the same pattern in l1b. The metadata calls that value the geometric mean while the code computes the plain midpoint, (min + max) / 2. Changes this would effect all 3 data levels.
the axis labels are only unique for the standard rates. @lacoak21 let me know how far the uniqueness should extend.

lacoak21 · 2026-10-08T19:45:16Z

Okay, after fixing the suggestions there are 3 things that still need to be addressed:

OK thanks for this summary

I think we should updated the attrs to have
h_summed
he4_summed
fe_summed
as well as
h_sectored
he4_sectored
fe_sectored

then in the code, when grabbing the attrs we can tack on summed or sectored. Is that easy to do?

Huh that is concerning. This may have to be a question for the HIT team. I will check the HIT algorithm document. We may have to email Grant and Eric from the HIT team.

3.Ah good catch. Each variable needs to have a unique axis label.

lacoak21 · 2026-10-08T20:38:17Z

OK yeah from what I could find from the latest hit alg document it says geometric mean.

https://lasp.colorado.edu/galaxy/spaces/IMAP/pages/146768943/IMAP+Algorithm+Document+from+Instrument+Teams

lacoak21

After you fix number 1 and 2 in the last comment its good to go!

lacoak21 · 2026-10-12T19:08:12Z

energy_mean issue #3562

daralynnrhode added 2 commits September 21, 2026 17:08

adding hit l1b science cdf attr

482774a

adding he energy delta +- for istp compliance

f67072a

daralynnrhode added this to the September 2026 milestone Sep 21, 2026

daralynnrhode self-assigned this Sep 21, 2026

daralynnrhode added the Student Tasks suitable for student work label Sep 21, 2026

daralynnrhode added this to IMAP Sep 21, 2026

alphabatizing the attrs

f75f1f6

lacoak21 reviewed Sep 22, 2026

View reviewed changes

Comment thread imap_processing/cdf/config/imap_hit_l1b_variable_attrs.yaml Outdated

lacoak21 reviewed Sep 22, 2026

View reviewed changes

daralynnrhode requested a review from lacoak21 September 30, 2026 21:03

daralynnrhode changed the title ~~[WIP] Adding HIT l1b Science CDF Attributes~~ Adding HIT l1b Science CDF Attributes Sep 30, 2026

adding galaxy page link

f499745

lacoak21 requested a balanced review from Copilot October 7, 2026 19:13

Copilot started reviewing on behalf of lacoak21 October 7, 2026 19:14

View session

Copilot AI reviewed Oct 7, 2026

View reviewed changes

daralynnrhode and others added 3 commits October 7, 2026 14:48

Potential fix for pull request finding 'Mark optional parameter type …

f8d371b

…entry' Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Merge branch 'IMAP-Science-Operations-Center:dev' into hit_attributes

7fc0d25

addressing Luisa and Copilots comments

0d93c1b

lacoak21 approved these changes Oct 12, 2026

View reviewed changes

daralynnrhode and others added 3 commits October 13, 2026 11:18

Merge branch 'IMAP-Science-Operations-Center:dev' into hit_attributes

e5f9f68

Use product-specific attrs for summed and sectored L1B variables

28c6ddd

amking every l1b vairable have its own axis label plus adding a test

958a212

daralynnrhode merged commit 65c64f5 into IMAP-Science-Operations-Center:dev Oct 14, 2026
22 checks passed

daralynnrhode deleted the hit_attributes branch October 14, 2026 19:53

Conversation

daralynnrhode commented Sep 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Added the L1b Science Attributes for HIT

Overview

File changes

Testing

Uh oh!

Uh oh!

daralynnrhode commented Sep 22, 2026

Uh oh!

lacoak21 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bryan-harter commented Sep 23, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

daralynnrhode commented Oct 7, 2026

Uh oh!

lacoak21 commented Oct 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lacoak21 commented Oct 8, 2026

Uh oh!

lacoak21 left a comment

Choose a reason for hiding this comment

Uh oh!

lacoak21 commented Oct 12, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

daralynnrhode commented Sep 21, 2026 •

edited

Loading

lacoak21 commented Oct 8, 2026 •

edited

Loading