Add skills for data work - #131
Conversation
vahid-ahmadi
left a comment
There was a problem hiding this comment.
Both skills are well aimed — "where does this variable's value come from" and "how does it flow through the pipeline" are the two questions that actually cost people time, and splitting them so the tracing skill owns the formula side while the pipeline skill owns the stage side is the right seam. The ## CRITICAL: Verify Against Code framing up front, and describing the pipeline skill as "a starting point for investigation, not a reference manual", set the right expectation for docs that will drift.
The conceptual section earning its place before the navigation section is a good call too — "microdata values and weights are separate concerns" is precisely the confusion that sends people to the wrong stage.
1. Duplicate section number in the pipeline skill.
policyengine-us-data-pipeline-skill/SKILL.md has two ## 2. headings:
## 2. Pipeline Shape
## 2. Run ID and HuggingFace Artifact Linkage
## 3. Investigating a Variable — Navigation by Question
so the run-ID section is unreferenceable and everything after it is off by one relative to the reader's mental count. Renumber 2→3 onward, or drop the numbers entirely — the headings are descriptive enough to stand without them, and unnumbered sections don't rot when someone inserts one.
2. Worth confirming the two skills' cross-references point at the right names.
The pipeline skill has a ### Use the variable-tracing skill for the formula side section and a ## Related Skills block, which is exactly the right instinct given they're landing together. Just worth a check that the names used there match the frontmatter name: values (policyengine-us-data-pipeline, and whatever the tracing skill declares) rather than the directory names — the directories carry a -skill suffix that the frontmatter doesn't.
3. Two skills, one PR.
Both are additive and both have changelog entries, so this is fine — but they're independently reviewable, and the tracing skill (470 lines + a 189-line example) is the one most likely to attract wording feedback. If you'd rather not have the pipeline skill held up by that, splitting is easy. Your call.
4. Minor
- Both are registered in the two plugin lists in
marketplace.jsonconsistently — good, that's the easy thing to half-do. - The tracing skill ships
examples/snap-trace.md; the pipeline skill has no example. A single worked trace ("here'ssnapfrom raw CPS to calibrated weight") would make the cross-stage section much more concrete, and you already have most of it in the SNAP example next door. ### Verifying a formula against legal sourcesis a strong section to have included. Given it's the one step with a real correctness cost when skipped, it might deserve promotion nearer the top rather than sitting at the end of section 5.
Summary
policyengine-variable-tracing skill — Guides tracing variable definitions, formulas, and data lineage across country model packages (policyengine-us, policyengine-uk). Helps map H5 dataset values back to source code, navigate dependency trees, and understand cross-entity aggregation. Includes a worked example tracing
snapend-to-end.policyengine-us-data-pipeline skill — Guides investigation of how variables flow through the policyengine-us-data pipeline: from raw CPS/PUF survey data through QRF imputation, calibration targets (
target_config.yaml,policy_data.db), and L0 weight optimization. Covers locating pipeline diagnostics on HuggingFace and debugging aggregate mismatches.Both skills are designed as investigative starting points — they point you to the right files and functions, then emphasize reading the actual code to confirm.
Test plan
claude --plugin-dirand verify both skills trigger on relevant prompts🤖 Generated with Claude Code