Repository navigation
Conversation
Bumps [julia-actions/cache](https://github.com/julia-actions/cache) from 2 to 3. - [Release notes](https://github.com/julia-actions/cache/releases) - [Commits](julia-actions/cache@v2...v3) --- updated-dependencies: - dependency-name: julia-actions/cache dependency-version: '3' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [actions/checkout](https://github.com/actions/checkout) from 6 to 7. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v6...v7) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
…checkout-7 Bump actions/checkout from 6 to 7
…tions/cache-3 Bump julia-actions/cache from 2 to 3
Merge dev back into master
Add Zenodo DOI to citation metadata
CatalogEntry describes what can be fetched rather than what was measured, so it is a separate schema from EpiObservation. Its extent columns are nullable because fluview publishes no per-signal coverage. catalog.jl is a top-level file rather than part of src/sources/, since everything under sources/ standardizes into an EpiTable and this does not.
covidcast_meta answers an unrecognized geo_types or time_types with a 200 and an empty body, which is byte-identical to a valid query that matched nothing. So both are checked locally before a request is spent. geo_type is validated but not sent at all: GEO_TYPE_MAP treats hhs_region as an alias for hhs, and passing the alias upstream would return nothing.
min_time and max_time are packed in the row's own time_type, so they run through period_bounds. The opening bound comes from min_time and the closing one from max_time, which for a weekly row are six days apart. An unknown geo_type drops the row with a warning instead of throwing. hsa_nci reached this endpoint before it reached GEO_TYPE_MAP, and catalog is the function you would call to find that out. coerce_lag is renamed coerce_int, since num_locations reads through it too. The shared decode_response gets validate = false on its CSV.File call. CSV.jl (v0.10.16) validates the types dict's keys against the payload's real columns by default and throws when one is missing, and covidcast_meta has no geo_value column, so validate = false lets DelphiCovidcastMeta share the decoder unchanged instead of duplicating it.
fluview_meta reports one row for the whole table, not per-signal coverage, so these entries carry the signal, resolution, and level and leave every extent column missing. Omitting them entirely would read as though the package cannot fetch ILI.
Signal names had to be looked up in Delphi's documentation. catalog() reads them from covidcast_meta and appends the fluview signals, so discovery happens in the package. An empty covidcast reply is not an error on its own: a census_division filter matches nothing upstream but does match fluview, so emptiness is decided after both halves are in. source="fluview" skips the request entirely, since covidcast_meta has no fluview data_source to return.
catalog(source = "fluview") never talks to covidcast_meta, since that endpoint has no fluview data_source. An empty result on that path still threw NoResultsError naming a covidcast_meta query string, even though zero requests were made.
The Scope section promised covidcast_meta discovery as future work. It is here now, so it says what is actually still missing instead: no cache, and no pre-flight signal validation in fetch_covidcast. check_upstream gains a header check for covidcast_meta and an end-to-end catalog call, which puts the run at 10 requests. The Makefile said 5, which was already wrong before this change.
Task 7 fixed this same stale count twice in the Makefile but missed this third copy. check_upstream.jl now runs 10 requests, not 8.
Every output comment in the Discovering datasets block was written from the test fixture, not a live call, so none of the three matched what running the code actually prints. Reworked to length(cat), a filtered covid[1] that lands on a recognizable nssp signal instead of the true alphabetically-first beta_ source, and a print()-based source check — each value pasted from a real catalog() call just now.
- Note in catalog's docstring and the README that fluview rows are the signal list, not verified coverage (num_age_2 is a known empty case). - Stop pinning covidcast_meta values that move on every fixture refresh: keep the retired chng/day row exact, replace the live nssp/week and county-count assertions with invariants that still catch a real bug. Flag the drift risk in RefreshFixtures.yml's PR checklist. - Collapse empty_catalog to CatalogTable(CatalogEntry[]) now that the claim that it loses column types has been checked and is false. - Normalise a "*" or :* signal to nothing in catalog() so it means "no filter" instead of matching zero rows. - Narrow save's first argument to ::EpiTable so it rejects a CatalogTable at the call site instead of writing an Arrow file load() then refuses. - Add the fetch_covidcast hand-off the README's catalog section was missing, with real output from a live run.
Add catalog() to list the available datasets
The catalog work landed with 61 comment lines in a 306-line test file and 27 in src/catalog.jl. That is 16% and 10% against 7% and 5-7% in the files it sits beside. Each implementer was told to match the surrounding style, and this repo is comment-dense, so the density compounded task by task instead of being checked once. Removed: comments restating the assertion below them, comments justifying why a test exists, and prose already carried by the docstring of the function under test. Removed four section banners that only named the next function; kept the one marking the fluview endpoint boundary, which is how src/sources/delphi.jl uses them. Kept every comment carrying a fact the code cannot state: the measured wire behaviour at the top of catalog.jl, why the chng fixture rows can be pinned to exact dates while the nssp ones cannot, the packed-encoding decodes beside the dates they produce, and the row arithmetic behind the expected counts. Also straightens two README continuation lines that sat one space too deep. src/catalog.jl 27 -> 14 comment lines, test/test_catalog.jl 61 -> 7. No behaviour change; 1669 tests still pass.
Trim comments that restate code or justify decisions
Adds catalog(), and narrows save() to accept only an EpiTable. Julia treats the minor position as the breaking one below 1.0, so the save() change alone sets this at 0.2.0. The Zenodo pinning note no longer names a specific release. It quoted v0.1.0's version DOI as its example, which meant editing that line on every release for no gain, since the Zenodo record already lists one DOI per release.
Bump to v0.2.0
|
Comments on pull requests will not trigger Registrator, as it is disabled. Please try commenting on a commit or issue. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings
masterup todevfor the v0.2.0 release. 24 commits, no divergence:masteris 0 commits ahead ofdev.What lands
The catalog feature (13 commits).
catalog()lists every dataset the package can fetch, so signal names come from the package rather than Delphi's website. About 3,000 entries across 27 sources.One breaking change.
savenow takes anEpiTablerather than any Tables.jl source. It used to accept aCatalogTableand write a valid Arrow file thatloadthen refused to read back.The version bump to 0.2.0, in
Project.toml,CITATION.cff, and the README's BibTeX block.Work that was already on
devbut never reachedmaster(8 commits): the Zenodo DOI badge and metadata, and two dependabot action bumps.Verification
make check-upstreamholds across 7 contract checks and 3 end-to-end calls, with no drift.preon ubuntu.pkgversion(EpiDataKit)reports0.2.0, andprovenancestampsversion = "0.2.0"into saved Arrow files.Note that
CI.ymldoes not trigger ondevpushes, only onmasterpushes and pull requests. So this PR's run, and the push-triggered run after it merges, are the first CImastersees for any of this.After this merges
The package is in Julia's General registry, and only 0.1.0 is registered there. To publish 0.2.0, comment
@JuliaRegistrator registeron the merge commit. Registrator opens a registry PR, and when that merges TagBot creates thev0.2.0tag and the GitHub Release, which is what Zenodo watches.