Skip to content

feat(ui): Bulk Export workspace — scoped kick-off, filters, active exports - #538

Open
angela-helios wants to merge 9 commits into
mainfrom
feat/537-bulk-export-ui
Open

feat(ui): Bulk Export workspace — scoped kick-off, filters, active exports#538
angela-helios wants to merge 9 commits into
mainfrom
feat/537-bulk-export-ui

Conversation

@angela-helios

Copy link
Copy Markdown
Contributor

Closes #537

The pull-based companion to the Bulk Import workspace: a web UI over the server's own, already-shipped $export operation, replacing the "Export" coming-soon placeholder in the Batch & Data nav. First functional round of the new Export design.

/ui/bulk-export

  • Scope ("What are you exporting?"): Everything / Patients / Group (with a Group id field), matching the design's copy.
  • Resource types: checkbox grid sourced from the tenant's live resource-type list → _type. Nothing checked = every type.
  • Narrow it down: _elements, _typeFilter, and _since as presets (last day / 7 days / 4 weeks / all time) or a custom RFC 3339 instant.
  • Optional job name ("Diabetes registry 2024") and an Active-exports shortcut with the running count.

Self-calls that work under auth

Kick-offs, status polls, and cancels target this same server via the request's Host header, forwarding the caller's own Authorization and tenant — the export runs with the user's credentials, no service token needed (unlike the conformance self-fetch, which still needs HFS_OUTBOUND_BEARER_TOKEN). Relative Content-Location poll URLs resolve against the same base.

/ui/bulk-export/active

One card per job, newest first, with status chips (in progress / complete / failed / cancelled):

  • In progress: the poll's X-Progress verbatim + Cancel (DELETEs the job server-side). Cards htmx-refresh every 5s; each fetch performs exactly one server-side poll, and the completed card simply lacks the trigger — the same no-JS polling pattern as the import workspace.
  • Complete: file count, elapsed time, and the completion manifest's output files as download links.
  • Failed: the kick-off/poll error verbatim + Retry (same parameters, fresh kick-off).

Job state persists in the per-user settings document under the reserved byTenant subtree, like the import workspace.

Tests

The suite mounts the UI over a mock FHIR export backend on a real socket, so the Host-header self-calls loop back into the mock: kick-off query shape asserted at the API (_type/_elements/_since), Patient and Group paths, the 202-progress → 200-manifest lifecycle (polling starts and stops where it should), rejected kick-off → failed card → retry re-runs it, and cancel arriving as a DELETE. Full helios-ui suite green (incl. i18n parity ×3), clippy clean, --features ui builds.

Note: one implementation detail worth a look — the export form is parsed by hand via form_urlencoded because the repeated types checkboxes exceed what serde_urlencoded accepts.

…ports

The pull-based companion to the Bulk Import workspace (#537), driving
the server's own $export operation from the web UI and replacing the
Export placeholder in the Batch & Data nav.

- /ui/bulk-export: scope choice (everything / patients / group with a
  Group id), a resource-type checkbox grid sourced from the tenant's
  live resource types (becomes _type), narrowing filters (_elements,
  _typeFilter, _since as day/week/4-weeks presets or a custom RFC 3339
  instant), and an optional job name.
- Kick-offs and status polls are self-calls addressed via the request's
  Host header, with the caller's own Authorization and tenant forwarded
  -- the export runs with the user's credentials, so the workspace works
  under auth without a service account. Relative Content-Location poll
  URLs resolve against the same base.
- /ui/bulk-export/active: one card per job (newest first) with status
  chips; in-progress cards htmx-refresh every 5s, each fetch performing
  exactly one server-side poll (202 records X-Progress, 200 stores the
  completion manifest's output files and stops the polling trigger).
  Complete cards list the files as download links with elapsed time;
  failed cards show the error verbatim with Retry (same parameters,
  fresh kick-off); Cancel DELETEs the job server-side.
- Job state lives in the per-user settings document under the reserved
  byTenant subtree, like the import workspace.
- The export form is parsed by hand (form_urlencoded): the repeated
  checkbox fields are more than serde_urlencoded accepts.

Tests mount the UI over a mock FHIR export backend on a real socket so
the Host-header self-calls loop back into the mock: kick-off query
shape (_type/_elements/_since), patient and group paths, the
202-progress -> 200-manifest polling lifecycle, rejected kick-off ->
failed card -> retry, and cancel reaching the API as a DELETE.
@angela-helios

Copy link
Copy Markdown
Contributor Author

Screenshots of the workspace working end-to-end against a live server built from this branch (sqlite, 73 seeded resources) — real $export kick-offs, not mocks.

Export builder — named job, three resource types checked, Since = last 7 days:

form

Active Exports, happy path — the job completed in 6s; the card stopped polling on its own and lists the completion manifest's file as a download link (only Patient had data among the checked types):

complete

Failure path — a Group-scope export against a group that doesn't exist: the card lands as Failed with the server's OperationOutcome verbatim and a Retry that re-runs the same parameters:

failed

Side observation while producing these, not this PR's concern: GET /Group/{missing}/$export answers 500 with a generic internal-error OperationOutcome rather than a 404 — worth a small consumer-side issue.

…lint

CI's newer toolchain flags the must_use attribute async-trait 0.1.89
emits in its expansion (clippy::double_must_use, promoted by
-D warnings) on every #[async_trait] trait, first surfacing in
helios-observability's DashboardProvider. 0.1.92's expansion is clean.
No source changes; the lock adds syn v3 alongside v2 as its dependency.
Same adjustment Import needed in #528: the Bulk Export workspace made
the sidebar's Export item a real link, so the chrome spec expects a
navLink instead of a soon-item. SQL-on-FHIR remains the section's only
placeholder.
The Export and Import page headers stacked their action buttons under
the title; page-head--row (the existing flex variant the resources
workspace uses) puts them top-right, matching the design. The giant
gap reported under the Export header turned out to be a Playwright
full-page stitching artifact with the fixed nav rail, not a layout
bug -- the screenshots are retaken at a tall viewport instead.
@angela-helios

Copy link
Copy Markdown
Contributor Author

Follow-up on a layout question from review: the large empty band under the header in the earlier screenshots was a Playwright full-page-capture stitching artifact (the fixed nav rail confuses the scroll-and-stitch), not real layout — live DOM shows the standard 20px. What was real: the header action buttons stacked under the title. Fixed by using the existing page-head--row flex variant on the four bulk workspace pages, matching the design(actions top-right). Screenshots above re-captured artifact-free at a tall viewport.

@codecov

codecov Bot commented Aug 13, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.42820% with 29 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
crates/ui/src/bulk_export.rs 92.20% 29 Missing ⚠️

📢 Thoughts on this report? Let us know!

@smunini

smunini commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Resource-type picker isn't actually FHIR-version aware

Reviewing where the export form's resource-type checkboxes come from, I hit a discrepancy worth deciding on. Note this is pre-existing behavior this PR inherited, not something introduced here — but the Bulk Export page is a new consumer of it, so it seems like the right moment to flag it.

The chain today

crates/ui/src/bulk_export.rs:265

let resource_types = state
    .compartments
    .resource_type_names(&rt.id, helios_fhir::FhirVersion::default())
    .await;

CompartmentCatalog::resource_type_names (crates/ui/src/compartments.rs:131) takes the first cached CompartmentDefinition (sorted by code, so Device) and maps resource[].code. Each definition enumerates the full type set for its version, so any one works as a registry.
CompartmentCatalog::definitions fetches GET /CompartmentDefinition from the server's own API over loopback, cached per (tenant, version).
→ Storage is seeded at startup from the version-specific bundles data/compartment-definitions-{r4,r4b,r5,r6}.json (seed_spec_compartment_definitions, crates/persistence/src/search/seeder.rs:274).

The discrepancy

The underlying data is version-specific, but two things flatten it before it reaches the page:

  1. The page passes FhirVersion::default() (R4) rather than the request version rv.0 — matching the three existing callers in lib.rs:973,995,1020 (Search, Queries, Resources). So the UI's version selector doesn't change the type list.
  2. HttpConformanceSource::fetch ignores its version argument entirely (_version, crates/ui/src/conformance.rs:63) and just issues GET {base}/CompartmentDefinition. The FhirVersion argument therefore functions only as a cache key.

Net effect at runtime: the list reflects whatever the server seeded — config.default_fhir_version (crates/hfs/src/main.rs:1012, i.e. HFS_FHIR_VERSION) — regardless of what the user picks in the version selector.

The bundles do differ meaningfully:

Bundle Resource types Compartments
R4 145 5
R4B 140 5
R5 157 5
R6 126 6 (adds Group)

Why it isn't visible in tests

StaticConformanceSource::fetch is keyed by (resource_type, version) (crates/ui/src/conformance.rs:203), so the test double honors the version the production HTTP source discards. A test can't currently surface the mismatch.

Options

  • Leave as-is — correct for a single-version deployment, which is the common case. If so, worth a comment at resource_type_names recording that the version argument is a cache key only, so the next reader doesn't assume it filters.
  • Make it real — pass rv.0 at the call sites and have HttpConformanceSource::fetch propagate the version to the self-call. That's a cross-cutting change touching all four callers, so probably its own issue rather than scope creep here.

No objection to merging on this — just want the behavior to be a decision rather than an accident.

…keys test

The bulk-submit JWKS work brought in the pure-Rust rsa crate for RS384
public-JWK derivation, tripping the Security Audit on every PR:
RUSTSEC-2023-0071 (Marvin Attack timing sidechannel) has no fixed
release, and jwe.rs already rejects RSA-OAEP for exactly this reason.
The JWKS handler's own doc promises an empty key set for non-ES384
algorithms -- the code now honors it. Both copies of the derivation
(rest's derive_public_jwk, ui's signing_kid) return None for RS384
with the rationale in place; an RS384 deployment registers its key
out-of-band, the assertion goes out without a kid, and the rsa crate
leaves the lockfile.

Also updates the UI keys test the redirect change left behind: the
route permanently redirects to /.well-known/bulk-submit-jwks.json now,
and the old 404/200 assertions were failing Test Rust on every PR's
merge ref.
… only

Steve's review of the export page traced the chain: the page passes
FhirVersion::default() like the three existing callers, and the
production HttpConformanceSource discards its version argument, so the
list always reflects the server's seeded version (HFS_FHIR_VERSION)
regardless of the sidebar selector. Correct on a single-version
deployment; now documented at resource_type_names and the new call
site so the behavior is a decision rather than an accident. Making the
version real end-to-end across all four callers is filed separately.
@angela-helios

Copy link
Copy Markdown
Contributor Author

Good catch, and agreed on the framing — making it a decision. Taken as option 1 now, option 2 filed:

  • The cache-key-only semantics are now documented at resource_type_names and at this page's call site (8126eb1e9), so the next reader doesn't assume the argument filters.
  • The end-to-end fix (propagate the version through HttpConformanceSource::fetch, switch all four callers to rv.0, and a test that flips the version — the static double already honors it, production needs to catch up) is ui: make the conformance-backed resource-type lists FHIR-version aware #562. Cross-cutting enough that it deserves its own change, as you said.

Also resolved the conflicts: merged main (which brought #539 in) and carried the #560 fix onto this branch so the merge ref is green regardless of merge order — rsa is out of this branch's lock and the keys test asserts the redirect contract.

angela-helios and others added 2 commits August 18, 2026 01:23
…ns idempotent

Both fallouts of the root add-picker auto-opening on an empty document
(#547):

- axe can now see the value[x] choice selects that used to hide inside
  the collapsed details, and they never had an accessible name — each
  carries an aria-label built from the declarer and the pick-type
  message.
- The editor-controls specs clicked the panel summary to open it, which
  now toggles an already-open picker closed and strands the fill behind
  an invisible filter. The page object gains openAddPanel() (click only
  when closed) and the value[x] spec checks the open attribute before
  toggling.

Verified locally: the affected editor-controls and a11y editor specs
pass against the rebuilt binary (9/9).
# Conflicts:
#	crates/ui/assets/app.css
#	locales/de/main.ftl
#	locales/en/main.ftl
#	locales/es/main.ftl
@angela-helios

Copy link
Copy Markdown
Contributor Author

CI status: everything is green here except Test FHIRPath, and that one is infrastructure, not code — it has landed on github-agent4 three re-runs in a row, and that runner's disk is full enough that the job dies before checkout (There is not enough space on the disk while downloading actions/checkout, three attempts). Because it fails in seconds, the agent frees up first and re-captures each re-queued job, so further re-runs don't help.

The #564 target-GC can't rescue it either — it runs inside the job, after the checkout that never happens. It needs a manual prune of the agent's workspace (or pausing the agent so the job schedules elsewhere). Happy to re-trigger the job as soon as the disk has room; the rest of the run is already green (Linting, Test Rust, Test Python, Security Audit, Code Coverage).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(ui): Bulk Export workspace — scoped kick-off, filters, active exports with progress

2 participants