Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# AI assistant guidance

- When investigating a discrepant Microcosm calibration target, follow [the shared investigation workflow](docs/ai/workflows/investigate-microcosm-target.md) and its linked role-specific reviews.
- Before starting the application for Cross-dataset work, configure exactly one artifact location as described in [the Cross-dataset application configuration](docs/cross-dataset-api.md#configure-the-application).
- Before starting the application for Cross-dataset work, configure exactly one DIR or URL artifact location for each country you will use, as described in [the Cross-dataset application configuration](docs/cross-dataset-api.md#configure-the-application).
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Claude Code guidance

- When investigating a discrepant Microcosm calibration target, follow [the shared investigation workflow](docs/ai/workflows/investigate-microcosm-target.md) and its linked role-specific reviews.
- Before starting the application for Cross-dataset work, configure exactly one artifact location as described in [the Cross-dataset application configuration](docs/cross-dataset-api.md#configure-the-application).
- Before starting the application for Cross-dataset work, configure exactly one DIR or URL artifact location for each country you will use, as described in [the Cross-dataset application configuration](docs/cross-dataset-api.md#configure-the-application).
179 changes: 179 additions & 0 deletions LANE_E4_REPORT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,179 @@
# Lane E4 report

## Outcome

Cross-dataset artifacts are now selected independently for US, UK, and Belgium.
The unsuffixed configuration remains the US contract, UK and Belgium use
country-suffixed variables, and each country has an independent cached reader.
Every configured reader validates that its manifest contains the selected
jurisdiction before any view is served.

`country` now reaches every Cross-dataset API and client query. Cross-dataset is
visible in every selectable country's navigation, while an unconfigured country
fails closed into the page's existing unavailable state with a message naming
the country and applicable environment variables. The overview and fact views
use bundle-provided source labels and group metadata instead of US-specific
presentation assumptions. Country switches also discard bundle-specific fact,
source, and filter URL state rather than applying it to the next country.

## File-by-file changes

- `AGENTS.md`: clarifies that each country used by Cross-dataset work must have
exactly one configured directory or URL.
- `CLAUDE.md`: mirrors the per-country application-start configuration rule.
- `docs/cross-dataset-api.md`: documents all country-specific variables, the
per-country exclusivity rule, the live Belgium URL, jurisdiction validation,
and the `country=us|uk|be` API parameter.
- `frontend/app/api/microcosm/cross-dataset/route.ts`: parses `country` with the
shared country parser and selects that country's reader before dispatching any
Cross-dataset view.
- `frontend/app/api/microcosm/cross-dataset/route.test.ts`: proves a BE-only
configuration serves Belgium and leaves default/explicit US requests in the
existing 503-style unconfigured state.
- `frontend/components/layout/country-context.tsx`: clears Cross-dataset's
bundle-specific view and filters when the global country selector changes,
while preserving query state on other pages.
- `frontend/components/layout/country-context.test.ts`: covers both sides of
that country-switch URL behavior.
- `frontend/components/layout/nav-items.ts`: removes the `usOnly` gate from the
Cross-dataset item.
- `frontend/components/layout/nav-items.test.ts`: verifies the unchanged label
and route and visibility for US, UK, and Belgium.
- `frontend/components/microcosm/cross-dataset-view.tsx`: scopes overview query
keys and API requests by country; derives geography, sample, and grouping
controls from the bundle; validates filter selections against the active
bundle; preserves the capped-relative-error definition; and removes
US-specific page copy.
- `frontend/components/microcosm/cross-dataset-facts-view.tsx`: threads country
through summary, catalog, and detail requests, query keys, and links, and does
not reuse one country's placeholder data for another.
- `frontend/lib/cross-dataset/artifact.ts`: validates and retains manifest
`jurisdictions`, then fails closed when none of the selected country's accepted
codes appears in the manifest.
- `frontend/lib/cross-dataset/artifact.test.ts`: covers exact jurisdiction
matches, the UK/GB alias, missing/mismatched jurisdictions, and malformed
jurisdiction arrays.
- `frontend/lib/cross-dataset/fact-presentation.ts`: includes country in catalog
parameters and generated catalog/detail URLs, with the current selector state
taking precedence over stale URL country state.
- `frontend/lib/cross-dataset/fact-presentation.test.ts`: covers country parsing,
overrides, and URL preservation.
- `frontend/lib/cross-dataset/presentation.ts`: uses bundle source/group labels,
preserves bundle source order, derives available controls from group
dimensions, and places country in group-to-fact links.
- `frontend/lib/cross-dataset/presentation.test.ts`: replaces US-ordering and
hard-coded-label expectations with bundle-driven presentation and
country-aware link coverage.
- `frontend/lib/cross-dataset/source.ts`: resolves independent US/UK/BE
directory, URL, and expected-run settings; enforces directory/URL exclusivity
per country; supplies the jurisdiction check; and caches one reader per
country/configuration.
- `frontend/lib/cross-dataset/source.test.ts`: covers each environment-variable
family, per-country conflicts and missing configuration, simultaneous-country
configuration, expected run IDs, jurisdiction rejection, cache isolation, and
the trimmed Belgium bundle.
- `frontend/lib/cross-dataset/fixtures/be-frontend-bundle/manifest.json`: trimmed
manifest fixture based on `.lane-fixtures/be-frontend-bundle`, retaining the
real bundle shape, hashes, 726-fact count, three source IDs, and
`jurisdictions: ["BE"]`.
- `frontend/lib/cross-dataset/fixtures/be-frontend-bundle/summary.json`: matching
real-shape summary fixture with the three Belgium source labels. The source
`.lane-fixtures` directory is not included.
- `LANE_E4_REPORT.md`: this report.

## Environment and jurisdiction contract

| Country | Directory | Base URL | Optional expected run ID | Accepted manifest code |
| --- | --- | --- | --- | --- |
| US | `CROSS_DATASET_ARTIFACT_DIR` | `CROSS_DATASET_ARTIFACT_BASE_URL` | `CROSS_DATASET_EXPECTED_RUN_ID` | `US` |
| UK | `CROSS_DATASET_ARTIFACT_DIR_UK` | `CROSS_DATASET_ARTIFACT_BASE_URL_UK` | `CROSS_DATASET_EXPECTED_RUN_ID_UK` | `UK` or `GB` |
| Belgium | `CROSS_DATASET_ARTIFACT_DIR_BE` | `CROSS_DATASET_ARTIFACT_BASE_URL_BE` | `CROSS_DATASET_EXPECTED_RUN_ID_BE` | `BE` |

For each country, configuring both its directory and base URL is invalid.
Different countries may be configured simultaneously. The expected-run variable
for the selected country retains the prior exact run-ID match semantics. A
missing jurisdiction list also fails closed. A mismatch raises the same
503-rendered `ArtifactError` family, with `stale_artifact` and text of the form:

```text
Cross-dataset bundle is for jurisdictions BE, not US.
```

An unconfigured Belgium request, for example, names both applicable location
variables:

```text
Cross-dataset artifacts are not configured for BE. Set CROSS_DATASET_ARTIFACT_DIR_BE or CROSS_DATASET_ARTIFACT_BASE_URL_BE.
```

## Verification

Two independent read-only reviews covered the server/API path and the
UI/navigation/country-switch path. Both completed with no remaining findings.
The protected `frontend/app/layout.tsx`, `frontend/app/globals.css`, and package
scripts are unchanged. `git diff --check` produced no output.

All gates ran from `frontend/`.

### `bun test`

```text
bun test v1.3.11 (af24e281)

219 pass
4 todo
0 fail
855 expect() calls
Ran 223 tests across 30 files. [407.00ms]
```

### `bun run lint`

```text
$ tsc --noEmit
```

### `IS_WEBPACK_TEST=1 CROSS_DATASET_ARTIFACT_DIR_BE=/Users/maxghenis/PolicyEngine/_worktrees/caldiag-e4/frontend/lib/cross-dataset/fixtures/be-frontend-bundle bun run build`

```text
Route (app)
┌ ○ /
├ ○ /_not-found
├ ƒ /api/hf-webhook
├ ƒ /api/microcosm
├ ƒ /api/microcosm/compare
├ ƒ /api/microcosm/cross-dataset
├ ƒ /api/microcosm/releases
├ ƒ /api/microcosm/staging/compare
├ ƒ /api/microcosm/staging/run
├ ƒ /api/microcosm/staging/runs
├ ƒ /api/microcosm/staging/target-diagnostics
├ ƒ /api/microcosm/target-diagnostics
├ ƒ /api/microcosm/target-investigation
├ ƒ /api/microcosm/target-tree
├ ƒ /api/microcosm/target-treemap
├ ƒ /api/microcosm/variable
├ ○ /icon.svg
├ ○ /microcosm
├ ○ /microcosm/compare
├ ○ /microcosm/datasets
├ ƒ /microcosm/model-coverage
├ ○ /microcosm/pipeline
├ ○ /microcosm/staging
├ ƒ /microcosm/targets
└ ○ /microcosm/variables


○ (Static) prerendered as static content
ƒ (Dynamic) server-rendered on demand
```

The successful production build configured exactly one artifact location: the
local Belgium fixture via `CROSS_DATASET_ARTIFACT_DIR_BE`. It used the unchanged
`build` script with Next's webpack selector because the literal Turbopack path
attempts prohibited network/worker-port operations in this lane sandbox. The
green run compiled successfully, ran TypeScript, generated all 20 static pages,
and collected build traces. No build workaround or package-script change is
committed.

LANE E4 DONE
35 changes: 27 additions & 8 deletions docs/cross-dataset-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,12 +55,21 @@ snapshot IDs and has a SHA-256 recorded in the manifest.

## Configure the application

Set exactly one of `CROSS_DATASET_ARTIFACT_DIR` or
`CROSS_DATASET_ARTIFACT_BASE_URL` before starting the application for
Cross-dataset work. The Cross-dataset API has no default artifact location: if
neither variable is set, the application process starts, but the Cross-dataset
page is unavailable and its API returns HTTP 503. Setting both variables is
also invalid.
Configure at most one local directory or remote base URL for each country. The
unsuffixed variables remain the US configuration so existing deployments keep
working; UK and Belgium use country suffixes:

| Country | Local bundle | Remote bundle | Optional pinned run |
| --- | --- | --- | --- |
| US | `CROSS_DATASET_ARTIFACT_DIR` | `CROSS_DATASET_ARTIFACT_BASE_URL` | `CROSS_DATASET_EXPECTED_RUN_ID` |
| UK | `CROSS_DATASET_ARTIFACT_DIR_UK` | `CROSS_DATASET_ARTIFACT_BASE_URL_UK` | `CROSS_DATASET_EXPECTED_RUN_ID_UK` |
| Belgium | `CROSS_DATASET_ARTIFACT_DIR_BE` | `CROSS_DATASET_ARTIFACT_BASE_URL_BE` | `CROSS_DATASET_EXPECTED_RUN_ID_BE` |

Setting both the directory and URL for the same country is invalid. Configuring
different countries at the same time is valid. If neither location is set for
the selected country, the application process still starts, but that country's
Cross-dataset page is unavailable and its API returns HTTP 503 naming the
applicable variables.

For a bundle on the local filesystem, set `CROSS_DATASET_ARTIFACT_DIR` to the
generated frontend bundle directory. `CROSS_DATASET_EXPECTED_RUN_ID` is
Expand All @@ -80,12 +89,22 @@ export CROSS_DATASET_ARTIFACT_BASE_URL=https://example.org/evaluation-run/fronte
make dev
```

For example, the published Belgium bundle can be selected with:

```bash
export CROSS_DATASET_ARTIFACT_BASE_URL_BE=https://huggingface.co/datasets/policyengine/microcosm-evaluation/resolve/main/be/evaluation-f28ca06a0b0d2baf13c87f2f/frontend/
make dev
```

A missing, stale, partial, malformed, or hash-mismatched bundle returns HTTP
503 rather than serving mixed results.
503 rather than serving mixed results. The API also verifies the selected
country against the manifest and fails closed on a mismatch: US requires `US`,
Belgium requires `BE`, and UK accepts either `UK` or `GB`.

## Read the API

`GET /api/microcosm/cross-dataset` accepts these views:
`GET /api/microcosm/cross-dataset` accepts `country=us|uk|be` (default `us`)
and these views:

- `view=summary` (default): source-level score, coverage, capability statuses,
unsupported reasons, period treatments, and target-performance buckets.
Expand Down
57 changes: 57 additions & 0 deletions frontend/app/api/microcosm/cross-dataset/route.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
import { fileURLToPath } from "node:url";

import { expect, test } from "bun:test";

import { GET } from "./route";

const ENV_KEYS = [
"CROSS_DATASET_ARTIFACT_DIR",
"CROSS_DATASET_ARTIFACT_BASE_URL",
"CROSS_DATASET_EXPECTED_RUN_ID",
"CROSS_DATASET_ARTIFACT_DIR_BE",
"CROSS_DATASET_ARTIFACT_BASE_URL_BE",
"CROSS_DATASET_EXPECTED_RUN_ID_BE",
] as const;
const BELGIUM_FIXTURE_DIRECTORY = fileURLToPath(
new URL("../../../../lib/cross-dataset/fixtures/be-frontend-bundle/", import.meta.url),
);

test("threads country to the configured Cross-dataset reader", async () => {
const previousEnvironment = Object.fromEntries(
ENV_KEYS.map((key) => [key, process.env[key]]),
) as Record<(typeof ENV_KEYS)[number], string | undefined>;

try {
for (const key of ENV_KEYS) delete process.env[key];
process.env.CROSS_DATASET_ARTIFACT_DIR_BE = BELGIUM_FIXTURE_DIRECTORY;

const belgiumResponse = await GET(
new Request("http://example.test/api/microcosm/cross-dataset?view=summary&country=be"),
);
expect(belgiumResponse.status).toBe(200);
expect(await belgiumResponse.json()).toMatchObject({
jurisdictions: ["BE"],
fact_count: 726,
sources: [
{ label: "EUROMOD BE_2025 on EU-SILC (JRC country report 2025)" },
{ label: "Microcosm-BE v0.4 × Axiom rules engine" },
{ label: "Microcosm-BE v0.4 × EUROMOD BE_2025" },
],
});

for (const url of [
"http://example.test/api/microcosm/cross-dataset?view=summary",
"http://example.test/api/microcosm/cross-dataset?view=summary&country=us",
]) {
const response = await GET(new Request(url));
expect(response.status).toBe(503);
expect(await response.json()).toMatchObject({ artifact_error: "partial_artifact" });
}
} finally {
for (const key of ENV_KEYS) {
const value = previousEnvironment[key];
if (value === undefined) delete process.env[key];
else process.env[key] = value;
}
}
});
4 changes: 3 additions & 1 deletion frontend/app/api/microcosm/cross-dataset/route.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,17 @@ import { NextResponse } from "next/server";
import { ArtifactError } from "@/lib/cross-dataset/artifact";
import { crossDatasetApiResponse } from "@/lib/cross-dataset/query";
import { configuredCrossDatasetReader } from "@/lib/cross-dataset/source";
import { parseCountry } from "@/lib/microcosm/latest-artifact";

export const revalidate = 300;
export const runtime = "nodejs";

export async function GET(request: Request) {
const country = parseCountry(new URL(request.url).searchParams.get("country"));
try {
const response = await crossDatasetApiResponse(
request.url,
configuredCrossDatasetReader(),
configuredCrossDatasetReader(country),
);
return NextResponse.json(response.body, { status: response.status });
} catch (error) {
Expand Down
27 changes: 27 additions & 0 deletions frontend/components/layout/country-context.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
import { expect, test } from "bun:test";

import { countrySwitchUrl } from "./country-context";

test("country switching clears bundle-specific Cross-dataset state", () => {
const switched = new URL(
countrySwitchUrl(
"https://example.test/microcosm/datasets?country=us&view=fact&fact_key=us-only&source=us-model",
"be",
),
);

expect(switched.pathname).toBe("/microcosm/datasets");
expect(switched.searchParams.toString()).toBe("country=be");
});

test("country switching preserves route state outside Cross-dataset", () => {
const switched = new URL(
countrySwitchUrl(
"https://example.test/microcosm/targets?country=us&variable=income_tax",
"uk",
),
);

expect(switched.pathname).toBe("/microcosm/targets");
expect(switched.searchParams.toString()).toBe("country=uk&variable=income_tax");
});
18 changes: 15 additions & 3 deletions frontend/components/layout/country-context.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,15 @@ export function isCountry(value: string | null): value is Country {

const STORAGE_KEY = "microcosm-country";

export function countrySwitchUrl(currentUrl: string, next: Country): string {
const url = new URL(currentUrl);
if (url.pathname.endsWith("/microcosm/datasets")) {
url.search = "";
}
url.searchParams.set("country", next);
return url.toString();
}

function persistCountry(country: Country) {
try {
window.localStorage.setItem(STORAGE_KEY, country);
Expand Down Expand Up @@ -51,11 +60,14 @@ export function CountryProvider({ children }: { children: ReactNode }) {
}, []);

const setCountry = (next: Country) => {
if (next === country) return;
setCountryState(next);
persistCountry(next);
const url = new URL(window.location.href);
url.searchParams.set("country", next);
window.history.replaceState(window.history.state, "", url);
window.history.replaceState(
window.history.state,
"",
countrySwitchUrl(window.location.href, next),
);
};

return (
Expand Down
Loading
Loading