chore(lint): sync the contract schema from the backend

[mivds]

Problem

go/internal/lint/schema/soda_data_contract_json_schema_1_0_0.json was branched from soda-core's copy in May and has drifted from the schema the backend validates every contract against on ingestion. sodacli contract lint therefore passed contracts that the backend then rejected — the opposite of what a linter is for.

Change

The embedded schema is now the backend's canonical contract schema, byte-identical to it apart from one docs-link fix (below). What that adds over the old copy:

• four dataset-level keys — check_attributes, diagnostics, failed_rows, reconciliation
• seven check types — aggregate_diff, duplicate_diff, freshness_diff, row_count_diff, group_by, hierarchy, reconciliation

No check type is lost. The 167 schema paths that disappear are all $defs reorganization (generic_check_properties, format, numeric_range, and the threshold if/then conditionals), which the backend expresses differently. soda_runner and soda_agent are both in the upstream schema now, so the backwards-compatible alias from PLATL-650 survives the swap.

Byte-identical is the point: the next sync is cp with nothing to remember. An earlier revision of this PR kept additionalProperties: false at the root, reasoning that a linter should catch typo'd keys; that deviation is gone. The backend sets it to true so a contract written for a newer server does not fail against an older one, and the lint schema should not be stricter than the thing it predicts.

Unknown keys deeper in the document are still rejected — the schema carries additionalProperties: false in 67 places, covering check bodies, thresholds and reconciliation blocks. Columns are permissive, because their definition is an allOf and a false inside one branch would reject keys defined by sibling branches.

Worth knowing what that leaves uncovered: soda-core's parser reads only the root keys it knows and silently ignores the rest (no warning at any log level — the sole exception is an unknown threshold key, which warns), and the backend accepts them by design. So a typo'd root key is reported by nothing. That is the backend's deliberate forward-compatibility trade, and this PR follows it rather than diverging from it.

The lint tests no longer assert what the schema means

This repo holds a copy of the backend's schema; what that schema means is the backend's business. Re-deriving it here bought nothing and made every sync a test-editing exercise — a real cost, since a legitimate upstream change fails CI here until someone comes and loosens the assertions. That is part of why the copy sat four months stale.

Seven tests are removed on those grounds. Three were added earlier in this PR (AcceptsDatasetLevelContractOptions, AllowsUnknownRootKeysLikeTheBackend, RejectsUnknownKeyInsideACheck); four predate it (MissingColumnName, AcceptsSodaRunner, AcceptsSodaAgentLegacyAlias, RejectsBothSodaRunnerAndSodaAgent, from 92ed81e and 284d7f3). The soda_runner pair shows the problem plainly: PLATL-650 added the key to this copy and a test for it, the backend has since added the key upstream, so those tests now pass for a reason unrelated to why they were written and would break if the backend restructured that oneOf.

Six tests remain, all covering this repo's code rather than the schema's content:

test	what it covers
ValidContract	the embedded schema compiles and a valid contract passes
YAMLParseError	malformed YAML is reported at the document root
EmptyFile	an empty file is invalid
LintFiles_Mixed	one result per input, in input order
ReportsErrorPathsForAnInvalidContract	the jsonschema-to-LintError translation: every error carries a message and a $-rooted path
SegmentsToPath	JSON path formatting

ReportsErrorPathsForAnInvalidContract replaces what the removed invalid-contract cases incidentally covered. It asserts path and message shape, not which key is wrong, and leans only on required: ["dataset", "columns"] — the one thing identical across every copy and schema version compared here.

One content fix on top of the sync

The upstream schema's check-attributes description pointed at https://beta.docs.soda.io/…, a host that does not resolve at all. It now points at the published page, https://docs.soda.io/soda-v4/manage-issues/check-and-dataset-attributes#attributes-in-checks (verified 200, and the anchor matches the ## Attributes in checks heading), which is the form every other link in the schema uses.

Fixed upstream too — sodadata/soda#13496 — so the next sync keeps the correction instead of reverting it. Until that merges, this file differs from master's copy by exactly that one line.

Testing

go build ./..., go vet ./... and go test ./... clean; the six remaining lint tests pass.

Validated on a 15-contract corpus — the backend's own contract-parser fixtures plus every complete contract example in the docs — by building the binary before and after and diffing the verdicts: identical, re-confirmed after both the root-permissiveness change and the test removal. The two contracts that fail are incomplete documentation snippets missing the required columns, which the old schema rejected too.

[m1n0]

Checked the swap: the embedded file is byte-equal to origin/master's soda-server/src/main/resources/contract/schema.json plus your one root edit, so the provenance claim holds. Built the binary and ran the contract from SCS-1441, where a customer put attributes on a metric check. Valid now, 6 errors before.

Two things before this lands.

The root additionalProperties: false deviation reintroduces the bug this PR fixes. A contract with a top-level kind: gets rejected by lint but is accepted by both Cloud and the engine:

kind: contract
dataset: ds/db/schema/t
columns:
  - name: id

lint says $: additional properties 'kind' not allowed, soda contract test says valid, and the canonical schema accepts it. kind is the collection dispatch key in soda-core at check_collections/base.py:237 and session.py:163, with values contract, data-standard, metric-monitoring. I'd keep the deviation, I agree a linter should catch typo'd keys, but add kind to the properties list so it's a two-line delta instead of one.

Second, nothing runs go test in CI. .github/workflows/ only has release.yml, tag-triggered, goreleaser. So TestLintFile_AcceptsDatasetLevelContractOptions never runs on a PR and the drift this fixes can come back without anyone noticing. A small test workflow here or straight after would close it.

Two smaller ones. The body says seven new check types, it's eleven. reconciliation is a top-level key rather than a check type, and fuzzy_duplicate, llm, metric_diff, reference_diff, rows_diff are missing from the list. And if the 15-contract corpus gives identical verdicts before and after, none of those fixtures exercised a newly-allowed key, so it's good regression evidence but doesn't show the fix works. The SCS-1441 contract would make a better fixture.

Nice catch on the dead ~/.soda/cache path, that had me looking too.

[m1n0]

Re-checked at bceaaba. kind is fixed, the schema is byte-identical to origin/master apart from the docs link, and my corpus is green: the SCS-1441 contract, attributes on metric and failed_rows at both levels, column_expression, remediation, group_by, check_attributes, the DET-107 additional: block. filter on a schema check is still correctly rejected. Good call fixing the link upstream in sodadata/soda#13496 too, that's the right order, though it's still open so a sync before it merges would revert your line.

The test deletions in bceaaba go too far.

I copied the old pre-sync schema over the new one and ran the current suite against it. It passes, all 6 tests. So nothing left in the suite can tell whether this PR's schema change happened at all. Revert the sync and the tests stay green.

TestLintFile_AcceptsSodaRunner, TestLintFile_AcceptsSodaAgentLegacyAlias and TestLintFile_RejectsBothSodaRunnerAndSodaAgent aren't assertions about the backend's schema. They're the soda_runner/soda_agent alias promise from PLATL-650, which is soda-cli's own commitment, and those tests are what stops a later sync quietly dropping it. I'd put all three back.

TestLintFile_AcceptsDatasetLevelContractOptions was the one test proving this PR does something. The SCS-1441 contract would be a good second fixture, it's four lines and it's the case that generated the ticket.

I verified all five deleted behaviours still hold at head, so this is lost coverage rather than a break. TestLintFile_ReportsErrorPathsForAnInvalidContract doesn't substitute: asserting that paths start with $ and messages are non-empty would pass even if the schema rejected every contract on earth.

Body's stale in the Change section. It still says the one deviation is the root keeping additionalProperties: false and cites TestLintFile_InvalidProperty as pinning it. b77622e undid both.

And still nothing runs go test in CI, which matters more now the suite is 6 tests instead of 11.

[mivds]

Thanks for the review @m1n0. I'll enable CI testing in a separate PR. Apart from that: I dropped contract validation tests since this is a copy of the schema from BE. Validation should live in one central place, not here.