CHANGELOG.md

Changelog

All notable changes to this reference implementation are documented here.

0.9.0 - 2026-09-10

This minor release adds the explicitly selected optional official SPSS Frontend 0.3 and changes emitted Python schema-extension identifiers. Older readers cannot load the new IDs; upgrade consumers before using new compiler output.

Added

• Implement explicitly selected official SPSS Frontend 0.3 over unchanged Plan 0.1/0.2 through the shared parser/binder, with a strict compile_spss_request boundary and opt-in live apply provenance. Support comments, dictionary-order TO, grouped commands, finite open RECODE ranges, additive ordered typed labels, inequality aliases, and canonical NOT lowering. Precedence is comparisons → NOT → AND → OR; parentheses override it. Tests cover all 90 effective normative cases plus request boundaries and in-place SQLite identity/metadata/audit/no-artifact checks. Default APIs and Python extension support/rejections remain unchanged.

Fixed

• Namespace new Python schema-change plans and SPSS output as openstatspec-python-schema-change-plan-v0.1 and openstatspec-python-schema-change-spss-v0.1, retaining exported constant names. The former openstatspec-transformation-plan-v0.3 remains accepted as a legacy Python extension with unchanged semantics, canonical JSON, and hashes; stored audits are not migrated. New compilation changes schema-plan hashes.
• Clarify that specification v0.5.0 has no official Plan 0.3. Its syntax-only Frontend 0.3 over Plan 0.1/0.2 is separate from Python schema extensions. Legacy acceptance and Python extension tests do not claim official conformance. See compatibility and migration.

The specification remains pinned to released v0.5.0 at exact commit 864e84479f554b8ee250ffed44c4dfb963750d4a; the required SPSS codec remains openstatspec-pyspssio==0.5.1.post2. No catalog migration, dependency change, new database support, or optional Transformation Workflow 0.3 implementation is included.

0.8.1 - 2026-09-10

Fixed

• Preserve other variables' shared value labels when replacing one variable's labels. Reject deletion of variables referenced by published derived lineage before any plan mutation, with variable_has_dependents.
• Enable and verify SQLite foreign keys on in-place apply connections.
• Read export values, dictionary metadata, import fidelity events and validation reflection from one native database snapshot; close it before writing SAV/ZSAV.
• Scope transformation label reads to the target variable's dataset. Unrelated malformed labels no longer fail the operation; malformed target-linked labels still fail, and shared/foreign-owned linked label sets remain supported.

Performance

• Remove quadratic suffix/name scans from canonical-plan binding and reuse one canonical plan encoding/hash per public canonical or SPSS apply.
• Remove redundant full case-row dictionary collections from database reads and imports. Results remain buffered; this is not a streaming API.
• Avoid the discarded core relation hash during SQLite workflow registration, retaining physical projection validation and derived-parent integrity checks. Execution-time hashes remain required; their loop avoids a redundant tuple copy.

Documentation and cleanup

• Document numeric create/replace, RECODE and separately provisioned target recipes, with bounded local SQLite measurements in the transformations manual.
• Remove an unreachable view-creation branch; workflow output remains materialized-only.

The specification pin, engine dependency, database/profile claims, catalog schema, public call signatures and Dolt-owned versioning policy remain unchanged.

0.8.0 - 2026-09-07

Changed

• Pin released specification v0.5.0 at exact commit 864e84479f554b8ee250ffed44c4dfb963750d4a, with normative status released. Capabilities identify adapter version 0.8.0 and explicitly select SAV/ZSAV 1.0 with database_io_policy=openstatspec-database-io-v1.
• Reads, inspection, validation, and SAV/ZSAV export no longer write database operation/fidelity audit records, including on failure. Export diagnostics and loss consent remain operation-scoped; export results omit operation_id.
• Default Dolt writes use the packaged tested exact-version policy for 2.2.2 and 2.2.3 without user-supplied evidence files. Unknown/untested versions remain blocked before mutation; explicit external overrides remain strict. Safety budgets are not claimed as proven native server limits.
• Dolt reads no longer require write-version declarations or the write variable-count ceiling. Missing SQLite files are rejected without creation.
• Exact Dolt 2.2.2/2.2.3 CI explicitly runs SELECT-only export success/failure tests using a separate fixture-admin account; normal service permissions and adapter permissions are unchanged. Wheel smoke installs resolve the required engine from PyPI rather than a source checkout.

No optional Transformation Workflow 0.3 implementation is claimed.

0.7.1 - 2026-08-12

Changed

• Pinned capability declarations, conformance fixtures, and release validation to the published OpenStatSpec specification v0.3.0 at exact commit cd8f198c68b849eb8ed018a894670a0904c2181d.

0.7.0 - 2026-08-11

Added

• Added root release workflow support for publishing the normative openstatspec-specification companion distribution, with independent artifact validation, smoke tests, and PyPI publishing.
• Carried the complete 0.6.0 transformation, PostgreSQL, and documentation work forward into the versioned 0.7.0 release from main.

0.6.0 - 2026-08-05

Added

• Completed Python parity for bounded in-place SPSS schema transformations: STRING, DELETE VARIABLES, string RECODE width validation, and durable delete/recreate physical-column bindings.
• Added PostgreSQL physical attribute-slot preflight and real-service regression coverage for delete/recreate at the column limit.

Changed

• Schema-changing SPSS plans use the v0.3 contract and fail closed for unsupported STRING/DELETE variable ranges.

0.5.0 - 2026-08-05

Fixed

• Removed the temporary *_catalog compatibility schema and made SAV/ZSAV import, validation, export, fidelity reporting, and in-place metadata edits use the normative UUID-keyed OpenStatSpec catalog exclusively.
• Existing databases that contain former compatibility relations must be remediated manually before further operations.
• Imports now reject dataset names that collide with normative UUID identifiers, cleanup drops only a physical table actually created by the failing import, and non-preserved SPSS compatible variable names are reported as an explicit loss requiring export consent.

Added

• Added bounded typed expressions with variable references, numeric literals, parentheses, =, <, <=, >, >=, AND, and OR. String comparison and v0.2 string assignment fail closed pending exact portable semantics.
• Added sequential SPSS-like COMPUTE and IF assignment operations plus FORMATS, VARIABLE LEVEL, and EXECUTE.
• Added atomic numeric target creation on SQLite and PostgreSQL. MySQL, MariaDB, and Dolt fail closed on target_mode=create; their targets must be provisioned physically and in the catalog by a separate versioned stage before this executor applies assignment and metadata operations.
• Added synthetic exact-program, catalog, failure-boundary, and pre-existing target regression coverage.

Changed

• Bumped the canonical transformation-plan and SPSS frontend contracts to v0.2; canonical JSON and hashes include every sequential operation.
• In-place apply records variable label, value labels, F print/write format, and measurement level only in the normative catalog.
• Dolt still requires an exact branch, exact HEAD, and clean working set; successful apply leaves an inspectable diff and never calls DOLT_COMMIT.
• Dolt declaration validation is implemented by the Python adapter. The specification remains a language-neutral source of JSON contracts and fixtures and is not installed as a Python distribution.

Specification basis

• Release validation is pinned to the untagged OpenStatSpec specification release candidate at immutable commit f2fdf687d8cb32b944ca55a3e9e7215ffc603019. Its specification_release remains null until that commit receives a stable tag.

0.4.0 — 2026-07-31

Added

• Added a public canonical transformation-plan in-place API accepting either typed plan objects or strictly validated mappings.
• Added apply-plan and install-in-place-schema CLI workflows alongside the compatible apply-spss command.
• Added explicit audit provenance distinguishing canonical plan documents from SPSS syntax sources.
• Added a transformation manual covering the canonical core, frontend boundary, execution invariants, CLI/API workflows, and extension rules.

Changed

• Moved the SPSS parser, binder, compiler, and convenience execution adapter to the dedicated openstatspec.frontends.spss package.
• Kept openstatspec.transform language-neutral with canonical plan, schema, and live-schema validation modules while preserving previous SPSS import paths.
• Reserved empty Stata and SAS frontend directories without exposing parsers, capabilities, CLI choices, or support claims.
• Strengthened pre-mutation target-type checks and target-scoped physical table identity guards. The executor continues to modify the same dataset and table without creating OpenStatSpec rollback, copy, snapshot, or history layers.

Specification basis

• Conformance and release validation continue to use OpenStatSpec specification release v0.2.0 at exact commit 79339ec3d8f8aa81789b7e85f6b8afa6f1374e50.

0.3.0 — 2026-07-31

Added

• Added a bounded SPSS-like RECODE, VARIABLE LABELS, and VALUE LABELS parser with stable diagnostics and conformance fixtures.
• Added canonical Transformation Plan serialization, RFC 8785 hashes, schema binding, deterministic typed operations, and an agent-facing CLI/API.
• Added product-neutral in-place execution with same dataset/table identity, direct data and metadata mutation, and compact operation audit. Dolt adds expected branch/HEAD and clean-working-set checks.

Changed

• Version history, diff, rollback, restoration, and commit remain database responsibilities; the transformer creates no derived/copy/snapshot/recovery layer and performs no DOLT_COMMIT.
• SQL server support now distinguishes conservative family claims from exact CI evidence: PostgreSQL 17.x/18.x at 17.10/18.4, MySQL 8.4.x/9.7.x at 8.4.11/9.7.2, and MariaDB 11.4.x/11.8.x/12.3.x at 11.4.12/11.8.8/12.3.2. Each service job verifies the live normalized version.
• Dolt remains an independent core profile, now claiming canonical stable versions >=2.2.2,<2.3.0 and running its complete service suite at exact versions 2.2.2 and 2.2.3 from immutable container-image pins. SQLite retains its core >=3.24.0,<4.0.0 and optional workflow >=3.35.0,<4.0.0 tiers. MSSQL remains unsupported and roadmap-only.

Specification basis

• CI, release validation, and machine-readable capabilities use OpenStatSpec specification release v0.2.0 at exact commit 79339ec3d8f8aa81789b7e85f6b8afa6f1374e50.

0.2.0 — 2026-07-30

This release extends the reference adapter while keeping imported source datasets immutable and all unsupported paths fail-closed.

Added

• An optional SQLite-only Transformation Workflow for immutable, versioned, parameterized SELECT definitions; materialized derived datasets; lineage and weights; public catalog APIs; and CLI operations.
• An independent Dolt core SQL profile for exact product version 2.2.2, with service-backed coverage, conservative adapter limits, complete compensating cleanup, and pre-DDL rejection of unsupported values. Transformation Workflow support on Dolt is not claimed.

Changed

• SAV and ZSAV validation now rewrites legacy compatible names consistently across type-2 records and subtype-13/subtype-14 long-name metadata.
• Transformation publication, interrupted-run reconciliation, retirement, and physical-removal recovery now preserve auditable state and fail closed on catalog or relation drift.
• Release automation builds distributions before test-only source checkouts and supports idempotent rebuilds of an existing release tag.

Specification basis

• CI and release validation are pinned to OpenStatSpec specification release v1.0.0-rc.1 at exact commit fef0dc6f4b17ff7141dad3f49d0524c63efbfed5.

0.1.0 — 2026-07-29

First public reference implementation of the OpenStatSpec strict wide-table SPSS profile.

Included

• Import of unencrypted SAV and ZSAV sources into one dedicated wide SQL table and its metadata catalog.
• Export of supported dataset semantics to SAV and ZSAV.
• SQLite, PostgreSQL, MySQL, and MariaDB profiles, including service-backed PostgreSQL 17/18, MySQL 8.4/9.7, and MariaDB 11.4/11.8/12.3 CI coverage.
• Read-only positive Dolt identity and working-set inspection. Operational Dolt writes use adapter-owned declaration validation and remain fail-closed because the adapter ships no concrete declaration claim. The specification remains language-neutral and is consumed only as JSON contracts and fixtures.
• Added explicit packaged/directory conformance-source injection and exact single-match binding across active Dolt product version, adapter implementation and version, and specification commit before mutation.
• Explicit catalog initialization/migration through initialize_catalog and openstatspec init; data operations never auto-create catalog relations.
• Read-only Dolt working-set evidence through dolt_state_snapshot and openstatspec dolt-state; core performs no Dolt version-control mutations.
• Preflight checks for target profile limits, atomic imports, validation, a command-line interface, and machine-readable capability and loss reports.

Important boundaries

• This release promises semantic round-trip equivalence only for supported features; it never promises byte-identical SAV/ZSAV output.
• The adapter requires explicit allow_loss consent before exporting known unsupported or lossy dictionary semantics.
• Multiple-response sets, variable alignment, variable sets, and custom attributes have explicit capability diagnostics; see the SAV profile.
• Encrypted SPSS files are not supported.
• Dolt adapter-envelope values are not claims about Dolt server limits. Failure recovery is compensating and reports deterministic residual inventory; a verified catalog receives a best-effort failed-operation audit when cleanup itself fails.
• Export publication uses a durable, audit-recorded prior-file backup through SQL finalization. Post-success backup retention is reported separately and does not rewrite a successful export as failed; pre-success failures restore the prior destination and close the running operation as failed.