[stugorf]

Problem and change

Relates to #15. Keep DataFusion's existing partition/default repartition settings and add keyword-only CypherEngine(..., target_partitions=None) for callers who measure their workload. Positive Python integers are accepted; booleans and invalid bounds are rejected. Refresh preserves the override. It controls partitions rather than threads or memory.

The controlled comparison covers partitions 1, 2 and the measured host's default 16, with round-robin repartition enabled and disabled: 24 complete workload cells and 528 unchanged golden checks, followed by 60 paired repeat cells. No alternative improves both workload geometric means without per-query regressions. The setting report retains every query and the trade-off. Thirty-six wide-scan runs over 500,000 rows with 4 KiB properties also pass; failed setup/checksum attempts are preserved and excluded. No adaptive scheduler or universal speedup is introduced.

Validation

Head 7461a7774d6cdf74bbd6961efa7ec86b229fa945; frozen parent 91d6b482a625b671c1e2eef923a4105331dc7ef4. Focused checks and regression details record commands, results, documentation and compatibility. The release receipt pins wheel/archive identity and 1,209 installed tests.

Both complete workloads pass unchanged goldens in memory and lazy modes. Retained first-pass geometric mean speedups are below (greater than one is faster). Per-query results and paired investigations remain in the audit archive.

Workload/mode	Main	Actual parent
Q30 memory	1.043	1.050
Q30 lazy	0.952	0.972
IC memory	0.982	0.994
IC lazy	0.994	1.013

Measurements use the same Apple M4 Max (16 cores, 128 GiB), macOS 26.6.2 arm64 and Python 3.13.17, with Q30 revision 6260689 and IC revision c185123. Reproduction instructions pin settings, data checksums, locks and commands. The frozen main baseline is code-identical to current main 90114f3; the equivalence check also verifies clean prospective main-target merges.

This is a coordinated draft. Preserve its frozen-head outcomes and apply the shared corrections in the qualified integration; changing the actual target requires requalification. The accepted combined engine source is 7d2ef3e; its current-main documentation merge is 5fe24cd.

User impact

CypherEngine gains keyword-only target_partitions=None. Positive integers are accepted, invalid values/booleans rejected, and refresh preserves the setting. Existing defaults remain; guides, API reference, stubs and a refresh example are updated.

Review ledger · Current contribution guide

Audit archive: 47,675,190 bytes; SHA-256 b3486fb30a34639c830b5d49353b3838f96369f95df959ac35f85511f7f8b2de. The small current-main supplement preserves the documentation update. Original absolute paths in receipts map to included records through the archive index.

Submitted by David Hughes (@stugorf), with AI Collaboration.

[prrao87]

This change merged into an intermediate branch, not main. It reached main through #55, which brought in the final integrated version of this work with every original commit kept on its branch. Follow-up fixes from review are listed there.