Gecode optimization benchmarks

Archived measurements for the Gecode optimization branch. Benchmark reports and historical measurement tools are maintained here separately from the solver source.

The reference is the offline report originally exported as benchmark-site/outputs/gecode-before-after/index.html. Its bytes are preserved in report/index.html, including local documentation and downloads. The final native study was completed on September 6, 2026. The report retains its original September 5 heading and does not measure the subsequent September 7 MiniZinc integration.

Results

Each solve used one thread and a ten-second limit. Automatic exploration and restart count against that limit. Values below are the largest sampled input size solved in both repetitions and all family variants, followed by an observed failed upper size.

Problem	Original solved / failed upper	Auto + race solved / failed upper	Configured solved / failed upper
Knapsack, items	33 / 34	2304 / 2336	2304 / 2336
Assignment, tasks	17 / 18	64 / 66	31 / 32
Facility location, sites	8 / 9	44 / 45	54 / 56
Bin packing, items	17 / 18	47 / 48	30 / 31
Production planning, periods	11 / 12	27 / 29	28 / 29
Routing, cities	21 / 22	21 / 22	21 / 22
Weighted queens	23 / 24	23 / 24	23 / 24

“Original” means the preserved optimization facade before the algorithm changes, not a pristine upstream Gecode release. The automatic cohort combines structural preprocessing, capacity dynamic programming, checked LP bounds, branching and sequential racing. The configured cohort uses frozen family presets. This comparison does not isolate the contribution of any one algorithm.

These are sampled size brackets, not runtime speedup ratios or mathematical maximum-size guarantees. Instance difficulty can be nonmonotonic. Production at 28 periods had mixed automatic results, leaving the 27/29 bracket unresolved at the one-period target. Original witnesses were independently checked; large instance optimality is backend-reported. The families also informed development and preset selection, so the study is not an independent holdout.

The method records the protocol, amendments and historical commands. The compressed final data contains all 115 frozen models, 356 measured observations and three runtime-loader probes, including original witnesses and source/library hashes. Earlier studies remain separate downloads within the report.

Read the report

Open the published benchmark report. It preserves the charts and measurements and adds the seven study phases, two captured compiler argument lists and all 360 process commands under “Reproducibility & documentation.” Solver commands are reconstructed from the frozen runner and records; a full shell/environment transcript was not captured. The optimized library at its historical path was subsequently rebuilt, so exact replay requires the historical binary identified in the command archive.

site/ contains the updated static HTML and command downloads. GitHub Pages publishes it on each push to main, using the unchanged report/ for shared documentation and data. To assemble the same site locally:

benchmark_dir=$(mktemp -d)
cp -R report/. "$benchmark_dir/"
cp -R site/. "$benchmark_dir/"

Open index.html in that directory. The command downloads also include a new ratio calculation that reads the saved archive without invoking a solver:

sh site/downloads/gecode-final-ratios.sh report/downloads/gecode-final.json.gz

Clone this repository and open report/index.html in a browser. The charts work offline without a server or JavaScript; the generated documentation uses its included local assets. Keep the entire report directory together.

The benchmark release also provides gecode-before-after.zip. Its SHA-256 is 424c659a7c0a739b2526f359ec3fccfa4daea3b32681b601fb9440c865d0c69c. Unzip it and open gecode-before-after/index.html.

Verify the saved evidence

Python 3.9 or newer and its standard library are sufficient:

python3 verify.py
python3 -B experiments/optimize/test_final_instances.py -v

The first command checks the immutable file manifest, published data hashes, all frozen JSON/TXT model hashes, observation counts, saved original witnesses and the reported size brackets. It also checks local links from the report entry page. It does not invoke a solver or independently prove optimality. The second command checks the deterministic generators against every input in the previous domain, extension prefixes and independent feasible witnesses.

Historical measurement tools

experiments/optimize preserves the final driver's source, phased runners, generators and supporting validation code. The two files in experiments/solver-bench provide the original independent model validators. The frozen files are hashed in provenance/immutable-files.json. They are retained for audit and use with the original environment.

The measurement commands in METHOD.md require the preserved original runtime, its manifests and the matching solver checkout. The driver builders use macOS shared libraries and recorded CMake compiler/link metadata. The runners also expect the historical build/capacity-scaling, build/comparison-algorithms, build/category-scaling and build/native-structure layouts. Those executables, build trees and phase-specific reports are not included here. Copying these scripts into a new checkout does not recreate that environment, and rerunning on another machine would produce a new measurement. Use the verification commands above to check the evidence distributed by this repository.

The source identities and artifact hashes are in provenance/archive.json and provenance/final-checkpoint.json.

License

The original Gecode license is retained. Individual source and embedded documentation notices continue to apply.