Standardization, extension of visualizations, and general updates - #3

Merged

fjwillemsen merged 240 commits into

mainfrom

standardization

Sep 3, 2025

Commits on Aug 20, 2024

Update the experiments setup json schema
jhozzova
committed
a3e1a15
Update experiments.py to work with a new schema
jhozzova
committed
594a87b
Update caching.py to work with a new schema
jhozzova
committed
f4fdac7
Add KTT tuning to runner.py
jhozzova
committed
f34a7f0
Update searchspace_statistics.py to work with a common output json schema
jhozzova
committed
aad790d
Update visualization scripts regarding new schema
jhozzova
committed
77289b0
Fix version number for kernel_tuner in pyproject.toml
jhozzova
committed
13a8ffc
Add example experiment files for KTT
jhozzova
committed
a28a240
Merge pull request #2 from jhozzova/new_schema_integration
fjwillemsen
authored
94c8193

Commits on Sep 25, 2024

Update
fjwillemsen
committed
6810ede

Commits on Oct 1, 2024

Wrote conversion script to new experiments file
fjwillemsen
committed
25397e8

Commits on Oct 10, 2024

Commits on Oct 11, 2024

Commits on Oct 12, 2024

Implemented experiments file generation
fjwillemsen
committed
92ad96f

Commits on Oct 16, 2024

Generate an experiments file using provided values and defaults
fjwillemsen
committed
d1b4310

Commits on Oct 22, 2024

Commits on Oct 23, 2024

Simplified validation of experiments file format
fjwillemsen
committed
53b1e70
Implemented T4 schema validator
fjwillemsen
committed
71f0cbb
Implemented conversion of time units based on T4 metadata
fjwillemsen
committed
b9259a1
Time unit conversion supports lists
fjwillemsen
committed
c647b87
Generalized several useful Searchspace statistics functions
fjwillemsen
committed
69a98af
Completed conversion of time based on units
fjwillemsen
committed
fd8f319
Baseline visualization in absolute plot, disabled automatic conversion of objective performance
fjwillemsen
committed
1b0ca47
Improvement to reporting non-overlapping time range
fjwillemsen
committed
d9bf8bc

Commits on Oct 25, 2024

Revamped plots specification in experiments files
fjwillemsen
committed
31bd3bb

Commits on Oct 26, 2024

Commits on Oct 27, 2024

Commits on Oct 29, 2024

Commits on Oct 30, 2024

Commits on Nov 2, 2024

Added a try-catch system to re-execute experiment caches that did not have enough overlap
fjwillemsen
committed
743482e

Commits on Nov 3, 2024

Changed error type
fjwillemsen
committed
81bc092

Commits on Nov 6, 2024

Updated comparison of hypertuned algorithms
fjwillemsen
committed
0bcfaf6
Extended VMin and colorbar range
fjwillemsen
committed
6edcbb5

Commits on Nov 7, 2024

Commits on Nov 20, 2024

Comparison of various BO methods
fjwillemsen
committed
ff75788

Commits on Mar 5, 2025

Commits on Mar 6, 2025

Added GEMM to hypertuning comparison
fjwillemsen
committed
50a464b
Added benchmark_hub repository as submodule
fjwillemsen
committed
ab937b7

Commits on Mar 7, 2025

Commits on Mar 8, 2025

Commits on Mar 9, 2025

Added tuning and inv. tuning of differential evolution and genetic algorithm
fjwillemsen
committed
55399b5

Commits on Mar 10, 2025

Added tuning and inv. tuning of PSO
fjwillemsen
committed
384cb21
Removed basinhopping and dual annealing
fjwillemsen
committed
8571416
Removed basinhopping and dual annealing
fjwillemsen
committed
c2f1fd7
With too few data points, return directly instead off smoothing
fjwillemsen
committed
97b73e2
Adjusted the cutoff percentile for hyperparameter tunoing
fjwillemsen
committed
7093271
Started over with hyperparamtuning, added dual_annealing and MLS tunings
fjwillemsen
committed
6c21d42

Commits on Mar 11, 2025

Updated hypertuner comparison
fjwillemsen
committed
6ac3ea5

Commits on Mar 12, 2025

Added genetic algorithm and simulated annealing tuned params
fjwillemsen
committed
f687932
Added tuned MLS
fjwillemsen
committed
d49967a
Put dual annealing back in comparison
fjwillemsen
committed
77f253b

Commits on Mar 13, 2025

Improved time margin calculation
fjwillemsen
committed
22446c1
Revised and tested benchmark kernels, added Nvidia A6000 bruteforced
fjwillemsen
committed
0787d7e
Added newly tuned basinhopping
fjwillemsen
committed
d4aaa76

Commits on Mar 14, 2025

Commits on Mar 15, 2025

Commits on Mar 17, 2025

Commits on Mar 18, 2025

Set compare_hypertuners_paper experiments file as in paper
fjwillemsen
committed
9c0ca13
Added new and improved plotting options for heatmaps
fjwillemsen
committed
74b746f
Updated benchmark_hub submodule
fjwillemsen
committed
1410ac1
Added experiments file and supporting files for comparing meta-strategies on hyperparameter problems
fjwillemsen
committed
9fc7383
Improved error reporting and code legibility
fjwillemsen
committed
a3f77fc
Added diff_evo tuning to comparison
fjwillemsen
committed
0eea8e4

Commits on Mar 19, 2025

Created new function in searchsspace_statistics for getting both the start and stop budget with a safe margin
fjwillemsen
committed
2b5371c

Commits on Mar 20, 2025

Commits on Mar 25, 2025

Updated hyperparameters
fjwillemsen
committed
1e2b2af
Improved error message and extended heatmap color range
fjwillemsen
committed
b618005
Updated dependencies and classifiers
fjwillemsen
committed
7b8d1eb
Applied linting to modified codebase
fjwillemsen
committed
b1e60ac
Updated benchmark hub
fjwillemsen
committed
8ad08f6
Bumped Kernel Tuner version
fjwillemsen
committed
9d3b0d1

Commits on Mar 26, 2025

Updated heatmap experiment files with new values
fjwillemsen
committed
4cd9394

Commits on May 8, 2025

Improved calculation of stopping point
fjwillemsen
committed
071d144

Commits on May 9, 2025

Improved type error message raised when having inner arrays in cache files
fjwillemsen
committed
fe8a23c
Improved W7800 cachefiles
fjwillemsen
committed
53125bb
Added W7800 to experiment files
fjwillemsen
committed
d295126

Commits on May 11, 2025

Commits on May 12, 2025

Commits on May 13, 2025

Commits on May 14, 2025

Implemented specification of custom labels
fjwillemsen
committed
a7e786c

Commits on May 15, 2025

Commits on May 16, 2025

Updated hyperparameters of PSO and simulated annealing
fjwillemsen
committed
c4fff3a

Commits on May 18, 2025

Updated hyperparameters of genetic algorithm
fjwillemsen
committed
e1294d3

Commits on May 19, 2025

Updated hyperparameters of firefly algorithm
fjwillemsen
committed
2c7fada
Minor changes to experiment files
fjwillemsen
committed
54146ed

Commits on May 20, 2025

Added median comparison to experiments file
fjwillemsen
committed
3473f2f
Added mean configurations to experiments file
fjwillemsen
committed
4a103a8

Commits on May 24, 2025

Commits on Jun 2, 2025

Adjusted plot sizes
fjwillemsen
committed
cd7277b
Implemented passing custom search method path
fjwillemsen
committed
5c16528

Commits on Jun 12, 2025

Added experiments file for comparing constrained to non-constrained strategies
fjwillemsen
committed
eac2e22

Commits on Jun 19, 2025

Added pyATF strategies to comparison
fjwillemsen
committed
870dad7

Commits on Jun 24, 2025

Introduced the 'cap to vmin' argument, allowing values under the heatmap vmin to be capped to the vmin (must be used with care)
fjwillemsen
committed
1039d05

Commits on Jun 25, 2025

New, more extensive color mapping and improved capping
fjwillemsen
committed
04de14b

Commits on Jun 26, 2025

Commits on Jun 27, 2025

Commits on Jun 28, 2025

Minor improvements to plotting and plot sizes
fjwillemsen
committed
8e3ef1b
Improvements to the experiment files
fjwillemsen
committed
7b6e611
Improved handling of NaNs in head2head comparison
fjwillemsen
committed
47e4dcb

Commits on Jun 30, 2025

Improvements to font size
fjwillemsen
committed
0c5b8a4
Use end time for head2head if not found
fjwillemsen
committed
6a9a50a

Commits on Jul 3, 2025

Switched to Numpy 2.0, code changes for compatibility
fjwillemsen
committed
8c2a4ba
Applied Ruff formatting
fjwillemsen
committed
101d283
Further improvements based on linting
fjwillemsen
committed
c36bf80
Further improvements based on linting
fjwillemsen
committed
3181010
Updated benchmark_hub submodule
fjwillemsen
committed
36ad35a
Further specification of linting
fjwillemsen
committed
1cc9cdb
Minor improvements based on tests
fjwillemsen
committed
348e91c
Adjusted tests to recent changes
fjwillemsen
committed
a0774d4
Adjusted head2head plot generation to be reselient to NaNs
fjwillemsen
committed
47e11b4
Added head2head plot to tests
fjwillemsen
committed
8229311
Minor improvements to head2head plots
fjwillemsen
committed
90d471e

Commits on Jul 18, 2025

Implemented quick and optional full validation of T4 files on loading, much improving performance
fjwillemsen
committed
f53955e
Improved to_valid_array performance
fjwillemsen
committed
6081542
Updated benchmarkhub
fjwillemsen
committed
c8a1f04

Commits on Jul 25, 2025

Improvements to docstrings
fjwillemsen
committed
9312c4c

Commits on Jul 28, 2025

Remove the second color (orange) to avoid confusion with the fourth (red)
fjwillemsen
committed
f78802b

Commits on Aug 7, 2025

Commits on Aug 9, 2025

Automatic compression and memory pressure reduction in results collection
fjwillemsen
committed
0feecd2

Commits on Aug 19, 2025

Commits on Sep 3, 2025