chore: upgrade to pulp 4

[andig]

Upgrades to pulp 4.0.0, which moves the model into a Rust core, returns solve statistics instead of a status code and stops bundling CBC.

• solve() returns LpSolveStats: Optimal now means proven, a run stopped by the clock reports TimeLimit and carries what it found in has_solution. CBC's "Optimal (within gap tolerance)" comes back as GapLimit and still counts as proven, the way pulp 3 read it. Each stage keeps the stats of the schedule it leaves in the variables, the sol_status workarounds are gone. The statuses the API reports are unchanged.
• CBC comes from the pulp[cbc] extra, CBC 2.10.13 on every platform, which replaces the Dockerfile's CBC download and the symlink into pulp's solver directory. OPTIMIZER_CBC_PATH selects another binary: on this macOS 27 machine the kernel kills the cbcbox binary on larger models, a Homebrew cbc works.
• Variables are created by the problem and a constraint cannot change once added, so the split adds its cost bound once and solve() rebuilds the model before it runs again. A variable no row uses comes back from the solver without a value and fails problem.valid(), so z_s_max_reached is only created for steps that have a demand.
• 140 tests in 9.0 s against 139 in 9.4 s on pulp 3.3.0. Model build 38 ms to 25 ms and MPS write 14 ms to 5 ms on a 6000 variable request, the solver dominates either way.

The continuity stage on the rollup branch shares variables between problem.copy() and the original, which pulp 4 rejects; it needs the copy's own variablesDict() when it lands.

🤖 Generated with Claude Code

[Petapton]

Hi, here are 2 issues I found with this branch:

• While building the container image, RUN echo | "$(/app/.venv/bin/python -c 'import pulp; print(pulp.COIN_CMD().path)')" | grep -q 'Version: 2.10.13' fails, since cbc reports dev as version

• When the processing time exceeds one probe share, I found that the call to self.problem.solve() in optimizer.py raises an exception. I'll delve into it later.

  Traceback

    ERROR in app: Exception on /optimize/charge-schedule [POST]
  Traceback (most recent call last):
    File "/app/.venv/lib/python3.13/site-packages/optimizer/app.py", line 265, in post
      result = optimizer.solve()
    File "/app/.venv/lib/python3.13/site-packages/optimizer/optimizer.py", line 908, in solve
      self._probe_then_split(tmpdir, deadline)
      ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
    File "/app/.venv/lib/python3.13/site-packages/optimizer/optimizer.py", line 830, in _probe_then_split
      self.stats = self.problem.solve(self._solver(tmpdir, timeLimit=probe))
                   ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/app/.venv/lib/python3.13/site-packages/pulp/core/lp_problem.py", line 839, in solve
      return solver.solve(self, **kwargs)
             ~~~~~~~~~~~~^^^^^^^^^^^^^^^^
    File "/app/.venv/lib/python3.13/site-packages/pulp/apis/core.py", line 236, in solve
      return self.actualSolve(lp, **kwargs)
             ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
    File "/app/.venv/lib/python3.13/site-packages/pulp/apis/coin.py", line 164, in actualSolve
      status, has_solution = self.solve_CBC(lp, **kwargs)
                             ~~~~~~~~~~~~~~^^^^^^^^^^^^^^
    File "/app/.venv/lib/python3.13/site-packages/pulp/apis/coin.py", line 257, in solve_CBC
      raise PulpSolverError(
      ...<2 lines>...
      )
  pulp.apis.core.PulpSolverError: Pulp: Error while trying to execute, use msg=True for more details/app/.venv/lib/python3.13/site-packages/cbcbox/cbc_dist/bin/cbc
  

[andig]

Thanks, both addressed in 3d81161.

Version check: cbcbox 2.935 ships a CBC devel build (CBC devel (git:146ce89)), no version line at all. The Dockerfile now only checks that the binary runs.

Probe exception: _probe_then_split now catches PulpSolverError on the probe and leaves the answer to the split, reported as split, probe failed. The split still has the rest of the clock, so a failed probe no longer takes the request down.

I could not reproduce the non-zero exit, though. On the random MIP and on every captured test case with a 0.5 s time limit, cbcbox exits 0 on "Stopped on time limit", on both linux/amd64 and linux/arm64, with 1 and 4 threads. pulp raises exactly that error only when cbc's exit code is non-zero. If you can run the failing request with msg=True or get the exit code (137 would be the gunicorn conf reaping the solver, 139 a crash), that would tell what stopped cbc.

🤖 Generated with Claude Code

[andig]

CI on this branch was red since the first commit, unrelated to the two points above: strict case 027 pinned which slot gets the partial 20 Wh charge, but the peak penalty only values the horizon maximum, so both schedules are worth exactly the same (cost 149.4, preference -0.285 on either). CBC 2.10 picked the late remainder, the devel build in cbcbox picks the early one. Reproduced on linux/amd64 and fixed by giving the case a c_priority, which makes the early fill the unique optimum on both solvers. Full suite passes with cbcbox on linux/amd64 and with CBC 2.10.13 locally.

🤖 Generated with Claude Code

[andig]

bc9200f drops the pulp[cbc] extra again and keeps the CBC 2.10.10 release build in the image (amd64, as on main; arm64 takes the Debian 2.10 package). pulp 4 resolves cbc from PATH, tests pass with it. Reason, measured on the 19 captured cases, time limit 10 s, 1 thread, best of 3:

	arm64	amd64
main, pulp 3, CBC 2.10.10	1.22 s
this branch, pulp 4, CBC 2.10.10	1.21 s
this branch, pulp 4, cbcbox	3.93 s	1.5× main

pulp 4 itself is neutral. The cbcbox solver is what costs, and the cause is the build, not pulp:

• cbcbox compiles the COIN-OR next branch (unreleased Clp/Cbc) as shared libraries with -ffp-contract=off on every component, by design for cross-platform reproducibility. On a pure LP (1500×900, ~1200 dual simplex iterations) its per-iteration speed is 2.8× slower than 2.10.10 on arm64 and 2.3× on amd64 with the generic x86-64 variant. The AVX2 variant, which cbcbox picks on AVX2 hosts, narrows that to 1.3×. arm64 only has the generic build.
• On top, the next branch does more root work on our models: 10 preprocessing passes, a 0.7 s feasibility pump, 19 cut passes, and 14 nodes / 1660 iterations on case 020 where 2.10.10 needs 6 / 1185. Case 020 on amd64: 2.10.10 0.70 s, cbcbox AVX2 1.13 s, cbcbox generic 2.96 s.
• Not the cause: OpenBLAS threading (same wall time with OPENBLAS_NUM_THREADS=1), debug symbols (none), assertions (one site).

So the extra is fine for a laptop, but production would pay 1.5 to 3× per solve for it.

🤖 Generated with Claude Code

[andig]

Correction on the cause split after looking at the instruction mix: -ffp-contract=off is a footnote. The 2.10.10 release build we run is a plain x86-64 build without FMA too, and the cbcbox AVX2 libClp even contains FMA instructions. The generic cbcbox dist (what arm64 and non-AVX2 amd64 hosts get) is a de-tuned build: no vector code, 4× the frame-pointer prologues, 28 % larger than the AVX2 build of the same source, and 2.65× slower per simplex iteration. The AVX2 dist is 1.14× slower per iteration. On top of either sits ~1.4× more root work in the next branch (1660 vs 1185 iterations, 14 vs 6 nodes on case 020). Build quality is the big lever, root work the floor.

🤖 Generated with Claude Code