Published Apache image rejects advertised external-payload sizes with HTTP 200 and PHP warning output

[rmcdaniel]

Observed

The published Server 2.3.1 image advertises a default 64 MiB runtime external-payload limit, but its PHP configuration retains post_max_size=8M and display_errors output. A direct authenticated POST /api/external-payloads/v1 with a valid 67,108,864-byte SDK-encoded Avro blob returns HTTP 200, not 201 or a structured rejection:

Warning: PHP Request Startup: POST Content-Length of 67108864 bytes exceeds the limit of 8388608 bytes ...
{"message":"The POST data is too large."}

Reproduced twice through the actual Apache/mod_php HTTP surface with the published image, not Laravel in-process tests. No upload reference was created; existing workflow and activity data remained intact. Server revision d9bed7072ae523978f16717f6b92bf669d5208f5, bundled Workflow 2.0.6. Image index sha256:9c52aa1252a4f19d7ec20adbf64e95d7b5bddf369412844f7b24e804c30eef2b.

Required Outcome

• Align the actual PHP/SAPI ingress limit with the explicitly configured runtime transport contract. Do not claim 64 MiB support merely by raising an INI value: verify the complete upload, fetch and retained-workflow consumption path with bounded memory.
• Prevent PHP startup/runtime warnings from corrupting API bodies or turning failures into apparent success.
• Oversized requests return the documented non-success status and structured response, including ordinary JSON requests and the external-upload endpoint.
• Add real published-image/Apache regression coverage. In-process Laravel tests alone do not exercise PHP startup limits.
• Exercise exact-limit, over-limit and concurrent requests, then verify ordinary requests and previously committed results still work. Preserve authentication, namespace isolation, admission and durable state.

Accepted follow-up from storage qualification. Investigate the complete size/memory boundary before publishing a fix so a single INI change does not conceal the next failure.

[rmcdaniel]

The size investigation confirms that raising the SAPI POST ceiling alone is insufficient. In the same disposable native runtime, changing only post_max_size=0, display_errors=Off, and log_errors=On lets the exact 64 MiB body reach the application, but an Apache request process is then killed by the memory cgroup. The client receives an empty reply. Cgroup memory.events records oom=1, oom_kill=1; other backends remain alive and health subsequently serves again.

This is evidence of a second constraint, not a recommendation to disable size checks or increase a production memory limit. The fix needs a bounded upload/read path and honest ingress limits, with SAPI-level tests. Do not publish an INI-only fix and claim the advertised maximum is qualified.

[rmcdaniel]

The fix is merged in #142 at 59b455d. All final source, Helm installation, database recovery and performance checks passed. Server 2.3.2 / Helm 0.1.81 publication is running: https://github.com/durable-workflow/server/actions/runs/34210652912 .

The actual built image has already passed concurrent 64 MiB workflow/activity HTTP results, fresh published PHP SDK consumption, and cold Server/MySQL restart with verified hashes. The final acceptance step is repeating that native proof with the registry-published image. This issue stays open until that user-installable artifact is verified; no production deployment is implied.

[rmcdaniel]

Verified and complete in Server 2.3.2, merged through #142 at 59b455d. Bundled Workflow 2.0.7; matching Helm chart 0.1.81. Publication and clean-client Helm installation passed.

The registry-downloaded image (no source overlays) was verified on linux/amd64:
durableworkflow/server@sha256:93b500a8a3d0edf6c10def344c4b68569904ac86a4034339f52d6175b17e3a66

Native scripts/regression/external-payload-http.php passed concurrent exact 64 MiB upload/fetch, SHA-256 integrity, structured oversized fixed/chunked rejection, ordinary JSON/form limits, retained workflow input/description, concurrent maximum workflow results, and standalone activity input/poll/result. A clean Composer consumer of PHP SDK 2.0.7 read both full results and verified the 50,331,630-byte decoded value.

After restarting both Server and MySQL, the script's --verify mode confirmed workflow/activity completion, reference integrity and full official Avro decoding. Another fresh published SDK process read both correct results. PHP heap remained 128 MiB, HTTP container 176 MiB/1 CPU, MySQL 400 MiB/1 CPU; zero OOM/OOM kills. The 5,786 cgroup max events were reclaim pressure, not evidence of spare headroom.

Temporary storage was disk-backed. This does not qualify tmpfs/concurrency limits for every deployment or remove the separate PHP outbound-upload gap in sdk-php#63. Large text validation can hold one decoded scalar, and temporary encoded/decoded snapshots require capacity planning. Production was not changed by this verification.