CAMEL-23703: tooling/camel-exe - native camel.exe bootstrap for Windows#24665
CAMEL-23703: tooling/camel-exe - native camel.exe bootstrap for Windows#24665ammachado wants to merge 29 commits into
Conversation
|
🌟 Thank you for your contribution to the Apache Camel project! 🌟 🐫 Apache Camel Committers, please review the following items:
|
c816c83 to
5f7ba72
Compare
|
🧪 CI tested the following changed modules:
🔬 Scalpel shadow comparison — Scalpel: 5 tested, 0 compile-only — current: 3 all testedMaveniverse Scalpel detected 5 affected modules (current approach: 3).
|
5f7ba72 to
6ed5fbe
Compare
|
For the reviewers: should this feature trigger a |
gnodet
left a comment
There was a problem hiding this comment.
Review: CAMEL-23703 — native camel.exe bootstrap for Windows
Verdict: COMMENT (no blocking issues found)
Confidence: HIGH
This is a well-structured contribution that adds a thin native Windows bootstrap (camel.exe) for the Camel CLI launcher, primarily to support WinGet's portable installer requirement for a genuine .exe command. The design choice to keep the exe as a pure forwarder to camel.bat (no Java discovery, no option parsing) is sound and minimizes the security and maintenance surface.
Strengths
- Clean C code: ~100 lines, well-commented, correct Windows API usage (wide-char throughout, proper handle cleanup,
ZeroMemoryinitialization). - Good test coverage: 5 behavioral tests covering argument forwarding, spaces in arguments, Unicode, exit code propagation, and spaced directory paths. The PowerShell-based UTF-8 capture approach for Unicode testing is clever.
- Proper module split: Separating
camel-exefromcamel-launcheravoids rebuilding the heavy jbang dependency graph for native bootstrap changes. - CI workflow: Path-filtered Windows CI jobs avoid unnecessary builds. Change detection using
git diff(instead ofdorny/paths-filter) avoids fork PR failures. - Documentation: Upgrade guide, module README, and native README are all present.
Findings
[MEDIUM] Silent truncation risk in _snwprintf_s — tooling/camel-exe/src/main/native/camel.c line 85. The _snwprintf_s call uses _TRUNCATE, which means if the formatted string exceeds cap, it silently truncates rather than failing. While the current +64 margin (31 chars of fixed overhead) makes this practically impossible for any realistic path, checking the return value for -1 (truncation indicator) would make the code more robust and prevent silent malformation. This is not a security issue (the user is trusted) but could cause confusing failures.
[LOW] MAX_PATH (260) limitation — tooling/camel-exe/src/main/native/camel.c line 54. On Windows 10+ with long path support enabled, paths can exceed 260 characters. The GetModuleFileNameW call with a fixed MAX_PATH buffer would fail for such installations. A dynamic allocation loop (doubling the buffer on ERROR_INSUFFICIENT_BUFFER) would future-proof this. Not a blocker since the vast majority of Windows installations operate within MAX_PATH.
[LOW] skip_argv0 does not handle escaped quotes — tooling/camel-exe/src/main/native/camel.c lines 33-49. The function stops at the first closing " when argv[0] is quoted, but Windows CommandLineToArgvW treats \" as a literal quote within a quoted argument. In practice, argv[0] comes from GetModuleFileNameW and will never contain escaped quotes, so this is purely academic.
[INFO] CI failure in camel-launcher-windows job — The camel-launcher-windows CI job fails, but the failure is in camel-management (a pre-existing test failure in an unrelated core module pulled in by -am). The camel-exe job passes cleanly. The launcher job's Maven command (-pl tooling/camel-exe,dsl/camel-jbang/camel-launcher -am install) pulls in the entire upstream dependency tree, which is why an unrelated failure surfaces.
[INFO] Scope overlap with merged PR #24666 — The first two commits (CAMEL-24053 .gitattributes fixes) were already included in merged PR #24666. After rebasing on latest main, these will become no-ops. This is expected stacking behavior and not an issue.
[INFO] POM packaging with test compilation — camel-exe uses <packaging>pom</packaging> but manually wires up build-helper-maven-plugin, maven-compiler-plugin, and maven-surefire-plugin for test compilation and execution. This is unconventional but pragmatic — there is no Java main source (the main artifact is the C-compiled exe), so pom packaging avoids the jar lifecycle overhead.
Scanner coverage
No static analysis tools (SonarCloud, SpotBugs, Error Prone) ran as part of this review. The C code (camel.c) falls outside the Java static analysis toolchain. Manual review was performed for memory safety, command injection, and Windows API correctness. This review does not replace specialized tools such as CodeRabbit or Sourcery.
Claude Code review on behalf of @gnodet
This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.
728b83b to
1f9243f
Compare
davsclaus
left a comment
There was a problem hiding this comment.
Thanks for this contribution @ammachado — the C code is clean, the test coverage is thorough (especially the Unicode capture workaround), and the module split makes sense.
The camel-launcher-windows CI job is currently failing: the assembled ZIP does not contain bin/camel.exe. The camel-exe job itself passes (native build + tests are green), so the issue is in the launcher integration — likely the maven-dependency-plugin:copy at generate-resources doesn't stage the file in time for the assembly, or the assembly doesn't pick it up. Worth investigating the phase ordering.
Minor observations (non-blocking):
- The assembly descriptor (
bin.xml) unconditionally includescamel.exefromtarget/. On non-Windows builds this is silently skipped (intended), so the enforcer profile is the safety net — which is fine. - The
camel-exemodule ordering intooling/pom.xmllooks correct (no downstream Java dependencies).
This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.
Claude Code on behalf of davsclaus
Apache Camel's Maven build now enforces JDK 21+ (enforce-java-version); the Windows launcher workflow provisioned JDK 17 and failed the enforcer. Bump the CI build JDK to 21. The launcher runtime requirement is unchanged (Java 17+). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…uncher Move camel.exe build and tests into tooling/camel-exe so Windows CI can validate the native bootstrap without building the full jbang plugin tree. camel-launcher now copies the exe artifact on Windows for assembly. Co-authored-by: Cursor <cursoragent@cursor.com>
Document why the native bootstrap lives in a standalone module and how it integrates with camel-launcher. Co-authored-by: Cursor <cursoragent@cursor.com>
dorny/paths-filter caused startup_failure on fork PRs (action not allowed). Use checkout + git diff instead; validated with actionlint. Co-authored-by: Cursor <cursoragent@cursor.com>
Install camel-buildtools before the targeted camel-exe verify job, and verify Unicode argument forwarding via a UTF-8 capture file instead of cmd.exe stdout which mangles non-ASCII on GitHub Actions runners. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
…ements - Fix preservesUnicodeArgument test: pass all args at once via %* instead of per-argument %~1 expansion, which corrupts non-ASCII characters through cmd.exe OEM code page conversion - Remove duplicate examples fileSet from launcher assembly descriptor - Merge stderr into stdout in test process helper to avoid pipe deadlock - Improve MAX_PATH error message in camel.c for debuggability - Tighten CI change detection to dsl/camel-jbang/camel-launcher/ to avoid unnecessary Windows builds for unrelated jbang submodule changes Co-authored-by: Cursor <cursoragent@cursor.com>
Keep purpose and architecture in the module README; trim the native README to MSVC setup, compiler flags, and release-gate details only. Co-authored-by: Cursor <cursoragent@cursor.com>
Add CamelLauncherWindowsExeIT to assert target/camel.exe is staged and bin/camel.exe is present in the Maven-assembled ZIP during release verify. Remove the camel-launcher-windows CI job; native bootstrap coverage stays in the camel-exe workflow. Co-authored-by: Cursor <cursoragent@cursor.com>
- Replace fixed MAX_PATH buffer with get_exe_dir() doubling loop for Windows 10+ long path support - Check _snwprintf_s and GetExitCodeProcess return values - Add failsGracefullyWhenCamelBatIsMissing test - Reduce CI fetch-depth from full clone to 50 commits - Add POM comment explaining manual test plugin wiring Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Renames the workflow, gates it on camel-launcher path changes too, and adds a job that builds the launcher with the native camel.exe and verifies bin/camel.exe is bundled in the release archive.
afc8e1c to
f81cda0
Compare
…ting JAR camel-launcher depends on camel-exe:exe purely so the assembly descriptor can stage bin/camel.exe; RepackageMojo.includeArtifact() only filtered by scope, so this non-jar dependency was getting embedded as a spurious library at BOOT-INF/lib/camel-exe-*.exe inside the self-executing jar. Only jar-type artifacts are valid Spring Boot loader libraries. Found while investigating the camel-launcher-windows CI failure reported on PR apache#24665 (missing bin/camel.exe in the packaged archive) - a separate, confirmed defect uncovered along the way, not the fix for that failure. Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
…dows PR apache#24665 review (davsclaus) reported the camel-launcher-windows job building successfully but the packaged archive missing bin/camel.exe. Investigation confirmed the dependency:copy -> enforcer -> assembly plumbing is correct: target/camel.exe exists on disk at prepare-package (enforcer passes), and an isolated reproduction of the exact Maven/ assembly-descriptor setup reliably produces the file in the archive outside this CI environment. The disappearance is therefore specific to the Windows runner, between the enforcer check and archive creation. Add a non-gating diagnostic step that checks whether target/camel.exe still exists post-build and inspects Windows Defender exclusions, threat detections, and the operational event log, to confirm or rule out AV quarantine of the freshly-compiled, unsigned executable before committing to a fix. Remove once the next CI run gives an answer. Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
The Defender diagnostic added previously ruled out AV quarantine: target/camel.exe was confirmed present and untouched immediately after the build, with zero Defender detections or exclusions. So whatever drops it happens inside assembly:single itself while writing the archive, not afterward. Add two more temporary diagnostic steps: - Full entry listing for both the zip and tar.gz (previously only the zip's presence/absence of a single entry was checked - this shows whether tar.gz is also missing it, and the actual entry path shape). - A standalone re-run of assembly:single with -X debug logging against the already-populated target/ dir (reusing the completed build's output rather than re-running the whole ~620-module reactor), to see what the DirectoryScanner/archiver actually did with the camel.exe fileSet. Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
camel-launcher's dependency closure is ~620 modules (it bundles camel-jbang-core plus several jbang plugins and kamelets), and the Windows job built them with plain sequential mvn (no -T), taking ~23 minutes for that step alone. Switch to mvnd (Apache's Maven daemon), which the project already uses for incremental PR builds on Linux, so the reactor builds in parallel instead of one module at a time. mvnd's windows-amd64 distribution is confirmed available from Apache downloads, but this is the first time this repo runs mvnd on a Windows runner: the build step moves from `shell: cmd` to `shell: bash` so the MSVC environment set by vcvars64.bat can be captured via `set` and re-exported for mvnd's daemon process, since cmd's `call` only affects the current cmd process. Not wired into the camel-exe job (its reactor is small enough that sequential mvn is not the bottleneck there). NOT YET VERIFIED on an actual Windows runner - intentionally held back from push until the current camel.exe / Windows Defender diagnostic investigation on this same job concludes, to avoid confounding that result with an unrelated build-tooling change. Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
The single quoted "call ... && set" string round-tripped through Git Bash's argv quoting (which backslash-escapes embedded quotes) and cmd.exe's /C parsing (which treats \" literally instead of unescaping it), corrupting the vcvars64.bat path and failing every run of this step on CI. Passing call/path/&&/set as separate argv elements avoids the mismatch since only the space-containing path needs quoting.
The mvnd wiring introduced Windows shell plumbing (bash env-porting of vcvars64 into mvnd) that kept failing on GHA Git Bash and, because the build never finished, blocked the temporary diagnostics that inspect the missing bin/camel.exe. mvnd is a build-speed optimization that is orthogonal to that packaging bug, so restore the known-good cmd + mvn step that built and packaged successfully, and revisit mvnd-on-Windows as a separate change.
…se dir maven-assembly-plugin's bin descriptor defaults includeBaseDirectory to true, so every entry is nested under camel-launcher-<version>/ (e.g. camel-launcher-4.22.0-SNAPSHOT/bin/camel.exe). The CI assertion checked for the literal string bin/camel.exe, which never matched that prefixed path, so it failed even though camel.exe was correctly present in every build. Also drops the temporary Defender/archive-listing/debug-assembly diagnostics added while chasing this false failure. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
davsclaus
left a comment
There was a problem hiding this comment.
Review: CAMEL-23703 — tooling/camel-exe - native camel.exe bootstrap for Windows
Well-structured PR. The module split (tooling/camel-exe separate from camel-launcher) is clean and well-justified. The C code is solid — proper error handling on all paths, Unicode support via wide-char APIs, and long-path support via a doubling-buffer loop.
Findings
[LOW] Potential path mismatch in CamelLauncherWindowsExeIT.binArchiveIncludesCamelExe() — The test uses archive.getEntry("bin/camel.exe") but the assembly descriptor (bin.xml) has no <includeBaseDirectory>false</includeBaseDirectory>, so Maven Assembly Plugin defaults to true — entries would be prefixed with camel-launcher-VERSION-bin/bin/camel.exe. The CI PowerShell assertion handles this correctly with wildcard */bin/camel.exe, but the Java test would return null from getEntry. This test is double-gated (@EnabledOnOs(OS.WINDOWS) + @EnabledIfSystemProperty), so it doesn't block CI — but a developer running mvn verify -Dcamel.launcher.requireWindowsExe=true on Windows could hit this.
Positive observations
- C code quality: Both AI-review comments from the earlier review (truncation risk, MAX_PATH) flagged issues that were already handled in the code.
- Tests: Good coverage — behavioral tests with a fake
camel.batcovering argument forwarding, Unicode preservation, exit code propagation, spaced paths, and graceful failure. - Repackager guard: The non-jar artifact filter in
RepackageMojoprevents the.exefrom being embedded inBOOT-INF/lib. Well-tested. - Conventions: Upgrade guide updated, license headers present, commit format correct.
This review does not replace specialized AI review tools (CodeRabbit, Sourcery) or static analysis (SonarCloud).
This review was generated by an AI agent (Claude Code on behalf of davsclaus) and may contain inaccuracies. Please verify all suggestions before applying.
| } | ||
|
|
||
| @Test | ||
| void binArchiveIncludesCamelExe() throws IOException { |
There was a problem hiding this comment.
[LOW] archive.getEntry("bin/camel.exe") uses an exact path, but the assembly descriptor (bin.xml) does not set <includeBaseDirectory>false</includeBaseDirectory> — Maven Assembly Plugin defaults to true, which prefixes entries with camel-launcher-VERSION-bin/. This means getEntry("bin/camel.exe") would return null.
The CI PowerShell assertion handles this correctly with wildcard matching (*/bin/camel.exe). Consider iterating entries with a suffix match here as well, or setting includeBaseDirectory to false in the assembly descriptor if that's the intended layout.
This test is double-gated (@EnabledOnOs(OS.WINDOWS) + @EnabledIfSystemProperty) so it doesn't affect CI, but would fail for a developer running mvn verify -Dcamel.launcher.requireWindowsExe=true on Windows.
There was a problem hiding this comment.
Good catch, same root cause as the CI assertion bug, fixed in f76de4f (suffix match instead of exact bin/camel.exe lookup, since includeBaseDirectory defaults to true in the assembly descriptor).
While digging into this I found a bigger gap worth flagging: the official release pipeline (Jenkinsfile.deploy) runs on ubuntu, and build-windows-exe / include-camel-exe only activate via <os><family>windows</family>. Neither camel.exe.requireWindowsExe nor camel.launcher.requireWindowsExe is set anywhere in that pipeline, so on an actual release the native compile is skipped, the camel.exe fileSet in bin.xml matches nothing, and the released camel-launcher-*-bin.zip ships without camel.exe, no error, just silently missing. That would break the Winget packaging from the other PR.
@davsclaus since you know the release process better than I do: is there already a place a Windows-hosted build step could hook into (e.g. a separate agent on the release Jenkins job), or should this be a post-release GitHub Actions job that builds the Windows artifact and uploads it alongside the rest of the release? Happy to put a PR together once there's a direction.
Claude Code on behalf of @ammachado
AI-generated review, so it may get things wrong. Please check any suggestion before you act on it.
archive.getEntry("bin/camel.exe") used an exact path, but the assembly
descriptor does not set includeBaseDirectory to false, so entries are
nested under camel-launcher-<version>/ and the lookup always returned
null. Match on a "/bin/camel.exe" suffix instead, mirroring the fix
already applied to the CI PowerShell assertion.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
gnodet
left a comment
There was a problem hiding this comment.
Good progress on the native camel.exe bootstrap. Previous review feedback has been addressed (truncation check on _snwprintf_s, get_exe_dir doubling loop for long paths, failsGracefullyWhenCamelBatIsMissing test). The C code is clean, the RepackageMojo fix is correct, and CI passes.
Two minor observations:
1. ZipFile handle leak in CI workflow (low)
In camel-launcher-windows.yml, the PowerShell assertion step chains:
$names = [System.IO.Compression.ZipFile]::OpenRead($zip.FullName).Entries.FullNameThe ZipArchive returned by OpenRead is IDisposable but is never assigned to a variable for disposal, leaking a file handle. Practical impact is minimal since this is the last CI step and the runner is ephemeral. Fix is trivial: assign to variable and wrap in try/finally with $archive.Dispose().
2. Potential test hang from blocking readAllBytes (low)
In CamelExeBootstrapTest.run(), p.getInputStream().readAllBytes() blocks until the process closes stdout. If camel.exe hangs, the 60-second waitFor timeout on the next line is never reached and the test thread hangs indefinitely. Additionally, the process is never destroyed on timeout failure (no finally block). Practical risk is low with the controlled fake bat scripts, but worth fixing defensively — consider reading the stream asynchronously or wrapping in assertTimeout.
This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.
Claude Code on behalf of @gnodet
Dispose the ZipArchive returned by ZipFile.OpenRead in the launcher CI workflow instead of leaking the handle, and drain camel.exe's stdout on a background thread in CamelExeBootstrapTest so the 60s waitFor timeout can actually fire (and the process gets destroyed) if the exe under test hangs, instead of blocking forever on readAllBytes waiting for the pipe to close. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Description
Slice of the larger CAMEL-23703 unified Camel CLI package distribution effort. This branch targets
main.It adds a native x64
bin/camel.exeto thecamel-launcherWindows distribution, alongsidebin/camel.bat.camel.exeis a thin bootstrap: it locates the adjacentcamel.bat, forwards all arguments verbatim (preserving spaces and Unicode), inherits the standard streams, and returns the child exit code. It performs no Java discovery or option parsing itself; that all remains incamel.bat.The executable exists so package managers that require a genuine executable command (such as WinGet's portable installer) can expose
cameldirectly. Direct users may keep invokingbin\camel.bat; both behave identically. Windows ARM64 is not yet supported.Module split
Native build logic lives in a new standalone module,
tooling/camel-exe, with no Java upstream dependencies:camel.csource, MSVC Maven profile, and attachedexeartifactCamelExeBootstrapTest(Windows-only behavioral tests)-Dcamel.exe.requireWindowsExe=truecamel-launcherconsumes thecamel-exe:exeartifact on Windows (copy intotarget/before assembly). User-visible distribution layout is unchanged:bin/camel.exestill ships insidecamel-launcher-*-bin.zip.Native bootstrap design
camel.exe(~130 lines of C) uses wide-char Windows APIs throughout:get_exe_dir()resolves the exe directory with a doubling-buffer loop, supporting paths beyondMAX_PATH(260) on Windows 10+ with long-path support enabledskip_argv0()stripsargv[0]from the raw command line, preserving the caller's quotingcmd.exe /S /C ""<dir>\camel.bat" <args>"to handle spaces and Unicode in paths_snwprintf_s,GetExitCodeProcess,CreateProcessW) are checked with diagnostics on failureCI
The Windows workflow (
.github/workflows/camel-launcher-windows.yml) uses path-filtered jobs with shallow clones:camel-exetooling/camel-exe/**mvn -pl tooling/camel-exe verify -Dcamel.exe.requireWindowsExe=trueThis avoids building the full jbang plugin tree on every native-bootstrap change.
Also included: upgrade-guide note in
camel-4x-upgrade-guide-4_22.adoc.Target
mainbranch)Tracking
Apache Camel coding standards and style
mvn clean install -DskipTestslocally from root folder and I have committed all auto-generated changes.AI-assisted contributions
Co-authored-bytrailers) and the PR description identifies the AI tool used.Claude Code on behalf of ammachado