Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .claude/skills/build-n-run-engine/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Build GPULlama3.java (this repo) with Maven, skipping tests for speed.

## Prerequisites

- `JAVA_HOME` set to JDK 21 or 25 (`java -version`)
- `JAVA_HOME` set to JDK 21, or JDK 22+ (22, 23, 24, 25, 26, 27, ...) (`java -version`)
- `TORNADOVM_HOME` set and `tornado --devices` succeeds — if not, run the `build-tornado` skill first
- `~/TornadoVM/setvars.sh` sourced in **this** shell (env vars don't persist across shells/tool calls)

Expand Down Expand Up @@ -51,7 +51,7 @@ git checkout <branch> && git pull
mvn clean install -DskipTests
```

For JDK 25 instead of the default JDK 21, ensure `JAVA_HOME` points at JDK 25 before running `make` — the pom auto-activates the `jdk25` profile from the detected JDK version, there is no separate `BACKEND=`-style flag.
For JDK 22 and newer (22, 23, 24, 25, 26, 27, ...) instead of the default JDK 21, ensure `JAVA_HOME` points at that JDK before building — the pom auto-activates the `jdk22plus` profile (activation range `[22,)`) from the detected JDK version, there is no separate `BACKEND=`-style flag. `jdk22plus` replaced the former per-version `jdk25`/`jdk26`/`jdk27` profiles with one consolidated profile that pins `tornadovm.version` to the locally-installed `5.2.1-jdk22plus-dev` artifact and compiles without `--enable-preview` (unnecessary from release 22 onward; only `--add-modules jdk.incubator.vector` is needed).

### Step 5: Verify

Expand Down
7 changes: 6 additions & 1 deletion .claude/skills/build-tornado/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,14 @@ Build TornadoVM from source. Available backends: `opencl` (default), `ptx`, `spi

## Prerequisites

- JAVA_HOME is set to jdk 21 or 25
- JAVA_HOME is set to jdk 21, or 22+ (22, 23, 24, 25, 26, 27, ...)
- `nvidia-smi` succeeds (GPUs visible)

Note: JDK 27 removed JVMCI entirely (openjdk/jdk#30834). TornadoVM's `jdk27` pom profile
vendors a same-named `jdk.internal.vm.ci` application module to compensate — see the
`jdk27-jvmci-removal` branch. `bin/compile --jdk jdk27` (and `make`) auto-detect this from
`JAVA_HOME`.

## Instructions

### Step 1: Verify Environment
Expand Down
11 changes: 10 additions & 1 deletion .github/workflows/deploy-maven-central.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,17 @@ jobs:
jdk:
- name: jdk21
java_version: 21.0.2-open
- name: jdk25
- name: jdk22plus
java_version: 25.0.2-open
# A single jdk22plus matrix entry now covers JDK 22 through 27+: the pom's
# jdk22plus profile (which replaced the former separate jdk25/jdk26/jdk27
# profiles) publishes one gpu-llama3:*-jdk22plus artifact that every JDK in
# that range resolves, mirroring TornadoVM's own consolidated jdk22plus SDK.
# Building it on 25.0.2-open is just this job's pick of JDK within the range,
# not a jdk25-specific artifact - no separate jdk26/jdk27 entries are needed.
# TornadoVM has not yet published GA 5.2.1-jdk22plus to Maven Central (only
# the "-dev" line exists today), so this entry will fail dependency resolution
# until that GA release lands - same caveat the old jdk26/jdk27 note carried.

steps:
- name: Checkout code
Expand Down
10 changes: 5 additions & 5 deletions .github/workflows/prepare-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ jobs:

SNIPPET_FILE="${{ runner.temp }}/dependency_snippet.md"
cat > "$SNIPPET_FILE" <<EOF
**JDK 21** (\`jdk21\` profile, auto-activates for JDK \`[21,25)\`):
**JDK 21** (\`jdk21\` profile, auto-activates for JDK \`[21,22)\`):
\`\`\`xml
<dependency>
<groupId>io.github.beehive-lab</groupId>
Expand All @@ -101,12 +101,12 @@ jobs:
</dependency>
\`\`\`

**JDK 25** (\`jdk25\` profile, auto-activates for JDK \`[25.0.2,)\`):
**JDK 22+** (\`jdk22plus\` profile, auto-activates for JDK \`[22,)\` — covers 22, 23, 24, 25, 26, 27 and later):
\`\`\`xml
<dependency>
<groupId>io.github.beehive-lab</groupId>
<artifactId>gpu-llama3</artifactId>
<version>${VERSION}-jdk25</version>
<version>${VERSION}-jdk22plus</version>
</dependency>
\`\`\`

Expand All @@ -117,9 +117,9 @@ jobs:
implementation 'io.github.beehive-lab:gpu-llama3:${VERSION}-jdk21'
\`\`\`

**JDK 25**:
**JDK 22+**:
\`\`\`groovy
implementation 'io.github.beehive-lab:gpu-llama3:${VERSION}-jdk25'
implementation 'io.github.beehive-lab:gpu-llama3:${VERSION}-jdk22plus'
\`\`\`
EOF

Expand Down
14 changes: 7 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
[![build JDK21](https://github.com/beehive-lab/GPULlama3.java/actions/workflows/build-and-run.yml/badge.svg)](https://github.com/beehive-lab/GPULlama3.java/actions/workflows/build-and-run.yml)
[![Maven Central](https://img.shields.io/maven-central/v/io.github.beehive-lab/gpu-llama3?&logo=apache-maven&color=blue)](https://central.sonatype.com/artifact/io.github.beehive-lab/gpu-llama3)
![Java 21](https://img.shields.io/badge/java-21-blue?logo=openjdk)
![Java 25](https://img.shields.io/badge/java-25-yellow?logo=openjdk)
![Java 22+](https://img.shields.io/badge/java-22%2B-yellow?logo=openjdk)
[![LangChain4j](https://img.shields.io/badge/LangChain4j-1.7.1+-purple?&logo=link&logoColor=white)](https://docs.langchain4j.dev/)
![NVIDIA](https://img.shields.io/badge/CUDA%20%7C%20PTX-supported-76B900?logo=nvidia)
![OpenCL](https://img.shields.io/badge/OpenCL-supported-blue?logo=khronos)
Expand Down Expand Up @@ -94,7 +94,7 @@ GPULlama3ChatModel model = GPULlama3ChatModel.builder()
### 📦 Maven

<!-- DEPENDENCY-SNIPPETS:START -->
**JDK 21** (`jdk21` profile, auto-activates for JDK `[21,25)`):
**JDK 21** (`jdk21` profile, auto-activates for JDK `[21,22)`):
```xml
<dependency>
<groupId>io.github.beehive-lab</groupId>
Expand All @@ -103,12 +103,12 @@ GPULlama3ChatModel model = GPULlama3ChatModel.builder()
</dependency>
```

**JDK 25** (`jdk25` profile, auto-activates for JDK `[25.0.2,)`):
**JDK 22+** (`jdk22plus` profile, auto-activates for JDK `[22,)` — covers 22, 23, 24, 25, 26, 27 and later):
```xml
<dependency>
<groupId>io.github.beehive-lab</groupId>
<artifactId>gpu-llama3</artifactId>
<version>1.0.0-jdk25</version>
<version>1.0.0-jdk22plus</version>
</dependency>
```

Expand All @@ -119,9 +119,9 @@ GPULlama3ChatModel model = GPULlama3ChatModel.builder()
implementation 'io.github.beehive-lab:gpu-llama3:1.0.0-jdk21'
```

**JDK 25**:
**JDK 22+**:
```groovy
implementation 'io.github.beehive-lab:gpu-llama3:1.0.0-jdk25'
implementation 'io.github.beehive-lab:gpu-llama3:1.0.0-jdk22plus'
```
<!-- DEPENDENCY-SNIPPETS:END -->

Expand All @@ -137,7 +137,7 @@ implementation 'io.github.beehive-lab:gpu-llama3:1.0.0-jdk25'

### Prerequisites

- **Java 21** — required for the Vector API & TornadoVM (Java 25 supported via the `-jdk25` artifact / `llamaTornado` script).
- **Java 21+** — required for the Vector API & TornadoVM. JDK 21 uses the `-jdk21` artifact; JDK 22 and newer (22, 23, 24, 25, 26, 27, ...) uses the `-jdk22plus` artifact / `llamaTornado` script.
- **[TornadoVM](https://github.com/beehive-lab/TornadoVM)** with an OpenCL, PTX, CUDA, or Metal backend. `llama-tornado`/`llamaTornado` auto-detect whichever backend your installed SDK was built with.
- **GCC/G++ 13+** — to build TornadoVM's native components.

Expand Down
77 changes: 69 additions & 8 deletions llama-tornado
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ The backend is auto-detected from the installed TornadoVM SDK
import argparse
import glob
import os
import re
import subprocess
import sys
import time
Expand Down Expand Up @@ -80,6 +81,21 @@ class LlamaRunner:
print("Note: check set_path in root dir -> source set_path")
sys.exit(1)

self.java_version = self._detect_java_version()

def _detect_java_version(self) -> Optional[int]:
"""Parse the JDK feature/major version out of `$JAVA_HOME/bin/java -version`,
e.g. 'openjdk version "21.0.2" ...' -> 21, 'openjdk version "27-ea" ...' -> 27."""
try:
output = subprocess.run(
[f"{self.java_home}/bin/java", "-version"],
capture_output=True, text=True, check=True,
).stderr
except (OSError, subprocess.CalledProcessError):
return None
match = re.search(r'version "(\d+)', output)
return int(match.group(1)) if match else None

def _validate_paths(self):
"""Validate that required paths exist."""
paths_to_check = {
Expand Down Expand Up @@ -134,19 +150,49 @@ class LlamaRunner:

def _build_base_command(self, args: argparse.Namespace) -> List[str]:
"""Build the base Java command with JVM options."""
# JDK 27+ removed the platform jdk.internal.vm.ci module entirely; TornadoVM ships a
# vendored same-named module for that case, which must go on the module-path (mirrors
# tornado.py's self.jvmci_absent). On JDK <=26 the platform module is still present and
# a same-named module on the module-path would cause a "two versions of module" error,
# so it's deliberately left off here -- see the --patch-module handling below instead.
jvmci_absent = self.java_version is not None and self.java_version >= 27
module_path_entries = [".", f"{self.tornado_sdk}/share/java/tornado"]
if jvmci_absent:
module_path_entries.append(f"{self.tornado_sdk}/share/java/jvmci")

cmd = [
f"{self.java_home}/bin/java",
"-server",
"-XX:+UnlockExperimentalVMOptions",
"-XX:+EnableJVMCI",
f"-Xms{args.heap_min}",
f"-Xmx{args.heap_max}",
"--enable-preview",
f"-Djava.library.path={self.tornado_sdk}/lib",
"-Djdk.module.showModuleResolution=false",
"--module-path",
self.module_path_colon_sep([".", f"{self.tornado_sdk}/share/java/tornado"]),
]
if jvmci_absent:
# JDK 27+ removed JVMCI entirely (openjdk/jdk#30834): -XX:+EnableJVMCI is now an
# unrecognized (fatal) VM option, and Panama/FFM is final so --enable-preview is
# unnecessary. Mirrors tornado.py's __JAVA_BASE_OPTIONS_NO_JVMCI__: set the saved
# property the vendored jdk.vm.ci.services.Services.checkJVMCIEnabled() reads
# (HotSpot's own +EnableJVMCI bookkeeping no longer exists to set it), and export the
# java.base internals the vendored jdk.internal.vm.ci module needs.
cmd.extend(
[
"-Djdk.internal.vm.ci.enabled=true",
"--add-exports", "java.base/jdk.internal.misc=jdk.internal.vm.ci",
"--add-exports", "java.base/jdk.internal.vm=jdk.internal.vm.ci",
"--add-exports", "java.base/jdk.internal.vm.annotation=jdk.internal.vm.ci",
"--add-exports", "java.base/jdk.internal.reflect=jdk.internal.vm.ci",
]
)
else:
cmd.extend(["-XX:+EnableJVMCI", "--enable-preview"])
cmd.extend(
[
f"-Xms{args.heap_min}",
f"-Xmx{args.heap_max}",
f"-Djava.library.path={self.tornado_sdk}/lib",
"-Djdk.module.showModuleResolution=false",
"--module-path",
self.module_path_colon_sep(module_path_entries),
]
)

# TornadoVM configuration
tornado_config = [
Expand Down Expand Up @@ -266,6 +312,21 @@ class LlamaRunner:
add_modules.append(BACKEND_MODULE_NAME[backend])
module_config.extend(["--add-modules", ",".join(add_modules)])

# JDK 22-26 still ship jdk.internal.vm.ci, but its jdk.vm.ci.* interfaces have drifted
# from the JDK-21 shape the reflection providers were compiled against (e.g.
# jdk.vm.ci.code.Architecture's constructor). Overlay the frozen JDK-21 classes so the
# loaded jdk.vm.ci.* matches what was compiled -- mirrors tornado.py's jvmci_patched
# branch, which `tornado --printJavaFlags` applies but this script builds its command
# independently of, so it has to be replicated here too.
if self.java_version is not None and 22 <= self.java_version <= 26:
module_config.extend(
[
"-Djdk.internal.vm.ci.enabled=true",
"--patch-module",
f"jdk.internal.vm.ci={self.tornado_sdk}/share/java/jvmci/jvmci-21.0.2.jar",
]
)

if getattr(args, "server", False):
main_class = "org.beehive.gpullama3.server.OpenAIServer"
elif getattr(args, "bench", False):
Expand Down
Loading
Loading