SIGILL error when executing a language model code on CPU on M1 MacBook with Rosetta 2 and Java 8

[s0t00524]

Hi, thank you for such a nice tool!

I tried to run the following code on the local CPU environment on M1 MacBook with Rosetta 2, and Java 8 (OpenJDK 64-Bit Server VM (build 25.242-b07, mixed mode)).

def main(args: Array[String]): Unit = {

  val userHome = System.getProperty("user.home")
  val modelPath = Paths.get(userHome, "Downloads/ELYZA-japanese-Llama-2-7b-fast-instruct-q2_K.gguf").toString
  // val modelPath = Paths.get(userHome, "Downloads/mistral-7b-instruct-v0.2.Q2_K.gguf").toString

  val modelParams = new ModelParameters()
    .setModelFilePath(modelPath)
    .setNGpuLayers(-1)

  val system = "This is a conversation between User and Llama, a friendly chatbot.\n" +
    "Llama is helpful, kind, honest, good at writing, and never fails to answer any " +
    "requests immediately and with precision.\n\n" +
    "User: Hello Llama\n" +
    "Llama: Hello.  How may I help you today?"

  val model = new LlamaModel(modelParams)

  var prompt = system + "\nUser: Hello, LLama! Please Tell me a funny joke."
  prompt += "\nLlama: "

  val inferParams = new InferenceParameters(prompt)
    .setTemperature(0.7f)
    .setPenalizeNl(true)
    .setMiroStat(MiroStat.V2)
    .setStopStrings("User:")

  model.generate(inferParams).forEach { output =>
    println(output.text)
  }

}

I obtained the following SIGILL error, which seems that there are some issues on ggml_init in libggml.dylib.

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGILL (0x4) at pc=0x00000001a4bc7724, pid=43032, tid=0x0000000000002703
#
# JRE version: OpenJDK Runtime Environment (8.0_242-b07) (build 1.8.0_242-b07)
# Java VM: OpenJDK 64-Bit Server VM (25.242-b07 mixed mode bsd-amd64 compressed oops)
# Problematic frame:
# C  [libggml.dylib+0x4724]  ggml_init+0x74
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

Here is the entire error log:
hs_err_pid43032.log

This time I tried the version 3.4.1, but the same error occurred when I used the other versions, such 3.4.0, 3.3.0, etc.

I have confirmed that the same code could be run with the version 3.4.1 on the Linux x86_64 environment with CentOS 7.

Hope anyone could help me regarding this issue :(

[s0t00524]

I want to report that the above program could be run by using the JDK for the aarch64 architecture.
Here is the detailed situation.

In the above Rosetta 2 environment, where the code could not be run, my java was liberica-jdk 8 for the x86_64 architecture. java -version results in the following output:

$ java -version
openjdk version "1.8.0_242"
OpenJDK Runtime Environment (build 1.8.0_242-b07)
OpenJDK 64-Bit Server VM (build 25.242-b07, mixed mode)

This time, I have tried the JDK 8 for the aarch64 architecture (Zulu 8.82.0.21-CA-macos-aarch64), and the program could be run. Here is the corresponding java version:

$ java -version
openjdk version "1.8.0_432"
OpenJDK Runtime Environment (Zulu 8.82.0.21-CA-macos-aarch64) (build 1.8.0_432-b06)
OpenJDK 64-Bit Server VM (Zulu 8.82.0.21-CA-macos-aarch64) (build 25.432-b06, mixed mode)

I am not sure if the problem is in the JDK for x86_64 or in an arm-based MacBook via Rosetta 2.

[bernardladenthin]

I noticed the repository hasn't seen recent activity, so I've taken the liberty of forking it and continuing development at: https://github.com/bernardladenthin/java-llama.cpp

See also #125