More apple metal woes

[Lewiscowles1986]

So I checked out other issues, this is a different issue altogether to any I can find so far

M1 Ultra 128GB RAM

> hey how are you doing?
Failed to generate a response: error response: status=500 body=unable to load runner: error waiting for runner to be ready: vllm-metal terminated unexpectedly: vllm-metal failed: r pid=3812)     super().__init__(
(APIServer pid=3812)   File "$HOME/.docker/model-runner/vllm-metal/lib/python3.12/site-packages/vllm/v1/engine/core_client.py", line 535, in __init__
(APIServer pid=3812)     with launch_core_engines(
(APIServer pid=3812)          ^^^^^^^^^^^^^^^^^^^^
(APIServer pid=3812)   File "$HOME/.docker/model-runner/vllm-metal/lib/python3.12/contextlib.py", line 144, in __exit__
(APIServer pid=3812)     next(self.gen)
(APIServer pid=3812)   File "$HOME/.docker/model-runner/vllm-metal/lib/python3.12/site-packages/vllm/v1/engine/utils.py", line 998, in launch_core_engines
(APIServer pid=3812)     wait_for_engine_startup(
(APIServer pid=3812)   File "$HOME/.docker/model-runner/vllm-metal/lib/python3.12/site-packages/vllm/v1/engine/utils.py", line 1057, in wait_for_engine_startup
(APIServer pid=3812)     raise RuntimeError(
(APIServer pid=3812) RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}

I have replaced my home directory with $HOME; so you dont need to debug that.

[ericcurtin]

@Lewiscowles1986 could you tell me what model you tried to use?

error waiting for runner to be ready

makes me suggest this was a super large model attempted to be loaded. Just want to know which one it was, would help to reproduce.

[Lewiscowles1986]

Hi @ericcurtin it was https://huggingface.co/google/gemma-4-31B-it