Unable to remove model using the inference-stack-deploy.sh script "Remove Model using deployment name" menu option . The removal process runs and reports success, but the model pod remains running and the deployment installed.
See the full process below using the qwen3-1.7 model as an example, starting from deploying the model:
user@master1:~/Enterprise-Inference/core$ ./inference-stack-deploy.sh
----------------------------------------------------------
| Intel AI for Enterprise Inference |
|---------------------------------------------------------|
| 1) Provision Enterprise Inference Cluster |
| 2) Decommission Existing Cluster |
| 3) Update Deployed Inference Cluster |
| 4) Brownfield Deployment of Enterprise Inference |
|---------------------------------------------------------|
Please choose an option (1, 2, 3 or 4):
> 3
-------------------------------------------------
| Update Existing Cluster |
|------------------------------------------------|
| 1) Manage Worker Nodes |
| 2) Manage LLM Models |
|------------------------------------------------|
Please choose an option (1 or 2):
> 2
-------------------------------------------------
| Manage LLM Models
|------------------------------------------------|
| 1) Deploy Model |
| 2) Undeploy Model |
| 3) List Installed Models |
| 4) Deploy Model from Hugging Face |
| 5) Remove Model using deployment name |
|------------------------------------------------|
Please choose an option (1, 2, 3, or 4):
> 1
Configuration file found, setting vars!
---------------------------------------
Metadata configuration file found, setting vars!
---------------------------------------
Proceeding with the setup of Fresh Kubernetes cluster: yes
Proceeding with the setup of Habana AI Operator: no
Proceeding with the setup of Ingress Controller: yes
Proceeding with the setup of Keycloak : yes
Proceeding with the setup of Apisix: yes
Proceeding with the setup of GenAI Gateway: no
Proceeding with the setup of Observability: no
Proceeding with the setup of Ceph cluster: no
Proceeding with the setup of Istio: no
NRI CPU Balloon Policy automatically enabled for CPU deployment
Using provided Huggingface token
Proceeding with the setup of Large Language Model (LLM): yes
Available Models for CPU Deployment:
21. meta-llama/Llama-3.1-8B-Instruct
22. meta-llama/Llama-3.2-3B-Instruct
23. deepseek-ai/DeepSeek-R1-Distill-Llama-8B
24. deepseek-ai/DeepSeek-R1-Distill-Qwen-32B
25. Qwen/Qwen3-1.7B
26. Qwen/Qwen3-4B-Instruct-2507
Enter the number of the CPU model you want to deploy/remove: 25
Deploying/removing CPU models: cpu-qwen3-1-7b
----- Input -----
Using provided CLUSTER URL: api.example.com
Using provided certificate file: /home/user/certs/cert.pem
Using provided key file: /home/user/certs/key.pem
Using provided keycloak client id: my-client-id
Using provided Keycloak admin username: your-keycloak-admin-user
Using provided Keycloak admin password
cpu_or_gpu is already set to c
Deploying models: cpu-qwen3-1-7b
NOTICE: You are initiating a model deployment. This will create the required services. Do you wish to continue? (y/n) y
Inference LLM Model is deployed successfully.
-------------------------------------------------------------------------------------
| AI LLM Model Deployment Complete! |
| The model is transitioning to a state ready for Inference. |
| This may take some time depending on system resources and other factors. |
| Please standby... |
--------------------------------------------------------------------------------------
Accessing Deployed Models for Inference
https://github.com/opea-project/Enterprise-Inference/blob/main/docs/accessing-deployed-models.md
Please refer to this comprehensive guide for detailed instructions.
user@master1:~/Enterprise-Inference/core$ kubectl get pods
NAME READY STATUS RESTARTS AGE
keycloak-0 1/1 Running 0 63m
keycloak-postgresql-0 1/1 Running 0 63m
keycloak-realm-setup-models-jll2t 0/1 Completed 0 4m50s
vllm-llama-3-2-3b-cpu-5f8cdf57c-25dq6 1/1 Running 0 57m
vllm-qwen-3-1-7b-cpu-549b9bd774-f2cmm 1/1 Running 0 4m15s
user@master1:~/Enterprise-Inference/core$ kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
vllm-llama-3-2-3b-cpu 1/1 1 1 57m
vllm-qwen-3-1-7b-cpu 1/1 1 1 4m45s
user@master1:~/Enterprise-Inference/core$ ./inference-stack-deploy.sh
----------------------------------------------------------
| Intel AI for Enterprise Inference |
|---------------------------------------------------------|
| 1) Provision Enterprise Inference Cluster |
| 2) Decommission Existing Cluster |
| 3) Update Deployed Inference Cluster |
| 4) Brownfield Deployment of Enterprise Inference |
|---------------------------------------------------------|
Please choose an option (1, 2, 3 or 4):
> 3
-------------------------------------------------
| Update Existing Cluster |
|------------------------------------------------|
| 1) Manage Worker Nodes |
| 2) Manage LLM Models |
|------------------------------------------------|
Please choose an option (1 or 2):
> 2
-------------------------------------------------
| Manage LLM Models
|------------------------------------------------|
| 1) Deploy Model |
| 2) Undeploy Model |
| 3) List Installed Models |
| 4) Deploy Model from Hugging Face |
| 5) Remove Model using deployment name |
|------------------------------------------------|
Please choose an option (1, 2, 3, or 4):
> 5
-------------------------------------------------
| Removing Model using Deployment name |
|------------------------------------------------|
Configuration file found, setting vars!
---------------------------------------
Metadata configuration file found, setting vars!
---------------------------------------
Proceeding with the setup of Fresh Kubernetes cluster: yes
Proceeding with the setup of Habana AI Operator: no
Proceeding with the setup of Ingress Controller: yes
Proceeding with the setup of Keycloak : yes
Proceeding with the setup of Apisix: yes
Proceeding with the setup of GenAI Gateway: no
Proceeding with the setup of Observability: no
Proceeding with the setup of Ceph cluster: no
Proceeding with the setup of Istio: no
NRI CPU Balloon Policy automatically enabled for CPU deployment
Using provided Huggingface token
Proceeding with the setup of Large Language Model (LLM): yes
----- Input -----
Using provided CLUSTER URL: api.example.com
Using provided certificate file: /home/user/certs/cert.pem
Using provided key file: /home/user/certs/key.pem
Using provided keycloak client id: my-client-id
Using provided Keycloak admin username: your-keycloak-admin-user
Using provided Keycloak admin password
cpu_or_gpu is already set to c
Some required arguments are missing. Prompting for input...
Proceeding with the setup of Fresh Kubernetes cluster: yes
Proceeding with the setup of Habana AI Operator: no
Proceeding with the setup of Ingress Controller: yes
Proceeding with the setup of Keycloak : yes
Proceeding with the setup of Apisix: yes
Proceeding with the setup of GenAI Gateway: no
Proceeding with the setup of Observability: no
Proceeding with the setup of Ceph cluster: no
Proceeding with the setup of Istio: no
Proceeding with the setup of NRI CPU Balloon Policy: yes
Using provided Huggingface token
Proceeding with the setup of Large Language Model (LLM): yes
----- Input -----
Using provided CLUSTER URL: api.example.com
Using provided certificate file: /home/user/certs/cert.pem
Using provided key file: /home/user/certs/key.pem
Using provided keycloak client id: my-client-id
Using provided Keycloak admin username: your-keycloak-admin-user
Using provided Keycloak admin password
cpu_or_gpu is already set to c
CAUTION: Removing the Inference LLM Model will also remove its associated services and resources, which may cause service downtime and potential data loss. This action is irreversible. Are you absolutely certain you want to proceed? (y/n) y
Enter the deployment name of the model you wish to deprovision: vllm-qwen-3-1-7b-cpu
Configuration file found, setting vars!
---------------------------------------
Metadata configuration file found, setting vars!
---------------------------------------
Setting up the Initial Environment...
Skipping system prerequisites check due to --skip-check argument.
Kubespray directory already exists, skipping clone.
Virtual environment already exists within Kubespray directory, skipping creation.
Attempting to activate the virtual environment...
Virtual environment activated successfully. Path: /home/user/Enterprise-Inference/core/kubespray/venv
Requirement already satisfied: pip in ./venv/lib/python3.10/site-packages (26.1.1)
Requirement already satisfied: ansible==9.13.0 in ./venv/lib/python3.10/site-packages (from -r requirements.txt (line 1)) (9.13.0)
Requirement already satisfied: cryptography==44.0.0 in ./venv/lib/python3.10/site-packages (from -r requirements.txt (line 3)) (44.0.0)
Requirement already satisfied: jmespath==1.0.1 in ./venv/lib/python3.10/site-packages (from -r requirements.txt (line 5)) (1.0.1)
Requirement already satisfied: netaddr==1.3.0 in ./venv/lib/python3.10/site-packages (from -r requirements.txt (line 7)) (1.3.0)
Requirement already satisfied: ansible-core~=2.16.14 in ./venv/lib/python3.10/site-packages (from ansible==9.13.0->-r requirements.txt (line 1)) (2.16.18)
Requirement already satisfied: cffi>=1.12 in ./venv/lib/python3.10/site-packages (from cryptography==44.0.0->-r requirements.txt (line 3)) (2.0.0)
Requirement already satisfied: jinja2>=3.0.0 in ./venv/lib/python3.10/site-packages (from ansible-core~=2.16.14->ansible==9.13.0->-r requirements.txt (line 1)) (3.1.6)
Requirement already satisfied: PyYAML>=5.1 in ./venv/lib/python3.10/site-packages (from ansible-core~=2.16.14->ansible==9.13.0->-r requirements.txt (line 1)) (6.0.3)
Requirement already satisfied: packaging in ./venv/lib/python3.10/site-packages (from ansible-core~=2.16.14->ansible==9.13.0->-r requirements.txt (line 1)) (26.2)
Requirement already satisfied: resolvelib<1.1.0,>=0.5.3 in ./venv/lib/python3.10/site-packages (from ansible-core~=2.16.14->ansible==9.13.0->-r requirements.txt (line 1)) (1.0.1)
Requirement already satisfied: pycparser in ./venv/lib/python3.10/site-packages (from cffi>=1.12->cryptography==44.0.0->-r requirements.txt (line 3)) (3.0)
Requirement already satisfied: MarkupSafe>=2.0 in ./venv/lib/python3.10/site-packages (from jinja2>=3.0.0->ansible-core~=2.16.14->ansible==9.13.0->-r requirements.txt (line 1)) (3.0.3)
Verifying Ansible Installation...
Ansible installed successfully
Enterprise Inference requirements installed.
Checking vault.yml for mandatory keys...
vault.yml exists and contains all mandatory keys. Skipping generation...
Additional files and directories copied to Kubespray directory.
Skipping infrastructure readiness check due to --skip-check argument.
Infrastructure readiness check completed successfully.
Starting galaxy collection install process
Nothing to do. All requested collections are already installed. If you want to reinstall them, consider using `--force`.
Removing Inference LLM Models...
Removing Inference LLM Models playbook...
Uninstalling the models...
[WARNING]: While constructing a mapping from /home/user/Enterprise-Inference/core/kubespray/playbooks/deploy-inference-models.yml, line 1963, column 11, found a duplicate dict key
(ignore_errors). Using last defined value only.
[WARNING]: While constructing a mapping from /home/user/Enterprise-Inference/core/kubespray/playbooks/deploy-inference-models.yml, line 1972, column 11, found a duplicate dict key
(ignore_errors). Using last defined value only.
[WARNING]: Skipping callback plugin 'ara_default', unable to load
PLAY [Deploy/Remove Inference LLM Models] ***************************************************************************************************************************************************
Wednesday 27 May 2026 15:51:09 -0500 (0:00:00.049) 0:00:00.049 *********
TASK [kubernetes-precheck : Check if kubectl is Available] **********************************************************************************************************************************
ok: [master1]
Wednesday 27 May 2026 15:51:10 -0500 (0:00:00.211) 0:00:00.260 *********
Wednesday 27 May 2026 15:51:10 -0500 (0:00:00.018) 0:00:00.279 *********
TASK [kubernetes-precheck : Check Kubernetes API server connectivity] ***********************************************************************************************************************
ok: [master1]
Wednesday 27 May 2026 15:51:10 -0500 (0:00:00.161) 0:00:00.441 *********
Wednesday 27 May 2026 15:51:10 -0500 (0:00:00.027) 0:00:00.468 *********
TASK [inference-tools : Ensure Python pip module is installed] ******************************************************************************************************************************
ok: [master1]
Wednesday 27 May 2026 15:51:11 -0500 (0:00:00.648) 0:00:01.117 *********
TASK [inference-tools : Install Kubernetes Python SDK] **************************************************************************************************************************************
ok: [master1]
Wednesday 27 May 2026 15:51:11 -0500 (0:00:00.682) 0:00:01.800 *********
TASK [inference-tools : Install Deployment Client] ******************************************************************************************************************************************
changed: [master1]
Wednesday 27 May 2026 15:51:12 -0500 (0:00:00.558) 0:00:02.359 *********
TASK [inference-tools : Ensure jq is installed] *********************************************************************************************************************************************
ok: [master1]
Wednesday 27 May 2026 15:51:13 -0500 (0:00:01.673) 0:00:04.032 *********
TASK [Print active tags for this task] ******************************************************************************************************************************************************
ok: [master1] => {
"ansible_run_tags": [
"uninstall-vllm-qwen-3-1-7b-cpu-cpu"
]
}
Wednesday 27 May 2026 15:51:13 -0500 (0:00:00.021) 0:00:04.054 *********
TASK [Display deployment configuration] *****************************************************************************************************************************************************
ok: [master1] => {
"msg": [
"===================================================================",
"Inference Model Deployment Configuration",
"===================================================================",
"CPU Deployment: false",
"GPU Deployment: false",
"NRI CPU Balloons: DISABLED",
"Models to Deploy: "
]
}
Wednesday 27 May 2026 15:51:14 -0500 (0:00:00.027) 0:00:04.081 *********
Wednesday 27 May 2026 15:51:14 -0500 (0:00:00.017) 0:00:04.099 *********
TASK [Create/Update Kubernetes Secret for Hugging Face Token] *******************************************************************************************************************************
ok: [master1]
Wednesday 27 May 2026 15:51:14 -0500 (0:00:00.578) 0:00:04.677 *********
TASK [Ensure Remote Directory Exists] *******************************************************************************************************************************************************
changed: [master1]
Wednesday 27 May 2026 15:51:14 -0500 (0:00:00.169) 0:00:04.846 *********
TASK [Sync dependency files to Deployment Nodes] ********************************************************************************************************************************************
changed: [master1] => (item={'src': '/home/user/Enterprise-Inference/core/kubespray/helm-charts', 'dest': '/tmp/helm-charts/'})
Wednesday 27 May 2026 15:51:15 -0500 (0:00:00.233) 0:00:05.080 *********
Wednesday 27 May 2026 15:51:15 -0500 (0:00:00.017) 0:00:05.098 *********
Wednesday 27 May 2026 15:51:15 -0500 (0:00:00.015) 0:00:05.114 *********
TASK [Remove EKS ingress template for VLLM when not on EKS] *********************************************************************************************************************************
changed: [master1]
Wednesday 27 May 2026 15:51:15 -0500 (0:00:00.128) 0:00:05.242 *********
Wednesday 27 May 2026 15:51:15 -0500 (0:00:00.018) 0:00:05.261 *********
Wednesday 27 May 2026 15:51:15 -0500 (0:00:00.021) 0:00:05.282 *********
TASK [Remove EKS ingress template for TEI when not on EKS] **********************************************************************************************************************************
changed: [master1]
Wednesday 27 May 2026 15:51:15 -0500 (0:00:00.125) 0:00:05.408 *********
Wednesday 27 May 2026 15:51:15 -0500 (0:00:00.017) 0:00:05.426 *********
Wednesday 27 May 2026 15:51:15 -0500 (0:00:00.017) 0:00:05.443 *********
TASK [Remove EKS ingress template for TEI Rerank when not on EKS] ***************************************************************************************************************************
changed: [master1]
Wednesday 27 May 2026 15:51:15 -0500 (0:00:00.124) 0:00:05.568 *********
Wednesday 27 May 2026 15:51:15 -0500 (0:00:00.022) 0:00:05.590 *********
Wednesday 27 May 2026 15:51:15 -0500 (0:00:00.019) 0:00:05.610 *********
TASK [Remove EKS ingress template for TGI when not on EKS] **********************************************************************************************************************************
changed: [master1]
Wednesday 27 May 2026 15:51:15 -0500 (0:00:00.162) 0:00:05.772 *********
Wednesday 27 May 2026 15:51:15 -0500 (0:00:00.015) 0:00:05.788 *********
Wednesday 27 May 2026 15:51:15 -0500 (0:00:00.016) 0:00:05.804 *********
Wednesday 27 May 2026 15:51:15 -0500 (0:00:00.015) 0:00:05.820 *********
Wednesday 27 May 2026 15:51:15 -0500 (0:00:00.015) 0:00:05.836 *********
Wednesday 27 May 2026 15:51:15 -0500 (0:00:00.015) 0:00:05.851 *********
Wednesday 27 May 2026 15:51:15 -0500 (0:00:00.015) 0:00:05.867 *********
Wednesday 27 May 2026 15:51:15 -0500 (0:00:00.015) 0:00:05.883 *********
Wednesday 27 May 2026 15:51:15 -0500 (0:00:00.016) 0:00:05.899 *********
Wednesday 27 May 2026 15:51:15 -0500 (0:00:00.015) 0:00:05.915 *********
Wednesday 27 May 2026 15:51:15 -0500 (0:00:00.016) 0:00:05.931 *********
Wednesday 27 May 2026 15:51:15 -0500 (0:00:00.023) 0:00:05.955 *********
Wednesday 27 May 2026 15:51:15 -0500 (0:00:00.021) 0:00:05.976 *********
Wednesday 27 May 2026 15:51:15 -0500 (0:00:00.017) 0:00:05.994 *********
Wednesday 27 May 2026 15:51:15 -0500 (0:00:00.016) 0:00:06.011 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.059) 0:00:06.070 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.024) 0:00:06.095 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.019) 0:00:06.114 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.017) 0:00:06.132 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.016) 0:00:06.149 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.016) 0:00:06.166 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.016) 0:00:06.182 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.019) 0:00:06.201 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.017) 0:00:06.219 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.016) 0:00:06.235 *********
TASK [env_proxy Contents] *******************************************************************************************************************************************************************
ok: [master1] => {
"env_proxy": {
"http_proxy": "",
"https_proxy": "",
"no_proxy": ""
}
}
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.021) 0:00:06.257 *********
TASK [Set proxy args if proxy is defined] ***************************************************************************************************************************************************
ok: [master1]
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.032) 0:00:06.289 *********
TASK [Print model_name_list] ****************************************************************************************************************************************************************
ok: [master1] => {
"model_name_list": ""
}
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.021) 0:00:06.311 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.022) 0:00:06.333 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.020) 0:00:06.354 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.062) 0:00:06.416 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.020) 0:00:06.437 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.017) 0:00:06.454 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.016) 0:00:06.471 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.015) 0:00:06.487 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.016) 0:00:06.503 *********
TASK [Check if "vllm-qwen-3-1-7b-cpu-cpu" Model is deployed] ********************************************************************************************************************************
changed: [master1]
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.171) 0:00:06.674 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.018) 0:00:06.693 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.017) 0:00:06.710 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.016) 0:00:06.726 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.015) 0:00:06.742 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.016) 0:00:06.758 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.016) 0:00:06.774 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.016) 0:00:06.790 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.015) 0:00:06.806 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.049) 0:00:06.855 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.016) 0:00:06.871 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.015) 0:00:06.887 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.015) 0:00:06.902 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.022) 0:00:06.924 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.022) 0:00:06.947 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.027) 0:00:06.974 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.021) 0:00:06.996 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.020) 0:00:07.016 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.021) 0:00:07.038 *********
Wednesday 27 May 2026 15:51:16 -0500 (0:00:00.017) 0:00:07.056 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.021) 0:00:07.077 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.018) 0:00:07.095 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.017) 0:00:07.112 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.050) 0:00:07.162 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.015) 0:00:07.178 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.016) 0:00:07.194 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.016) 0:00:07.210 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.015) 0:00:07.226 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.021) 0:00:07.248 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.018) 0:00:07.266 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.016) 0:00:07.283 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.016) 0:00:07.299 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.015) 0:00:07.314 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.016) 0:00:07.330 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.021) 0:00:07.352 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.018) 0:00:07.370 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.016) 0:00:07.386 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.016) 0:00:07.403 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.045) 0:00:07.448 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.015) 0:00:07.463 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.016) 0:00:07.480 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.015) 0:00:07.496 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.016) 0:00:07.512 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.016) 0:00:07.529 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.016) 0:00:07.545 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.017) 0:00:07.563 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.017) 0:00:07.581 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.016) 0:00:07.597 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.016) 0:00:07.614 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.016) 0:00:07.631 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.015) 0:00:07.647 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.015) 0:00:07.663 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.015) 0:00:07.679 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.044) 0:00:07.723 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.015) 0:00:07.739 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.015) 0:00:07.754 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.016) 0:00:07.771 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.015) 0:00:07.787 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.026) 0:00:07.813 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.020) 0:00:07.834 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.019) 0:00:07.854 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.020) 0:00:07.874 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.016) 0:00:07.890 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.015) 0:00:07.906 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.015) 0:00:07.922 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.015) 0:00:07.938 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.020) 0:00:07.959 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.025) 0:00:07.984 *********
Wednesday 27 May 2026 15:51:17 -0500 (0:00:00.069) 0:00:08.054 *********
Wednesday 27 May 2026 15:51:18 -0500 (0:00:00.021) 0:00:08.075 *********
Wednesday 27 May 2026 15:51:18 -0500 (0:00:00.020) 0:00:08.096 *********
Wednesday 27 May 2026 15:51:18 -0500 (0:00:00.019) 0:00:08.116 *********
Wednesday 27 May 2026 15:51:18 -0500 (0:00:00.020) 0:00:08.136 *********
Wednesday 27 May 2026 15:51:18 -0500 (0:00:00.020) 0:00:08.156 *********
Wednesday 27 May 2026 15:51:18 -0500 (0:00:00.026) 0:00:08.183 *********
Wednesday 27 May 2026 15:51:18 -0500 (0:00:00.021) 0:00:08.205 *********
Wednesday 27 May 2026 15:51:18 -0500 (0:00:00.020) 0:00:08.225 *********
Wednesday 27 May 2026 15:51:18 -0500 (0:00:00.020) 0:00:08.245 *********
TASK [List of Models to be Installed] *******************************************************************************************************************************************************
ok: [master1] => {
"model_name_list": ""
}
Wednesday 27 May 2026 15:51:18 -0500 (0:00:00.019) 0:00:08.265 *********
TASK [Clean up remote dependencies directory] ***********************************************************************************************************************************************
changed: [master1]
PLAY RECAP **********************************************************************************************************************************************************************************
master1 : ok=21 changed=9 unreachable=0 failed=0 skipped=120 rescued=0 ignored=0
Wednesday 27 May 2026 15:51:18 -0500 (0:00:00.117) 0:00:08.383 *********
===============================================================================
inference-tools : Ensure jq is installed --------------------------------------------------------------------------------------------------------------------------------------------- 1.67s
inference-tools : Install Kubernetes Python SDK -------------------------------------------------------------------------------------------------------------------------------------- 0.68s
inference-tools : Ensure Python pip module is installed ------------------------------------------------------------------------------------------------------------------------------ 0.65s
Create/Update Kubernetes Secret for Hugging Face Token ------------------------------------------------------------------------------------------------------------------------------- 0.58s
inference-tools : Install Deployment Client ------------------------------------------------------------------------------------------------------------------------------------------ 0.56s
Sync dependency files to Deployment Nodes -------------------------------------------------------------------------------------------------------------------------------------------- 0.23s
kubernetes-precheck : Check if kubectl is Available ---------------------------------------------------------------------------------------------------------------------------------- 0.21s
Check if "vllm-qwen-3-1-7b-cpu-cpu" Model is deployed -------------------------------------------------------------------------------------------------------------------------------- 0.17s
Ensure Remote Directory Exists ------------------------------------------------------------------------------------------------------------------------------------------------------- 0.17s
Remove EKS ingress template for TGI when not on EKS ---------------------------------------------------------------------------------------------------------------------------------- 0.16s
kubernetes-precheck : Check Kubernetes API server connectivity ----------------------------------------------------------------------------------------------------------------------- 0.16s
Remove EKS ingress template for VLLM when not on EKS --------------------------------------------------------------------------------------------------------------------------------- 0.13s
Remove EKS ingress template for TEI when not on EKS ---------------------------------------------------------------------------------------------------------------------------------- 0.13s
Remove EKS ingress template for TEI Rerank when not on EKS --------------------------------------------------------------------------------------------------------------------------- 0.12s
Clean up remote dependencies directory ----------------------------------------------------------------------------------------------------------------------------------------------- 0.12s
Get model registration Job logs ------------------------------------------------------------------------------------------------------------------------------------------------------ 0.07s
Get model registration Job logs ------------------------------------------------------------------------------------------------------------------------------------------------------ 0.06s
Display detected CPU topology for model deployment ----------------------------------------------------------------------------------------------------------------------------------- 0.06s
Wait for model registration Job to complete ------------------------------------------------------------------------------------------------------------------------------------------ 0.05s
Display model registration result ---------------------------------------------------------------------------------------------------------------------------------------------------- 0.05s
Inference LLM Model is removed successfully.
---------------------------------------------------------------------
| LLM Model Being Removed from Intel AI for Enterprise Inference! |
---------------------------------------------------------------------
user@master1:~/Enterprise-Inference/core$ kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
vllm-llama-3-2-3b-cpu 1/1 1 1 58m
vllm-qwen-3-1-7b-cpu 1/1 1 1 5m46s
user@master1:~/Enterprise-Inference/core$ kubectl get pods
NAME READY STATUS RESTARTS AGE
keycloak-0 1/1 Running 0 72m
keycloak-postgresql-0 1/1 Running 0 72m
vllm-llama-3-2-3b-cpu-5f8cdf57c-25dq6 1/1 Running 0 65m
vllm-qwen-3-1-7b-cpu-549b9bd774-f2cmm 1/1 Running 0 12m
Unable to remove model using the inference-stack-deploy.sh script "Remove Model using deployment name" menu option . The removal process runs and reports success, but the model pod remains running and the deployment installed.
See the full process below using the qwen3-1.7 model as an example, starting from deploying the model: