Configuring service worker without KVM

[luis-silva-idb]

Using code-interpreter with kvm is a no-go for me, as we're using VMware ESXi and it might disrupt vMotion and other stuff, so I've been trying to launch it without kvm enabled.

My setup:

• Host VM using Debian 12 running on a VMware ESXi cluster
• Using Docker Compose
• Copied the .env.example and only changed the SANDBOX_ACCESS_TOKEN variable

Right from the start, just setting the KVM_ENABLED variable to false didn't let me actually start the container, failed with the following error:

Error response from daemon: error gathering device information while adding custom device "/dev/kvm": no such file or directory

After commenting the device in the docker-compose.yaml file, it starts but fails:

sandbox-runner    | Starting Sandbox (direct NsJail, no microVM) on port 2000...
sandbox-runner    | [sandbox] WARNING: could not remount cgroupfs rw - NsJail cgroup isolation may fail
sandbox-runner    | mkdir: cannot create directory '/sys/fs/cgroup/init': Read-only file system
sandbox-runner    | [sandbox] Draining root cgroup (3 procs) into init/...
sandbox-runner    | /usr/local/bin/start-direct-sandbox.sh: line 19: /sys/fs/cgroup/init/cgroup.procs: No such file or directory
sandbox-runner    | /usr/local/bin/start-direct-sandbox.sh: line 19: /sys/fs/cgroup/init/cgroup.procs: No such file or directory
sandbox-runner    | /usr/local/bin/start-direct-sandbox.sh: line 19: /sys/fs/cgroup/init/cgroup.procs: No such file or directory
sandbox-runner    | [sandbox] Root cgroup procs after drain: 3
sandbox-runner    | [sandbox] WARNING: could not enable controllers on root (3 procs remain)
sandbox-runner    | /usr/local/bin/start-direct-sandbox.sh: line 24: /sys/fs/cgroup/cgroup.subtree_control: Read-only file system
sandbox-runner    | [sandbox] Removing 10 /proc submounts for fresh procfs support...
sandbox-runner    | [sandbox] WARNING: 10 /proc submounts remain
sandbox-runner    | unshare: unshare failed: Operation not permitted

At this point and without finding documentation to guide me, I turned to LLM for some guidance on the error and what I could do, it suggested adding the volume /sys/fs/cgroup:/sys/fs/cgroup:ro and adding privileged: true to the service definition.
After that, the error changed:

sandbox-runner    | Starting Sandbox (direct NsJail, no microVM) on port 2000...
sandbox-runner    | [sandbox] Remounted cgroupfs as rw
sandbox-runner    | [sandbox] Draining root cgroup (138 procs) into init/...
sandbox-runner    | [sandbox] Root cgroup procs after drain: 138
sandbox-runner    | [sandbox] Enabled +memory +pids on root cgroup.subtree_control
sandbox-runner    | [sandbox] No /proc submounts to remove
sandbox-runner    | FATAL: cannot bind /usr/lib
sandbox-runner    | bash: line 5: /usr/sbin/mount: No such file or directory

I got stuck after that, since every change suggested either didn't actually do anything or got me to some problems with the entrypoint.sh.

Is there any documentation on how to do this? I've searched across the repo and for any official documentation, but I haven't found enough information to go on.

I've posted this on discussions; if it's not the correct place, kindly direct me to where I can post it.

Thanks!