rancher-libvirt-node-driver

Rancher node driver for libvirt. The plugin binary is docker-machine-driver-libvirt; DriverName() is libvirt.

Features

• Connect with any libvirt URI (qemu+ssh, qemu+tls, xen+ssh, lxc://, …)
• Cloud credential for the hypervisor (URI, SSH key, password, or TLS)
• Boot from a cloud image already on the libvirt host (never downloaded or deleted)
• qcow2 overlay or full volume copy
• Libvirt network or host bridge
• BIOS or UEFI
• Cloud-init userdata merged with a generated guest SSH key
• Optional domain XML template for non-QEMU hypervisors
• Modular libvirt (virtqemud) socket fallback

The Rancher provision Job must reach libvirtd. Use a remote URI such as qemu+ssh://user@host/system.

Install

Host the linux-amd64 (and arm64) tarball where Rancher can download it. Set spec.url and spec.checksum in deploy/nodedriver.yaml. Apply with metadata.name: libvirt — a UI-created nd-* name breaks RKE2/K3s.

kubectl apply -f deploy/nodedriver.yaml

Driver args

Arg	Description	Required	Default
libvirt-uri	Libvirt connection URI	yes
libvirt-connection-ssh-key-contents	SSH private key for the libvirt host (qemu+ssh)	no
libvirt-connection-ssh-key-path	Path to that key (set by Rancher from the contents field)	no
libvirt-password	Password for SSH or TCP auth	no
libvirt-tls-ca-cert	PEM CA cert for qemu+tls	no
libvirt-tls-client-cert	PEM client cert for qemu+tls	no
libvirt-tls-client-key	PEM client key for qemu+tls	no
libvirt-known-hosts	SSH known_hosts contents or file	no
libvirt-no-verify	Skip TLS / SSH host-key verification	no	false
libvirt-base-image-path	Absolute path of the base image on the libvirt host	yes
libvirt-base-image-format	qcow2 or raw	no	qcow2
libvirt-disk-clone-mode	backing (overlay) or copy	no	backing
libvirt-storage-pool	Pool for the boot disk and cloud-init ISO	no	default
libvirt-network	Libvirt virtual network (ignored if a bridge is set)	no	default
libvirt-bridge	Host bridge instead of a libvirt network	no
libvirt-memory	Memory in MiB	no	4096
libvirt-cpu-count	vCPUs	no	2
libvirt-disk-size	Boot disk size in MiB	no	40000
libvirt-ssh-user	SSH user on the guest	no	ubuntu
libvirt-ssh-port	SSH port on the guest	no	22
libvirt-firmware	bios or uefi	no	bios
libvirt-userdata	Extra cloud-init (merged with the generated guest key)	no
libvirt-domain-xml	Domain XML Go template ({{.Name}}, {{.DiskPath}}, {{.MAC}}, …)	no	QEMU/KVM XML

Guest SSH keys are generated by the driver. The hypervisor login is the cloud credential.

Customize

Set these per node pool. The driver does not assume Ubuntu, a /var/lib/libvirt/images layout, or QEMU.

Select guest base image

Place a cloud-init image on the hypervisor. Point baseImagePath at it, match baseImageFormat, and set sshUser to that image’s default account. Add extra #cloud-config in userdata; the generated guest SSH key is merged in.

baseImagePath: /var/lib/libvirt/images/Rocky-9-GenericCloud.qcow2
baseImageFormat: qcow2
firmware: uefi
sshUser: cloud-user
userdata: |
  #cloud-config
  packages:
    - qemu-guest-agent

Custom storage pool

The image is never fetched. Use the absolute path on the target host, and the storage pool that should hold the new disk and cloud-init ISO. backing can overlay any host file; copy requires the image to already be a libvirt volume.

baseImagePath: /data/nfs/cloudimgs/sles15-sp6.qcow2
storagePool: nfs-pool
diskCloneMode: backing
diskSize: "50000"

Libvirt backend selection

uri is a full libvirt URI (qemu+ssh://…, qemu+tls://…, xen+ssh://…, lxc:///system, bhyve:///system). Default domain XML is QEMU/KVM. For other hypervisors, supply domainXml as a Go template. Include {{.CloudInitPath}} as a nocloud (cidata) disk so the guest key is injected.

Placeholders: {{.Name}}, {{.MemoryKiB}}, {{.VCPUs}}, {{.DiskPath}}, {{.CloudInitPath}}, {{.MAC}}, {{.Network}}, {{.Bridge}}, {{.Firmware}}.

# cloud credential
uri: xen+ssh://root@xen-host/system
# machine
domainXml: |
  <domain type='xen'>
    <name>{{.Name}}</name>
    <memory unit='KiB'>{{.MemoryKiB}}</memory>
    <vcpu>{{.VCPUs}}</vcpu>
    <os><type>hvm</type></os>
    <devices>
      <disk type='file' device='disk'>
        <source file='{{.DiskPath}}'/>
        <target dev='xvda' bus='xen'/>
      </disk>
      <disk type='file' device='cdrom'>
        <source file='{{.CloudInitPath}}'/>
        <target dev='xvdd' bus='xen'/>
        <readonly/>
      </disk>
      <interface type='network'>
        <mac address='{{.MAC}}'/>
        <source network='{{.Network}}'/>
      </interface>
    </devices>
  </domain>

Network

network attaches a libvirt virtual network. bridge attaches a host bridge and takes precedence if both are set. Rancher SSH bootstrap needs a guest IP reachable from the provision Job; RKE2 node-initiated bootstrap only needs outbound access from the VM.

bridge: br0
# or:
# network: default

Build

make test
make release VERSION=v0.1.0

Tarballs in dist/ contain only docker-machine-driver-libvirt. The binary is static (CGO_ENABLED=0).

In development

This v1 release places every machine on the single uri in the cloud credential. Kubernetes affinity and topology spread only apply to pods after those VMs are cluster nodes.

The plan for a v2 release would treat the credential as a pool of hypervisors: uris: [hv-a, hv-b, …]. On create, pick a host (fewest domains, then soft anti-affinity so control-plane machines do not stack), persist that URI on the machine so remove/state hit the same daemon, and fail over if the chosen host is full or unreachable. That is hypervisor placement, not a full maxSkew scheduler.

Until then, use one credential and machine pool per host.