Failure starting VMs after boot: "unable to map backing store for guest RAM: Cannot allocate memory"
After the upgrade to Bullseye (#17739 (closed)), Lizard is sometimes unable to start all VMs after a reboot. At least one if not more VMs fail to start with the following error:
zen@lizard ~ > sudo virsh start translate
error: Failed to start domain 'translate'
error: internal error: qemu unexpectedly closed the monitor: 2022-04-11T20:03:52.185594Z qemu-system-x86_64: -object memory-backend-file,id=pc.ram,mem-path=/dev/hugepages/libvirt/qemu/35-translate,prealloc=yes,size=5368709120,host-nodes=1,policy=bind: unable to map backing store for guest RAM: Cannot allocate memory
It's strange because there are available HugePages:
zen@lizard ~ [1]> grep Huge /proc/meminfo
AnonHugePages: 1105920 kB
ShmemHugePages: 0 kB
FileHugePages: 0 kB
HugePages_Total: 121100
HugePages_Free: 4557
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
Hugetlb: 248012800 kB
After waiting for some time (5-10min) and trying again it's eventually possible to start the missing VMs.
Sometimes all VMs will come up without errors.