VirGL 3D Acceleration on KVM with LibVirt and Nvidia Drivers

Nvidia drivers seem to be causing issues with the normal 3D acceleration libvirt configuration in virt-manager and Gnome Boxes. What happens is that the viewer returns a black/blank screen when the normal configuration to enable VirGL is applied -

<graphics type="spice">

  <listen type="none"/>

  <gl rendernode="/dev/dri/renderD128"/>

</graphics>

<video>

  <model type="virtio" heads="1" primary="yes">

    <acceleration accel3d="yes"/>

  </model>

  <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x0"/>

</video>

The error that it returns is -

qemu_spice_gl_scanout_texture: failed to get fd for texture

Looking around, this seems to be a problem due to EGL support being required for with the proprietary Nvidia drivers. The configuration that worked for me was this -

<graphics type="spice">

  <listen type="none"/>

</graphics>

<graphics type="egl-headless">

  <gl rendernode="/dev/dri/renderD128"/>

</graphics>

<video>

  <model type="virtio" heads="1" primary="yes">

    <acceleration accel3d="yes"/>

  </model>

  <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x0"/>

</video>

To check it is working inside the VM, use the following commands on the terminal in the VM -

$ dmesg |grep -i drm

[    5.326446] systemd[1]: Starting Load Kernel Module drm...

[    5.869884] [drm] pci: virtio-vga detected at 0000:00:01.0

[    5.871953] [drm] features: +virgl +edid

[    5.873360] [drm] number of scanouts: 1

[    5.873365] [drm] number of cap sets: 2

[    5.882054] [drm] cap set 0: id 1, max-version 1, max-size 308

[    5.882247] [drm] cap set 1: id 2, max-version 2, max-size 688

[    5.885640] [drm] Initialized virtio_gpu 0.1.0 0 for virtio0 on minor 0

[    5.891133] virtio_gpu virtio0: fb0: virtio_gpudrmfb frame buffer device

$ glxinfo |grep -i vir

    Device: virgl (0x1010)

OpenGL renderer string: virgl

Assembling a mdadm RAID volume with an external write-intent bitmap at boot on Fedora, CentOS and RHEL

When a drive drops out of an array, and then reintroduced, mdadm has to resync the array. This is long process, but can be sped up greatly by using a write-intent bitmap.

There are 2 types of bitmaps that can be created for an mdadm array - internal and external.

Internal bitmaps are written onto the disks in the array itself. It is the simplest and most straight-forward method and requires little setup and maintenance beyond initial creation. The drawback is that it affects write performance as the bitmap has to be continuously updated during writes.

Compared to internal bitmaps, external bitmaps have much less write performance penalty for mdadm RAID arrays. However, external bitmaps are difficult to set-up as they are files, which means that the RAID array has to be assembled after the partition containing the bitmap file has already been mounted. The reverse is normally the case - mdadm devices are usually assembled before partitions from /fstab are mounted.

After a lot of work reading documentation and testing out the behaviour of the system, here is a method that works (non-root RAID arrays only) on Fedora and derived distros such as RHEL and CentOS -

First add the bitmap to the array -

mdadm /dev/md15 --grow --bitmap=/boot/raid15.bitmap

If you place the bitmap outside of your rootfs (such as in /boot or another partition), add the line for that partition into the /etc/fstab.sys file for dracut to execute before RAID assembly. This step can be skipped if your external bitmap is on the rootfs.

echo "LABEL=boot /boot ext4 defaults 1 2" > /etc/fstab.sys

Scan the arrays for the mdadm.conf command to put into the conf file -

mdadm --detail --scan

Then, update the /etc/mdadm.conf file to assemble the array with the external bitmap -

ARRAY /dev/md15 UUID=<uuid> bitmap=/boot/raid15.bitmap

After this, we are ready to run dracut to regenerate the initramfs -

dracut -f

Finally, if SELinux is installed, mdadm won't be able to open the bitmap file properly. Use chcon to change the label on the file so that mdadm can open it -

chcon system_u:object_r:mdadm_var_run_t:s0 /boot/raid15.bitmap

That should be it. The md device should come back up on system boot. Check with -

cat /proc/mdstat

And make sure the array is assembled and that the bitmap is loaded.