• About
  • Manuals
  • Network
Confessions of a FreeBSD hacker Various tips & tricks about computers in general and FreeBSD in particular
Confessions of a FreeBSD hacker

Takeaways utilising VFIO in a KVM virtual guest

July 8, 2019 15:13 / Leave a Comment / Magnus Strahlert

Having a seamless experience with a host OS running a KVM-guest that gets full access to a GPU by PCI passthrough does come with some hurdles. These are some takeaways from setting it all up.

VFIO is a modern way of passing through PCI-devices from the host to a virtual machine. IOMMU must be enabled for this to work. Both in BIOS and in the kernel. For adding the kernel flags I found it easiest to use grubby for this.

Do note that grubby modifies the current grub.cfg which will be overwritten when grub2-mkconfig is run. That will occur when upgrading a kernel. So this will have to be re-run after a kernel upgrade.

grubby --args="intel_iommu=on" --update-kernel=/boot/vmlinuz-$(uname -r)

KVM guest

    • Install the KVM guest as usual but change bootmode to UEFI. This requires the package edk2-ovmf installed.
    • Then remove every virtual device having to do with display and spice
    • Change USB controller to accomodate usb3
    • Change storage and NIC to use virtio drivers
    • Add PCI hostdevice
      • Select the graphics card, usually 01:00.0 along with its HD audio device 01:00.1. Then by using an HDMI cable you get both display & sound!

/etc/dracut.conf.d/vfio.conf

add_drivers+="vfio vfio_iommu_type1 vfio_pci"

The GPU being passed through must be disabled from being used by the host OS.

/etc/modprobe.d/vfio.conf

options vfio-pci ids=10de:1401,10de:0fba
options vfio-pci disable_vga=1
options snd-hda-intel enable_msi=1
grubby --args="rd.driver.pre=vfio-pci" --update-kernel=/boot/vmlinuz-$(uname -r)
grubby --args="video=efifb:off" --update-kernel=/boot/vmlinuz-$(uname -r)
dracut -f --kver $(uname -r)

Without enabling MSI the sound can begin to stutter when there’s too much data going through the HDMI-cable. To verify if it’s enabled, check for a plus sign in the lspci -v -s 01:00.1|grep MSI: output.

	Capabilities: [68] MSI: Enable+ Count=1/1 Maskable- 64bit+

Keyboard & mouse

Found a software-KVM solution called Barrier (https://github.com/debauchee/barrier) to be most viable.

    • Installed as a client on the VM. It will autostart when having made a connection.
    • Installed as a server on the host OS. Went for letting the window manager autostart it upon login.
      • Tick use relative mouse moves
      • For some games lock-mode will have to be used (scroll-lock)
      • Non-ascii characters can be troublesome. Set meta = altgr and altgr = shift in the screens section.

USB-devices

As for handling USB-devices inside the VM there are some options. I went for udev and wrote a script to handle that. It’s available on https://github.com/mstrahlert/vfio-usb.

Posted in: KVM, Linux / Tagged: grub, kvm, linux, vfio, virtio

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

* Copy This Password *

* Type Or Paste Password Here *

Post Navigation

← Previous Post
 

Recent Posts

  • Takeaways utilising VFIO in a KVM virtual guest
  • Converting VM from ESXi to KVM with iSCSI storage
  • Deploying oVirt in self-hosted mode
  • Ansible playbook for handling perl on FreeBSD
  • iSCSI: Migrating from istgt to ctld

Tags

ansible backup carp chroot cluster cross-server esxi freebsd ftp ftpchroot gpt grub hast install iscsi kvm linux lun lvm mirror network nfs nfsroot ovirt pxe raid ramdrive raspberrypi snapshot ssd ssh terminalserver usb vfio virtio virtualbox vlan vmware volumes winxp zfs zpool zroot

Archives

  • July 2019
  • March 2018
  • February 2018
  • July 2017
  • February 2016
  • January 2015
  • August 2014
  • April 2013
  • March 2013
  • February 2013
  • January 2013
  • December 2012
  • September 2012
  • July 2012
  • April 2012
  • March 2012
  • November 2011
  • August 2011

Recent Comments

    Meta

    • Log in
    • Entries feed
    • Comments feed
    • WordPress.org
    © Copyright 2013-2016 - Magnus Strahlert
    Infinity Theme by DesignCoral / WordPress