Team OS : Your Only Destination To Custom OS !!

Welcome to TeamOS Community, Register or Login to the Community to Download Torrents, Get Access to Shoutbox, Post Replies, Use Search Engine and many more features. Register Today!

  • Minimum Required Ratio Alert!

    Notice! From Today Onwards Minimum Required Ratio is Set to 0.4 . Either Use Bonus to Increase Ratio or Become VIP in order to Download Torrents.

gaming on a vm with gpu passthrough analysis

monty cristo

Verified Member
Downloaded
647.9 GB
Uploaded
4.6 TB
Ratio
7.32
Goal: have one box many OS for gaming, namely XP, 7, 10 with GPU passthrough and decent performance

Candidates: VMware (workstation or vsphere), unRaid

TLDR Summary: VMware products wont passthrough nvidia gpus. Amd gpus have reset bug
Unraid good for dockers, nas and light gaming. Not recommended for heavy gaming or anything involving hardware layer like changing colour of ram, g-sync


For the longest time I have been interested in gaming on a VM and how this can be achieved so that you dont need more than one box for multiple OSes.
While vmware workstation does work, the software gpu implementation wont give you the native performance needed. Certainly the older the guest OS, the less support there is. Vmware vsphere wont passthrough nvidia cards. This is not hypervisor specific, this is a nvidia thing. Nvidia dont want you passing through consumer gpus they want you to buy their business GPUs.
Enter unraid, which is a linux based product which uses the KVM hypervisor that resides in the linux kernel. This will allow nvidia gpu passthrough, AMD cards still suffer from the reset bug, namely if you have a VM with a AMD gpu passthrough, once you shut down the VM then the passed through GPU isnt released until you reboot the server. So subsequent starts of the VM will fail.
Even with various XML tweaks to unraid VMs, Windows 10 performance in gaming was never quite acceptable. It could of course be the game engine, but I noticed quite a bit of lag in Grim dawn among others. Even with non gaming VMs there was quite a bit of slowdown when downloading torrents and then doing general browsing.

Here are my pros and cons of unraid

  • plex, radarr, deluge, pi etc dockers. Can use gpu transcoding now with plex
  • Gpu passthrough does work as long as you stick with Nvidia
  • TR 2950X precision boost wont work have to oc to 4.2
  • non win 10 VMs performance suffers - or shit like no usb 3 support for windows 7 or 1070/1080ti no windows 7 support black screen on boot
  • cant change colour of g.skill ram
  • ryzen master wont work
  • samsung magician wont work
  • avast wont work
  • max 7 cores for win 10 vm with numa enabled
  • front usb wont hot swap
  • vms not stable never know when going to break
  • not a true hypervisor platform too general really no snapshot support
  • complexity for little real benefit
  • most teamos builds wont work
  • geforce experience wont optimise games
  • xp and 7 too slow to be useful so only use gpu passthrough for win 10
  • benchmarking tools crash
  • couch potato doesnt work wont download movies
  • grim dawn performance bad, intro laggy and refresh rate < 100hz. All i had on was download machine to 2tb hdd
well done if you read this far :)

unRaid refugee
 

Cyler

Verified Member
Downloaded
5 GB
Uploaded
6.3 TB
Ratio
1291.58
Its a very interesting subject, and one I'm continuously working on for the last 3 years, both as a hobby and as a job requirement. I think you will have better luck in some dedicated forum for this rather than here. Level1 Techs might be a good start for you as Wendell and other people there, are very active when it comes to VM servers.

If I may though, a small clarification not so much for you as you seem to know, but for those who will read this.

"VMware products won't passthrough Nvidia GPUs" That is factually not right. Vmware player and pro, doesn't support ANY passthrough but, ESXi does. Fun fact, Xen and VMWare ESXi are the only two hypervisors that officially supports GPU passthrough. You will have some interesting reading to do if you research the Xen project (if you haven't). Citrix, for example, is based on Xen HyperVisor.

Also, AMD bug is a hit or miss situation and it doesn't affect all cards and CPU/mobo combos. AMD is actively trying and took great steps into solving this and you can find many workarounds to this depending on which OS, Hypervisor, and card used.

The main reason has to do more with the guest OS than with VM software. Once a card is used by the host OS, it can't be released to the virtual OS. Windows don't do well with PCI or driver sharing (and can't blame them with companies often writing bad code for drivers). That's why you need a type 1 Visor and it can't happen (easy) with type 2 visors. There is an exception and you can activate passthrough using 2 graphic cards in Hyper V. Windows Hyper V does support passthrough using DDA (Discrete Device Assignment). You can read more on Microsoft HyperV support documentation.

We also need to note that Nvidia doesn't play nice when it comes to "sharing" the GPU. They actively try to detect at a driver level to see if consumer-grade cards are used in a VM environment and try to block such use. This is where Nvidia Grid and Quadro/tesla cards come in. Maybe part of your issue is there.

Personally I don't have much experience using unraid as a virtual host for such projects and so I can't expand more than the small research I did and it looks like unraid doesn't have the best support for such projects. ESXi is what I use and experimenting heavily with SR IOV for my VMs.

Very nice to see that there are more souls out there that actively work on such projects. I would urge you to research for better VM solutions like ESXi/Vshpere and Xen or even HyperV and read on dedicated forums as you will find greater support, example cases and documentation from 1st hand knowledgeable people.
 
Last edited:

monty cristo

Verified Member
Downloaded
647.9 GB
Uploaded
4.6 TB
Ratio
7.32
@Cyler did you get esxi to work with amd cards then? Which ones do you have? 5700xt? I agree nice to know there are other virtualisation brothers out there :)
 

Cyler

Verified Member
Downloaded
5 GB
Uploaded
6.3 TB
Ratio
1291.58
I will be honest with you. With either AMD or Nvidia, it's 50-50% chance to work, and it depends on a lot of factors. CPU/MOBO combo, Graphics card chipset/driver used, and luck. For example, several 5xx AMDs are actually reflashed 4xx AMD cards and those often won't work. Nvidia has its fair share fo issues depending on the model/driver you use.

From what I can tell from mine and colleagues' experience, some combos work better than others and it's mostly due to drivers/bios way of coding and how Hypervisors / Linux will utilize them. Some manufacturers try to cut corners and that often prohibits a certain board from working. You can have the same chipset from 2 different vendors and one will work and the other won't.

That being said I was lucky so far and both AMD and Nvidia worked out fine (about 80% success on 1st to 3rd try) for the bigger part with both KVM/VFIO and ESXi tho I have to tell you, I do more compute/transcode/virtual desktops rather than gaming but I don't think its much of a difference. Part of the success was based on a). lots of available mobos/GCards to work/test with (thx to manufactures) and a LOT of online research.

It's more likely one will have issues that are a bios/mobo/card related rather than it KVM or ESXi related. If you plan to use Nvidia consumer (GTX etc), KVM is a lot better road as you will face problems with drivers, and through KVM tricks, you can convince the Nvidia driver that s not running as a passthrough to a VM. If you plan to use AMD either will do tho I find ESXi a bit better.

Note, always research about your specific combo of cpu/mobo/card to see other people's experience and workarounds with Either KVM or ESXi or even Hyper-V, don't neglect that. Hyper V has a lot of power tho it does need some PowerShell tricks from time to time.

Sorry if I can't be more specific but subjects such as this are not easy to discuss with few lines in a web forum such as this.

Wish you the best of luck.
 
Loading...
Top