Virtualization
Virtualization isolates software from hardware. Hypervisors orchestrate virtual environments. Hypervisor allows Virtual Machines (VM) to execute instructions incompatible with underlying architecture/hardware. Intel and AMD virtualization technologies VT-x and AMD-V.
QEMU/NEMU/KVM/Lib Virt may be familiar. All of these tools have specific roles.
| Tool | Tool Family | Purpose |
| KVM(Kernel-based Virtual Machine) | Virtualization | – VM manipulation deamon. Instead of relying on emulation, KVM uses the CPU’s built-in Virtualization extension to run virtual machines (VT-x or AMD-V) |
| Lib Virt | Virtualization | – Virtualization management (API, CLI, libvirtd) |
| QEMU(Quick EMUlator) | Emulation | – Emulates CPU and peripherals. It converts guest instructions (like ARM) into hardware-compatible ones (for instance, x86).– Supports all virtualizations/emulations.– It’s infamous for being slow |
| NEMU | Emulation | – NEMU is a QEMU fork that exploits contemporary CPUs with robust virtualization characteristics to speed up QEMU.– NEMU focuses on KVM compatibility to use the CPU’s virtualization extension and simplify instruction translation.– It doesn’t support all hardware,– QEMU is slower. |
Components can be stacked.
Now… AI and GPUs
Virtualization and containerization for AI
Docker is recommended for AI workloads. A key aspect of AI workload management is ensuring every library is compiled with the correct CUDA accelerators (cuBLAS, cuDNN) and Driver.
Playing with Deep Learning framework and CUDA version will give you a headache.
Virtualization lets you break and rebuild your environment. Adding containerized technologies can help you achieve reproducibility and flexibility across frameworks.
Nvidia recognized this issue when they launched NGC.
Virtualize GPUs
When we discussed virtualization layers, we didn’t address GPUs because they’re not on every hardware platform. GPUs can be virtualized in several ways:
- emulation using nvemulate (NVIDIA) or OpenCL device emulator (AMD) instruction emulation (cuda or opencl).
- Using an intermediate interpreter, you can virtualize the GPU (vGPU) or a piece of it. In this situation, GPU access is virtualized, not direct. Citrix, vmware, NVIDIA GRID for VDI, NVIDIA Quadro Virtual Data Center Workstation, NVIDIA Virtual Compute Server (vCS),…) are market solutions.
- Direct access to physical hardware through passthrough-ization. VFIO will assign the guest VM’s GPU’s physical address. This option improves performance (between 1 and 2 percents).
CPUs/GPUs virtualization trap
Why PCI Express pass-through limits?
Virtualization’s restrictions must be defined. KVM virtualization assigns CPU cycles to each VM. At each CPU cycle, the kernel may shift the Virtual Machine from the attached CPU to another CPU.
PCI Express pass-through means the KVM won’t mimic the GPU address but will transfer the instruction to the GPU’s hardware address. The hypervisor gives the virtual machine PCI-e BUS access to the GPU. This causes the Hypervisor to use the GPU uncontrolled, giving you complete access to the RAW hardware.
CPU is a sports car, GPU a truck
CPU throughput limited (indeed CPU as around a 10x slower throughput than GPUs). Simple operations are sometimes slower on a CPU than on a GPU.
Why carry your bag from your sports car to a truck if it’s the only thing you need to move from Berlin to Paris? So it’s a perpetual tradeoff between CPU and GPU. Thankfully, the AI framework core developer arbitraged again.
CPU performance, does it matter when utilizing a GPU?
It does. CPUs are primary bottlenecks when moving data from CPU to GPU using PCie.
Welcome to the world of DomainRooster, where roosters (and hens) rule the roost! We're a one-stop shop for all your entrepreneurial needs, bringing together domain names and website hosting, and all the tools you need to bring your ideas to life. With our help, you'll soar to new heights and hatch great success. Think of us as your trusty sidekick, always there to lend a wing and help you navigate the sometimes-complex world of domain names and web hosting. Our team of roosters are experts in their fields and are always on hand to answer any questions and provide guidance. So why wait? Sign up today and join the ranks of the world's greatest entrepreneurs. With DomainRooster, the sky's the limit! And remember, as the saying goes, "Successful people do what unsuccessful people are not willing to do." So don't be afraid to take that leap of faith - DomainRooster is here to help you reach for the stars. Caw on!
