Today I thought I’d write about CPU scheduling in VMware. CPU scheduling pertains to how VMware’s hypervisor (ESX and ESXi) allocates CPU and helps to answer questions like:
- How many CPU’s should a VM have?
- Is there an optimum number of processors / cores to allocate a VM?
- How do I get the best performance from my CPUs in VMware?
- Can you allocate too many CPUs to a VM?
- What is the best practice for adding CPUs to VMs?
Since I’ve had to explain this topic a number of times, rather than scrambling around for a pen and paper to draw diagrams, I figured that it would be easier to have a concise article explaining this stuff, so I can refer people next time I’m asked how to allocate the correct number of CPUs to a VM.
How VMware CPU Scheduling Works
The first thing you have to understand when allocating CPUs to a VM, is how VMware CPU scheduling works. CPU scheduling is the process VMware uses to allocate physical CPU time slices to vCPUs in VMs. Let me explain with an example:
You have an ESX / ESXi host that has 8 cores. This host has 10 VMs running on it, each with 1 vCPU.
How is that possible? Well, the vCPUs in the VMs each take turns to use the physical cores. Because there are 8 cores, up to 8 VMs can use CPU resources at a time; The other 2 VMs will have to wait their turn. This swapping behaviour is normal and happens to processes in Windows, hence it doesn’t cause VMs to crash, but may slow them down. The longer they have to wait, the slower they will go.
Can Adding Too Many CPUs To A VM Make VMware Go Slow?
Now that you know how CPU scheduling works, let me introduce a new concept:
A VM with multiple vCPUs has to wait for that many physical CPU cores to become free before it can gain access to the ESX / ESXi hosts CPU resources
This is crucial information to help us understand how many CPUs a VM should have. Let’s consider an example in which a VM runs faster with fewer CPUs:
There is a VMware ESX / ESXi host which has 4 physical cores. This host runs 5 VMs, each with 1 vCPU. Here’s how each VM (in this example) accesses the resources over time:
Now imagine how that would look if VM number 4 had been allocated an extra CPU:
As you can see, VM number 4 is forced to wait until 2 cores are simultaneously available, because it has 2 vCPUs. Consequently (in this example) the VM will run slower with more CPUs.
What’s The Best Practice For Deciding How Many CPUs To Add To A VM?
So now you are probably thinking, How do I know how many CPUs I should allocate my VM – How many CPUs is too many? Well, the best practice is to allocate 1 vCPU (core, if you’re using ESXi 5) to a VM then do performance testing to ensure that CPU utilization in Windows is acceptable under load. If you have a VM that needs more, add another vCPU and test your VM again. If you have a complex environment, with many VMs requiring multiple CPUs, then you may need to plan your resources better.
The more vCPUs you allocate to a VM, the more likely it is to be in a position whereby it is waiting for a spare CPU. Consider this, if you give your VM the same number of CPU cores that your host has, then it would only take one of those CPU cores to be in use by another VM to make the VM not be able to use any CPU resources.