With vSphere 4U2 and vSphere 4.1 you have the option of using an advanced configuration setting to change the "virtual cores per socket" to allow thread count needs to have a lesser impact on OS and application licensing. The advanced configuration parameter name is "cpuid.coresPerSocket" (default 1) and acts as a divisor for the virtual hardware setting "CPUs" which must be an integral multiple of the "cpuid.coresPerSocket" value. More on the specifics and limitations of this setting can be found in "Chapter 7, Configuring Virtual Machines" (page 79) of the vSphere Virtual Machine Administrator Guide for vSphere 4.1. [Note: See also VMware KB1010184.]
The value of "cpuid.coresPerSocket" is effectively ignored when "CPUs" is set to 1. In case "cpuid.coresPerSocket" is an imperfect divisor, the power-on operation will fail with the following message in the VI Client's task history:
[caption id="attachment_1726" align="aligncenter" width="328" caption="Virtual core count is imperfect divisor of CPUs"]
[/caption]If virtual machine logging is enabled, the following messages (only relevant items listed) will appear in the VM's log (Note: CPUs = 3, cpuid.coresPerSocket = 2):
Nov 08 14:17:43.676: vmx| DICT virtualHW.version = 7
Nov 08 14:17:43.677: vmx| DICT numvcpus = 3
Nov 08 14:17:43.677: vmx| DICT cpuid.coresPerSocket = 2
Nov 08 14:17:43.727: vmx| VMMon_ConfigMemSched: vmmon.numVCPUs=3
Nov 08 14:17:43.799: vmx| NumVCPUs 3
Nov 08 14:17:44.008: vmx| Msg_Post: Error
Nov 08 14:17:44.008: vmx| [msg.cpuid.asymmetricalCores] The number of VCPUs is not a multiple of the number of cores per socket of your VM, so it cannot be powered on.----------------------------------------
Nov 08 14:17:44.033: vmx| Module CPUID power on failed.
While the configuration guide clearly states (as Jason Boche rightly pointed out in his blog):
The number of virtual CPUs must be divisible by the number of cores per socket. The coresPerSocketsetting must be a power of two.- Virtual Machine Configuration Guide, vSphere 4.1
We've found that "cpuid.coresPerCPU" simply needs to be a perfect divisor of the "CPUs" value. This tracks much better with prior versions of vSphere where "odd numbered" socket/CPU counts were allowed, so therefore odd numbers of cores-per-CPU allowed provided the division of CPUs by coresPerCPU is integral. Suffice to say, if the manual says "power of two" (1, 2, 4, 8, etc.) then those are likely the only "supported" configuration available. Any other configuration that "works" (i.e. 3, 5, 6, 7, etc.) will likely be unsupported by VMware in the event of a problem.
That said, odd values of "cpuid.coresPerCPU" do work just fine. Since SOLORI has a large number of AMD-only eco-systems, it is useful to test configurations that match the physical core count of the underlying processors (i.e. 2, 3, 4, 6, 8, 12). For instance, we were able to create a single, multi-core virtual CPU with 3-cores (CPUs = 3, cpuid.coresPerSocket = 3) and run Windows Server 2003 without incident:
[caption id="attachment_1727" align="aligncenter" width="403" caption="Windows Server 2003 with virtual "tri-core" CPU"]
[/caption]It follows, then, that we were likewise able to run a 2P virtual machine with a total of 6-cores (3-per CPU) running the same installation of Windows Server 2003 (CPUs = 6, cpuid.coresPerSocket = 3):
[caption id="attachment_1728" align="aligncenter" width="403" caption="Virtual Dual-processor (2P), Tri-core (six cores total)"]
[/caption]Here are the relevant vmware log messages associated with this 2P, six total core virtual machine boot-up:
Nov 08 14:54:21.892: vmx| DICT virtualHW.version = 7
Nov 08 14:54:21.893: vmx| DICT numvcpus = 6
Nov 08 14:54:21.893: vmx| DICT cpuid.coresPerSocket = 3
Nov 08 14:54:21.944: vmx| VMMon_ConfigMemSched: vmmon.numVCPUs=6
Nov 08 14:54:22.009: vmx| NumVCPUs 6
Nov 08 14:54:22.278: vmx| VMX_PowerOn: ModuleTable_PowerOn = 1
Nov 08 14:54:22.279: vcpu-0| VMMon_Start: vcpu-0: worldID=530748
Nov 08 14:54:22.456: vcpu-1| VMMon_Start: vcpu-1: worldID=530749
Nov 08 14:54:22.487: vcpu-2| VMMon_Start: vcpu-2: worldID=530750
Nov 08 14:54:22.489: vcpu-3| VMMon_Start: vcpu-3: worldID=530751
Nov 08 14:54:22.489: vcpu-4| VMMon_Start: vcpu-4: worldID=530752
Nov 08 14:54:22.491: vcpu-5| VMMon_Start: vcpu-5: worldID=530753
It's clear from the log that each virtual core spawns a new virtual machine monitor thread within the VMware kernel. Confirming the distribution of cores from the OS perspective is somewhat nebulous due to the mismatch of the CPU's ID (follows the physical CPU on the ESX host) and the "arbitrary" configuration set through the VI Client. CPU-z shows how this can be confusing:
[caption id="attachment_1729" align="aligncenter" width="407" caption="CPU#1 as described by CPU-z"]
[/caption][caption id="attachment_1730" align="aligncenter" width="406" caption="CPU#2 as described by CPU-z"]
[/caption]Note that CPU-z identifies the first 4-cores with what it calls "Processor #1" and the remaining 2-cores with "Processor #2" - this appears arbitrary due to CPU-z's "knowledge" of the physical CPU layout. In (virtual) reality, this assessment by CPU-z is incorrect in terms of cores per CPU, however it does properly demonstrate the existence of two (virtual) CPUs. Here's the same VM with a "cpuid.coresPerSocket" of 6 (again, not 1, 2, 4 or 8 as supported ):
[caption id="attachment_1731" align="aligncenter" width="405" caption="CPU-z demonstrating a 1P, six-core virtual CPU"]
[/caption]Note again that CPU-z correctly identifies the underlying physical CPU as an Opteron 2376 (2.3GHz quad-core) but shows 6-cores, 6-threads as configured through VMware. Note also that the "grayed-out" selection for "Processor #1" demonstrates a single processor is enumerated in virtual hardware. [Note: VMware's KB1030067 demonstrates accepted ways of verifying cores per socket in a VM.)
How does this help with per-CPU licensing in a virtual world? It effectively evens the playing field between physical and virtual configurations. In the past (VI3 and early vSphere 4) multiple virtual threads were only possible through the use of additional virtual sockets. This paradigm did not track with OS licensing and CPU-socket-aware application licensing since the OS/applications would recognize the additional threads as CPU sockets in excess of the license count.
With virtual cores, the underlying CPU configuration (2P, 12 total cores, etc.) can be emulated to the virtual machine layer and deliver thread-count parity to the virtual machine. Since most per-CPU licenses speak to the physical hardware layer, this allows for parity between the ESX host CPU count and the virtual machine CPU count, regardless of the number of physical cores.
Also, in NUMA systems where core/socket/memory affinity is a potential performance issue, addressing physical/virtual parity is potentially important. This could have performance implications for AMD 2400/6100 and Intel 5600 systems where 6 and 12 cores/threads are delivered per physical CPU socket.
Login doesn't really continue, but in fact, ends with the following error:
