Shanghai Economics 101 - Continued

Let's look at some more real world applications of what we've learned from the VMmark results for Nehalem and what it means in a practical comparison. We'll award Nehalem-EP's SMT a 25% bonus for in our comparisons when vCPU/core count is taken into the measurement. In a 6:1 consolidation, this means 60 vCPU's for 2P Nehalem and 48 vCPU's for Shanghai. Using this bias, the following cost characteristics are revealed for VM's with average memory footprints of 1.5GB, for the Nehalem-EP 3.2GHz system:

Nehalem-EP Configuration	Street $	1536MB VM's, 1 vCPU's	Max vCPU's (6/c)	Cost/VM
2P/8C, Nehalem-EP, W5580 3.2GHz, 6.4GT QPI with 24GB DDR3/1333	$7,017.69	13	60	$539.82
2P/8C, Nehalem-EP, W5580 3.2GHz, 6.4GT QPI with 48GB DDR3/1066	$7,755.99	28	60	$277.00
2P/8C, Nehalem-EP, W5580 3.2GHz, 6.4GT QPI with 72GB DDR3/800	$8,708.19	42	60	$207.34
2P/8C, Nehalem-EP, W5580 3.2GHz, 6.4GT QPI with 96GB DDR3/1066	$21,969.99	57	60	$385.44
2P/8C, Nehalem-EP, W5580 3.2GHz, 6.4GT QPI with 144GB DDR3/800	$30,029.19	60	60	$500.49
2 x 2P/8C, Nehalem-EP, W5580 3.2GHz, 6.4GT QPI with 144GB DDR3/800	$60,058.38	120	120	$500.49

We'll compare this to a Shanghai 2P system at 3.1GHz vs. the Nehalem-EP system:

Shanghai 2P/HT3 Configuration	Street $	1536MB VM's, 1 vCPU's	Max vCPU's (6/c)	Cost/VM	Savings per VM	Savings %
2P/8C Shanghai, 2393 SE, 3.1GHz, 4.4GT HT3 with 32GB DDR2/800	$5,892.12	18	48	$327.34	$212.48	39.36%
2P/8C Shanghai, 2393 SE, 3.1GHz, 4.4GT HT3 with 48GB DDR2/800	$6,352.12	28	48	$226.86	$50.14	18.10%
2P/8C Shanghai, 2393 SE, 3.1GHz, 4.4GT HT3 with 64GB DDR2/533	$6,462.52	37	48	$174.66	$32.68	15.76%
2P/8C Shanghai, 2393 SE, 3.1GHz, 4.4GT HT3 with 80GB DDR2/667	$8,422.12	47	48	$179.19	$28.14	13.57%
2P/8C Shanghai, 2393 SE, 3.1GHz, 4.4GT HT3 with 96GB DDR2/667	$11,968.72	48	48	$249.35	$136.09	35.31%
2P/8C Shanghai, 2393 SE, 3.1GHz, 4.4GT HT3 with 128GB DDR2/533	$14,300.92	48	48	$297.94	$202.55	40.47%
2 x 2P/8C Shanghai, 2393 SE, 3.1GHz, 4.4GT HT3 with 128GB DDR2/533	$28,601.83	96	96	$297.94	$202.55	40.47%

And against a 4P Shanghai 3.1GHz system vs. the Nehalem-EP system:

Shanghai 4P/HT3 Configuration	Street $	1536MB VM's, 1 vCPU's	Max vCPU's (6/c)	Cost/VM	Savings per VM	Savings %
4P/16C Shanghai, 8393 SE, 3.1GHz, 4.4GT HT3 with 48GB DDR2/800	$13,789.58	28	96	$492.49	$47.34	8.77%
4P/16C Shanghai, 8393 SE, 3.1GHz, 4.4GT HT3 with 64GB DDR2/800	$14,189.58	37	96	$383.50	-$106.50	-38.45%
4P/16C Shanghai, 8393 SE, 3.1GHz, 4.4GT HT3 with 80GB DDR2/800	$14,589.58	47	96	$310.42	-$103.08	-49.72%
4P/16C Shanghai, 8393 SE, 3.1GHz, 4.4GT HT3 with 96GB DDR2/800	$17,512.87	57	96	$307.24	$78.20	20.29%
4P/16C Shanghai, 8393 SE, 3.1GHz, 4.4GT HT3 with 160GB DDR2/667	$26,413.87	96	96	$275.14	$225.34	45.02%
4P/16C Shanghai, 8393 SE, 3.1GHz, 4.4GT HT3 with 256GB DDR2/533	$33,410.47	96	96	$348.03	$152.46	30.46%

Note the relatively high start-up cost of the 4P system cause some problems economically for 4P in the 64 and 80GB configurations. This shows where "modern consolidation rules" favor 2P systems where low consolidation ratios (high memory-to-CPU ratios) prevail. In fact, the economics for real world applications of 4P consolidations do not prove advantageous until 96GB of memory is allocated in an Microsoft SQL Server consolidation. Note that the cost savings associated with license reuse become much more significant in these systems (not accounted for in this material cost comparison.)

Shanghai 4P/HT3 Configuration	Street $	12288MB VM's, 4 vCPU's	Max vCPU's (3/c)	Cost/VM	Savings per VM	Savings %
4P/16C Shanghai, 8393 SE, 3.1GHz, 4.4GT HT3 with 96GB DDR2/800	$17,512.87	7	48	$2,501.84	$636.73	20.29%
4P/16C Shanghai, 8393 SE, 3.1GHz, 4.4GT HT3 with 160GB DDR2/667	$26,413.87	12	48	$2,201.16	$2,088.73	48.69%
4P/16C Shanghai, 8393 SE, 3.1GHz, 4.4GT HT3 with 256GB DDR2/533	$33,410.47	12	48	$2,784.21	$1,219.69	30.46%

Disclaimer: the following is for comparative analysis and represents pricing that has neither been established nor disclosed in any way to SOLORI. These numbers have been derived by taking the current price of Opteron 8393SE and adding 15%. Please do not e-mail me looking for Istanbul processors.

Assuming Istanbul ships at a 15% premium, clock-for-clock, the "SQL consolidation" example, the "value" of Istanbul becomes fairly apparent:

Istanbul 4P/HT3 Configuration	Street $ (est.)	12288MB VM's, 4 vCPU's	Max vCPU's (3/c)	Cost/VM	Savings per VM	Savings %
4P/24C Istanbul, 8493 SE, 3.1GHz, 4.4GT HT3 with 96GB DDR2/800	$19,340.68	7	72	$2,762.95	$375.62	11.97%
4P/24C Istanbul, 8493 SE, 3.1GHz, 4.4GT HT3 with 160GB DDR2/667	$28,241.68	13	72	$2,172.44	$2,117.45	49.36%
4P/24C Istanbul, 8493 SE, 3.1GHz, 4.4GT HT3 with 256GB DDR2/533	$35,238.28	18	72	$1,957.68	$2,046.21	51.11%

Perhaps a more interesting real world case is consolidation of Windows Server 2008 requiring 6GB RAM and 2 vCPU per VM. This makes a good use case for Istanbul 2P systems vs. Nehalem-EP in Server 2008 consolidations:

Istanbul 2P/HT3 Configuration	Street $ (est.)	6144MB VM's, 2 vCPU's	Max vCPU's (6/c)	Cost/VM	Savings per VM	Savings %
2P/12C Istanbul, 2493 SE, 3.1GHz, 4.4GT HT3 with 32GB DDR2/800	$6,294.04	4	72	$1,573.51	$765.72	32.73%
2P/12C Istanbul, 2493 SE, 3.1GHz, 4.4GT HT3 with 48GB DDR2/800	$6,754.04	7	72	$964.86	$143.14	12.92%
2P/12C Istanbul, 2493 SE, 3.1GHz, 4.4GT HT3 with 64GB DDR2/533	$6,864.44	10	72	$686.44	$105.21	13.29%
2P/12C Istanbul, 2493 SE, 3.1GHz, 4.4GT HT3 with 80GB DDR2/667	$8,824.04	12	72	$735.34	$56.32	7.11%
2P/12C Istanbul, 2493 SE, 3.1GHz, 4.4GT HT3 with 96GB DDR2/667	$12,370.64	15	72	$824.71	$639.96	43.69%
2P/12C Istanbul, 2493 SE, 3.1GHz, 4.4GT HT3 with 128GB DDR2/533	$14,702.84	20	72	$735.14	$570.47	43.69%
2 x 2P/12C Istanbul, 2493 SE, 3.1GHz, 4.4GT HT3 with 128GB DDR2/533	$29,405.68	41	144	$717.21	$588.41	45.07%

The economics of consolidation systems does not appear to favor 4P systems with the exception of some very special cases where access to multiple-threads and multi-gigabytes of memory are needed by the virtual machines. All other workloads seem to favor 2P systems in $/VM computations. Since 2P systems can be completely turned-off in DRS-driven DPM (VMware) the power saving promised in "better" 4P systems has dubious application.

Perhaps this explains the absence of road-mapped 8P systems from AMD and Intel for their new architectures. If socket G34 delivers 2P and 4P within the same $/CPU cost, these equations will change in favor of 4P systems. Why? The dollar cost of memory, and sliding performance scale as memory buses are loaded, allows 4P systems to out-perform 2P systems with the same memory footprint. Hence, G34 is AMD's future "performance oriented" platform.

What's more, it looks like Intel has really done their homework - zeroing-in on the price/performance/capacity sweet-spot that makes the economics most favorable (their 72GB x 2P platform). However, using some statistics provided by the University of Chicago still shows a $/VM advantage in Opteron's favor - skewing today's usage towards the 32GB platform.

Istanbul 2P/HT3 Configuration	Street $ (est.)	1569MB VM's, 1.45 vCPU's	Max vCPU's (2.33/c)	Cost/VM	Savings per VM	Savings %
2P/12C Istanbul, 2493 SE, 3.1GHz, 4.4GT HT3 with 32GB DDR2/800	$6,294.04	18	27.96	$349.67	$190.15	35.23%
2P/12C Istanbul, 2493 SE, 3.1GHz, 4.4GT HT3 with 48GB DDR2/800	$6,754.04	19	27.96	$355.48	$129.27	26.67%
2P/12C Istanbul, 2493 SE, 3.1GHz, 4.4GT HT3 with 64GB DDR2/533	$6,864.44	19	27.96	$361.29	$182.98	33.62%
2P/12C Istanbul, 2493 SE, 3.1GHz, 4.4GT HT3 with 80GB DDR2/667	$8,824.04	19	27.96	$464.42	$79.84	14.67%
2P/12C Istanbul, 2493 SE, 3.1GHz, 4.4GT HT3 with 96GB DDR2/667	$12,370.64	19	27.96	$651.09	$722.04	52.58%
2P/12C Istanbul, 2493 SE, 3.1GHz, 4.4GT HT3 with 128GB DDR2/533	$14,702.84	19	27.96	$773.83	$1,102.99	58.77%
2 x 2P/12C Istanbul, 2493 SE, 3.1GHz, 4.4GT HT3 with 128GB DDR2/533	$29,405.68	38	55.92	$773.83	$1,102.99	58.77%