Quick Take: Q1 DRAM Price Follow-up, 8GB DDR3 Below Target

In September 2009 we predicted that average 8GB DIMM prices (DDR2 and DDR3) would reach $565/stick by year end (with DDR3 being higher than DDR2) and at now we're seeing the reversal of fortunes for DDR2. At year end, the average price for benchmark DDR2/DDR3 was $591 retail, with promotional pricing pushing that below$550 as predicted. Today, we're seeing DDR3 begin to overtake DDR2 in the 8GB ECC category, dropping below $510/stick, while DDR2 climbs to $550/stick (promotional, on $625/stick retail.)

In 4GB ECC configurations, DDR2 enjoys only a slight retail advantage (13%) while promotional pricing (likely due to inventory reduction initiatives) are providing a bit better value short term. However, the price gap is only 1/2 the power gap, with DDR3 delivering a greater than 35% reduction in power over its DDR2 equivalent (about $1.25/year/stick at $0.10/kWh). The honeymoon is almost over for DDR2.

Benchmark Server (Spot) Memory Pricing - Dual Rank DDR2 Only
DDR2 Reg. ECC Series (1.8V)	Price Jun '09	Price Sep '09	Price Dec '09	Price Mar '10
KVR800D2D4P6/4G 4GB 800MHz DDR2 ECC Reg with Parity CL6 DIMM Dual Rank, x4 (5.400W operating)	$100.00	$117.00 up 17%	$140.70 up 23% promo	$128.90 ($151 retail)
KVR667D2D4P5/4G 4GB 667MHz DDR2 ECC Reg with Parity CL5 DIMM Dual Rank, x4 (5.940W operating)	$80.00	$103.00 up 29%	$97.99 down 5%	$128.74 ($149 retail)
KVR667D2D4P5/8G 8GB 667MHz DDR2 ECC Reg with Parity CL5 DIMM Dual Rank, x4 (7.236W operating)	$396.00	$433.00	$433.00 (promo)	$550.00 (Promo price, retail $625)

Benchmark Server (Spot) Memory Pricing - Dual Rank DDR3 Only
DDR3 Reg. ECC Series (1.5V)	Price Jun '09	Price Sep '09	Price Dec '09	Price Mar '10
KVR1333D3D4R9S/4G 4GB 1333MHz DDR3 ECC Reg w/Parity CL9 DIMM Dual Rank, x4 w/Therm Sen (3.960W operating)	$138.00	$151.00 up 10%	$135.99 down 10%	$150.74 ($170 retail)
KVR1066D3D4R7S/4G 4GB 1066MHz DDR3 ECC Reg w/Parity CL7 DIMM Dual Rank, x4 w/Therm Sen (5.085W operating)	$132.00	$151.00 up 15%	$137.59 down 9% (promo)	$150.74 ($170 retail)
KVR1066D3D4R7S/8G 8GB 1066MHz DDR3 ECC Reg w/Parity CL7 DIMM Dual Rank, x4 w/Therm Sen (4.110W operating)	$1035.00	$917.00 down 11.5%	$667.00 down 28%	$506.59 (retail $584, avail. 3/15)
KVR1333D3D4R9S/8GHA 8GB 1333MHz DDR3 ECC Reg CL9 DIMM 2R x4 w/TS Server Hynix A (4.635W operating)				$584.00

SOLORI's Take: With strong DDR3 demand and short-falls in DDR2 supply (according to DRAMeXchange), the only thing keeping DDR3 prices above DDR2 at this point is demand and inventory. As Q2/2010 introduces a rush of new workstation and server products based on DDR3 systems, the DRAM production ramp will eventually stabilize demand somewhere towards the end of Q3/2010. Meanwhile, technology companies like VMware, Microsoft, Intel and AMD are betting on new infrastructure spending on operating systems, virtualization and hardware refresh to drive-up economic market factors. If the global economic crisis deepens, this anticipated spending spree could be short-lived and its impact shallow.

Quick Take: HP Blade Tops 8-core VMmark w/OC'd Memory

HP's ProLiant BL490c G6 server blade now tops the VMware VMmark table for 8-core systems - just squeaking past rack servers from Lenovo and Dell with a score of 24.54@17 tiles: a new 8-core record. The half-height blade was equipped with two, quad-core Intel Xeon X5570 (Nehalem-EP, 130W TDP) and 96GB ECC Registered DDR3-1333 (12x 8GB, 2-DIMM/channel) memory.

In our follow-up, we found that HP's on-line configuration tool does not allow for DDR3-1333 memory so we went to the street for a comparison. For starters, we examined the on-line price from HP with DDR3-1066 memory and the added QLogic QMH2462 Fiber Channel adapter ($750) and additional NC360m dual-port Gigabit Ethernet controller ($320) which came to a grand total of $28,280 for the blade (about $277/VM, not including Blade chassis or SAN storage).

Stripping memory from the build-out results in a $7,970 floor to the hardware, sans memory. Going to the street to find 8GB sticks with DDR3-1333 ratings and HP support yielded the Kingston KTH-PL313K3/24G kit (3x 8GB DIMMs) of which we would need three to complete the build-out.  At $4,773 per kit, the completed system comes to $22,289 (about $218/VM, not including chassis or storage) which may do more to demonstrate Kingston's value in the market place rather than HP's penchant for "over-priced" memory.

Now, the interesting disclosure from HP's testing team is this:

[caption id="attachment_1203" align="aligncenter" width="450" caption="Notes from HP's VMmark submission."][/caption]

While this appears to boost memory performance significantly for HP's latest run (compared to the 24.24@17 tiles score back in May, 2009) it does so at the risk of running the Nehalem-EP memory controller out of specification - essentially, driving the controller beyond the rated load. It is hard for us to imagine that this specific configuration would be vendor supported if used in a problematic customer installation.

SOLORI's Take:Those of you following closely may be asking yourselves: "Why did HP choose to over-clock the  memory controller in this run by pushing a 1066MHz, 2DPC limit to 1333MHz?"  It would appear the answer is self-evident: the extra 6% was needed to put them over the Lenovo machine. This issue raises a new question about the VMmark validation process: "Should out of specification configurations be allowed in the general benchmark corpus?" It is our opinion that VMmark should represent off-the-shelf, fully-supported configurations only - not esoteric configuration tweaks and questionable over-clocking practices.

Should there be as "unlimited" category in the VMmark arena? Who knows? How many enterprises knowingly commit their mission critical data and processes to systems running over-clocked processors and over-driven memory controllers? No hands? That's what we thought... Congratulations anyway to HP for clawing their way to the top of the VMmark 8-core heap...

Quick Take: DRAM Price Follow-Up

As anticipated, global DRAM prices have continued their upward trend through September, 2009. We reported on August 4, 2009 about the DDR3 and DDR2 price increases that - coupled with a short-fall in DDR3 production - have caused a temporary shift of the consumer market towards DDR2-based designs.

Last week, the Inquirer also reported that DRAM prices were on the rise and that the trend will result in parity between DDR2 and DDR3 prices. MaximumPC ran the Inquirer's story urging its readers to buy now as the tide rises on both fronts. DRAMeXchange is reporting a significant revenue gain to the major players in the DRAM market as a result of this well orchestrated ballet of supply and demand. The net result for consumers is higher prices across the board as the DDR2/DDR3 production cross-over point is reached.

SOLORI's Take: DDR2 is a fading bargain in the server markets, and DIMM vendors like Kingston are working to maintain a stable source of DDR2 components through the end of 2009. While still Looking at our benchmark tracking components, we project 8GB DIMMs to average $565/DIMM by the end of 2009. In the new year, expect 8GB/DDR2 to hit $600/DIMM by the end of H2/2010 with lower pricing on 8GB/DDR3-1066 - in the $500/DIMM range (if supply can keep up with new system demands created by continued growth in the virtualization market.)

Benchmark Server Memory Pricing
DDR2 Series (1.8V)	Price Jun '09	Price Sep '09	DDR3 Series (1.5V)	Price Jun '09	Price Sep '09
4GB 800MHz DDR2 ECC Reg with Parity CL6 DIMM Dual Rank, x4 (5.4W)	$100.00	$117.00 up 17%	4GB 1333MHz DDR3 ECC Reg w/Parity CL9 DIMM Dual Rank, x4 w/Therm Sen (3.96W)	$138.00	$151.00 up 10%
4GB 667MHz DDR2 ECC Reg with Parity CL5 DIMM Dual Rank, x4 (5.94W)	$80.00	$103.00 up 29%	4GB 1066MHz DDR3 ECC Reg w/Parity CL7 DIMM Dual Rank, x4 w/Therm Sen (5.09W)	$132.00	$151.00 up 15%
8GB 667MHz DDR2 ECC Reg with Parity CL5 DIMM Dual Rank, x4 (7.236W)	$396.00	$433.00 up 9%	8GB 1066MHz DDR3 ECC Reg w/Parity CL7 DIMM Dual Rank, x4 w/Therm Sen (6.36W)	$1035.00	$917.00 down 11.5%

SOLORI's 2nd Take: Samsung has been driving the DRAM roller coaster in an effort to dominate the market. With Samsung's 40-nm 2Gb DRAM production ramping by year end, the chip maker's infulence could create a disruptive position in the PC and server markets by driving 8GB/DDR3 prices into the sub-$250/DIMM range by 2H/2010. Meanwhile Hynix, the #2 market leader, chases with 40-nm 1Gb DDR3 giving Samsung the opportunity to repeat its 2008/2009 gambit in 2010 making it increasingly harder for competitors to get a foot-hold in the DDR3 market.

Samsung has their eye on the future with 16GB and 32GB DIMMs already exhibited with 50-nm 2Gb parts claiming a 20% power savings over the current line of memory. With 40-nm 2Gb parts, Samsung is claiming up to 30% additional power savings. To put this into perspective, eight 32GB DIMMs would could about 60% of the power consumed by 32 8GB DIMMs (requiring a 4P+ server). In a virtualization context, this is enough memory to enable 100 virtual machines with 2.5GB of memory each without over subscription. Realistically, we expect to see 16GB DDR3 DIMMs at $1,200/DIMM by 2H/2010 - if everything goes according to plan.

Quick Take: Dell/Nehalem Take #2, 2P VMmark Spot

The new 1st runner-up spot for VMmark in the “8 core” category was taken yesterday by Dell's R710 - just edging-out the previous second spot HP ProLiant BL490 G6 by 0.1% - a virtual dead heat. Equipped with a pair of Xeon X5570 ($1386/ea, bulk list) and 96GB registered DDR3/1066 (12x8GB), the 2U, rack mount R710 weighs-in with a tile ratio of 1.43 over 102 VMs. :

• Dell R710 w/redundant high-output power supply, ($18,209)


• 2 x Intel Xeon X5570 Processors (included)


• 96GB ECC DDR3/1066 (12×8GB) (included)


• 2 x Broadcom NexXtreme II 5709 dual-port GigabitEthernet w/TOE (included)


• 1 x Intel PRO 1000VT quad-port GigabitEthernet (1x PCIe-x4 slot, $529)


• 3 x QLogic QLE2462 FC HBA (1x PCIe slot, $1,219/ea)


• 1 x LSI1078 SAS Controller (on-board)


• 8 x 15K SAS OS drive, RAID10 (included)


• Required ProSupport package ($2,164)


• Total as Configured: $24,559 ($241/VM, not including storage)

Three Dell/EMC CX3-40f arrays were used as the storage backing of the test. The storage system included 8GB cache, 2 enclosures and 15, 15K disks per array delivering 19 LUNs at about 300GB each. Intel's Hyper-Threading and  "Turbo Boost" were enabled for 8-thread, 3.33GHz core clocking as was VT; however embedded SATA and USB were disabled as is common practice.

At about $1,445/tile ($241/VM) the new “second dog” delivers its best at a 20% price premium over Lenovo's "top dog" - although the non-standard OS drive configuration makes-up a half of the difference, with Dell's mandatory support package making-up the remainder. Using a simple RAID1 SAS and eliminating the support package would have droped the cost to $20,421 - a dead heat with Lenovo at $182/VM.

Comparing the Dell R710 the 2P, 12-core benchmark HP DL385 G6 Istanbul system at 15.54@11 tiles:

• HP DL385 G6  ($5,840)


• 2 x AMD 2435 Istanbul Processors (included)


• 64GB ECC DDR2/667 (8×8GB) ($433/DIMM)


• 2 x Broadcom 5709 dual-port GigabitEthernet (on-board)


• 1 x Intel 82571EB dual-port GigabitEthernet (1x PCIe slot, $150/ea)


• 1 x QLogic QLE2462 FC HBA (1x PCIe slot, $1,219/ea)


• 1 x HP SAS Controller (on-board)


• 2 x SAS OS drive (included)


• $10,673/system total (versus $14,696 complete from HP)

Direct pricing shows Istanbul’s numbers at $1,336/tile ($223/VM) which is  a 7.5% savings per-VM over the Dell R710. Going to the street - for memory only - changes the Istanbul picture to $970/tile ($162/VM) representing a 33% savings over the R710.

SOLORI's Take: Istanbul continues to offer a 20-30% CAPEX value proposition against Nehalem in the virtualization use case - even without IOMMU and higher memory bandwidth promised in upcoming Magny-Cours. With the HE parts running around $500 per processor, the OPEX benefits are there for Istanbul too. It is difficult to understand why HP wants to charge $900/DIMM for 8GB PC-5300 sticks when they are available on the street for 50% less - that's a 100% markup. Looking at what HP charges for 8GB DDR3/1066 - $1,700/DIM - they are at least consistent. HP's memory pricing practice makes one thing clear - customers are not buying large memory configurations from their system vendors...

On the contrary, Dell appears to be happy to offer decent prices on 8GB DDR3/1066 with their R710 at approximately $837/DIMM - almost par with street prices.  Looking to see if this parity held up with Dell's AMD offerings, we examined the prices offered with Dell's R805: while - at $680/DIMM - Dell's prices were significantly better than HP's, they still exceeded the market by 50%. Still, we were able to configure a Dell R805 with AMD 2435's for much less than the equivalent HP system:

• Dell R805 w/redundant power ($7,214)


• 2 x AMD 2435 Istanbul Processors (included)


• 64GB ECC DDR2/667 (8×8GB) ($433/ea, street)


• 4 x Broadcom 5708 GigabitEthernet (on-board)


• 1 x Intel PRO 100oPT dual-port GigabitEthernet (1x PCIe slot, included)


• 1 x QLogic QLE2462 FC HBA (1x PCIe slot, included)


• 1 x Dell PERC SAS Controller (on-board)


• 2 x SAS OS drive (included)


• $10,678/system total (versus $12,702 complete from Dell)

This offering from Dell should be able to deliver equivalent performance with HP's DL385 G6 and likewise savings/VM compared to the Nehalem-based R710. Even at the $12,702 price as delivered from Dell, the R805 represents a potential $192/VM price point - about $50/VM (25%) savings over the R710.

Quick Take: DDR3 Prices on the Rise

In the current server-class arms race, Intel and AMD have secured separate quarters: Intel's rival QPI architecture coupled to a 3-channel DDR3 memory bus and functional hyper-threading cores (top bin parts) holds the pure performance sector; while AMD's improved Istanbul cores can be delivered 6 at a time and paired with inexpensive DDR2 memory to achieve better price-performance (acquisition). Both solutions deliver about the same economies in power consumption under virtualized loads.

All in all, the Twin² with Xeon L5520 CPUs is the best platform for those seeking an affordable server with an excellent performance/watt ratio at an affordable price. On the other hand, if performance/price is the most important criterion followed by performance/watt, we would probably opt for the six-core Opteron version of the Twin². Supermicro has "a blade killer" avialable with the Twin², especially for those people who like to keep the hardware costs low.

John De Gelas, AnantTech, July 22, 2009

[caption id="" align="alignright" width="335" caption="Global DDR2 and DDR3 Capacity"][/caption]

Meanwhile, the cost differential between DDR3 and DDR2 continues to widen due to increased demand in the notebook sector and reduced supply (capacity). According to DRAMeXchange, the trend will continue into Q4/09 as suppliers are expected to commit up to 30% of capacity to DDR3 by that time.

At the same time, DDR3 prices continue to inch up, by 5% in July, while DDR2 prices have appeared to bottom-out. This trend in DDR3 pricing is consistent across all speed ratings (1066/1333/1600) and, despite artificial downward price pressure from Samsung, has managed to drift upward 20% since May, 2009.

[caption id="" align="alignright" width="311" caption="DDR3 Price Trend, May to August, 2009"][/caption]

Because low-end, lower-priced 2GB DDR3/1066 ($60/stick) memory shows little advantage over 2GB DDR2/800 ($35/stick), the 70% price premium keeps DDR2 in demand. With the added economic pressures of the world economy and cautious growth outlook of manufacturing sector, the cross-over from DDR2 to DDR3 will come at a significant cost: either to the consumer or the supplier.

Until the cross-over, DDR2-based systems will continue to be a favorite in price sensitive applications (i.e. where total system cost plays a significant role in purchasing decisions.) As an example of this economic inequality, let's take the HP DL380 G6 and DL385 G6 as a comparison. Adding 16GB to the DL380 adds about $760 to the price tag (4x4GB DDR3-1066), while adding the same amount of memory to the DL385 adds only $410 (4x4GB DDR2-800). This comparison demonstrates an 85% price premium of DDR3 versus DDR2, a bit higher (percentage wise) than the desktop norm of 70%.

SOLORI's Take: While the cost of memory in desktop systems typically represents a small portion of the overall system cost, the same can not be said for virtualization systems where entry configurations weigh-in at 16GB and often run from 48GB to 72GB in "fully loaded" systems. This, as our calculus has shown, is where the sweet-spot of $/VM is delivered.

In such configurations, the cost of DDR3 memory can tripple the system cost ($6,370 for 2P, L5506 w/12x4GB DDR3-1066R vs. $5,201 for 2P 2427 w/12x4GB DDR2-800). Moving to the higher memory footprint in 2P systems is typically not cost effective because core count cannot keep-up with the memory needs of the virtual machine inventory. However, if it were possible to utilize additional memory in the 2P platform, our benchmark 8GB DDR3-1066 versus DDR2-667 price comparison tells another story. At $900/stick, the cost of 8GB DDR3 is still a 235% premium over 8GB DDR2, making 96GB DDR3 systems (2P Xeon w/HT) nearly $6,200 per server more costly than their DDR2 counterparts (2P Istanbul) based on memory pricing alone.

SOLORI's 2nd Take: We're hoping to see Tyan and Supermicro release SR5690 chipset-based systems - promised in Q3/2009 - to take advantage of this pricing trend and round-out the Istanbul offering before Q1/2010 ushers-in the next wave of multi-core systems. With 10G prices on the decline, we think today's virtualization applications make Istanbul+IOMMU a good price-performance and price-feature fit in the 32-64GB memory footprint space, leaving Nehalem-EP with only the performance niche to its credit. The only question is: where is SR5690?

Clarification: Nehalem-EP and DDR3

I have seen a lot of contrasting comments about Nehalem-EP and memory speed on the community groups - especially in the area of supported speed ratings: often in the context of comparison to Opteron's need to reduce supported DIMM speed ratings based on slot population. While it is true Nehalem's 3-channel design allows for a mixture of performance (800/1066/1333) and capacity, it does not allow for both.

Here are the rules (from Intel's "Intel Xeon Processor 5500 Series Datasheet, Volume 2") based on DIMM per Channel (DPC):

• 1-DPC = Support DDR3-1333 (if DIMM supports DDR3-1333)
  
  
  
  • KVR1333D3D4R9S/4G - $169/ea
  
  
  • 12GB/CPU max. @ $507/CPU (24GB/system max.)
  
  
  
  


• 2-DPC = Support DDR3-1066 (if all DIMMs are rated DDR3-1066 or higher)
  
  
  
  • KVR1066D3D4R7S/4G - $138/ea
  
  
  • 24GB/CPU max. @ $828/CPU (48GB/system max.)
  
  
  • KVR1066D3Q4R7S/8G - $1,168/ea
  
  
  • 48GB/CPU max. @ $7,008/CPU (96GB/system max.)
  
  
  • "96GB Memory (12x8GB), 1066MHz Dual Ranked RDIMMs for 2 Processors,Optimized [add $15,400]" - Dell
  
  
  
  


• 3-DPC = Support DDR3-800 only (if all DIMMs are rated DDR3-800 or higher)
  
  
  
  • KVR1066D3D4R7S/4G - $138/ea
  
  
  • 36GB/CPU max. @ $1,242/CPU (72GB/system max.)
  
  
  • "144GB Memory (18x8GB), 800MHz Dual Ranked RDIMMs for 2 Processors,Optimized [add $22,900]" - Dell
  
  
  
  

When the IMC detects the presence of 1, 2 or 3 DIMMs, these speed limits are imposed, regardless of the capabilities of the DIMM. A couple of other notable exceptions exist:

• When one 4-rank DIMM is used, it must be populated in DIMM slot0 of a given channel (farthest from CPU);


• Mixing of 4-rank DIMMs in one channel and 3-DIMMs in other channel (3-DPC) on the same CPU socket is not allowed - forcing BIOS to disable on the 4-rank channel;


• RDIMM
  
  
  
  • Single-rank DIMM: 1-DPC, 2-DPC or 3-DPC
  
  
  • Dual-rank DIMM: 1-DPC, 2-DPC or 3-DPC
  
  
  • Quad-rank DIMM: 1-DPC or 2-DPC
  
  
  
  


• UDIMM
  
  
  
  • Single-rank DIMM: 1-DPC or 2-DPC
  
  
  • Dual-rank DIMM: 1-DPC or 2-DPC
  
  
  • Quad-rank DIMM: n/a
  
  
  
  

Speed freaks be warned!