Quick-Take: Is Your Marriage a Happy One?

I came across a recent post by Chad Sakac (VP, VMware Alliance at EMC) discussing the issue of how vendors drive customer specifications down from broader goals to individual features or implementation sets (I'm sure VCE was not in mind at the time.) When it comes to vendors insist on framing the "client argument" in terms of specific features and proprietary approaches, I have to agree that Chad is spot on. Here's why:

First, it helps when vendors move beyond the "simple thinking" of infrastructure elements as a grid of point solutions and more of an "organic marriage of tools" - often with overlapping qualities. Some marriages begin with specific goals, some develop them along the way and others change course drastically and without much warning. The rigidness of point approaches rarely accommodates growth beyond the set of assumptions that created the it in the first place. Likewise, the "laser focus" on specific features detracts from the overall goal: the present and future value of the solution.

When I married my wife, we both knew we wanted kids. Some of our friends married and "never" wanted kids, only to discover a child on the way and subsequent fulfillment through raising them. Still, others saw a bright future strained with incompatibility and the inevitable divorce. Such is the way with marriages.

Second, it takes vision to solve complex problems. Our church (Church of the Highlands in Birmingham, Alabama) takes a very cautious position on the union between souls: requiring that each new couple seeking a marriage give it the due consideration and compatibility testing necessary to have a real chance at a successful outcome. A lot of "problems" we would encounter were identified before we were married, and when they finally popped-up we knew how to identify and deal with them properly.

Couples that see "counseling" as too obtrusive (or unnecessary) have other options. While the initial investment of money are often equivalent, the return on investment is not so certain. Uncovering incompatibilities "after the sale" provides for difficult and too often a doomed outcome (hence, 50% divorce rate.)

This same drama plays out in IT infrastructures where equally elaborate plans, goals and unexpected changes abound. You date (prospecting and trials), you marry (close) and are either fruitful (happy client), disappointed (unfulfilled promises) or divorce. Often, it's not the plan that failed but the failure to set/manage expectations and address problems that causes the split.

Our pastor could not promise that our marriage would last forever: our success is left to God and the two of us. But he did help us to make decisions that would give us a chance at a fruitful union. Likewise, no vendor can promise a flawless outcome (if they do, get a second opinion), but they can (and should) provide the necessary foundation for a successful marriage of the technology to the business problem.

Third, the value of good advice is not always obvious and never comes without risk. My wife and I were somewhat hesitant on counseling before marriage because we were "in love" and were happy to be blind to the "problems" we might face. Our church made it easy for us: no counseling, no marriage. Businesses can choose to plot a similar course for their clients with respect to their products (especially the complex ones): discuss the potential problems with the solution BEFORE the sale or there is no sale. Sometimes this takes a lot of guts - especially when the competition takes the route of oversimplification. Too often IT sales see identifying initial problems (with their own approach) as too high a risk and too great an obstacle to the sale.

Ultimately, when you give due consideration to the needs of the marriage, you have more options and are better equipped to handle the inevitable trials you will face. Whether it's an unexpected child on the way, or an unexpected up-tick in storage growth, having the tools in-hand to deal with the problem lessens its severity. The point is, being prepared is better than the assumption of perfection.

Finally, the focus has to be what YOUR SOLUTION can bring to the table: not how you think your competition will come-up short. In Chad's story, he's identified vendors disqualifying one another's solutions based on their (institutional) belief (or disbelief) in a particular feature or value proposition. That's all hollow marketing and puffery to me, and I agree completely with his conclusion: vendors need to concentrate on how their solution(s) provide present and future value to the customer and refrain from the "art" of narrowly framing their competitors.

Features don't solve problems: the people using them do. The presence (or absence) of a feature simply changes the approach (i.e. the fallacy of feature parity). As Chad said, it's the TOTALITY of the approach that derives value - and that goes way beyond individual features and products. It's clear to me that a lot of counseling takes place between Sakac's EMC team and their clients to reach those results. Great job, Chad, you've set a great example for your team!

Quick-Take: ZFS and Early Disk Failure

Anyone who's discussed storage with me knows that I "hate" desktop drives in storage arrays. When using SAS disks as a standard, that's typically a non-issue because there's not typically a distinction between "desktop" and "server" disks in the SAS world. Therefore, you know I'm talking about the other "S" word - SATA. Here's a tale of SATA woe that I've seen repeatedly cause problems for inexperienced ZFS'ers out there...

When volumes fail in ZFS, the "final" indicator is data corruption. Fortunately, ZFS checksums recognize corrupted data and can take action to correct and report the problem. But that's like treating cancer only after you've experienced the symptoms. In fact, the failing disk will likely begin to "under-perform" well before actual "hard" errors show-up as read, write or checksum errors in the ZFS pool. Depending on the reason for "under-performing" this can affect the performance of any controller, pool or enclosure that contains the disk.

Wait - did he say enclosure? Sure. Just like a bad NIC chattering on a loaded network, a bad SATA device can occupy enough of the available service time for a controller or SAS bus (i.e. JBOD enclosure) to make a noticeable performance drop in otherwise "unrelated" ZFS pools. Hence, detection of such events is an important thing. Here's an example of an old WD SATA disk failing as viewed from the NexentaStor "Data Sets" GUI:

[caption id="attachment_1660" align="aligncenter" width="450" caption="Something is wrong with device c5t84d0..."][/caption]

Device c5t84d0 is having some serious problems. Busy time is 7x higher than counterparts, and its average service time is 14x higher. As a member of a RAIDz group, the entire group is being held-back by this "under-performing" member. From this snapshot, it appears that NexentaStor is giving us some good information about the disk from the "web GUI" but this assumption would not be correct. In fact, the "web GUI" is only reporting "real time" data so long as the disk is under load. In the case of a lightly loaded zpool, the statistics may not even be reported.

However, from the command shell, historic and real-time access to per-device performance is available. The output of "iostat -exn" shows the count of all errors for devices since the last time counters were reset, and average I/O loads for each:

[caption id="attachment_1662" align="aligncenter" width="450" caption="Device statistics from 'iostat' show error and I/O history."][/caption]

The output of iostat clearly shows this disk has serious hardware problems. It indicates hardware errors as well as transmission errors for the device recognized as 'c5t84d0' and the I/O statistics - chiefly read, write and average service time - implicate this disk as a performance problem for the associated RAIDz group. So, if the device is really failing, shouldn't there be a log report of such an event? Yes, and here's a snip from the message log showing the error:

[caption id="attachment_1663" align="aligncenter" width="450" caption="SCSI error with ioc_status=0x8048 reported in /var/log/messages for failing device."][/caption]

However, in this case, the log is not "full" with messages of this sort. In fact, it only showed-up under the stress of an iozone benchmark (run from the NexentaStor 'nmc' console). I can (somewhat safely) conclude this to be a device failure since at least one other disk in this group is of the same make, model and firmware revision of the culprit. The interesting aspect about this "failure" is that it does not result in a read, write or checksum error for the associated zpool. Why? Because the device is only loosely coupled to the zpool as a constituent leaf device, and it also implies that the device errors were recoverable by either the drive or the device driver (mapping around a bad/hard error.)

Since these problems are being resolved at the device layer, the ZFS pool is "unaware" of the problem as you can see from the output of 'zpool status' for this volume:

[caption id="attachment_1661" align="aligncenter" width="450" caption="Problems with disk device as yet undetected at the zpool layer."][/caption]

This doesn't mean that the "consumers" of the zpool's resources are "unaware" of the problem, as the disk error has manifested itself in the zpool as higher delays, lower I/O through-put and subsequently less pool bandwidth. In short, if the error is persistent under load, the drive has a correctable but catastrophic (to performance) problem and will need to be replaced. If, however, the error goes away, it is possible that the device driver has suitably corrected for the problem and the drive can stay in place.

SOLORI's Take: How do we know if the drive needs to be replaced? Time will establish an error rate. In short, running the benchmark again and watching the error counters for the device will determine if the problem persists. Eventually, the errors will either go away or they wont. For me, I'm hoping that the disk fails to give me an excuse to replace the whole pool with a new set of SATA "eco/green" disks for more lab play. Stay tuned...

SOLORI's Take: In all of its flavors, 1.5Gbps, 3Gbps and 6Gbps, I find SATA drives inferior to "similarly" spec'd SAS for just about everything. In my experience, the worst SAS drives I've ever used have been more reliable than most of the SATA drives I've used. That doesn't mean there are "no" good SATA drives, but it means that you really need to work within tighter boundaries when mixing vendors and models in SATA arrays. On top of that, the additional drive port and better typical sustained performance make SAS a clear winner over SATA (IMHO). The big exception to the rule is economy - especially where disk arrays are used for on-line backup - but that's another discussion...

Quick-Take: NexentaStor AD Shares in 100% Virtual SMB

Here's a maintenance note for SMB environments attempting 100% virtualization and relying on SAN-based file shares to simplify backup and storage management: beware the chicken-and-egg scenario on restart before going home to capture much needed Zzz's. If your domain controller is virtualized and it's VMDK file lives on SAN/NAS, you'll need to restart SMB services on the NexentaStor appliance before leaving the building.

Here's the scenario:

1. An afterhours SAN upgrade in non-HA environment (maybe Auto-CDP for BC/DR, but no active fail-over);


2. Shutdown of SAN requires shutdown of all dependent VM's, including domain controllers (AD);


3. End-user and/or maintenance plans are dependent on CIFS shares from SAN;


4. Authentication of CIFS shares on NexentaStor is AD-based;

Here's the typical maintenance plan (detail omitted):

1. Ordered shutdown of non-critical VM's (including UpdateManager, vMA, etc.);


2. Ordered shutdown of application VM's;


3. Ordered shutdown of resource VM's;


4. Ordered shutdown of AD server VM's (minus one, see step 7);


5. Migrate/vMotion remaining AD server and vCenter to a single ESX host;


6. Ordered shutdown of ESX hosts (minus one, see step 8);


7. vSphere Client: Log-out of vCenter;


8. vSphere Client: Log-in to remaining ESX host;


9. Ordered shutdown of vCenter;


10. Ordered shutdown of remaining AD server;


11. Ordered shutdown of remaining ESX host;


12. Update SAN;


13. Reboot SAN to update checkpoint;


14. Test SAN update - restoring previous checkpoint if necessary;


15. Power-on ESX host containing vCenter and AD server (see step 8);


16. vSphere Client: Log-in to remaining ESX host;


17. Power-on AD server (through to VMware Tools OK);


18. Restart SMB service on NexentaStor;


19. Power-on vCenter;


20. vSphere Client: Log-in to vCenter;


21. vSphere Client: Log-out of ESX host;


22. Power-on remaining ESX hosts;


23. Ordered power-on of remaining VM's;

A couple of things to note in an AD environment:

1. NexnetaStor requires the use of AD-based DNS for AD integration;


2. AD-based DNS will not be available at SAN re-boot if all DNS servers are virtual and only one SAN is involved;


3. Lack of DNS resolution on re-boot will cause a failure for DNS name based NTP service synchronization;


4. NexentaStor SMB service will fail to properly initialize AD credentials;


5. VMware 4.1 now pushes AD authentication all the way to ESX hosts, enabling better credential management and security but creating a potential AD dependency as well;


6. Using auto-startup order on the remaining ESX host for AD and vCenter could automate the process (steps 17 & 19), however, I prefer the "manual" approach after a SAN upgrade in case the upgrade failure is detected only after ESX host is restarted (i.e. storage service interaction in NFS/iSCSI after upgrade).

SOLORI's Take: This is a great opportunity to re-think storage resources in the SMB as the linchpin to 100% virtualization.  Since most SMB's will have a tier-2 or backup NAS/SAN (auto-sync or auto-CDP) for off-rack backup, leveraging a shared LUN/volume from that SAN/NAS for a backup domain controller is a smart move. Since tier-2 SAN's may not have the IOPs to run ALL mission critical applications during the maintenance interval, the presence of at least one valid AD server will promote a quicker RTO, post-maintenance, than coming up cold. [This even works with DAS on the ESX host]. Solution - add the following and you can ignore step 15:

3a. Migrate always-on AD server to LUN/volume on tier-2 SAN/NAS;

24. Migrate always-on AD server from LUN/volume on tier-2 SAN/NAS back to tier-1;

Since even vSphere Essentials Plus has vMotion now (a much requested and timely addition) collapsing all remaining VM's to a single ESX host is a no brainer. However, migrating the storage is another issue which cannot be resolved without either a shutdown of the VM (off-line storage migration) or Enterprise/Enterprise Plus version of vSphere. That is why the migration of the AD server from tier-2 is reserved for last (step 17) - it will likely need to be shutdown to migrate the storage between SAN/NAS appliances.

Quick Take: Q1 DRAM Price Follow-up, 8GB DDR3 Below Target

In September 2009 we predicted that average 8GB DIMM prices (DDR2 and DDR3) would reach $565/stick by year end (with DDR3 being higher than DDR2) and at now we're seeing the reversal of fortunes for DDR2. At year end, the average price for benchmark DDR2/DDR3 was $591 retail, with promotional pricing pushing that below$550 as predicted. Today, we're seeing DDR3 begin to overtake DDR2 in the 8GB ECC category, dropping below $510/stick, while DDR2 climbs to $550/stick (promotional, on $625/stick retail.)

In 4GB ECC configurations, DDR2 enjoys only a slight retail advantage (13%) while promotional pricing (likely due to inventory reduction initiatives) are providing a bit better value short term. However, the price gap is only 1/2 the power gap, with DDR3 delivering a greater than 35% reduction in power over its DDR2 equivalent (about $1.25/year/stick at $0.10/kWh). The honeymoon is almost over for DDR2.

Benchmark Server (Spot) Memory Pricing - Dual Rank DDR2 Only
DDR2 Reg. ECC Series (1.8V)	Price Jun '09	Price Sep '09	Price Dec '09	Price Mar '10
KVR800D2D4P6/4G 4GB 800MHz DDR2 ECC Reg with Parity CL6 DIMM Dual Rank, x4 (5.400W operating)	$100.00	$117.00 up 17%	$140.70 up 23% promo	$128.90 ($151 retail)
KVR667D2D4P5/4G 4GB 667MHz DDR2 ECC Reg with Parity CL5 DIMM Dual Rank, x4 (5.940W operating)	$80.00	$103.00 up 29%	$97.99 down 5%	$128.74 ($149 retail)
KVR667D2D4P5/8G 8GB 667MHz DDR2 ECC Reg with Parity CL5 DIMM Dual Rank, x4 (7.236W operating)	$396.00	$433.00	$433.00 (promo)	$550.00 (Promo price, retail $625)

Benchmark Server (Spot) Memory Pricing - Dual Rank DDR3 Only
DDR3 Reg. ECC Series (1.5V)	Price Jun '09	Price Sep '09	Price Dec '09	Price Mar '10
KVR1333D3D4R9S/4G 4GB 1333MHz DDR3 ECC Reg w/Parity CL9 DIMM Dual Rank, x4 w/Therm Sen (3.960W operating)	$138.00	$151.00 up 10%	$135.99 down 10%	$150.74 ($170 retail)
KVR1066D3D4R7S/4G 4GB 1066MHz DDR3 ECC Reg w/Parity CL7 DIMM Dual Rank, x4 w/Therm Sen (5.085W operating)	$132.00	$151.00 up 15%	$137.59 down 9% (promo)	$150.74 ($170 retail)
KVR1066D3D4R7S/8G 8GB 1066MHz DDR3 ECC Reg w/Parity CL7 DIMM Dual Rank, x4 w/Therm Sen (4.110W operating)	$1035.00	$917.00 down 11.5%	$667.00 down 28%	$506.59 (retail $584, avail. 3/15)
KVR1333D3D4R9S/8GHA 8GB 1333MHz DDR3 ECC Reg CL9 DIMM 2R x4 w/TS Server Hynix A (4.635W operating)				$584.00

SOLORI's Take: With strong DDR3 demand and short-falls in DDR2 supply (according to DRAMeXchange), the only thing keeping DDR3 prices above DDR2 at this point is demand and inventory. As Q2/2010 introduces a rush of new workstation and server products based on DDR3 systems, the DRAM production ramp will eventually stabilize demand somewhere towards the end of Q3/2010. Meanwhile, technology companies like VMware, Microsoft, Intel and AMD are betting on new infrastructure spending on operating systems, virtualization and hardware refresh to drive-up economic market factors. If the global economic crisis deepens, this anticipated spending spree could be short-lived and its impact shallow.

Quick-Take: AMD Dodeca-core Opteron, Real Soon Now

In a recent blog, John Fruehe recounted a few highlights from the recent server analyst event at AMD/Austin concerning the upcoming release of AMD's new 12-core (dodeca) Opteron 6100 series processor - previously knows as Magny-Cours. While not much "new" was officially said outside of NDA privilege, here's what we're reading from his post:

1. Unlike previous launches, AMD is planning to have "boots on the ground" this time with vendors and supply alignments in place to be able to ship product against anticipated demand. While it is now well known that Magny-Cours has been shipping to certain OEM and institutional customers for some time, our guess is that 2000/8000 series 6-core HE series have been hard to come by for a reason - and that reason has 12-cores not 6;

Obviously the big topic was the new AMD Opteron™ 6000 Series platforms that will be launching very soon.  We had plenty of party favors – everyone walked home with a new 12-core AMD Opteron 6100 Series processor, code name “Magny-Cours”.

- Fruehe on AMD's pending launch

2. Timing is right! With Intel's Nehalem-EX 8-core and Core i7/Nehalem-EP 6-core being demoed about, there is more pressure than ever for AMD to step-up with a competitive player. Likewise, DDR3 is neck-and-neck with DDR2 in affordability and way ahead with low-power variants that more than compensate for power-hungry CPU profiles. AMD needs to deliver mainstream performance in 24-cores and 96GB DRAM within the power envelope of 12-cores and 64GB to be a player. With 1.35V DDR3 parts paired to better power efficiency in the 6100, this could be a possibility;

We demonstrated a benchmark running on two servers, one based on the Six-Core AMD Opteron processor codenamed “Istanbul,” and one 12-core “Magny-Cours”-based platform.  You would have seen that the power consumption for the two is about the same at each utilization level.  However, there is one area where there was a big difference – at idle.  The “Magny-Cours”-based platform was actually lower!

- AMD's Fruehe on Opteron 6100's power consumption

3. Performance in scaled virtualization matters - raw single-threaded performance is secondary. In virtual architectures, clusters of systems must perform as one in an orchestrated ballet of performance and efficiency seeking. For some clusters, dynamic load migration to favour power consumption is a priority - relying on solid power efficiency under high load conditions. For other clusters, workload is spread to maximize performance available to key workloads - relying on solid power efficiency under generally light loads. For many environments, multi-generational hardware will be commonplace and AMD is counting on its wider range of migration compatibility to hold-on to customers that have not yet jumped ship for Intel's Nehalem-EP/EX.

"We demonstrated Microsoft Hyper-V running on two different servers, one based on a Quad-Core AMD Opteron processor codenamed “Barcelona” (circa 2007) and a brand new “Magny-Cours”-based system. ...companies might have problems moving a 2010 VM to a 2007 server without limiting the VM features. (For example, in order to move a virtual machine from an Intel  “Nehalem”-based system to a “Harpertown” [or earlier] platform, the customer must not enable nested paging in the “Nehalem” virtual machine, which can reduce the overall performance of the VM.)"

- AMD's Fruehe, extolling the virtues of Opteron generational compatibility

SOLORI's Take: It would appear that Magny-Cours has more under the MCM hood than a pair of Istanbul processors (as previously charged.) To manage better idle performance and constant power performance in spite of a two-to-one core ratio and similar 45nm process, AMD's process and feature set must include much better power management as well, however, core speed is not one of them. With the standard "Maranello" 6100 series coming in at 1.9, 2.1 and 2.2 GHz with an HE variant at 1.7GHz and SE version running at 2.3GHz, finding parity in an existing cluster of 2.4, 2.6 and 2.8 GHz six-core servers may be difficult. Still, Maranello/G34 CPUs will be at 85, 115 and 140W TDP.

That said, Fruehe has a point on virtualization platform deployment and processor speed: it is not necessary to trim-out an entire farm with top-bin parts - only a small portion of the cluster needs to operate with top-band performance marks. The rest of the market is looking for predictable performance, scalability and power efficiency per thread. While SMT makes a good run at efficiency per thread, it does so at the expense of predictable performance. Here's hoping that AMD's C1E (or whatever their power-sipping special sauce will be called) does nothing to interfere with predictable performance...

As we've said before, memory capacity and bandwidth (as a function of system power and core/thread capacity) are key factors in a CPU's viability in a virtualization stack. With 12 DIMM slots per CPU (3-DPC, 4-channel), AMD inherits an enviable position over Intel's current line-up of 2P solutions by being able to offer 50% more memory per cluster node without resorting to 8GB DIMMs. That said, it's up to OEM's to deliver rack server designs that feature 12 DIMM per CPU and not hold-back with only 8 DIMM variants. In the blade and 1/2-size market, cramming 8 DIMM per board (effectively 1-DPC for 2P Magny-Cours) can be a challenge let alone 24 DIMMs! Perhaps we'll see single-socket blades with 12 DIMMs (12-cores, 48/96GB DDR3) or 2P blades with only one 12 DIMM memory bank (one-hop, NUMA) in the short term.

SOLORI's 2nd Take: It makes sense that AMD would showcase their leading OEM partners because their success will be determined on what those OEM's bring to market. With VDI finally poised to make a big market impact, we'd expect to see the first systems delivered with 2-DPC configurations (8 DIMM per CPU, economically 2.5GB/core) which could meet both VDI and HPC segments equally. However, with Window7 gaining momentum, what's good for HPC might not cut it for long in the VDI segment where expectations of 4-6 VM's per core at 1-2GB/VM are mounting.

Besides the launch date, what wasn't said was who these OEM's are and how many systems they'll be delivering at launch. Whoever they are, they need to be (1) financially stronger than AMD, (2) in an aggressive marketing position with respect to today's key growth market (server and desktop virtualization), and (3) willing to put AMD-based products "above the fold" on their marketing and e-commerce initiatives. AMD needs to "represent" in a big way before a tide of new technologies makes them yesterday's news. We have high hopes that AMD's recent "perfect" execution streak will continue.

Quick Take - VMware PartnerExchange 2010: Day 3

With about 2.5 hours of sleep and the VCP410 test looming, Tuesday took on a different tone than the previous two days. My calendar was full:

• 5:30 - Wake-up and check e-mail/blog/systems back in CST


• 7:00am - Breakfast at the VMware Experience Hall


• 8:30am - Keynote with Carl Eschenbach, EVP Worldwide Sales & Field Ops, VMware


• 10:00am - ESXi Convergence Roadmap Session


• 11:15am - View4 Workload Sizing and Testing


• 12:15pm - Lunch in the VMware Alumni Lounge


• 1:30pm - vSphere4 Advanced Configuration Topics


• 3:45pm - VCP410 Test


• 5:15pm - View Composer Tips, Tricks and Best Practices


• 6:45pm - Check-in at home


• 7:00pm - Update blog and check/respond to e-mail

First, the keynote with Carl and the gang was awesome! VMware took a really aggressive attitude towards the competition, including Citrix (virtual desktop) and Microsoft (virtual data center). To sum up the conversation: VMware's intent on carrying on the Q4/09 momentum through 2011, extending its lead in virtual data center and cloud computing capabilities. But Carl's not happy with just the traditional server market, and VMware wants to own the virtual desktop space - virtually putting the squeeze on Citrix as the walls close-in around them.

With over 60-70% of net new Citrix VDI builds being deployed on VMware's ESX servers, it make me wonder why Citrix would drive its XenApp customers to VDI - in the form of XenDesktop4 - by offering a 2-for-1 trade-in program. Isn't this like asking their own clients to reconsider the value proposition of XenApp - in essence turning vendor-locked accounts into a new battleground with VMware? If the momentum shifts towards VMware View4.x and VMware accelerates the pace on product features (including management and integration) as suggested by Carl's aggressive tone today, where does that leave XenDesktop and Citrix?

[caption id="attachment_1439" align="alignright" width="150" caption="The VMware Express Mobile Data Center"][/caption]

The VMware Express: Coming to a City Near You!

VMware introduced its "data center on wheels" to the PartnerExchange 2010 audience today and I got a chance to get on-board and take a look. The build-out was clean and functional with 60+ Gbps of external interconnects waiting for a venue. Inside the Express was a data center, a conference room and several demonstration stations showing VMware vSphere and View4 demos.

[caption id="attachment_1438" align="alignleft" width="113" caption="6 ton Mobile A/C for Express' Data Center"][/caption]

VMware rolled-out the red carpet to the PartnerExchange 2010 attendees. To the right - above the 5-th wheel - is the conference room. All the way in the back (to the left) is the data center portion. In-between the data center and the conference room lies the demonstration area with external plasma screens in kick-panels displaying slide deck and demonstration materials.

[caption id="attachment_1444" align="alignright" width="92" caption="The Express' Diesel Generator - Capable of Powering Things for Nearly 2-days"][/caption]

Up front - behind the cab - rests 6-tons of air conditioning mounted to the front of the trailer. This keeps the living area inside habitable and the data center (about 70-80 sq. ft.) cool enough to run some serious virtualization equipment. Mounted directly behind the driver's cabin is the diesel generator - capable of powering the entire operation for better than 40-hours when external power is unavailable. Today, however, the VMware Express was taking advantage of "house power" provided by Mandalay Bay's conference center.

Where the rubber met the road was inside the data center: currently occupied by an EMC/Cisco rack and a rack powered by MDS/NetApp/Xsigo. Both featured 12TB of raw storage and high-density Nehalem-EP solutions. In the right corner, the heavyweight EMC/Cisco bundle was powered by Cisco's UCS B-series platform featuring eight Nehalem 2P blades per 6U chassis fed by a pair of Cisco 4900-series converged switches. In the left corner, the super middleweight MDS Micro QUADv-series"mini-blade" chassis featuring an eight Nehalem 2P blades per two 2U chassis fed by a pair of Xsigo I/O directors delivering converged network and SAN tunneled over infiniband interconnects.

[caption id="attachment_1441" align="alignleft" width="300" caption="Two-of-Three Racks are Currently Occupied by EMC/Cisco and MDS/NetApp/Xsigo"][/caption]

It will be interesting to see how the drive arrays survive the journey as the VMware Express travels across the country over the next year. Meanwhile, this tractor trailer is packing 60-blades worth of serious virtualization hardware destined for a town near you. VMware is currently looking for additional sponsors from the partner community to expand its tour; and access to the VMware Express will be prioritized based on partner status and/or sponsorship.

VCP410 Test Passed - Waiting for Official Notification

With the VCP410 test in the books, I'm now waiting for official notification from VMware of my VCP4 status. According to my "Examination Score Report" I should receive notice from VMware within 30-days having met all of the requirements for "VMware Certified Professional on vSphere 4 Certification" and testing above the Certified Instructor minimums.

As a systems and network architect, I found the "interface related questions" somewhat more challenging than the "design and configure" related fare. However, the test was pretty well balanced and left me with well over 25 minutes to go back over questions I'd checked for review and finalize those answers. I logged-out of the exam with 18 minutes left on the clock. My recommendation to those looking to pass the VCP410:

1. Work with vSphere in a hands-on capacity for several days before taking the test, making good mental notes related to interface operations inside and outside of vCenter


2. Know the minimums and maximums for ESX and vCenter configurations


3. Understand storage zoning, masking and configuration


4. Go over the VCP blueprint on your own before seeking additional assistance


5. Remember the test is on GA release and not "current" release so "correct" answers may differ slightly from "reality"


6. Get more than 2.5 hours sleep the night before you take the exam


7. Schedule the exam in the morning - while you're fresh - not the afternoon following meetings, etc.


8. Dig into topics on the VCP Forum online

That about does it for day number three in Las Vegas, Nevada. It's time to shuffle-up and deal!

Quick-Take: It's Time to Upgrade from ESX to ESXi

That's right, I said upgrade from ESX to ESXi - not ESX 3.x to vSphere, but ESX (any version) to vSphere's ESXi! Ever since VMware PartnerExchange 2009 (April), SOLORI has been advising clients and prospects to focus on ESXi and move away from ESX-based hosts - you know, the one with the "Linux" service console. Personally, I'm glad VMware has strengthened their message about the virtues of ESXi and the direction of their flagship product.

ESXi has a superior architecture and we encourage customers to deploy ESXi as part of any new vSphere deployment. Our future posts will compare ESX 4 and ESXi 4 in detail on topics like hardware compatibility list, performance, and management to demonstrate that ESXi is either on par with or superior than ESX. But for now, here are some key points you should know about ESXi vs. ESX:

1. The functionality and performance of VMware ESX and ESXi are the same; the difference between the two hypervisors resides in their packaging architecture and operational management. VMware ESXi is the latest hypervisor architecture from VMware. It has an ultra thin footprint with no reliance on a general-purpose OS, setting a new bar for security and reliability (learn more).


2. In the future, ESXi’s superior architecture will be the exclusive focus of VMware's development efforts.


3. New and existing customers are highly encouraged to deploy ESXi. Many Fortune 100 companies have already standardized on the ESXi platform.

- VMware Blog, June, 2009

Not unfamiliar with the VI3 version of ESXi, its ease of installation, configuration and management and smaller footprint, I was one of about 10 participants in an "ESXi BoF Breakout Session" with Charu Chaubal of VMware. While discussing vSphere's ESXi with Charu, I never heard the words "ESXi is a superior architecture," but I did get a clear message that ESXi was the way of the future. From that point on, it seemed as though any efforts concentrated on (net new) ESX deployments was going to be "time wasted."

However, it was clear by the whispered tone about ESXi's virtues that the timing was not right for the real message to be spoken aloud. Remember, this was the launch of vSphere and "ESXi" was strongly associated with "ESXi Free" - not a clear marketing message when license sales and adoption curves are on the line. Perhaps that's why the "talking points" at PEX2009 always suggested that ESX was the "flagship" hypervisor and ESXi was "targeted for embedded and OEM" deployments.

In practical terms, migration from ESX 3.x to vSphere/ESXi didn't make a lot of sense for many large or institutional customers at the time due to the lack of third-party driver parity between ESX and ESXi in vSphere. However, for net new installations where thrid-party drivers and service console agents were not a concern, the message about ESXi's superiority was getting lost. Thomas Paine once said "what we obtain too cheap, we esteem too lightly" and I'd attribute the slow uptake on ESXi to the perception that it was somehow inferior to ESX - a misconception owed to it being offered in a "free" version.

Price is what you pay; value is what you get.

- Warren Buffet

By Warren Buffet's standards, ESXi is a tremendous value. For the novice entering bare-metal virtualization for the first time, the relative ease with which ESXi installs to a USB flash drive allows for a trial experience that is both quick and reassuring. I'd argue that ESXi's tiny memory footprint and generous hardware support make it the easiest platform for testing bare-metal virtualization. As I've blogged about, ESXi takes about 14 minutes to install to USB flash - from initial CD-ROM boot to configuration console.

Anyone experienced with ESX will have encountered service and performance affecting IRQ sharing issues with the service console. The hours discovering, tracking and mitigating these kinds of issues after a BIOS update or hardware refresh can never be reclaimed. The fact is these problems simply do not exist for ESXi - something many ESXers will appreciate.

I've attended a great deal of WebEx sessions on VMware products over the last year and I'm still hearing hushed tones and uncertainty about the role of ESXi versus ESX. To be clear, ESXi is being talked about as "enterprise ready," but much of the focus is still on ESX. These overtones were still present in an "vSphere: Install, Configure and Manage" course I recently attended to qualify for VCP410. While our instructors were very knowledgeable and experienced, there seemed to be much less confidence when discussing ESXi versus ESX. In fact, the lab guide clearly states:

If you are new to VMware vSphere and you do not have any special needs for more advanced features, use ESXi.

- Page 599, Module 13, Installing VMware ESX and ESXi

The manual - and VMware's choice of message here - seems to indicate that ESX has "more advanced features" than ESXi. While the "advanced features" VMware is talking about are service console related, it leaves many regarding ESXi as the inferior product in sharp contrast to today's message. If the statement "ESXi’s superior architecture will be the exclusive focus of VMware's development efforts" isn't writing on the wall for the rest of you, here's a list of VMware's new talking points on ESXi:

• Improved Reliability and Security


• Fewer Patches


• Less Disk Space


• Streamlined Deployment and Configuration


• Reduced Management Overhead


• Next Generation Hypervisor


• Superior Architecture


• Drastically Reduced Hypervisor Footprint


• Smaller Code Base, Smaller Attach Surface


• Certified on over 1,000 Server Systems - including USB keys


• New, Operating System Independent

In contrast, the ESX platform is being re-imaged. Here are some new talking points about ESX:

• The Older Architecture


• Relies on Linux OS for Resource Management


• 20x Larger on-disk Footprint


• More Complex to Configure


• Console OS Administration for Configuration and Diagnostics


• Prone to Arbitrary Code Execution (Console OS)

For many of us familiar with both ESXi and ESX, nothing here is really new. The only real change is the message: build your eco-system around ESXi...

SOLORI's Take: It's clear to me that VMware took inventory of its customers and chose to lead with ESX when vSphere was released. I suspect this was a practical decision due to the overwhelming numbers of ESX hosts already installed. However, the change in marketing and positioning we're witnessing signals that we're moving toward a time when ESX will be openly considered a dead-end variant.

When will ESX be phased-out? That's up to market forces and VMware, but the cloud loves efficiency and ESXi is certainly more resource efficient and compartmentalized than its brother ESX. Furthermore, VMware has to maintain two development and support chains with ESX and ESXi and Darwin likes ESXi. If I had to bet, I wouldn't put my money on seeing an ESX version of the next major release. In any case, when ESX is gone VMware can stop having to make excuses for the "linux console" and the implications that VMware is somehow "based on Linux."

Quick Take: VirtualBox adds Live Migra... uh, Teleportation

Sun announced the 3.1.0 release of its desktop hypervisor - VirtualBox - with their own version of live virtual machine host migration called "Teleporting." Teleporting, according to the user's manual, is defined as:

"moving a virtual machine over a network from one VirtualBox host to another, while the virtual machine is running. This works regardless of the host operating system that is running on the hosts: you can teleport virtual machines between Solaris and Mac hosts, for example."

Teleportation operates like an in-place replacement of a VM's facilities, requiring that the "target" host has a virtual machine in VirtualBox with exactly the same hardware settings as the "source" VM. The source and target VM's must also share the same storage, etc. and must use either the same VirtualBox accessible iSCSI targets or some other network storage (NFS or SMB/CIFS) - and no snapshots.

"The hosts must have fairly similar CPUs. While VirtualBox can simulate some CPU features to a degree, this does not always work. Teleporting between Intel and AMD CPUs will probably fail with an error message."

The recipe for teleportation begins on the target and is given in an example, leveraging VirtualBox's VBoxManage command syntax:

VBoxManage modifyvm  --teleporter on --teleporterport

On the source, the running virtual machine is modified according to the following:

VBoxManage controlvm  teleport --host  --port

For testing, same-host teleportation is allowed (source and target equal loopback). Obviously a ready and clean-up script would be involved to copy the settings to a target location, provide the teleport maintenance and clean-up the former VM configuration that is obsoleted in the teleportation. In the case of an error, the running VM stays running on the source host, and the target VM fails to initialize.

SOLORI's Take: This represents the writing on the wall for VMware and vMotion. Perhaps the shift from VMotion to vMotion telegraphs the reduced value VMware already sees in the "now standard" feature. Adding vMotion to vSphere Essentials and Essentials Plus would garner a lot of adoption from the SMB market that is moving quickly to Hyper-V over Citrix and VMware. With VirtualBox's obvious play in desktop virtualization - where minimalist live migration features would be less of a burden - VMware's market could quickly become divided in 2010 with some crafty third-party integration along with open VDI. It's a ways off, but the potential is there...

Quick Take: Red Hat and Microsoft Virtual Inter-Op

This week Red Hat and Microsoft announced support of certain of their OSes as guests in their respective hypervisor implementations: Kernel Virtual Machine (KVM) and Hyper-V, respectively. This comes on the heels of Red Hat's Enterprise Server 5.4 announcement last month.

KVM is Red Hat's new hypervisor that leverages the Linux kernel to accelerate support for hardware and capabilities. It was Red Hat and AMD that first demonstrated live migration between AMD and Intel-based hypervisors using KVM late last year - then somewhat of a "Holy Grail" of hypervisor feats. With nearly a year of improvements and integration into their Red Hat Enterprise Server and Fedora "free and open source" offerings, Red Hat is almost ready to strike-out in a commercially viable way.

Microsoft now officially supports the following Red Hat guest operating systems in Hyper-V:

Red Hat Enterprise Linux 5.2, 5.3 and 5.4

Red Hat likewise officially supports the following Microsoft quest operating systems in KVM:

Windows Server 2003, 2008 and 2008 R2

The goal of the announcement and associated agreements between Red Hat and Microsoft was to enable a fully supported virtualization infrastructure for enterprises with Red Hat and Microsoft assets. As such, Microsoft and Red Hat are committed to supporting their respective products whether the hypervisor environment is all Red Hat, all Hyper-V or totally heterogeneous - mixing Red Hat KVM and Microsoft Hyper-V as necessary.

"With this announcement, Red Hat and Microsoft are ensuring their customers can resolve any issues related to Microsoft Windows on Red Hat Enterprise Virtualization, and Red Hat Enterprise Linux operating on Microsoft Hyper-V, regardless of whether the problem is related to the operating system or the virtualization implementation."

- Red Hat press release, October 7, 2009

Many in the industry cite Red Hat's adoption of KVM as a step backwards [from Xen] requiring the re-development of significant amount of support code. However, Red Hat's use of libvirt as a common management API has allowed the change to happen much more rapidly that critics assumptions had allowed. At Red Hat Summit 2009, key Red Hat officials were keen to point out just how tasty their "dog food" is:

Tim Burke, Red Hat's vice president of engineering, said that Red Hat already runs much of its own infrastructure, including mail servers and file servers, on KVM, and is working hard to promote KVM with key original equipment manufacturer partners and vendors.

And Red Hat CTO Brian Stevens pointed out in his Summit keynote that with KVM inside the Linux kernel, Red Hat customers will no longer have to choose which applications to virtualize; virtualization will be everywhere and the tools to manage applications will be the same as those used to manage virtualized guests.

- Xen vs. KVM, by Pam Derringer, SearchDataCenter.com

For system integrators and virtual infrastructure practices, Red Hat's play is creating opportunities for differentiation. With a focus on light-weight, high-performance, I/O-driven virtualization applications and no need to support years-old established processes that are dragging on Xen and VMware, KVM stands to leap-frog the competition in the short term.

SOLORI's Take: This news is good for all Red Hat and Microsoft customers alike. Indeed, it shows that Microsoft realizes that its licenses are being sold into the enterprise whether or not they run on physical hardware. With 20+:1 consolidation ratios now common, that represents a 5:1 license to hardware sale for Microsoft, regardless of the hypervisor. With KVM's demonstrated CPU agnostic migration capabilities, this opens the door to an even more diverse virtualization infrastructure than ever before.

On the Red Hat side, it demonstrates how rapidly Red Hat has matured its offering following the shift to KVM earlier this year. While KVM is new to Red Hat, it is not new to Linux or aggressive early adopters since being added to the Linux kernel as of 2.6.20 back in September of 2007. With support already in active projects like ConVirt (VM life cycle management), OpenNebula (cloud administration tools), Ganeti, and Enomaly's Elastic Computing Platform, the game of catch-up for Red Hat and KVM is very likely to be a short one.

Quick Take: Nehalem/Istanbul Comparison at AnandTech

Johan De Gelas and crew present an interesting comparison of Dunnington, Shanghai, Istanbul and Nehalem in a new post at AnandTech this week. In the test line-up are the "top bin" parts from Intel and AMD in 4-core and 6-core incarnations:

• Intel Nehalem-EP Xeon, X5570 2.93GHz, 4-core, 8-thread


• Intel "Dunnington" Xeon, X7460, 2.66GHz, 6-core, 6-thread


• AMD "Shanghai" Opteron 2389/8389, 2.9GHz, 4-core, 4-thread


• AMD "Istanbul" Opteron 2435/8435, 2.6GHz, 6-core, 6-thread

Most importantly for virtualization systems architects is how the vCPU scheduling affects "measured" performance. The telling piece comes from the difference in comparison results where vCPU scheduling is equalized:

[caption id="attachment_1276" align="aligncenter" width="450" caption="AnandTech's Quad Sockets v. Dual Sockets Comparison. Oct 6, 2009."][/caption]

When comparing the results, De Gelas hits on the I/O factor which chiefly separates VMmark from vAPUS:

The result is that VMmark with its huge number of VMs per server (up to 102 VMs!) places a lot of stress on the I/O systems. The reason for the Intel Xeon X5570's crushing VMmark results cannot be explained by the processor architecture alone. One possible explanation may be that the VMDq (multiple queues and offloading of the virtual switch to the hardware) implementation of the Intel NICs is better than the Broadcom NICs that are typically found in the AMD based servers.

Johan De Gelas, AnandTech, Oct 2009

This is yet another issue that VMware architects struggle with in complex deployments. The latency in "Dunnington" is a huge contributor to its downfall and why the Penryn architecture was a dead-end. Combined with 8 additional threads in the 2P form factor, Nehalem delivers twice the number of hardware execution contexts than Shanghai, resulting in significant efficiencies for Nehalem where small working data sets are involved.

When larger sets are used - as in vAPUS - the Istanbul's additional cores allows it to close the gap to within the clock speed difference of Nehalem (about 12%). In contrast to VMmark which implies a 3:2 advantage to Nehalem, the vAPUS results suggest a closer performance gap in more aggressive virtualization use cases.

SOLORI's Take: We differ with De Gelas on the reduction in vAPUS' data set to accommodate the "cheaper" memory build of the Nehalem system. While this offers some advantages in testing, it also diminishes one of Opteron's greatest strengths: access to cheap and abundant memory. Here we have the testing conundrum: fit the test around the competitors or the competitors around the test. The former approach presents a bias on the "pure performance" aspect of the competitors, while the latter is more typical of use-case testing.

We do not construe this issue as intentional bias on AnandTech's part, however it is another vector to consider in the evaluation of the results. De Gelas delivers a report worth reading in its entirety, and we view this as a primer to the issues that will define the first half of 2010.

Quick Take: HP Blade Tops 8-core VMmark w/OC'd Memory

HP's ProLiant BL490c G6 server blade now tops the VMware VMmark table for 8-core systems - just squeaking past rack servers from Lenovo and Dell with a score of 24.54@17 tiles: a new 8-core record. The half-height blade was equipped with two, quad-core Intel Xeon X5570 (Nehalem-EP, 130W TDP) and 96GB ECC Registered DDR3-1333 (12x 8GB, 2-DIMM/channel) memory.

In our follow-up, we found that HP's on-line configuration tool does not allow for DDR3-1333 memory so we went to the street for a comparison. For starters, we examined the on-line price from HP with DDR3-1066 memory and the added QLogic QMH2462 Fiber Channel adapter ($750) and additional NC360m dual-port Gigabit Ethernet controller ($320) which came to a grand total of $28,280 for the blade (about $277/VM, not including Blade chassis or SAN storage).

Stripping memory from the build-out results in a $7,970 floor to the hardware, sans memory. Going to the street to find 8GB sticks with DDR3-1333 ratings and HP support yielded the Kingston KTH-PL313K3/24G kit (3x 8GB DIMMs) of which we would need three to complete the build-out.  At $4,773 per kit, the completed system comes to $22,289 (about $218/VM, not including chassis or storage) which may do more to demonstrate Kingston's value in the market place rather than HP's penchant for "over-priced" memory.

Now, the interesting disclosure from HP's testing team is this:

[caption id="attachment_1203" align="aligncenter" width="450" caption="Notes from HP's VMmark submission."][/caption]

While this appears to boost memory performance significantly for HP's latest run (compared to the 24.24@17 tiles score back in May, 2009) it does so at the risk of running the Nehalem-EP memory controller out of specification - essentially, driving the controller beyond the rated load. It is hard for us to imagine that this specific configuration would be vendor supported if used in a problematic customer installation.

SOLORI's Take:Those of you following closely may be asking yourselves: "Why did HP choose to over-clock the  memory controller in this run by pushing a 1066MHz, 2DPC limit to 1333MHz?"  It would appear the answer is self-evident: the extra 6% was needed to put them over the Lenovo machine. This issue raises a new question about the VMmark validation process: "Should out of specification configurations be allowed in the general benchmark corpus?" It is our opinion that VMmark should represent off-the-shelf, fully-supported configurations only - not esoteric configuration tweaks and questionable over-clocking practices.

Should there be as "unlimited" category in the VMmark arena? Who knows? How many enterprises knowingly commit their mission critical data and processes to systems running over-clocked processors and over-driven memory controllers? No hands? That's what we thought... Congratulations anyway to HP for clawing their way to the top of the VMmark 8-core heap...

Quick Take: DRAM Price Follow-Up

As anticipated, global DRAM prices have continued their upward trend through September, 2009. We reported on August 4, 2009 about the DDR3 and DDR2 price increases that - coupled with a short-fall in DDR3 production - have caused a temporary shift of the consumer market towards DDR2-based designs.

Last week, the Inquirer also reported that DRAM prices were on the rise and that the trend will result in parity between DDR2 and DDR3 prices. MaximumPC ran the Inquirer's story urging its readers to buy now as the tide rises on both fronts. DRAMeXchange is reporting a significant revenue gain to the major players in the DRAM market as a result of this well orchestrated ballet of supply and demand. The net result for consumers is higher prices across the board as the DDR2/DDR3 production cross-over point is reached.

SOLORI's Take: DDR2 is a fading bargain in the server markets, and DIMM vendors like Kingston are working to maintain a stable source of DDR2 components through the end of 2009. While still Looking at our benchmark tracking components, we project 8GB DIMMs to average $565/DIMM by the end of 2009. In the new year, expect 8GB/DDR2 to hit $600/DIMM by the end of H2/2010 with lower pricing on 8GB/DDR3-1066 - in the $500/DIMM range (if supply can keep up with new system demands created by continued growth in the virtualization market.)

Benchmark Server Memory Pricing
DDR2 Series (1.8V)	Price Jun '09	Price Sep '09	DDR3 Series (1.5V)	Price Jun '09	Price Sep '09
4GB 800MHz DDR2 ECC Reg with Parity CL6 DIMM Dual Rank, x4 (5.4W)	$100.00	$117.00 up 17%	4GB 1333MHz DDR3 ECC Reg w/Parity CL9 DIMM Dual Rank, x4 w/Therm Sen (3.96W)	$138.00	$151.00 up 10%
4GB 667MHz DDR2 ECC Reg with Parity CL5 DIMM Dual Rank, x4 (5.94W)	$80.00	$103.00 up 29%	4GB 1066MHz DDR3 ECC Reg w/Parity CL7 DIMM Dual Rank, x4 w/Therm Sen (5.09W)	$132.00	$151.00 up 15%
8GB 667MHz DDR2 ECC Reg with Parity CL5 DIMM Dual Rank, x4 (7.236W)	$396.00	$433.00 up 9%	8GB 1066MHz DDR3 ECC Reg w/Parity CL7 DIMM Dual Rank, x4 w/Therm Sen (6.36W)	$1035.00	$917.00 down 11.5%

SOLORI's 2nd Take: Samsung has been driving the DRAM roller coaster in an effort to dominate the market. With Samsung's 40-nm 2Gb DRAM production ramping by year end, the chip maker's infulence could create a disruptive position in the PC and server markets by driving 8GB/DDR3 prices into the sub-$250/DIMM range by 2H/2010. Meanwhile Hynix, the #2 market leader, chases with 40-nm 1Gb DDR3 giving Samsung the opportunity to repeat its 2008/2009 gambit in 2010 making it increasingly harder for competitors to get a foot-hold in the DDR3 market.

Samsung has their eye on the future with 16GB and 32GB DIMMs already exhibited with 50-nm 2Gb parts claiming a 20% power savings over the current line of memory. With 40-nm 2Gb parts, Samsung is claiming up to 30% additional power savings. To put this into perspective, eight 32GB DIMMs would could about 60% of the power consumed by 32 8GB DIMMs (requiring a 4P+ server). In a virtualization context, this is enough memory to enable 100 virtual machines with 2.5GB of memory each without over subscription. Realistically, we expect to see 16GB DDR3 DIMMs at $1,200/DIMM by 2H/2010 - if everything goes according to plan.

Quick Take: Magny-Cours Spotted, Pushed to 3GHz for wPrime

Andreas Galistel at NordicHardware posted an article showing a system running a pair of engineering samples of the Magny-Cours processor running at 3.0GHz. Undoubtedly these images were culled from a report "leaked" on XtremeSystems forums showing a "DINAR2" motherboard with SR5690 chipset - in single and dual processor installation - running Magny-Cours at the more typical pre-release speed of 1.7GHz.

We know that Magny-Cours is essentially a MCM of Istanbul delivered in the rectangular socket G34 package. One thing illuminating about the two posts is the reported "reduction" in L3 cache from 12MB (6MB x 2 in MCM) to 10MB (2 x 5MB in MCM). Where did the additional cache go? That 's easy: since a 2P Magny-Cours installation is essentially a 4P Istanbul configuration, these processors have the new HT Assist feature enabled - giving 1MB of cache from each chip in the MCM to HT Assist.

"wPrime uses a recursive call of Newton's method for estimating functions, with f(x)=x²-k, where k is the number we're sqrting, until Sgn(f(x)/f'(x)) does not equal that of the previous iteration, starting with an estimation of k/2. It then uses an iterative calling of the estimation method a set amount of times to increase the accuracy of the results. It then confirms that n(k)²=k to ensure the calculation was correct. It repeats this for all numbers from 1 to the requested maximum."

- wPrime site

Another thing intriguing about the XtremeSystems post in particular is the reported wPrime 32M and 1024M completion times. Compared to the hyper-threading-enabled 2P Xeon W5590 (130W TDP) running wPrime 32M at 3.33GHz (3.6GHz turbo)  in 3.950 seconds, the 2P 3.0GHz Magny-Cours completed wPrime 32M in an unofficial 3.539 seconds - about 10% quicker while running a 10% slower clock. From the myopic lens of this result, it would appear AMD's choice of "real cores" versus hyper-threading delivers its punch.

SOLORI's Take: As a "reality check" we can compared the reigning quad-socked, quad-core Opteron 8393 SE result in wPrime 32M and wPrime 1024M at 3.90 and 89.52  seconds, respectively. Adjusted for clock and core count versus its Shanghai cousin, the Magny-Cours engineering samples - at 3.54 and 75.77 seconds, respectively - turned-in times about 10% slower than our calculus predicted. While still "record breaking" for 2P systems, we expected the Magny-Cours/Istanbul cores to out-perform Shanghai clock-per-clock - even at this stage of the game.

Due to the multi-threaded nature of the wPrime benchmark, it is likely that the HT Assist feature - enabled in a 2P Magny-Cours system by default - is the cause of the discrepancy. By reducing the available L3 cache by 1MB per die - 4MB of L3 cache total - HT Assist actually could be creating a slow-down. However, there are several things to remember here:

• These are engineering samples qualified for 1.7GHz operation


• Speed enhancements were performed with tools not yet adapted to Magny-Cours


• The author indicated a lack of control over AMD's Cool 'n Quiet technology which could have made "as tested" core clocks somewhat lower than what CPUz reported (at least during the extended tests)


• It is speculated that AMD will release Magny-Cours at 2.2GHz (top bin) upon release, making the 2.6+ GHz results non-typical


• The BIOS and related dependencies are likely still being "baked"

Looking at the more "typical" engineering sample speed tests posted on the XtremeSystems' forum tracks with the 3.0GHz overclock results at a more "typical" clock speed of 2.6GHz for 2P Magny-Cours: 3.947 seconds and 79.625 seconds for wPrime 32M and 1024M, respectively. Even at that speed, the 24-core system is on par with the 2P Nehalem system clocked nearly a GHz faster. Oddly, Intel reports the W5590  as not supporting "turbo" or hyper-threading although it is clear that Intel's marketing is incorrect based on actual testing.

Assuming Magny-Cours improves slightly on its way to market, we already know how 24-core Istanbul stacks-up against 16-thread Nehalem in VMmark and what that means for Nehalem-EP. This partly explains the marketing shift as Intel tries to position Nehalep-EP as a destined for workstations instead of servers. Whether or not you consider this move a prelude to the ensuing Nehalem-EX v. Magny-Cours combat to come or an attempt to keep Intel's server chip power average down by eliminating the 130W+ parts from the "server" list,  Intel and AMD will each attempt win the war before the first shot is fired. Either way, we see nothing that disrupts the price-performance and power-performance comparison models that dominate the server markets.

[Ed: The 10% difference is likely due to the fact that the author was unable to get "more than one core" clocked at 3.0GHz. Likewise, he was uncertain that all cores were reliably clocking at 2.6GHz for the longer wPrime tests. Again, this engineering sample was designed to run at 1.7GHz and was not likely "hand picked" to run at much higher clocks. He speculated that some form of dynamic core clocking linked to temperature was affecting clock stability - perhaps due to some AMD-P tweaks in Magny-Cours.]

Quick Take: Dell/Nehalem Take #2, 2P VMmark Spot

The new 1st runner-up spot for VMmark in the “8 core” category was taken yesterday by Dell's R710 - just edging-out the previous second spot HP ProLiant BL490 G6 by 0.1% - a virtual dead heat. Equipped with a pair of Xeon X5570 ($1386/ea, bulk list) and 96GB registered DDR3/1066 (12x8GB), the 2U, rack mount R710 weighs-in with a tile ratio of 1.43 over 102 VMs. :

• Dell R710 w/redundant high-output power supply, ($18,209)


• 2 x Intel Xeon X5570 Processors (included)


• 96GB ECC DDR3/1066 (12×8GB) (included)


• 2 x Broadcom NexXtreme II 5709 dual-port GigabitEthernet w/TOE (included)


• 1 x Intel PRO 1000VT quad-port GigabitEthernet (1x PCIe-x4 slot, $529)


• 3 x QLogic QLE2462 FC HBA (1x PCIe slot, $1,219/ea)


• 1 x LSI1078 SAS Controller (on-board)


• 8 x 15K SAS OS drive, RAID10 (included)


• Required ProSupport package ($2,164)


• Total as Configured: $24,559 ($241/VM, not including storage)

Three Dell/EMC CX3-40f arrays were used as the storage backing of the test. The storage system included 8GB cache, 2 enclosures and 15, 15K disks per array delivering 19 LUNs at about 300GB each. Intel's Hyper-Threading and  "Turbo Boost" were enabled for 8-thread, 3.33GHz core clocking as was VT; however embedded SATA and USB were disabled as is common practice.

At about $1,445/tile ($241/VM) the new “second dog” delivers its best at a 20% price premium over Lenovo's "top dog" - although the non-standard OS drive configuration makes-up a half of the difference, with Dell's mandatory support package making-up the remainder. Using a simple RAID1 SAS and eliminating the support package would have droped the cost to $20,421 - a dead heat with Lenovo at $182/VM.

Comparing the Dell R710 the 2P, 12-core benchmark HP DL385 G6 Istanbul system at 15.54@11 tiles:

• HP DL385 G6  ($5,840)


• 2 x AMD 2435 Istanbul Processors (included)


• 64GB ECC DDR2/667 (8×8GB) ($433/DIMM)


• 2 x Broadcom 5709 dual-port GigabitEthernet (on-board)


• 1 x Intel 82571EB dual-port GigabitEthernet (1x PCIe slot, $150/ea)


• 1 x QLogic QLE2462 FC HBA (1x PCIe slot, $1,219/ea)


• 1 x HP SAS Controller (on-board)


• 2 x SAS OS drive (included)


• $10,673/system total (versus $14,696 complete from HP)

Direct pricing shows Istanbul’s numbers at $1,336/tile ($223/VM) which is  a 7.5% savings per-VM over the Dell R710. Going to the street - for memory only - changes the Istanbul picture to $970/tile ($162/VM) representing a 33% savings over the R710.

SOLORI's Take: Istanbul continues to offer a 20-30% CAPEX value proposition against Nehalem in the virtualization use case - even without IOMMU and higher memory bandwidth promised in upcoming Magny-Cours. With the HE parts running around $500 per processor, the OPEX benefits are there for Istanbul too. It is difficult to understand why HP wants to charge $900/DIMM for 8GB PC-5300 sticks when they are available on the street for 50% less - that's a 100% markup. Looking at what HP charges for 8GB DDR3/1066 - $1,700/DIM - they are at least consistent. HP's memory pricing practice makes one thing clear - customers are not buying large memory configurations from their system vendors...

On the contrary, Dell appears to be happy to offer decent prices on 8GB DDR3/1066 with their R710 at approximately $837/DIMM - almost par with street prices.  Looking to see if this parity held up with Dell's AMD offerings, we examined the prices offered with Dell's R805: while - at $680/DIMM - Dell's prices were significantly better than HP's, they still exceeded the market by 50%. Still, we were able to configure a Dell R805 with AMD 2435's for much less than the equivalent HP system:

• Dell R805 w/redundant power ($7,214)


• 2 x AMD 2435 Istanbul Processors (included)


• 64GB ECC DDR2/667 (8×8GB) ($433/ea, street)


• 4 x Broadcom 5708 GigabitEthernet (on-board)


• 1 x Intel PRO 100oPT dual-port GigabitEthernet (1x PCIe slot, included)


• 1 x QLogic QLE2462 FC HBA (1x PCIe slot, included)


• 1 x Dell PERC SAS Controller (on-board)


• 2 x SAS OS drive (included)


• $10,678/system total (versus $12,702 complete from Dell)

This offering from Dell should be able to deliver equivalent performance with HP's DL385 G6 and likewise savings/VM compared to the Nehalem-based R710. Even at the $12,702 price as delivered from Dell, the R805 represents a potential $192/VM price point - about $50/VM (25%) savings over the R710.

Quick-Take: VMworld 2009 Wrap-Up

VMworld 2009 in San Franciso started off with a crash and a fist fight, but ended without further incident. If you're looking for what happened, it would be hard to beat Duncan Epping's link-summary of the San Francisco VMworld 2009 at Yellow-Bricks, so we won't even try. Likewise, Chad Sakacc has some great EMC view point on his Virtualgeek blog, and - fresh from his new book release - Scott Lowe has some great detail about the VMworld keynotes, events and sessions he attended.

There is a great no-spin commentary on VMworld's "softer underbelly" on Jon William Toigo's Drunken Data blog - especially the post about Xsigo's participation in VMworld 2009. Also, Brian Madden has a great wrap-up video of of interviews from the VMworld floor including VMware's Client Virtualization Platform (CVP) and the software implementation of Teradici's PC-over-IP.

AMD's IOMMU was on display using a test mule with two 12-core 6100 processors and a SR5690 chipset. The targets were a FirePro graphics card and a Solarflare 10GE NIC. For IOMMU-based virtualization to have broad appeal, hardware device segmentation must be supported in a manner compatible with vMotion (live migration.) No segmentation was hinted at in AMD's demo (for FirePro), but the fact that vSphere+IOMMU+Magny-Cours equated to enough stability to be openly demonstrating the technology says a lot about the maturity of AMD's upcoming chips and chipsets. On the other hand, Solarflare's demonstration previewed - in 10GE - what could be possible in a future version of IOV for GPU's:

"The flexible vNIC demonstration will highlight the Solarstorm server adapter’s scalable, virtualized architecture, supporting 100s of virtual machines and 1000s of vNICs. The Solarstorm vNIC architecture provides flexible mapping of vNICs, so that each guest OS can have its own vNIC, as well as traffic management, enabling prioritization and isolation of IP flows between vNICs."

- Solarflare Press Release

SOLORI's Take: The controversy surrounding VMware's "focus" on the VMware "sphere" of products was a non-starter. The name VMworld does not stand for "Virtualization World" - it stands for "VMware World" and denying competitor's "marketing access" to that venue seems like a reasonable restriction. While it may seem like a strong-arm tactic to some, insisting that vendors/partners are there "for VMworld only" - and hence restricting cross-marketing efforts in and around the venue - makes it more difficult for direct competitors to play the "NASCAR-style marketing" (as Toigo calls it) game.

VMworld is a showcase for technologies driving the virtualization eco-system as seen from VMware's perspective. While there are a growing number of competitors for virtualization mind-share, VMware's pace and vision - to date - has been driven by careful observation of use-case more so than innovation for innovation's sake. It is this attention to business need that has made VMware successful and what defines VMworld's focus - and it is in that light that VMworld 2009 looks like a great success.

http://www.networkworld.com/news/2009/090309-vmworld-vmware-roundup.html

Quick Take: AristaNetworks Announces vEOS for vNDS

Arista Networks - the Menlo Park, CA "cloud networking solutions" company - has announced the first competitor to Cisco's Nexus 1000V virtual, distributed switch for VMware's vSphere. Called simply vEOS (aka Virtualized Extensible Operating System) the virtual machine application integrates into the VMware vNetwork Distributed Switch (vNDS) framework to accomplish its task.

The current advantages posed by the vNDS architecture are preservation of a VM's network state across vMotion events and port configuration simplicity. Enhancements for the vEOS virtual machine will include QoS (with TX/RX limiting), ACL enforcement (when used with Arista's 7000 series switch family), CLI configuration and management, distributed port profiles, port profile inheritance, VMware port mirroring, SNMP v3 RW, syslog exports, active standby control plane, SSH/telnet access to CLI, "hitless" control plane upgrades, non-disruptive installation and integrated vSwitch upgrade workflow.

Another advantage of vEOS for adopters of Arista's 7000-series 10Gbase-T switches is its derrivation from the switch's own EOS code base. The EOS image is monolithic with respect to feature set, so there are no "trains to catch" for compatibility, et al. Michael Morris from NetworkWorld has an article about the vEOS announcement along with some additional information about Arista's plans and how to get your hands on a beta version of vEOS... See Arista's official press release about vEOS.

SOLORI's Take: While CNA's are great, 10Gbase-T provides a less "process disruptive" access to greater network bandwidth. Arista's move with vEOS is challenging both Cisco's presence in the VMware market and its vision for network convergence. Just like Ethernet won-out over "superior" technologies based on its simplicity, we're looking at 10Gbase-T's drop-in simplicity (CAT5e support, 100/1G/10G auto-negotiate, etc.) to drive obvious market share for related products. With the 2010 battle lines drawn in the virtualization platform market, the all-important network segment will be as much a scalability factor as it is a budgetary one.

When using the coming "massive" virtualization potential of 2P/4P hardware being released in 2010 (see our Quick Take on 48-core virtualization) the obvious conduit for network traffic into 180+ VM consolidations is 10GE. The question: will 10Gbase-T deliver order-of-magnitude economies of scale needed to displace the technical advantages of DCE/CNA-based networking?

Quick Take: HP Plants the Flag with 48-core VMmark Milestones

Following on the heels of last month we predicted that HP could easily claim the VMmark summit with its DL785 G6 using AMD's Istanbul processors:

If AMD’s Istanbul scales to 8-socket at least as efficiently as Dunnington, we should be seeing some 48-core results in the 43.8@30 tile range in the next month or so from HP’s 785 G6 with 8-AMD 8439 SE processors. You might ask: what virtualization applications scale to 48-cores when $/VM is doubled at the same time? We don’t have that answer, and judging by Intel and AMD’s scale-by-hub designs coming in 2010, that market will need to be created at the OEM level.

Well, HP didn't make us wait too long. Today, the PC maker cleared two significant VMmark milestones: crossing the 30 tile barrier in a single system (180 VMs) and exceeding the 40 mark on VMmark score. With a score of 47.77@30 tiles, the HP DL785 G6 - powered by 8 AMD Istanbul 8439 SE processors and 256GB of DDR2/667 memory - set the bar well beyond the competition and does so with better performance than we expected - most likely due to AMD's "HT assist" technology increasing its scalability.

Not available until September 14, 2009, the HP DL785 G6 is a pricey competitor. We estimate - based on today's processor and memory prices - that a system as well appointed as the VMmark-configured version (additional NICs, HBA, etc) will run at least $54,000 or around $300/VM (about $60/VM higher than the 24-core contender and about $35/VM lower than HP's Dunnnigton "equivalent").

SOLORI's Take: While the September timing of the release might imply a G6 with AMD's SR5690 and IOMMU, we're doubtful that the timing is anything but a coincidence: even though such a pairing would enable PCIe 2.0 and highly effective 10Gbps solutions. The modular design of the DL785 series - with its ability to scale from 4P to 8P in the same system - mitigates the economic realities of the dwindling 8P segment, and HP has delivered the pinnacle of performance for this technology.

We are also impressed with HP's performance team and their ability to scale Shanghai to Istanbul with relative efficiency. Moving from DL785 G5 quad-core to DL785 G6 six-core was an almost perfect linear increase in capacity (95% of theoretical increase from 32-core to 48-core) while performance-per-tile increased by 6%. This further demonstrates the "home run" AMD has hit with Istanbul and underscores the excellent value proposition of Socket-F systems over the last several years.

Unfortunately, while they demonstrate a 91% scaling efficiency from 12-core to 24-core, HP and Istanbul have only achieved a 75% incremental scaling efficiency from 24-cores to 48-cores. When looking at tile-per-core scaling using the 8-core, 2P system as a baseline (1:1 tile-to-core ratio), 2P, 4P and 8P Istanbul deliver 91%, 83% and 62.5% efficiencies overall, respectively. However, compared to the %58 and 50% tile-to-core efficiencies of Dunnington 4P and 8P, respectively, Istanbul clearly dominates the 4P and 8P performance and price-performance landscape in 2009.

In today's age of virtualization-driven scale-out, SOLORI's calculus indicates that multi-socket solutions that deliver a tile-to-core ratio of less than 75% will not succeed (economically) in the virtualization use case in 2010, regardless of socket count. That said - even at a 2:3 tile-to-core ratio - the 8P, 48-core Istanbul will likely reign supreme as the VMmark heavy-weight champion of 2009.

SOLORI's 2nd Take: HP and AMD's achievements with this Istanbul system should be recognized before we usher-in the next wave of technology like Magny-Cours and Socket G34. While the DL785 G6 is not a game changer, its footnote in computing history may well be as a preview of what we can expect to see out of Magny-Cours in 2H/2010. If 12-core, 4P system price shrinks with the socket count we could be looking at a $150/VM price-point for a 4P system: now that would be a serious game changer.

Quick Take: 6-core "Gulftown" Nehalem-EP Spotted, Tested

TechReport is reporting on a Taiwanese overclocker who may be testing a pair of Nehalem 6-core processors (2P) slated for release early in 2010. Likewise, AlienBabelTech mentions a Chinese website, HKEPC, that has preliminary testing completed on the desktop (1P) variant of the 6-core. While these could be different 32nm silicon parts, it is more likely - judging from the CPU-Z outputs and provided package pictures - that these are the same sample SKUs tested as 1P and 2P LGA-1366 components.

What does this mean for AMD and the only 6-core shipping today? Since Intel's still projecting Q2/2010 for the server part, AMD has a decent opportunity to grow market share for Istanbul. Intel's biggest rival will be itself - facing a wildly growing number of SKU's in across its i-line from i5, i7, i8 and i9 "families" with multiple speed and feature variants. Clearly, the non-HT version would stand as a direct competitor to Istanbul's native 6-core SKUs. Likewise, Istanbul may be no match for the 6-core Nehalem with HT and "turbo core" feature set.

However, with an 8-core "Beckton" Nehalem variant on the horizon, it might be hard to understand just where the Gulftown fits in Intel's picture. Intel faces a serious production issue, filling fab capacity with 4-core, 6-core and 8-core processors, each with speed, power, socket and HT variants from which to supply high-speed, high-power SKUs and lower-speed, low-power SKUs for 1P, 2P and 4P+ destinations. Doing the simple math with 3 SKU's per part Intel would be offering the market a minimum of 18 base parts according to their current marketing strategy: 9 with HT/turbo, 9 without HT/turbo. For socket LGA-1366, this could easily mean 40+ SKUs with 1xQPI and 2xQPI variants included (up from 23).

SOLORI's take: Intel will have to create some interesting "crippling or pricing tricks" to keep Gulftown from canibalizing the Gainstown market. If they follow their "normal" play book, we prodict the next 10-months will play out like this:

1. Initially there will be no 8-core product for 1P and 2P systems (LGA-1366), allowing for artificially high margins on the 8-core EX chip (LGA-1567), slowing the enevitable canibalization of the 4-core/2P market, and easing production burdens;


2. Intel will silently and abruptly kill Itanium in favor of "hyper-scale" Nehalem-EX variants;


3. Gulftown will remain high-power (90-130W TDP) and be positioned against AMD's G34 systems and Magny-Cours - plotting 12-core against 12-thread;


4. Intel creates a "socket refresh" (LGA-1566?) to enable "inexpensive" 2P-4P platforms from its Gulftown/Beckton line-up in 2H/2010 (ostensibly to maintain parity with G34) without hurting EX;


5. Revised, lower-power variants of Gainstown will be positioned against AMD's C32 target market;


6. Intel will cut SKUs in favor of higher margins, increasing speed and features for "same dollar" cost;


7. Non-HT parts will begin to disappear in 4-core configurations completely;


8. Intel's AES enhancements in Gulftown will allow it to further differentiate itself in storage and security markets;

It would be a mistake for Intel to continue growing SKU count or provide too much overlap between 4-core HT and 6-core non-HT offerings. If purchasing trends soften in 4Q/09 and remain (relatively) flat through 2Q/10, Intel will benefit from a leaner, well differentiated line-up. AMD has already announced a "leaner" plan for G34/C32. If all goes well at the fabs, 1H/2010 will be a good ole fashioned street fight between blue and green.