In-the-Lab: NexentaStor vs. Grub

In this In-the-Lab segment we're going to look at how to recover from a failed ZFS version update in case you've become ambitious with your NexentaStor installation after the last Short-Take on ZFS/ZPOOL versions. If you used the "root shell" to make those changes, chances are your grub is failing after reboot. If so, this blog can help, but before you read on, observe this necessary disclaimer:

NexentaStor is an appliance operating system, not a general purpose one. The accepted way to manage the system volume is through the NMC shell and NMV web interface. Using a "root shell" to configure the file system(s) is unsupported and may void your support agreement(s) and/or license(s).

That said, let's assume that you updated the syspool filesystem and zpool to the latest versions using the "root shell" instead of the NMC (i.e. following a system update where zfs and zpool warnings declare that your pool and filesystems are too old, etc.) In such a case, the resulting syspool will not be bootable until you update grub (this happens automagically when you use the NMC commands.) When this happens, you're greeted with the following boot prompt:

grub>

Grub is now telling you that it has no idea how to boot your NexentaStor OS. Chances are there are two things that will need to happen before your system boots again:

1. Your boot archive will need updating, pointing to the latest checkpoint;


2. Your master boot record (MBR) will need to have grub installed again.

We'll update both in the same recovery session to save time (this assumes you know or have a rough idea about your intended boot checkpoint - it is usually the highest numbered rootfs-nmu-NNN checkpoint, where NNN is a three digit number.) The first step is to load the recovery console. This could have been done from the "Safe Mode" boot menu option if grub was still active. However, since grub is blown-away, we'll boot from the latest NexentaStor CD and select the recovery option from the menu.

Import the syspool

Then, we login as "root" (empty password.) From this "root shell" we can import the existing (disks connected to active controllers) syspool with the following command:

# zpool import -f syspool

Note the use of the "-f" card to force the import of the pool. Chances are, the pool will not have been "destroyed" or "exported" so zpool will "think" the pool belongs to another system (your boot system, not the rescue system). As a precaution, zpool assumes that the pool is still "in use" by the "other system" and the import is rejected to avoid "importing an imported pool" which would be completely catastrophic.

With the syspool imported, we need to mount the correct (latest) checkpointed filesystem as our boot reference for grub, destroy the local zfs.cache file (in case the pool disks have been moved, but still all there), update the boot archive to correspond to the mounted checkpoint and install grub to the disk(s) in the pool (i.e. each mirror member).

List the Checkpoints

# zfs list syspool

From the resulting list, we'll pick our highest-numbered checkpoint; for the sake of this article let's say it's "rootfs-nmu-013" and mount it.

Mount the Checkpoint

# mkdir /tmp/syspool
# mount -F zfs syspool/rootfs-nmu-013 /tmp/syspool

Remove the ZPool Cache File

# cd /tmp/syspool/etc/zfs
# rm -f zpool.cache

Update the Boot Archive

# bootadm update-archive -R /tmp/syspool

Determine the Active Disks

# zpool status syspool

For the sake of this article, let's say the syspool was a three-way mirror and the zpool status returned the following:

  pool: syspool
 state: ONLINE
  scan: resilvered 8.64M in 0h0m with 0 errors on Tue Nov 16 12:34:40 2010
config:
        NAME           STATE     READ WRITE CKSUM
        syspool        ONLINE       0     0     0
          mirror-0     ONLINE       0     0     0
            c6t13d0s0  ONLINE       0     0     0
            c6t14d0s0  ONLINE       0     0     0
            c6t15d0s0  ONLINE       0     0     0

errors: No known data errors

This enumerates the three disk mirror as being composed of disks/slices c6t13d0s0, c6t14d0s0 and c6t15d0s0. We'll use that information for the grub installation.

Install Grub to Each Mirror Disk

# cd /tmp/syspool/boot/grub
# installgrub -f -m stage[12] c6t13d0s0
# installgrub -f -m stage[12] c6t14d0s0
# installgrub -f -m stage[12] c6t15d0s0

Unmount and Reboot

# umount /tmp/syspool
# sync
# reboot

Now, the system should be restored to a bootable configuration based on the selected system checkpoint. A similar procedure can be found on Nexenta's site when using the "Safe Mode" boot option. If you follow that process, you'll quickly encounter an error - likely intentional and meant to elicit a call to support for help. See if you can spot the step...

Short-Take: ZFS version ZPOOL Versions

As features are added to ZFS - the ZFS (filesystem) code may change and/or the underlying ZFS POOL code may change. When features are added, older versions of ZFS/ZPOOL will not be able to take advantage of these new features without the ZFS filesystem and/or pool being updated first.

Since ZFS filesystems exist inside of ZFS pools, the ZFS pool may need to be upgraded before a ZFS filesystem upgrade may take place. For instance, in ZFS pool version 24, support for system attributes was added to ZFS. To allow ZFS filesystems to take advantage of these new attributes, ZFS filesystem version 4 (or higher) is required. The proper order to upgrade would be to bring the ZFS pool up to at least version 24, and then upgrade the ZFS filesystem(s) as needed.

Systems running a newer version of ZFS (pool or filesystem) may "understand" an earlier version. However, older versions of ZFS will not be able to access ZFS streams from newer versions of ZFS.

For NexentaStor users, here are the current versions of the ZFS filesystem (see "zfs upgrade -v"):

VER  DESCRIPTION
---  --------------------------------------------------------
 1   Initial ZFS filesystem version
 2   Enhanced directory entries
 3   Case insensitive and File system unique identifier (FUID)
 4   userquota, groupquota properties
 5   System attributes

For NexentaStor users, here are the current versions of the ZFS pool (see "zpool upgrade -v"):

VER  DESCRIPTION
---  --------------------------------------------------------
 1   Initial ZFS version
 2   Ditto blocks (replicated metadata)
 3   Hot spares and double parity RAID-Z
 4   zpool history
 5   Compression using the gzip algorithm
 6   bootfs pool property
 7   Separate intent log devices
 8   Delegated administration
 9   refquota and refreservation properties
 10  Cache devices
 11  Improved scrub performance
 12  Snapshot properties
 13  snapused property
 14  passthrough-x aclinherit
 15  user/group space accounting
 16  stmf property support
 17  Triple-parity RAID-Z
 18  Snapshot user holds
 19  Log device removal
 20  Compression using zle (zero-length encoding)
 21  Deduplication
 22  Received properties
 23  Slim ZIL
 24  System attributes
 25  Improved scrub stats
 26  Improved snapshot deletion performance

As versions change, upgrading the ZFS pool and filesystem is possible using the respective upgrade command. To upgrade all imported ZFS pools, issue the following command as root:

zpool upgrade -a

Likewise, to upgrade the ZFS filesystem(s) inside the pool and all child filesystems, issue the following command as root:

zfs upgrade -r -a

The new ZFS features available to these pool and filesystem version(s) will now be available to the upgraded pools/filesystems.

Short-Take: iSCSI+Nexenta, Performance Notes

Here are a few performance tips for running iSCSI with NexentaStor in a Windows environment:

1. When using the Windows iSCSI Software Initiator with some workloads, disabling the Nagle Algorithm on the Windows Server is sometimes recommended;


2. Tuning TCP window and iSCSI parameters on each side of the connection can deliver better performance;


3. VMware part of the equation? Adjusting the way VMware handles congestion events could be useful;


4. On NexentaStor, disable the Nagle Algorithm with a value of "1" (default, 4095, enabled)

For storage applications where latency is a paramount issue, these hints just might help...

Quick-Take: NexentaStor AD Shares in 100% Virtual SMB

Here's a maintenance note for SMB environments attempting 100% virtualization and relying on SAN-based file shares to simplify backup and storage management: beware the chicken-and-egg scenario on restart before going home to capture much needed Zzz's. If your domain controller is virtualized and it's VMDK file lives on SAN/NAS, you'll need to restart SMB services on the NexentaStor appliance before leaving the building.

Here's the scenario:

1. An afterhours SAN upgrade in non-HA environment (maybe Auto-CDP for BC/DR, but no active fail-over);


2. Shutdown of SAN requires shutdown of all dependent VM's, including domain controllers (AD);


3. End-user and/or maintenance plans are dependent on CIFS shares from SAN;


4. Authentication of CIFS shares on NexentaStor is AD-based;

Here's the typical maintenance plan (detail omitted):

1. Ordered shutdown of non-critical VM's (including UpdateManager, vMA, etc.);


2. Ordered shutdown of application VM's;


3. Ordered shutdown of resource VM's;


4. Ordered shutdown of AD server VM's (minus one, see step 7);


5. Migrate/vMotion remaining AD server and vCenter to a single ESX host;


6. Ordered shutdown of ESX hosts (minus one, see step 8);


7. vSphere Client: Log-out of vCenter;


8. vSphere Client: Log-in to remaining ESX host;


9. Ordered shutdown of vCenter;


10. Ordered shutdown of remaining AD server;


11. Ordered shutdown of remaining ESX host;


12. Update SAN;


13. Reboot SAN to update checkpoint;


14. Test SAN update - restoring previous checkpoint if necessary;


15. Power-on ESX host containing vCenter and AD server (see step 8);


16. vSphere Client: Log-in to remaining ESX host;


17. Power-on AD server (through to VMware Tools OK);


18. Restart SMB service on NexentaStor;


19. Power-on vCenter;


20. vSphere Client: Log-in to vCenter;


21. vSphere Client: Log-out of ESX host;


22. Power-on remaining ESX hosts;


23. Ordered power-on of remaining VM's;

A couple of things to note in an AD environment:

1. NexnetaStor requires the use of AD-based DNS for AD integration;


2. AD-based DNS will not be available at SAN re-boot if all DNS servers are virtual and only one SAN is involved;


3. Lack of DNS resolution on re-boot will cause a failure for DNS name based NTP service synchronization;


4. NexentaStor SMB service will fail to properly initialize AD credentials;


5. VMware 4.1 now pushes AD authentication all the way to ESX hosts, enabling better credential management and security but creating a potential AD dependency as well;


6. Using auto-startup order on the remaining ESX host for AD and vCenter could automate the process (steps 17 & 19), however, I prefer the "manual" approach after a SAN upgrade in case the upgrade failure is detected only after ESX host is restarted (i.e. storage service interaction in NFS/iSCSI after upgrade).

SOLORI's Take: This is a great opportunity to re-think storage resources in the SMB as the linchpin to 100% virtualization.  Since most SMB's will have a tier-2 or backup NAS/SAN (auto-sync or auto-CDP) for off-rack backup, leveraging a shared LUN/volume from that SAN/NAS for a backup domain controller is a smart move. Since tier-2 SAN's may not have the IOPs to run ALL mission critical applications during the maintenance interval, the presence of at least one valid AD server will promote a quicker RTO, post-maintenance, than coming up cold. [This even works with DAS on the ESX host]. Solution - add the following and you can ignore step 15:

3a. Migrate always-on AD server to LUN/volume on tier-2 SAN/NAS;

24. Migrate always-on AD server from LUN/volume on tier-2 SAN/NAS back to tier-1;

Since even vSphere Essentials Plus has vMotion now (a much requested and timely addition) collapsing all remaining VM's to a single ESX host is a no brainer. However, migrating the storage is another issue which cannot be resolved without either a shutdown of the VM (off-line storage migration) or Enterprise/Enterprise Plus version of vSphere. That is why the migration of the AD server from tier-2 is reserved for last (step 17) - it will likely need to be shutdown to migrate the storage between SAN/NAS appliances.

ZFS Pool Import Fails After Power Outage

The early summer storms have taken its toll on Alabama and UPS failures (and short-falls) have been popping-up all over. Add consolidated, shared storage to the equation and you have a recipe for potential data loss - at least this is what we've been seeing recently. Add JBOD's with separate power rails and limited UPS life-time and/or no generator backup and you've got a recipe for potential data loss.

Even with ZFS pools, data integrity in a power event cannot be guaranteed - especially when employing "desktop" drives and RAID controllers with RAM cache and no BBU (or perhaps a "bad storage admin" that has managed to disable the ZIL). When this happens, NexentaStor (an other ZFS storage devices) may even show all members in the ZFS pool as "ONLINE" as if they are awaiting proper import. However, when an import is attempted (either automatically on reboot or manually) the pool fails to import.

From the command line, the suspect pool's status might look like this:

root@NexentaStor:~# zpool import
pool: pool0
id: 710683863402427473
state: ONLINE
action: The pool can be imported using its name or numeric identifier.
config:
        pool0        ONLINE
          mirror-0   ONLINE
            c1t12d0  ONLINE
            c1t13d0  ONLINE
          mirror-1   ONLINE
            c1t14d0  ONLINE
            c1t15d0  ONLINE

Looks good, but the import it may fail like this:

root@NexentaStor:~# zpool import pool0
cannot import 'pool0': I/O error

Not good. This probably indicates that something is not right with the array. Let's try to force the import and see what happens:

Nope. Now this is the point where most people start to get nervous, their neck tightens-up a bit and they begin to flip through a mental calendar of backup schedules and catalog backup repositories - I know I do. However, it's the next one that makes most administrators really nervous when trying to "force" the import:

root@NexentaStor:~# zpool import -f pool0
pool: pool0
id: 710683863402427473
status: The pool metadata is corrupted and the pool cannot be opened.
action: Restore the file in question if possible.  Otherwise restore the
        entire pool from backup.
cannot import 'pool0': I/O error

Really not good. Did it really suggest going to backup? Ouch!.

In this case, something must have happened to corrupt metadata - perhaps the non-BBU cache on the RAID device when power failed. Expensive lesson learned? Not yet. The ZFS file system still presents you with options, namely "acceptable data loss" for the period of time accounted for in the RAID controller's cache. Since ZFS writes data in transaction groups and transaction groups normally commit in 20-30 second intervals, that RAID controller's lack of BBU puts some or all of that pending group at risk. Here's how to tell by testing the forced import as if data loss was allowed:

root@NexentaStor:~# zpool import -nfF pool0
Would be able to return data to its state as of Fri May 7 10:14:32 2010.
Would discard approximately 30 seconds of transactions.

or

root@NexentaStor:~# zpool import -nfF pool0
WARNING: can't open objset for pool0

If the first output is acceptable, then proceeding without the "n" option will produce the desired effect by "rewinding" the last couple of transaction groups (read ignoring) and imported the "truncated" pool. The "import" option will report the exact number of "seconds" worth of data that cannot be restored. Depending on the bandwidth and utilization of your system, this could be very little data or several MB worth of transaction(s).

What to do about the second option? From the man pages on "zpool import" Sun/Oracle says the following:

zpool import [-o mntopts] [ -o property=value] ... [-d dir | -c cachefile] [-D] [-f] [-R root] [-F [-n]]-a

Imports all pools found in the search directories. Identical to the previous command, except that all pools with a sufficient number of devices available are imported. Destroyed pools, pools that were previously destroyed with the “zpool destroy” command, will not be imported unless the-D option is specified.

-o mntopts

Comma-separated list of mount options to use when mounting datasets within the pool. See zfs(1M) for a description of dataset properties and mount options.

-o property=value

Sets the specified property on the imported pool. See the “Properties” section for more information on the available pool properties.

-c cachefile

Reads configuration from the given cachefile that was created with the “cachefile” pool property. This cachefile is used instead of searching for devices.

-d dir

Searches for devices or files in dir. The -d option can be specified multiple times. This option is incompatible with the -c option.

-D

Imports destroyed pools only. The -f option is also required.

-f

Forces import, even if the pool appears to be potentially active.

-F

Recovery mode for a non-importable pool. Attempt to return the pool to an importable state by discarding the last few transactions. Not all damaged pools can be recovered by using this option. If successful, the data from the discarded transactions is irretrievably lost. This option is ignored if the pool is importable or already imported.

-a

Searches for and imports all pools found.

-R root

Sets the “cachefile” property to “none” and the “altroot” property to “root”.

-n

Used with the -F recovery option. Determines whether a non-importable pool can be made importable again, but does not actually perform the pool recovery. For more details about pool recovery mode, see the -F option, above.

No real help here. What the documentation omits is the "-X" option. This option is only valid with the "-F" recovery mode setting, however it is NOT well documented suffice to say it is the last resort before acquiescing to real problem solving... Assuming the standard recovery mode "depth" of transaction replay is not quite enough to get you over the hump, the "-X" option gives you an "extended replay" by seemingly providing a scrub-like search through the transaction groups (read "potentially time consuming") until it arrives at the last reliable transaction group in the dataset.

Lessons to be learned from this excursion into pool recovery are as follows:

1. Enterprise SAS good; desktop SATA could be a trap


2. Redundant Power + UPS + Generator = Protected; Anything else = Risk


3. SAS/RAID Controller + Cache + BBU = Fast; SAS/RAID Controller + Cache - BBU = Train Wreck

The data integrity functions in ZFS are solid when used appropriately. When architecting your HOME/SOHO/SMB NAS appliance, pay attention to the hidden risks of "promised performance" that may walk you down the plank towards a tape backup (or resume writing) event. Better to leave the 5-15% performance benefit on the table or purchase adequate BBU/UPS/Generator resources to supplant your system in worst-case events. In complex environments, a pending power loss can be properly mitigated through management supervisors and clever scripts: turning down resources in advance of total failure. How valuable is your data???

NexentaStor CIFS Shares with Active Directory Authentication

Sharing folders in NexentaStor is pretty easy in Workgroup mode, but Active Directory integration takes a few extra steps.  Unfortunately, it's not (yet) as easy as point-and-click, but it doesn't have to be too difficult either. (The following assumes/requires that the NexentaStor appliance has been correctly configured-in and joined-to Active Directory.)

[caption id="attachment_1562" align="alignright" width="170" caption="Typical user and group permissions for a local hard disk in Windows."][/caption]

Let's examine the case where a domain admin group will have "Full Control" of the share, and "Everyone" will have read/execute permissions. This is a typical use case where a single share contains multiple user directories under administrative control. It's the same configuration as local disks in a Windows environment. For our example, we're going to mimic this setup using a CIFS share from a NexentaStor CE appliance and create the basic ACL to allow for Windows AD control.

For this process to work, we need to join the NexentaStor appliance to the Active Directory Domain. The best practice is to create the machine account in AD first, assign control user/group rights (if possible) and then attempt to join. It is IMPORTANT that the host name and DNS configuration of the NexentaStor appliance match domain norms, or things will come crashing to a halt pretty quickly.

That said, assuming that your DC is 1.1.1.1 and your BDC is 1.1.1.2 with a "short" domain of "SOLORI" and a FQDN of "SOLORI.MSFT" your NexentaStor's name server configuration (Settings->Network->Name Servers) would look something like this:

This is important because the AD queries will pull service records from the configured domain name servers. If these point to an "Internet" DNS server, the AD entries may not be reflected in that server's database and AD authentication (as well as join) will fail.

The other way the NexentaStor appliance knows what AD Domain to look into is by its own host name. For AD authentication to work properly, the NexentaStor host name must reflect the AD domain. For example, if the FQDN of your AD domain is "SOLORI.MSFT" then your domain name on the appliance would be configured like this (Appliance->Basic Settings->Domainname):

The next step is to create the machine account in AD using "Active Directory Users and Computers" administrator's configuration tool. Find your domain folder and right-click "Computers" - select New->Computer from the menu and enter the computer name (no domain). The default user group assigned to administrative control should be Domain Admins. Since this works for our example, no changes are necessary so click "OK" to complete.

Now it's time to join the AD domain from NexentaStor. Any user with permissions to join a machine to the domain will do. Armed with that information, drill down to Data Management->Shares->CIFS Server->Join AD/DNS Server and enter the AD/DNS server. AD server, AD user and user password into the configuration box:

If your permissions and credentials are good, your NexentaStor appliance is not a member of your domain. As such, it can now identify AD users and groups by unique gid and uid data created from AD. This gid and uid information will be used to create our ACLs for the CIFS share.

To uncover the gid for the "Domain Admins" and "Domain Users" groups, we issue the following from the NexentaStor NMC (CLI):

nmc@san01:/$ idmap dump -n | grep "Domain Admins"

wingroup:Domain Admins@solori.msft     ==      gid:3036392745

nmc@san01:/$ idmap dump -n | grep "Domain Users"
wingroup:Domain Users@solori.msft     ==      gid:1238392562

Now we can construct a CIFS share (with anonymous read/write disabled) and apply the Domain Admin gid to an ACL - just click on the share, and then click "(+) Add Permissions for Group":

[caption id="attachment_1568" align="aligncenter" width="300" caption="Applying administrative permissions with the AD group ID for Domain Admins."][/caption]

We do similarly with the Domain User gid:

[caption id="attachment_1569" align="aligncenter" width="300" caption="Applying the Domain User gid to CIFS share ACL."][/caption]

Note that the "Domain Users" group gets only "execute" and "read" permissions while the "Domain Admins" group gets full control - just like the local disk! Now, with CIFS sharing enabled and the ACL suited to our AD authentication, we can access the share from any domain machine provided our user is in the Domain Users or Admins group.

Administrators can now create "personal" folders and assign detailed user rights just as they would do with any shared storage device. The only trick is in creating the initial ACL for the CIFS share - as about - and you've successfully integrated your NexentaStor appliance into you AD domain.

NOTE: If you're running Windows Server 2008 (or SBS 2008) as your AD controller, you will need to update the share mode prior to joining the domain using the following command (from root CLI):

# sharectl set -p lmauth_level=2 smb

NOTE: I’ve also noticed that, upon reboot of the appliance (i.e. after a major update of the kernel/modules) your ephemeral id mapping takes some time to populate during which time authentication failures to CIFS shares can fail. This appears to have something to do with the state of ephemeral-to-SID mapping after re-boot.

To enable the mapping of unresolvable SIDs, do the following:

$ svccfg -s idmap setprop config/unresolvable_sid_mapping = boolean: true
$ svcadm refresh idmap

In-the-Lab: Full ESX/vMotion Test Lab in a Box, Part 2

In Part 1 of this series we introduced the basic Lab-in-a-Box platform and outlined how it would be used to provide the three major components of a vMotion lab: (1) shared storage, (2) high speed network and (3) multiple ESX hosts. If you have followed along in your lab, you should now have an operating VMware ESXi 4 system with at least two drives and a properly configured network stack.

In Part 2 of this series we're going to deploy a Virtual Storage Appliance (VSA) based on an open storage platform which uses Sun's Zetabyte File System (ZFS) as its underpinnings. We've been working with Nexenta's NexentaStor SAN operating system for some time now and will use it - with its web-based volume management - instead of deploying OpenSolaris and creating storage manually.

Part 2, Choosing a Virtual Storage Architecture

To get started on the VSA, we want to identify some key features and concepts that caused us to choose NexentaStor over a myriad of other options. These are:

• NexentaStor is based on open storage concepts and licensing;


• NexentaStor comes in a "free" developer's version with 4TB 2TB of managed storage;


• NexentaStor developer's version includes snapshots, replication, CIFS, NFS and performance monitoring facilities;


• NexentaStor is available in a fully supported, commercially licensed variant with very affordable $/TB licensing costs;


• NexentaStor has proven extremely reliable and forgiving in the lab and in the field;


• Nexenta is a VMware Technology Alliance Partner with VMware-specific plug-ins (commercial product) that facilitate the production use of NexentaStor with little administrative input;


• Sun's ZFS (and hence NexentaStor) was designed for commodity hardware and makes good use of additional RAM for cache as well as SSD's for read and write caching;


• Sun's ZFS is designed to maximize end-to-end data integrity - a key point when ALL system components live in the storage domain (i.e. virtualized);


• Sun's ZFS employs several "simple but advanced" architectural concepts that maximize performance capabilities on commodity hardware: increasing IOPs and reducing latency;

While the performance features of NexentaStor/ZFS are well outside the capabilities of an inexpensive "all-in-one-box" lab, the concepts behind them are important enough to touch on briefly. Once understood, the concepts behind ZFS make it a compelling architecture to use with virtualized workloads. Eric Sproul has a short slide deck on ZFS that's worth reviewing.

ZFS and Cache - DRAM, Disks and SSD's

Legacy SAN architectures are typically split into two elements: cache and disks. While not always monolithic, the cache in legacy storage typically are single-purpose pools set aside to hold frequently accessed blocks of storage - allowing this information to be read/written from/to RAM instead of disk. Such caches are generally very expensive to expand (when possible) and may only accomodate one specific cache function (i.e. read or write, not both). Storage vendors employ many strategies to "predict" what information should stay in cache and how to manage it to effectively improve overall storage throughput.

[caption id="attachment_968" align="aligncenter" width="405" caption="New cache model used by ZFS allows main memory and fast SSDs to be used as read cache and write cache, reducing the need for large DRAM cache facilities."][/caption]

In-the-Lab: Full ESX/vMotion Test Lab in a Box, Part 1

There are many features in vSphere worth exploring but to do so requires committing time, effort, testing, training and hardware resources. In this feature, we'll investigate a way - using your existing VMware facilities - to reduce the time, effort and hardware needed to test and train-up on vSphere's ESXi, ESX and vCenter components. We'll start with a single hardware server running VMware ESXi free as the "lab mule" and install everything we need on top of that system.

Part 1, Getting Started

To get started, here are the major hardware and software items you will need to follow along:

Recommended Lab Hardware Components

• One 2P, 6-core AMD "Istanbul" Opteron system


• Two 500-1,500GB Hard Drives


• 24GB DDR2/800 Memory


• Four 1Gbps Ethernet Ports (4x1, 2x2 or 1x4)


• One 4GB SanDisk "Cruiser" USB Flash Drive


• Either of the following:
  
  
  
  • One CD-ROM with VMware-VMvisor-Installer-4.0.0-164009.x86_64.iso burned to it
  
  
  • An IP/KVM management card to export ISO images to the lab system from the network
  
  
  
  

Recommended Lab Software Components

• One ISO image of NexentaStor 2.x (for the Virtual Storage Appliance, VSA, component)


• One ISO image of ESX 4.0


• One ISO image of ESXi 4.0


• One ISO image of VCenter Server 4


• One ISO image of Windows Server 2003 STD (for vCenter installation and testing)

For the hardware items to work, you'll need to check your system components against the VMware HCL and community supported hardware lists. For best results, always disable (in BIOS) or physically remove all unsupported or unused hardware- this includes communication ports, USB, software RAID, etc. Doing so will reduce potential hardware conflicts from unsupported devices.

The Lab Setup

We're first going to install VMware ESXi 4.0 on the "test mule" and configure the local storage for maximum use. Next, we'll create three (3) machines two create our "virtual testing lab" - deploying ESX, ESXi and NexentaStor running directly on top of our ESXi "test mule." All subsequent tests VMs will be running in either of the virtualized ESX platforms from shared storage provided by the NexentaStor VSA.

[caption id="attachment_929" align="aligncenter" width="450" caption="ESX, ESXi and VSA running atop ESXi"][/caption]

Next up, quick-and-easy install of ESXi to USB Flash...

Nexenta Turns 2.0

Beth Pariseau, Senior News Writer for SearchStorage.com, has an interview with Nexenta CEO Evan Powell about the release of NexentaStor 2.0 today. The open source storage vendor is making some fundamentally "enterprise focused" changes to its platform in this release by adding active-active high availability features and 24/7 phone support.

"Version 2.0 is Nexenta's attempt to "cross the chasm" between the open-source community and the traditional enterprise. Chief among these new features is the ability to perform fully automated two-way high availability between ZFS server nodes. Nexenta has already made synchronous replication and manual failover available for ZFS, which doesn't offer those features natively, Powell said. With the release of Nexenta's High Availability 1.0 software, failover and failback to the secondary server can happen without human intervention."

- SearchStorage.com

In a related webinar and conference call today, Powell reiterated Nexenta's support of open storage saying, "we believe that you should own your storage. Legacy vendors want to lock you into their storage platform, but with Nexenta you can take your storage to any platform that speaks ZFS." Powell sees Nexenta's anti-lock-in approach as part of their wider value proposition. When asked about de-duplication technology, he referred to Sun's prototyped de-duplication technology and the promise to introduce it into the main line this summer.

In-the-Lab: VSA Shoot-out

We're cooking with gas over the next two weeks, making performance contrasts using similar resource build-outs of:

1. FreeNAS (0.7 nightly build)


2. OpenFiler 2.3


3. NexentaStor (1.1.5 dev)


4. SunStorage (VSA)

All configured as VSA's under ESXi with pass-thru access to local attached storage. Why do this? ESX provides some supervisory analysis that makes quick lab analysis easy. Also, VSA storage is a 2009 killer-app and absolutely necessary for targeted micro-deployments.

These are not indepth storage studys, but an out-of-the-box analysis of what to expect and where to spend your time=money. Stay tuned...