Las Solanas CRM Web Hosting Site Gallery Virtualization Forum DataCore Software

Storage Virtualization | FAQs & Discussions How To Build A High Performance H/A iSCSI SAN using IBM System x3655 Servers In this example, we will build a scaleable iSCSI SAN storage array featuring Synchronous Data Mirroring for Extreme High Availability using IBM System x (a.k.a. xSeries) servers. First of all, we will need some mechanism to turn an x86 Server into a SAN Storage Array. To that end, we will use a unique software package from DataCore Software Corporation called SANmelody™. This Storage Virtualization software has the logic needed to implement a SAN Storage Processor and offers many features associated with High-End storage systems. Two of these will be of interest to us here: High-Performance through caching and an advanced real-time I/O subsystem High-Availability through Synchronous Data Mirroring over iSCSI and/or Fibre Channel Building The Storage Processor Hardware The choice of the IBM System x3655 for this example was based on the server's expansion slots and internal drive slots. The x3655 comes in flavours of either eight 2.5" or six 3.5" internal drives. The 3.5" slots are hot swappable and can accommodate the large 500GB SATA drives. The server offers 4 expansion slots (2 PCI Express x8, 1 PCI Express x4, and a PCI-X 133 MHz) allowing us to add in additional RAID controllers, Gig-E and / or FC cards. We begin by selecting processor and memory options. We decide on a good price/performance balance and choose an AMD Dual-Core Opteron 2.4GHz and add in 4GB of RAM. The server's memory will be used as storage processor cache.

Next we choose the disks. The x3655 offers a choice between SAS and SATA drives. In the interest of maximizing capacity, we will load all 6 3.5" Hot-Swap drive slots with 500GB SATA drives. While not strictly necessary to implement data protection in this HA configuration, we will optionally upgrade the internal storage controller to a ServeRAID-8K, for implementing RAID-5 on the internal drives. More on this later.

The x3655 has two integrated Gig-E ports which can serve as iSCSI targets. We will add two additional Gig-E HBA's, bringing our iSCSI port count to 4.

Finally, we add in an optional redundant power supply. I would recommend adding a UPS, as well.

The following IBM System x3655 server was configured using IBM's online store. Date of this configuration: November 12, 2006.

Total cost of the server as configured: $8070.00.

Turning The Server Into A SAN Storage Array

So we've configured our high-performance SAN storage processor hardware with 4GB of cache, 4 Gig-E ports, and 3.0TB of raw storage, all in a 2U enclosure. The one thing that's missing is the logic to turn this standard high-power server into a SAN: the SAN volume, caching, and LUN masking management, as well as the drivers to run the Gig-E ports as front-end iSCSI "targets".

For this functionality we will use a unique software package from DataCore Software Corporation called SANmelody™. This software will turn the IBM System x3655 into a SAN storage array far more powerful and performant than any storage appliance — but for less money.

SANmelody™ runs on a standard, familiar Windows OS platform. We will first perform a clean install of Windows OS (such as XP Pro SP2 or 2003 R2) on the x3655 server. Then we will install the SANmelody software. Upon rebooting the machine, the SANmelody software will load its target drivers on any available ports (Gig-E for iSCSI, FC on any Fibre Channel HBAs) and once the SANmelody services are started, the server will behave as a storage processor. The SANmelody software will drive the Windows OS as a real-time process, implementing I/O Polling on the iSCSI and FC ports.

The SANmelody software is managed from the MMC (Microsoft Management Console), under the storage section. The interface is intuitive and an online help system provides a "Quick Start" tutorial to get us going. If you can manage a Windows server, you can manage a SANmelody™ SAN.

Thus far, we've built a scaleable iSCSI SAN storage array for very little money. This single storage processor model features 4 iSCSI front end ports, 4GB of cache, and 3TB of SATA storage. This iSCSI SAN array still has two slots available so we could add an external RAID controller to increase our capacity, as well as add additional Gig-E ports or even a Fibre Channel HBA. This SAN will cost significantly less than, say, an EqualLogic PS50E (limited to only 3 iSCSI front end ports, 1GB of cache and 1.7TB of SATA storage), but will largely outperform it.

Still, like the PS50E and other similar appliances, our Single Storage Processor SAN is a Single Point of Failure (SPOF). To address our High Availability requirement, we turn again to the DataCore SANmelody software.

Implementing Extreme High-Availability

The traditional midrange storage array achieves High-Availability through redundancy: two storage processors share a common backplane over which they can mirror their data (i.e. Mirrored Write Caching). Most of the traditional arrays implement mirrored write caching over a traditional media, such as a shared FC bus on the backplane. Both storage processors are attached to the physical spindles in a "dual-ported" fashion. While at first this appears to be an excellent means of protecting the data and keeping it available, the storage array itself is a single point of failure. I have heard numerous accounts first-hand from customers who have had downtime on their highly-available systems. A few have involved broken water pipes flooding an inadequately insulated data center. One involved a failed fibre channel drive in a StorageTek-badged array. The dual-ported drive's failed behavior caused a "LIP Storm"; both storage processors to begin taking down RAID groups.

Such interruptions to production could be avoided by simply separating the dual storage processors, each with its own physical storage attached. Mirrored write caching could be implemented by a Fibre Channel or iSCSI inter-connect. This is exactly what we propose to do using the SANmelody software. The SANmelody software has an optional "Auto Failover" feature that creates a partnership between two SANmelody servers. For any given virtual volume, a synchronously mirrored pair can be created, effectively a RAID-1 mirror. If any component fails (cable, HBA, SANmelody server, etc.), the application servers can fail over seamlessly to the surviving SANmelody node and continue to access their data. The SANmelody failover feature is an intelligent implementation with similar characteristics one would find on a mid-range or Tier-1 system. However, SANmelody takes High-Availability to the extreme, separating the storage processors and physical spindles.

SANmelody's synchronous data mirroring is implemented on a virtual volume basis. This is to say that you are not required to mirror all of the storage -- you only mirror those volumes for which you want extreme availability.

Of course, two SANmelody nodes also means an increase in the number of front end ports and double the processing power and spindles. You can load balance by distributing the volumes between the two processors.

We will create a second storage array using another, similarly configured IBM System x3655. We will separate the two servers, either across the data center, across the building, or across campus, depending on the availability of real estate and fibre access. SANmelody can use both iSCSI and FC to implement mirror channels for the Synchronous Data Mirroring. Rather than invest in FC technology, we will use iSCSI for the mirroring. Although we could interconnect the two SANmelody storage arrays over our IP SAN, it is preferable to isolate the mirrored write traffic, so I propose we use a crossover cable between two of our Gig-E ports. Thus, we've used one iSCSI port on each of the SANmelody servers for the mirror traffic, leaving us 3 front end iSCSI ports per storage processor, or a total of 6 front-end iSCSI ports. And don't forget, we still have 2 expansion slots available should we need them.

Advantages Of This System Over A Similarly Priced EqualLogic PS50E

Taking the SANmelody™ software approach to developing a SAN virtual infrastructure has numerous advantages over the appliance approach. Comparing, for example, the similarly priced low-end EqualLogic PS50E appliance with our High Availability x3655 / SANmelody™ configuration, several benefits are evident.

• The similarly priced PS50E is a SPOF (Single Point of Failure) with its single storage processor. Our SANmelody configuration offers Extreme High Availability as the two mirrored SANmelody storage processors can be physically separated.
• Higher performance. Our SANmelody™ configuration has two storage processors, each loaded with 4GB of cache. The PS50E is a single storage processor with only 1GB of RAM.
• The PS50E is limited to 3 Gig-E Network Ports which are used as iSCSI targets. Our SANmelody™ H/A solution comes with 4 Gig-E ports per server, or 8 ports total. We use two ports for the synchronous mirroring, leaving us 6 front end ports total -- twice as many as the similarly priced PS50E. But we still have 2 additional expansion slots available on each of our servers. We can use them to add additional external storage and / or add additional iSCSI or Fibre Channel front end connections, as we like.
• Any type of drive supported: SATA, SAS, Ultra320 SCSI, Fibre Channel, FireWire, USB2... any drive that can be seen in the Windows Disk Management screen of the Microsoft Management Console. The EqualLogic PS50E is limited to 250GB SATA-II drives.
• Our SANmelody™ configuration can scale beyond the PS50E's maximum of 14 x 250GB drives. Our configuration features 6TB raw, 3TB synchronously mirrored. We also have PCI slots available for adding additional RAID controllers to attach SATA or SAS enclosures. The similarly priced PS50E includes 7 x 250GB disks (1.75TB raw) with 7 empty disk slots allowing you to scale to a maximum capacity 14 x 250GB drives or 3.5TB.
• Additional SANmelody™ software license key features can be turned on to implement Remote Replication. Upgrading the SANmelody™ software is a simple license key upgrade that does not require downtime. SANmelody™ is truly scaleable and will grow as our needs grow.
• Truly modular: the software is in not bound to the server hardware or server vendor. If you are an IBM shop, you'll appreciate that this SANmelody™ server integrates into your shop as just another IBM System x server running Windows XP Pro.

This last point is particularly important. Although the PS50E and our SANmelody™ H/A configuration have similar price points, the EqualLogic appliance features "built-in obsolescence"; SANmelody™ software will keep its value long after the PS50E is end-of-life. As new features are introduced in SANmelody™, support-paying customers will get the upgrades for free.

A few years down the road as server platforms continue to evolve, and as the x3655 has come to end-of-life, we can transfer our SANmelody™ software to a newer, more powerful server. As new disk technologies come to market, our SANmelody™ server will be ready to take advantage of them. With SANmelody's Synchronous Data Mirroring, you can even plan the migration to the new server or technologies without downtime.

On the other hand, the appliance approach limits you to the feature set and technologies available at the time of purchase. An EqualLogic appliance you purchase today won't be able to adapt to use new technologies as they become available. And when the first three years of support have run out and you get the support renewal bill, it will be time to forklift the PS50E and replace it with something else. You'll have to buy the entire feature set all over again.

Las Solanas Consulting offers this information as a service and is not a DataCore or IBM reseller. DataCore does not publish their prices on the web. If you would like more information, please contact DataCore™ or a DataCore reseller for pricing information.

If you want to take a test drive of the SANmelody software, you can download a free, no-obligation 30-day evaluation. The evaluation software includes iSCSI and FC support, as well as support for their unique Thin Provisioning feature. You'll see for yourself how SANmelody out-performs the traditional storage vendor's SANs or Storage Arrays.

© 2003-2006 LAS SOLANAS CONSULTING. ALL RIGHTS RESERVED. | Terms Of Use | Various trademarks held by their respective owners.