Las Solanas Consulting

Storage Virtualization | FAQs & Discussions

Build a Business Continuity 4.8TB iSCSI SAN Solution with Tiered Storage using two HP DL180 G6 Servers

Author: Tim Warden

Date: October 8, 2009

BUILDING A SANmelody™ SAN

The recent acquisitions of the two leading independent iSCSI SAN appliance vendors by Dell [EqualLogic] and HP [LeftHand] have created a vacuum leaving many customers and their resellers looking for alternative solutions offering greater flexibility and needed scalability to meet growth and changing future requirements. If you count yourself among those looking for an alternative, look no further: we have your solution here. And if you're one of those looking for a better way to do SAN storage control, read on.

In this cookbook, we will use two HP DL180 G6 Windows 2008 servers to build a flexible, scalable, highly available iSCSI SAN Storage Appliance. The servers will run DataCore's SANmelody software package, which will turn them into dedicated storage controllers. SANmelody implements a rich feature set, including both iSCSI and Fibre Channel targets, Read/Write Caching, Thin Provisioning, Snapshots, Active/Active Synchronous Mirroring for Business Continuity, and Asynchronous Replication for Disaster Recovery.

SANmelody can be installed on any Windows 2003 or 2008 server platform, regardless of vendor, model, CPU architecture, whether rackmount, blade or VM. If Windows runs on the platform, SANmelody can run on the platform. The choice of the HP DL180 G6 server in this cookbook is arbitrary. We could just as easily build this solution with a DL380, or a Dell R710 or an IBM System x3650... or any combination thereof!

We'll configure each server with 2.4TB of storage for a total of 4.8TB. The two servers will be configured as "SANmelody Partners": Active/Active controllers that can synchronously mirror their volumes. Some vendors call this "Network RAID". DataCore calls it Synchronous Network Mirroring with Auto Failover and Failback.

The DL180 can be configured with several internal drive chassis. For this SAN, we can use the HP 8-bay LFF (Large Form Factor) chassis to give us 2.4TB of storage per controller. (HP also offers a 25 disk chassis for Small Form Factor disks — see DataCore Cookbook #20 for details on that configuration.)

As for disks, we'll be using eight 300GB 6Gb/s 15K RPM SAS drives yielding 2.4TB. In fact, we can use any of HP's Large Form Factor drives, including their lower cost SATA drives, and even their high performance SDD (Solid State) drives.

The internal disks attach via eight 6Gb/s SAS connectors to an HP P410 RAID controller configured to support 512MB of BBWC and RAID 1, 1+0, 5 and 50.

We'll build each server with a single Quad-Core CPU — we won't trouble ourselves too much with CPU performance. A 2GHz processor will be perfectly suitable for our purposes — after all, storage control is not a compute-intensive exercise.

As for RAM, 4GB would be just fine, but SANmelody implements a best-of-breed cache engine that will allocate up to 80% of the available memory for caching I/O's. RAM is cheap, so let's load this baby up with 12GB.

The DL180 G6 comes with 2 embedded 1Gb NICs. We'll add an additional quad-port NIC card, yielding 6 NICs total which can be dedicated as iSCSI Targets, iSCSI Initiators, Synchronous Mirroring channels, or used for Asynchronous Replication. We'll also take HP's ILO option, allowing total control of the server (hardware and software) via web browser.

The DL180 G6 comes standard with three PCI Express slots and a few "Riser" options. We've used two of the slots for the RAID controller and the quad-port NIC. The additional slot will come in useful when we need to expand storage capacity or add additional targets: we can easily add an external RAID controller, additional NICs or even Fibre Channel HBA's to expand the power and capacity of the SANmelody nodes.

Redundant 460W power supplies will be plugged into an optional UPS assuring that power failures don't result in loss of cached writes, allowing the UPS-aware SANmelody software to flush cache and shutdown properly.

In the next chapter we enumerate the parts for the two DL180 G6 servers.

HP DL180 G6 SERVER HARDWARE

HP DL180 G6 8-bay

QTYDESCRIPTION
1HP ProLiant DL180 G6 Rack-Mountable Server
- includes HP Embedded SATA Controller
- includes HP Embedded HP NC362i Integrated Dual Port Gigabit Server Adapter
1Quad-Core Intel® Xeon® Processor E5504 (2.00GHz, 4M Cache, 80 Watts, 800MHz)
1HP 12GB PC3-10600E 6x2GB 2Rank Memory
1HP NC364T PCI Express Quad Port Gigabit Server Adapter — Low Profile
1HP P410/512 BBWC Smart Array Controller
1HP 8-Bay Drive Cage
8HP 300GB 6G Hot Plug 3.5 SAS 15,000rpm Dual Port Hard Drive
2HP 460W HE 12V Hot Plug AC Power Supplies
2HP 1.83m 10A C13-UL US Power Cords
1HP Lights-Out 100i Remote Management
1HP Dedicated Management Port
1Microsoft® Windows® 2008 Server, Standard Edition 5 CALs
1Protected by HP Services, 3-Year Next Day Part Replacement, 0-Year Labor, 0-Year Onsite support with next business day response.
1HP Care Pack, 3 Years, 4 Hours, 13x5, Hardware, DL18x

DON'T LIKE MY CHOICE OF DISKS?

Then change it. That's the beauty of a software SAN solution... like they say at Burger King, "Have it your way..." HP has an impressive list of drives available for the DL180 G6, including high-speed Solid State Drives. And although these drives are manufactured on the same assembly lines as the so-called "Tier 1 Enterprise" drives sold by the traditional SAN vendors, as you know they are far more economical.

SAS Hot Plug 3.5" Enterprise (ENT) Drives
HP 600GB 6G SAS 15K 3.5" DP ENT HDD
HP 450GB 6G SAS 15K 3.5" DP ENT HDD
HP 450GB 3G 15K 3.5" DP SAS ENT HDD
HP 300GB 6G SAS 15K 3.5" DP ENT HDD
HP 300GB 3G SAS 15K 3.5" DP HDD
HP 146GB 3G SAS 15K 3.5" DP HDD
Hot Plug Midline (MDL) Drives
HP 1TB 3G SAS 7.2K 3.5" DP MDL HDD
HP 750GB 3G SAS 7.2K 3.5" DP MDL HDD
HP 1TB 3G SATA 7.2K 3.5" MDL HDD
HP 750GB 3G SATA 7.2K 3.5" MDL HDD
HP 500GB 3G SATA 7.2K 3.5" MDL HDD
HP 250GB 3G SATA 7.2K 3.5" MDL HDD
HP 60GB 3G SATA 3.5in MDL 1y Wty SSD (SOLID STATE)
HP 120GB 3G SATA 3.5in MDL 1y Wty SSD (SOLID STATE)

For this SAN, we chose the LFF 8-bay chassis, but HP has a 25-bay (Small Form Factor) 2.5" drive cage we could use to increase spindle density, performance and capacity. If you prefer the 2.5" drives, again you have many choices...

SAS Hot Plug SFF Enterprise (ENT) Drives
HP 300GB 6G SAS 10K SFF DP ENT HDD
HP 300GB 3G SAS 10K SFF DP ENT HDD
HP 146GB 6G SAS 15K SFF DP ENT HDD
HP 146GB 3G SAS 15K SFF DP ENT HDD
HP 146GB 6G SAS 10K SFF DP ENT HDD
HP 146GB 3G SAS 10K SFF DP ENT HDD
HP 72GB 6G SAS 15K SFF DP ENT HDD
HP 72GB 3G SAS 15K SFF DP ENT HDD
HP 72GB 3G SAS 10K SFF DP ENT HDD
Hot Plug Midline (MDL) Drives
HP 500GB 6G SAS 7.2K SFF DP MDL HDD
HP 500GB 3G SATA 7.2K 2.5in MDL HDD
HP 160GB 3G SATA 7.2K 2.5in MDL HDD
HP 60GB 3G SATA 2.5in MDL 1y Wty SSD (SOLID STATE)
HP 120GB 3G SATA 2.5in MDL 1y Wty SSD (SOLID STATE)

The most important point to keep in mind is that SANmelody can use _any_ MBR disk you can see in the Windows Device Manager. That means SATA, SAS, SDD, FC, iSCSI, SCSI, you name it!

Yes, you can even attach an older SAN appliance to your SANmelody storage server and virtualize it... give it an advanced feature set its manufacturer never intended. Try doing that with a traditional SAN storage array!

WANT FC WITH THAT? BIGGIE-SIZE IT WITH 10Gb?

Again, the software approach to SAN design gives you the greatest flexibility. You choose the options you want. DataCore's SANmelody software supports any IP stack the Windows Device Manager presents, whether 1Gb or 10Gb. The software includes an FC option with Target/Initiator drivers that supports most QLogic and Emulex Fibre Channel HBA's (Including the HP-rebranded models). In fact, SANmelody customers know that DataCore was the first on the block with an 8Gb Fibre Channel SAN — and they were able to take advantage of the capability by downloading the free Product Service Pack DataCore made available shortly after those first 8Gb cards hit the market.

DataCore has recently announced support for FCoE as well, and again, our customers will be able to leverage advances in technology via the power of architecting their SAN around portable software.

BUSINESS CONTINUITY

In these days of twits and twitter, blokes that blog and virtual attention spans, 24/7 is the norm and it's a catastrophe if your Blackberry server is down for an hour. Small shops have the same business imperatives as their Goliath enterprise counterparts and everything depends on the datacenter. Keeping your job means keeping the datacenter running.

Most IT professionals have a fundamental understanding of availability when it comes to servers: if it's a critical application, put it in a cluster. This is particularly evidenced with virtual server hosts — If a single ESX or XenServer goes down, all of its VMs and their applications will go down with it. The common practice is thus to use two or more virtualization hosts in a cluster and use advanced features such as VMware HA, XenMotion, etc., to keep the applications available.

And as your cluster servers depend intrinsically on the underlying SAN infrastructure, we'd better make sure it is also highly available. The astute reader has already figured out that the DL160 G6 server we have built here is a single point of failure. Even with its redundant power supplies and fans, and its battery-backed RAID controllers, an environmental issue can take the SAN out.

The traditional SAN vendors address High Availability via redundancy: dual storage controllers dual-porting the same disks, dual power, redundant cooling... all in a Highly-Available 19"-rackable chassis that is in and of itself a Single Point of Failure.

Perhaps we'd be better off taking the same approach to our storage that we take with our virtual server hosts: true redundancy — physically autonomous controllers, each with their own disks implementing an active-active network RAID1.

Active-Active Business Continuity

And that's just what DataCore's synchronous mirroring feature does. Two SANmelody servers are put into a "partnership". They share a common configuration and each actively presents and mirrors the SAN volumes. The mirroring takes place between the two SANmelody controllers over iSCSI or Fibre Channel. The SANmelody administrator has complete control over which volumes are to be mirrored, and whether the volumes will be simple mirrors or active/active multipath mirrors. A multipath mirror is actively presented by both controllers and provides the highest level of availability possible: true business continuity.

And because we're taking advantage of lower cost server drives and SANmelody's Thin Provisioning pools, we've made this redundancy an affordable reality — a mere fraction of what we would pay for similar functionality from one of the traditional storage vendors.

SANmelody™ — POWERFUL, FLEXIBLE, FEATURE RICH

DataCore SANmelody

Now let's add the software to turn these two HP DL180 G6 servers into a high-performance, feature rich SAN: let's add the SANmelody software.

Each of these servers has 2.4TB of raw storage, and depending on how we create our RAID stripes, we'll have at most about 2TB of usable space left. SANmelody is sold as feature-packed bundles, based on capacity. The VMS bundle will manage up to 2TB of usable capacity — we'll need two licenses, one for each server.

QTYSKUDESCRIPTION
2VMS-ESR-200-BSVDataCore VMS Feature Packed 2TB License for 1 Node
2VMS-EWR-GLD-BSVAnnual Maintenance & Support

These feature pack licenses include iSCSI, Storage Pools with RAID Striping, Thin Provisioning, Snapshots, Active-Active Synchronous Network Mirroring with Auto Failover and Failback, Asynchronous IP Mirroring, and Volume Migration. Fibre Channel is available as an option.

COST-EFFECTIVE STORAGE EXPANSION

Once you've made the initial investment in your SAN, it's just a matter of time before you'll need to add additional storage. And adding storage with the DataCore approach is as easy as adding shelves of low-cost disks attached to a RAID controller.

DataCore is very flexible with their licensing model. With their "Carry Forward Value Protection" policy, additional capacity and capabilities can be licensed at any time without a penalty for buying it later when you actually need it — you'll just pay the difference between the licenses. The Storage Pool manager allows you to have multiple pools per SANmelody node so as you add shelves, you can tier the storage by spindle type, RAID type or application.

As the two partner nodes work independently and are asymmetric, you can add features or capacity to one without necessarily adding to the other. In terms of attaching additional shelves of storage, this is a very important consideration between the DataCore solution and some of the "RAIN" or network RAID solutions available from other vendors. With some network RAIN clusters, although you can have nodes of different capacities, each node will only be able to use the capacity equal to that of the smallest node in the cluster. Ouch.

With DataCore, you can add a new shelf of 450GB or 600GB drives to either of the SANmelody servers and be able to use all of the new storage, regardless of how the other node is configured.

HP P411 SAS RAID Controller To begin our expansion, we'll add an HP P411 external RAID controller and attach an MSA60 or MSA70 shelf with disks. The P411 has two external SAS ports and each card can control up to 100 spindles.

The MSA60 shelf holds 12 of the same LFF disks we listed in the tables earlier and we can attach up to 8 of these shelves to the P411. We can tier the storage anyway we like, using large capacity SATA drives or high performance SAS. There are endless configurations possible, but if we were using all 1TB SATA drives in the 8 shelves, that would yield about 98.4TB of raw storage! If we stick with the same 300GB SAS drives we've used in this configuration that will give us over 30TB raw.

HP MSA60 300GB SAS 15K Drives

This is a far more cost effective approach to storage expansion than, say, "adding another node", as some of the appliance vendors (such as HP's own Lefthand) would recommend. It will also be much easier to manage, and use less electricity.

QTYDESCRIPTION
1Configurable - HP StorageWorks 60 Modular Smart Array Up to 12 SAS/SATA/SAS 3" MDL drives; SAS host interface supports 1+3 cascading off single x4 SAS port
12HP 300GB Hot Plug 3.5 SAS 15,000rpm Dual Port Hard Drive
1HP P411/512 BBWC Smart Array Controller

SOFTWARE... A BYTE OF PHILOSOPHY

At first, the idea of implementing a SAN using software running on standard x86/x64 servers may seem a bit unorthodox to the reader. In reality, it's just the natural evolution of the storage industry — one the traditional storage vendors are understandably reluctant to embrace.

If you think back throughout the history of IT, you'll realize that many of the technologies we commonly use began as closed, hardware-based appliances... and then evolved into software running on common computing devices. Whether we're talking copy machines, old fashioned telephones, stereos, VCRs... or proprietary databases on proprietary minicomputers or mainframes, the world accepted that was how it was done.

But with each of these technologies, at some point in time a brilliant engineer — maybe a college student working on a term project — wrote an "emulator" on a computer. Nifty idea, but nothing to be taken seriously, right? Right?

Ah, but someone — a visionary, if you will — would see the potential in that nifty idea and the engineering team would keep working at it until the emulator had a richer feature set than the thing it was emulating. Oh, and by the way, it was running on a hardware platform destined to follow Moore's law, getting faster and faster with each generation.

From 8086's to today's multi-core Gigahertz processors, we've come a long way. Memory speeds, and peripheral busses... PCI, PCI-X, PCI Express, PCIe 2.0. Let's face it: the closed appliance approach can never keep up with software leveraging the incessant advances in open server architectures. And so eventually the traditional, closed architecture appliance goes the way of the dinosaur.

Think about it — no one considers Microsoft Word to be a typewriter emulator, but isn't that effectively what word processors are? And VoIP? Telephone emulation, implemented as software running on common servers?

And as much as VMware insists that ESX is not an emulator... well, just what is a hypervisor? It's an emulator, emulating physical server hardware, and doing a darned good job of it, I might add... with a powerful feature set you just can't find on a physical server. And it's software. (Amusing that it was a college project... Stanford University. The visionary? Mendel, right? Doing for x86 servers what IBM did back in the 60s with their VM operating system...)

Funny how the SAN storage industry has been the last bastion of closed, proprietary hardware products. Particularly funny, because if you "look under the hood" of the vast majority of those traditional hardware SANs and storage arrays, you'll find server boards and software. Indeed, the EMC CX, which boasts a very large installed base, is built on Dell motherboards running Windows. Their CX3 is based on Microsoft Windows XP-Embedded, the newer CX4 is running Windows 2003 Storage Server edition — you can even see the Microsoft license holograms on the back of the chassis. And most of the iSCSI appliances sold today? Usually just hacked Linux systems, many based on CentOS.

So what is DataCore doing so different? Simple. DataCore has broken with tradition and offers SAN storage control as a software package that you can install on your favorite server platform... even if that server is a virtual machine. And is it fast? Better fasten your seatbelts.

NEXT STEPS TO STORAGE FREEDOM

Whether you're in the market for a low-cost iSCSI SAN appliance or a tier-one high-performance SAN, you should investigate the DataCore offering. DataCore's approach to the storage industry is truly unique, from their software-only solutions to their "Carry Forward Value Protection Program".

SANmelody is packaged as feature-rich, usable-capacity-based bundles. Capacity can be expanded and additional features turned on by simple and cost-effective license upgrades with no "upgrade penalties".

DataCore resells SANmelody through their network of authorized resellers who are trained and certified to architect and deploy complete DataCore SAN solutions using industry standard servers such as the DL180 G6 we've discussed here.

For more information or to download a 30-day evaluation of SANmelody, visit www.datacore.com.