Storage Virtualization | FAQs & Discussions

How To Migrate NTFS Volumes From A LeftHand SAN To Some Other Storage Array

A White Paper by Tim Warden; August 31, 2008

DATA MIGRATION — ADMIN'S NIGHTMARE, VENDOR'S LOCK

Ask anyone who's been doing shared or "SAN" storage for more than a generation's worth of hardware and they'll tell you. The experienced storage admins know to ask the question, the inexperienced discover the pain after the P.O.'s been signed. "How are we going to get our data off the old array and onto the new?"

Alas, the Storage Industry has been the last bastion of monolithic, closed architectures. In spite of all the industry talk and promises surrounding Storage Area Networks and standards, the SAN vendors products don't inter-operate, and a heterogeneous SAN environment is little more than several "Storage Islands" all physically plugged into the same fabric. Your EMC CX-340 won't talk to your DS-4200, your DS-4200 won't talk to your HP MSA-2000, your HP MSA-2000 don't know nuttin' 'bout your EqualLogic, your EqualLogic won't give the time of day to your LeftHand, and your LeftHand doesn't know what your right hand is doing.

The major vendors can offer you the "tried and true" traditional solution: what's left of your budget spent on professional services and downtime.

If you're using an advanced Server Virtualization product like VMware's ESX 3.5, you can take advantage of their "Storage VMotion" feature to move your machines and their data from one platform to another. It's a nice feature, but requires all that SAN data to be associated with VMs on an ESX platform.

But what about all those NTFS volumes you have on your faithful LeftHand Networks SAN that's gone End-Of-Life? Or maybe you've outgrown that trusty LeftHand appliance and just need more capacity... and you were shocked when you saw the price sticker for the upgrade? Perhaps the newer LeftHand higher capacity nodes weren't compatible with your older, lower capacity node and you discovered you wouldn't be able to take full advantage of the additional new capacity, so you're looking for an alternate solution, a "best of both worlds"?

In this white paper we detail a step-by-step solution for migrating NTFS volumes off of a LeftHand Networks storage appliance (the "old SAN" device) and onto another storage vendor's array (the "new SAN" device"). Our example migration uses a LeftHand appliance for the "old SAN" device, but this procedure applies equally to any backend storage, whether it be EqualLogic, Clariion, HP, IBM, HDS, or whatever. The same holds true for the "new SAN" device. The "new SAN" could be one of the aforementioned vendor's arrays, or it could simply be LUNs created using a RAID controller card (such as those made by LSI, Adaptec, or Promise) and direct attached storage.

MIGRATION VIA STORAGE PROXY

The secret to data migration between heterogeneous SANs with minimal downtime is the use of a Proxy Agent. You're already familiar with proxies... maybe you've used them in your datacenter for certain of your internet services, such as providing internet web access to your intranet users. The proxy acts as a pass-through — an intermediary, if you will — between the service requestor and the service provider. The proxy offers the same service as the provider, in many cases extending or enhancing the service, such as by improving performance via caching.

The data migration solution we propose here involves placing an advanced storage controller between your SAN storage arrays and your SAN clients in a "storage proxy" role.

THE PROCEDURE

Identify an x86 platform you can use for the Storage Proxy server ("proxy") and deploy a clean install of Microsoft Windows on the server. If your new SAN storage is also iSCSI, you can use a VM for the proxy server, as I have done in this demo. (In fact, in a few of the screenshots, you will notice the "new SAN" proxy volumes have a VMware VolumeID... I'm simply using vmdk-based disks as the new SAN storage.)

Install the Microsoft iSCSI Software Initiator on the proxy server and add your LeftHand appliance as a target. The proxy server appears to the LeftHand appliance as just another Windows server with its software iSCSI Initiator.

Determine the IQN or identifier of the proxy server — by default, it is displayed as the Initiator Node Name in the MS iSCSI Initiator control panel.
On the LeftHand SAN appliance, add the proxy server as a New Authentication Group.
On the proxy server, add the LeftHand appliance as a target.

Install the DataCore SANmelody™ storage array software on the proxy server. This storage controller software will implement the proxy service.

The installation is wizard-based. You will need to apply the license file, and then advance through the wizard's dialogs accepting defaults and dismissing any warnings to permit the product's drivers to be installed.
After the wizard completes, restart the proxy server to complete the installation and then open the MMC (the Windows management console) to access SANmelody and start the SAN controller. Click the "Start Service" button to launch SANmelody.

Identify the first server ("target server") whose volumes you want to migrate ("target volumes") and jot down the drive letters and identify their corresponding LUNs (Logical Unit Numbers) on the LeftHand appliance.

Identify the LUNs of Volumes on Server

Create corresponding LUNs on the new SAN to which you wish to migrate the target volumes. Make sure they are at least the same size or larger than the source volumes. Present those LUNs to the proxy server and discover them in the proxy server's Disk Management.

Discovering LUNs from New SAN

SANmelody™ has an elegant "Storage Pooling" feature that makes provisioning new storage a trivial "right-click". Most DataCore customers prefer using Thin Provisioned storage pools, and ordinarily I would be showing this feature in our white paper. However, in the interest of clarity in this data migration discussion, we will forego using Storage Pools and presume the LUNs we discover are coming from some other SAN storage array — that this exercise is strictly data migration. To highlight this point, we will proxy both those volumes from the old SAN and those from the new SAN. In this way, you could even retire the SANmelody server after the migration project has been completed.

In order to proxy volumes in SANmelody, the volumes must be formatted with an NTFS file system. Right-click each of the new SAN volumes and select "New Partition..." Create a single primary partition on the volume and do a "Quick Format" of NTFS, but do not assign a drive letter — we don't want to mount the volumes on the proxy server.
Add the newly discovered and formatted volumes from the new SAN into SANmelody's volume inventory. To do so, click SANmelody's "Protected Volumes" snap-in to see the list of NTFS volumes available for proxy.
Right-click over each of the two volumes you just formatted and select "Protect File System Volume".
The two volumes from the new SAN will now appear in SANmelody's volume inventory.
In SANmelody's Virtual Volume snap-in, create "virtual volumes" from the proxied volumes and name them according to your department's standard or your fancy. In this example, I am naming them based on the original target volumes from the LeftHand SAN to which they correspond, and adding a suffix of "-SS" because I intend to use them as snapshot destination volumes.

Identify a short (e.g. 15 minute) window when you can pause or shutdown the first client server ("target server") whose volumes you want to migrate off of the LeftHand appliance. Don't stop the server yet!

Use the iSCSI Initiator on the target server to discover the SANmelody target portal, and add the target server to SANmelody's Application Server inventory.

Add SANmelody iSCSI Target

In the iSCSI Initiator control panel under the "Targets" tab, login to the SANmelody target. The status should change to "Connected".
In SANmelody's Application Server snap-in, create a new application server to represent the target server.
Double-click the application server to open its Properties dialog. Select the "Channels" tab and add in the IQN from the target server.

Pause or stop the application.

Stopping SQL2005

Once the service has stopped, un-mount the target volume or volumes you wish to migrate.

Unmounting Volumes

Un-mount the volumes by removing their drive letters; this effectively flushes and quiesces the volumes.
Use the iSCSI Initiator control panel on the target server to log-out from the LeftHand appliance. This step is necessary because even after you have removed the authentication group from the volume list, the LeftHand node will continue to present the volumes to the target server.

Use the LeftHand management console to un-map the target volume or volumes from the target server, and remap them to the proxy server.

Right-click the Volume List and select "Edit Volume List".
In the ensuing dialog, select "Authentication Groups" and remove access for the group which represents the target server.
Then click "Add..." to add in the SANmelody proxy server group.
In the Edit Volume List dialog, Click "OK" to dismiss the dialog and apply the changes.

Discover the target volumes now presented to the SANmelody proxy server by the LeftHand appliance and add them to SANmelody's volume inventory.

On the proxy server, select the iSCSI Initiator control panel's Targets tab. Click "Refresh" and then Log On to the newly discovered targets.
On the proxy server, rescan to discover the target volumes in Disk Management.
On the proxy server, use SANmelody's "Protected Volumes" snap-in to add the target volumes by proxy to SANmelody's volume inventory.
Under SANmelody's Storage Server snap-in, you can verify that the volumes have indeed been added. You will note that the VolumeID indicates the volumes are from the LeftHand appliance.
Create Virtual Volumes for the old SAN's target volumes, just as we did earlier for the snapshot volumes of the new SAN, naming them appropriately.

Use SANmelody's Snapshot manager to take snapshots for each of the proxied target volumes. The proxied target volume will be the snapshot source, and the corresponding volume on the new SAN will serve as the snapshot destination.

Snapshot LeftHand Volumes

Optionally you can rename the snapshot relationship from its default of Snapshot1 to something more descriptive.
Select the corresponding volume from the new SAN to be the destination or snapshot volume
Finally, enable the snapshots. This effectively "takes the picture". SANmelody snapshots are "Copy-On-Write", pointer-based. Enabling a snapshot is instantaneous and the snapshot is immediately available for use, an exact bit-for-bit copy of the source volume at the time the snapshot was made.
In the Application Servers snap-in, map the snapshot volumes to the target server. SANmelody snapshot volumes behave like any other volumes and by default will mount read/write. These snapshot volumes — based on storage from the new SAN — will now become our production volumes.
Click "Apply Changes" to commit the configuration changes we've made; SANmelody will then actively present the proxied snapshot LUNs to the target server.

On the target server, discover and mount the snapshots of the proxied volumes as presented by the SANmelody server and un-pause or restart the application.

In Disk Management, rescan to discover the volumes. Note that if we right-click over one of the disks, we can bring up its properties dialog, and see the volume is now on a DataCore SANmelody device.
Re-mount the volumes by re-applying their corresponding drive letters.
Restart the service or application.
Verify that the data is available and coherent.

At this point, we have had downtime for the application of perhaps 10-15 minutes, depending on how fast you work. (In my lab, I was able to switch the two target volumes in just under 25 minutes — all the while painstakingly taking screenshots and pasting them into this white paper while performing the procedure.) The server has returned to production and we are ready to begin the data migration.

For each snapshot relationship, create a "Complete Image" clone of the source volume. This copies or migrates all the block-level data for the volumes from the source LeftHand device over to the snapshot volume of the new storage device. Depending on the size of the volumes, this step can take some time. You may decide to clone the volumes one at a time, and you can even throttle the speed of the migration so as not to impede the performance on your most speed-critical volumes.

Clone Volumes Across SANs

Go to lunch, take in a movie, surprise your spouse by coming home early for once, spend some quality time with the kids... Why not? Production is up, the users are as happy as users ever are, and SANmelody is migrating your storage quietly in the background.

Once the Complete Image clone has been established, we can break the snapshot relationship and remove the target source volumes. Production will continue on the new SAN volumes.

For each volume, confirm the status of the Complete Image.
Those volumes indicating a "Complete Image" status of "Established" have been completely migrated. Their snapshot relationships are no longer needed and can be deleted. To delete the snapshot, first Disable it. You can safely ignore the warning message.
Once the snapshot has been disabled, you can then right-click the relationship again and select "Delete" to remove the snapshot relationship.
You may then remove the original LeftHand source volumes from the SANmelody server. In the Virtual Volumes snap-in, right click the source volumes and select "Delete". As the volumes are managed by proxy, the delete will not in any way delete or modify the original data.
After deleting the virtual volumes and applying changes, you may now un-proxy the LeftHand source volumes by selecting "Delete" over the corresponding volumes in the Storage Server snap-in.
You may now use the LeftHand management console to un-map the volumes from the SANmelody proxy server. As we have not at all modified the LeftHand source volumes, those volumes will represent a snapshot or backup of the target server's data at the time you stopped the service. You may decide to keep them for a short time as a backup, or you may safely delete them and recover the space.

When you are ready, identify the next server whose volumes you want to migrate and repeat the procedure beginning at Step 4 above.

Once all volumes have been migrated from the old SAN to the new SAN, you can safely decommission or re-deploy the LeftHand appliance.

CONCLUSION

In this example we've proven out a method for migrating NTFS volumes from a LeftHand Networks SAN appliance (or any other SAN storage array) to some other storage device.

Our downtime for the entire migration project has been limited to 10-15 minutes per server, regardless of the size and number of volumes.

And did you notice the performance boost? SANmelody is built for high performance. SANmelody uses the server's memory as storage processor cache, and the I/O drivers take full advantage of the Windows kernel's deterministic I/O subsystem to deliver real-time performance. Need for speed? SANmelody's your ticket.

SANmelody — A UNIQUE STORAGE CONTROLLER

The fact that SANmelody is built as services or "daemons" on a standard, ubiquitous operating system (Windows) gives the product a truly unique feature not found in other SAN storage products: the ability to be not only a full-featured FC/iSCSI SAN storage array, but also to be a SAN storage client. Any disk visible to the SANmelody server, including SAN LUNs from your LeftHand appliance, can be "virtualized" by SANmelody and presented over Fibre Channel and/or iSCSI targets. This unique ability springs from SANmelody's roots — its development is based on DataCore's flagship SANsymphony™ Storage Virtualization platform.

BEYOND NTFS — MIGRATING ANY STORAGE

If your environment is rich and includes diverse volume types (lvm, ext3, HFS+, or raw storage), DataCore's SANsymphony™ product includes a complete volume proxy mechanism that can proxy any storage presented. Whether the volumes are AIX, HP/UX, Solaris, Netware, Mac OS, or other, SANsymphony™ offers many tools for migrating and virtualizing the entire storage environment. With SANsymphony™, you can federate all those storage islands and share storage between all your SAN devices regardless of vendor or model: LeftHand, EqualLogic, HP EVA or MSA, IBM DS, EMC DMX, CX, AX and T-Rex... SANsymphony™ homogenizes the storage environment and provides a common Tier-1 feature set to all your shared storage devices.

NEXT STEPS TO EASY STORAGE MIGRATION

Ready to start your migration project?

SANmelody™ is sold uniquely through DataCore's network of Value Added Resellers. For more information on the SANmelody™ product and to find an authorized reseller in your area, please visit www.datacore.com.

Las Solanas Consulting is not a DataCore or LeftHand Networks reseller. This white paper is published for informational purposes only. Please contact DataCore™ or a DataCore reseller for pricing information on SANmelody.