Transporte sans bouger (Moving FlexClone volumes without splitting them)

pch-bigsur4.jpg

I’m somewhat of a music buff. I spent most of my teenage years with headphones glued to my ears, and eventually made my way into college radio as a DJ. I was even once able to interview Layzie Bone from Bone Thugs and Harmony!

(I was DJ Sledgehammer)

https://studentmedia.ncsu.edu/web/?q=training/wknc/how-to-be-a-good-dj

I like most every type of music, from hip hop to rock to indie to bluegrass. For a while, I was really into a band from England called Stereolab. In particular, this song:

It’s a song about technology and how humanity becomes disconnected from one another. The title translates to “Transported without Moving” and its themes still resonate today.

But, for this blog, I wanted to apply “transporte san bouger” to an ONTAP feature that allows you to transport data without moving it – volume rehost – and the limitation it currently has on moving FlexClone volumes.

cluster::*> vol rehost -vserver DEMO -volume clone -destination-vserver NFS 

Error: command failed: Cannot rehost volume "clone" on Vserver "DEMO" because the volume is a clone volume.

What is a FlexClone volume?

To answer this question, let’s start with “what is a FlexVol volume”?

A FlexVol is NetApp’s way of presenting a file system to clients in ONTAP software. Each FlexVol volume has its own unique file system ID and can host data via NAS (via NFS and/or SMB) or SAN (via FCP/iSCSI with LUNs). It’s a logical container for data that allows storage administrators to carve out space for their end users without provisioning the entire array to everyone.

FlexVol volumes can host some very important production data, such as code repositories and databases. Sometimes, people need to be able to work with that data without impacting the production data or taking up a bunch of extra storage space. A FlexClone provides an instant copy of a FlexVol volume, based on a snapshot, that only takes up space for inode pointers to data blocks. Thus, you could clone a FlexVol volume that has 10TB of data in less than a second and use up less than 1MB of additional space.

The clone can be read-only or read-write, depending on what you use it for. If it’s a read-write clone, then any data that is written/deleted/overwritten in the clone will use space. But the initial clone? Negligible. Check out the space usage on the aggregate before and after the clone (physical used):

cluster::*> aggr show-space -aggregate-name aggr1_node1

Aggregate : aggr1_node1

Feature Used Used%
-------------------------------- ---------- ------
Volume Footprints 278.4GB 3%
Aggregate Metadata 2.65GB 0%
Snapshot Reserve 0B 0%
Total Used 281.0GB 3%

Total Physical Used 265.6GB 3%


cluster::*> vol clone create -vserver DEMO -flexclone clone -type RW -parent-vserver DEMO -parent-volume flexvol -junction-active true -foreground true -junction-path /clone
[Job 13563] Job succeeded: Successful


cluster::*> vol show -vserver DEMO -volume flexvol,clone -fields used,aggregate
vserver volume    aggregate used
------- ------    ----------- ------
DEMO    clone     aggr1_node1 1.77GB
DEMO    flexvol   aggr1_node1 1.79GB


cluster::*> aggr show-space -aggregate-name aggr1_node1

Aggregate : aggr1_node1

Feature Used Used%
-------------------------------- ---------- ------
Volume Footprints 283.0GB 4%
Aggregate Metadata 2.64GB 0%
Snapshot Reserve 0B 0%
Total Used 285.6GB 4%

Total Physical Used 265.6GB 3%

Some use cases I’ve heard of are:

That’s not nearly all you can do with clones. Leave feedback in the comments with how  you use clones!

So what’s the problem?

A FlexClone, however, for all its usefulness, isn’t super flexible when it comes to moving it around. In addition to volume rehost being unable to move a FlexClone, a non-disruptive volume move of a clone will result in splitting the clone, as well as negating any storage efficiencies you might have!

cluster::*> volume move start -vserver DEMO -volume clone -destination-aggregate aggr1_node2

Warning: Volume will no longer be a clone volume after the move and any associated space efficiency savings will be lost. Do you want to proceed? {y|n}: n

Yuck.

Fortunately, NetApp has some smart people like Jeff Steiner and Florian Feldhaus, who came up with a way to move a clone around without splitting it – thereby retaining the storage efficiencies!

How? With another cool ONTAP feature – SnapMirror.

What’s a SnapMirror?

SnapMirror is a disaster recovery technology that will replicate – at the block level – an exact copy of data to a destination site. That destination could be remote, or it could be on the same cluster. But, it can also be used to move FlexClone volumes around without losing storage efficiencies!

Wait… why would I want to move a FlexClone?

It might not seem like you’d want to move a FlexClone volume, but there are some reasons. For one, you can’t rehost a clone to a new SVM. So, if you were testing in a SVM and wanted to move your volumes to a new SVM and retain the clones, you would be stuck.

Another use case would be moving from ONTAP running in 7-Mode to clustered ONTAP. Using SnapMirror to move data is much faster in many cases than a file-based copy.

Great! How do I do it?

The basic steps are:

  1. Create a SnapMirror of the parent volume (requires cluster/SVM peering, which is all super easy to do in System Manager now)
  2. Ensure the clone volume has a common snapshot for a snapmirror resync operation
  3. Clone the destination volume
  4. Issue a snapmirror resync on the clone volumes (via CLI)

That’s it! Once you’ve done that, you can break the snapmirror, delete the source volume and clone after a final resync for a quick cutover to the new destination.

Here’s where I did it:

Create a SnapMirror of the parent volume

Use a “mirror” relationship type and “MirrorAllSnapshots” as the policy.

clone-mirror1

clone-mirror3

Ensure the clone volume has a common snapshot for a snapmirror resync operation

You can find the snapshot copies under the “Volumes” menu by clicking on the volume for more details and viewing the “Snapshot copies” tab.

clone-mirror-common-snap

Clone the destination volume

Be sure to select the common snapshot.

clone-vol1

Issue a snapmirror resync on the clone volumes (via CLI)

Since we’re kind of “cheating” here by resyncing a relationship that doesn’t actually exist, we have to use the CLI.

sm-resync.png

Once this is done, the relationship will show up in System Manager:

sm-resync-ocsm

Once that transfer completes, I can update the mirror when I’m ready to cutover, quiesce and break the mirror, point clients the new clone and delete the old one from the source SVM. Or, I can keep both around if I like. Pretty nifty, eh?

Any caveats?

There are a couple that immediately come to mind…

  • FlexClone doesn’t currently work with FlexGroup volumes
  • FlexClone and SnapMirror both require licenses

Questions? Comments? Hit me up in the comments!

Advertisements

Life hack: Change the UID of all files in NAS volume… without actually changing anything!

146538-640x313

There are a ton of hidden gems in ONTAP that go unnoticed because they get added without a lot of fanfare. For example, the volume recovery queue got added in 8.3 and no one really knew what it was, what it did, or why the volumes they deleted didn’t seem to actually get deleted for 24 hours.

I keep my ears open for these features so I can promote them and I ran across a pretty slick, simple gem while at the NetApp Converge (sales kick off) conference, from an old colleague in my support days that now does SE work. (Shout out to Maarten Lippmann!)

But, features are only as good as their use cases.

Here’s the scenario…

Let’s say you have a Git code repository with millions of files and its files are owned by a number of different people that one of your developers wants to access and make changes to. They don’t have access to some of those files by way of permissions, but there are way too many to re-permission effectively and in a timely manner. Plus, if you change the access to these files, you might break the code repo horribly.

So, how do you:

  • Create a usable copy of the entire code repo in a reasonable amount of time without eating up a ton of space
  • Assign a new owner to all the files in the volume quickly and easily
  • Keep the original repo intact

It’s pretty easy in ONTAP, actually – In fact, it’s a single command. All you need is a FlexClone license and you can make an instant copy of a volume with a new file owner without impacting the source volume and without using up any new space. Additionally, if you wanted to keep those changes, you can split the clone into its own unique volume.

In the following example, I have an existing volume that has a ton of files and folders, all owned by root:

[root@XCP nfs4]# ls -la
total 8012
d------r-x. 102 root root 8192 Apr 11 11:41 .
drwxr-xr-x. 5 root root 4096 Apr 12 17:20 ..
----------. 1 root root 0 Apr 11 11:29 file
d---------. 1002 root root 77824 Apr 11 11:47 topdir_0
d---------. 1002 root root 77824 Apr 11 11:47 topdir_1
...
d---------. 1002 root root 77824 Apr 11 11:47 topdir_99

I want the new owner of the files in the cloned volume to be a user named “prof1” and the GID to be 1101.

cluster::*> getxxbyyy getpwbyname -node ontap9-tme-8040-01 -vserver DEMO -username prof1
 (vserver services name-service getxxbyyy getpwbyname)
pw_name: prof1
pw_passwd:
pw_uid: 1100
pw_gid: 1101
pw_gecos:
pw_dir:
pw_shell:

So, I do the following:

cluster::*> vol clone create -vserver DEMO -flexclone clone -type RW -parent-vserver DEMO -parent-volume flexvol -junction-active true -foreground true -junction-path /clone -uid 1100 -gid 1101
[Job 12606] Job succeeded: Successful

cluster::*> vol show -vserver DEMO -volume clone -fields clone-volume,clone-parent-name,clone-parent-vserver
vserver volume clone-volume clone-parent-vserver clone-parent-name
------- ------ ------------ -------------------- -----------------
DEMO clone true DEMO flexvol

That command took literally 10 seconds to complete. There are over 1.8 million objects in that volume.

cluster::*> df -i /vol/clone
Filesystem iused ifree %iused Mounted on Vserver
/vol/clone/ 1824430 4401487 29% /clone DEMO

Then, I check the owner of the files:

cluster::*> vserver security file-directory show -vserver DEMO /clone/nfs4

Vserver: DEMO
 File Path: /clone/nfs4
 File Inode Number: 96
 Security Style: unix
 Effective Style: unix
 DOS Attributes: 10
 DOS Attributes in Text: ----D---
Expanded Dos Attributes: -
 UNIX User Id: 1100
 UNIX Group Id: 1101
 UNIX Mode Bits: 5
 UNIX Mode Bits in Text: ------r-x
 ACLs: NFSV4 Security Descriptor
 Control:0x8014
 DACL - ACEs
 ALLOW-user-prof1-0x1601ff-FI|DI|IO
 ALLOW-user-student1-0x21-FI|DI|IO
 ALLOW-group-ProfGroup-0x1200a9-FI|DI|IO|IG
 ALLOW-EVERYONE@-0x1200a9

cluster::*> vserver security file-directory show -vserver DEMO /clone/nfs4/topdir_99

Vserver: DEMO
 File Path: /clone/nfs4/topdir_99
 File Inode Number: 3556
 Security Style: unix
 Effective Style: unix
 DOS Attributes: 10
 DOS Attributes in Text: ----D---
Expanded Dos Attributes: -
 UNIX User Id: 1100
 UNIX Group Id: 1101
 UNIX Mode Bits: 0
 UNIX Mode Bits in Text: ---------
 ACLs: NFSV4 Security Descriptor
 Control:0x8004
 DACL - ACEs
 ALLOW-user-prof1-0x1601ff-FI|DI
 ALLOW-user-student1-0x21-FI|DI
 ALLOW-group-ProfGroup-0x1200a9-FI|DI|IG

And from the client:

[root@XCP nfs4]# pwd
/clone/nfs4

[root@XCP nfs4]# ls -la
total 8012
d------r-x. 102 1100 1101 8192 Apr 11 11:41 .
drwxr-xr-x. 5 1100 1101 4096 Apr 12 17:20 ..
----------. 1 1100 1101 0 Apr 11 11:29 file
d---------. 1002 1100 1101 77824 Apr 11 11:47 topdir_0
d---------. 1002 1100 1101 77824 Apr 11 11:47 topdir_1
d---------. 1002 1100 1101 77824 Apr 11 11:47 topdir_10
d---------. 1002 1100 1101 77824 Apr 11 11:47 topdir_11
d---------. 1002 1100 1101 77824 Apr 11 11:47 topdir_12

It shouldn’t be that easy, should it?

If I wanted to split the volume off into its own volume (such as when a dev makes changes and wants to keep them, but doesn’t want to change the source volume):

cluster::*> vol clone split
 estimate show start status stop

If I want to delete the clone after I’m done, I just run “volume destroy.”

Questions? Hit me up in the comments!