Logical vs. Used Space in ONTAP 9.4

1eqdbk

In storage, the race to zero usually takes on two different faces:

  • Performance/latency
  • Space used/storage efficiencies

While it may seem counter-intuitive for a company that sells data storage to want people to have to use less of it, this is what people want. They don’t want to feel like they’re being double-charged for inefficiencies in how the storage filesystem operates or for duplicate files. It’s now gone from a “nice to have” feature to a necessity that every vendor must have.

NetApp has been one of the leaders in storage efficiency since the ONTAP 7.3 days, but they’ve recently stepped up their game by shaving even more off the top for space savings without having to sacrifice storage performance to do it.

Included in the storage efficiencies offered by ONTAP:

  • Volume Deduplication (inline and post-process)
  • Aggregate-level/cross volume deduplication (inline and post-process)
  • Data compaction
  • Compression (inline and post-process)
  • FabricPool

Sometimes, things like clones, snapshots and space guarantees are also factored in to storage efficiency, but that depends on your definition.

But I’m not here to tout the values of storage efficiency. I’m here to tell you about a new, relatively unknown feature added in ONTAP 9.4 that was brought to my attention by one of NetApp’s Sales Engineers, Riley Johnson.

So, what is it?

One of the main challenges we’ve seen is that while you like to save space in your cluster as a storage administrator, you may not necessarily like your end users/customers *knowing* you’re saving space, because that eats into your overall bottom line. Service providers, for example, might want to charge on data ingested rather than data that has been reduced, since storage efficiencies can make the actual amount of data being written to a volume wildly unpredictable and hard to track and if you ever needed to unpack that data, you might not have room.

For example… I can provision a 1TB volume and write 1.2TB to it, but with storage efficiencies, my storage system might report only 800GB used. If that data ever needed to inflate for whatever reason, then I could find myself painted into a corner. Before ONTAP 9.4, if my storage efficiencies saved me 400GB in space, then the client would see only 800GB used despite actually writing 1.2TB. ONTAP 9.4, however, introduced a volume-level option to change how space is reported back to clients when storage efficiencies are in use. (FlexVol only)

 [-is-space-reporting-logical {true|false}] - Logical Space Reporting
 If this parameter is specified, the command displays information only about the volumes that have logical space reporting enabled or disabled as specified. When space is reported logically, ONTAP reports the volume space such that all the physical space saved by the storage efficiency features are also as reported as used. This parameter is not supported on FlexGroups or Infinite Volumes.

By default, this option is off. But when you enable it, ONTAP will hide the fact that efficiencies have saved space from clients that run a df or other space reporting utility, and will instead report back the actual data amount that was written to the volume.

There are also several other options added to the volume level to help report space used vs. logical space used.

[-logical-used {<integer>[KB|MB|GB|TB|PB]}] - Logical Used Size
 If this parameter is specified, the command displays information only about the volume or volumes that have the specified logical used size. This value includes all the space saved by the storage efficiency features along with the physically used space. This does not include Snapshot reserve but does consider Snapshot spill. This parameter is not supported on FlexGroups or Infinite Volumes.

[-logical-used-percent <percent_no_limit>] - Logical Used Percentage
 If this parameter is specified, the command displays information only about the volume or volumes that have the specified logical used percentage. This parameter is not supported on FlexGroups or Infinite Volumes.

[-logical-available {<integer>[KB|MB|GB|TB|PB]}] - Logical Available Size
 If this parameter is specified, the command displays information only about the volume or volumes that have the specified logical available size. This value is the amount of free space currently available considering space saved by the storage efficiency features as being used. This does not include Snapshot reserve. This parameter is not supported on FlexGroups or Infinite Volumes.

[-logical-used-by-afs {<integer>[KB|MB|GB|TB|PB]}] - Logical Size Used by Active Filesystem
 If this parameter is specified, the command displays information only about the volume or volumes that have the specified logical size used by the active file system. This value differs from logical-used by the amount of Snapshot spill that exceeds Snapshot reserve. This parameter is not supported on FlexGroups or Infinite Volumes.

[-logical-used-by-snapshots {<integer>[KB|MB|GB|TB|PB]}] - Logical Size Used by All Snapshots
 If this parameter is specified, the command displays information only about the volume or volumes that have the specified logical size used across all Snapshot copies. This value differs from size-used-by-snapshots by the space saved by the storage efficiency features across the Snapshot copies. This parameter is not supported on FlexGroups or Infinite Volumes.

I proved this out on one of my volumes in a cluster running ONTAP 9.4. As you can see, this volume has a discrepancy in used vs. logical space:

vserver volume               used   logical-used is-space-reporting-logical
------- -------------------- ------ ------------ --------------------------
SVM1    xcp_hardlinks_source 8.97GB 15.89GB      false

From a NFS client, this is what I see with logical space reporting set to false:

[root@XCP /]# df -h
Filesystem                     Size Used Avail Use% Mounted on
10.x.x.x:/xcp_hardlinks_source 9.5G 9.0G 542M  95% /xcphardlink

As you can see, the client is reporting the “used” field from ONTAP – not “logical-used.” That means it’s not taking storage efficiency savings into account. If I’m an end user and I see that, I think “oh goodie! I have more space left!” even though I’ve actually used 15.9GB in logical space.

When “-is-space-reporting-logical” is set to true, the client will see the space as if no storage efficiencies were applied:

cluster ::*> vol modify -vserver SVM1 -volume xcp_hardlinks_source -is-space-reporting-logical true
Volume modify successful on volume xcp_hardlinks_source of Vserver SVM1.
[root@XCP /]# df -h | grep xcphardlink
Filesystem                     Size Used Avail Use% Mounted on
10.x.x.x:/xcp_hardlinks_source 17G  16G  542M  97% /xcphardlink

Fuzzy math

What we also see is that the “size” reported back is essentially logical used + available, so if you provisioned 10GB in a volume and have enabled logical space reporting, your clients will see a larger size than what the volume actually has. In the above, the size reporting is 17GB, which is roughly 16GB (used) + 542MB (available), rounded up.

As a result, reporting the logical space needs to be a decision made based on what you want your end users to see.

So where did all that space go?

On this volume, I had all available storage efficiencies enabled. As a result, there is roughly a 7GB discrepancy in logical space and used space after efficiencies are applied. We can see the difference with the following command:

vol show -fields size,logical-used,used,available,logical-available -is-flexgroup false -is-constituent false -vserver SVM -volume volname

Or with:

cluster::*> volume show-space -vserver SVM1 -volume xcp_hardlinks_source

Vserver: SVM1
 Volume Name: xcp_hardlinks_source
 Volume MSID: 2163226536
 Volume DSID: 1932
 Vserver UUID: 05e7ab78-2d84-11e6-a796-00a098696ec7
 Aggregate Name: aggr1_node1
 Aggregate UUID: 4b7f701e-ee9a-43db-980f-ba6f1ea7a8cb
 Hostname: ontap9-tme-8040-01
 User Data: 8.96GB
 User Data Percent: 90%
 Deduplication: 12KB
 Deduplication Percent: 0%
 Temporary Deduplication: -
 Temporary Deduplication Percent: -
 Filesystem Metadata: 7.02MB
 Filesystem Metadata Percent: 0%
 SnapMirror Metadata: -
 SnapMirror Metadata Percent: -
 Tape Backup Metadata: -
 Tape Backup Metadata Percent: -
 Quota Metadata: -
 Quota Metadata Percent: -
 Inodes: 3.88MB
 Inodes Percent: 0%
 Inodes Upgrade: -
 Inodes Upgrade Percent: -
 Snapshot Reserve: 512MB
 Snapshot Reserve Percent: 5%
 Snapshot Reserve Unusable: -
Snapshot Reserve Unusable Percent: -
 Snapshot Spill: -
 Snapshot Spill Percent: -
 Performance Metadata: 1.74MB
 Performance Metadata Percent: 0%
 Total Used: 9.47GB
 Total Used Percent: 95%
 Total Physical Used Size: 9.03GB
 Physical Used Percentage: 90%
 Logical Used Size: 15.89GB
 Logical Used Percent: 159%
 Logical Available: 542.1MB

We can also see efficiencies at the aggregate level:

cluster::*> aggr show-efficiency -aggregate aggr1_node1

Name of the Aggregate: aggr1_node1
 Node where Aggregate Resides: node1
Logical Size Used by Volumes, Clones, Snapshot Copies in the Aggregate: 3.19TB
 Total Physical Used: 140.9GB
 Total Storage Efficiency Ratio: 23.20:1
 Total Data Reduction Logical Used: 268.0GB
 Total Data Reduction Physical Used: 137.0GB
 Total Data Reduction Efficiency Ratio: 1.95:1
 Logical Space Used for All Volumes: 269.8GB
 Physical Space Used for All Volumes: 139.3GB
 Space Saved by Volume Deduplication: 58.33GB
Space Saved by Volume Deduplication and pattern detection: 122.7GB
 Volume Deduplication Savings ratio: 1.83:1
 Space Saved by Volume Compression: 7.85GB
 Volume Compression Savings ratio: 1.03:1
Space Saved by Inline Zero Pattern Detection: 64.33GB
 Volume Data Reduction SE Ratio: 1.94:1
 Logical Space Used by the Aggregate: 149.1GB
 Physical Space Used by the Aggregate: 140.9GB
 Space Saved by Aggregate Data Reduction: 8.21GB
 Aggregate Data Reduction SE Ratio: 1.06:1
 Logical Size Used by Snapshot Copies: 2.93TB
 Physical Size Used by Snapshot Copies: 3.48GB
 Logical Size Used by FlexClone Volumes: 1.77GB
 Physical Sized Used by FlexClone Volumes: 364.5MB
Snapshot And FlexClone Volume Data Reduction SE Ratio: 781.17:1
 Snapshot Volume Data Reduction Ratio: 860.45:1
 FlexClone Volume Data Reduction Ratio: 4.97:1
 Number of Volumes Offline: -
 Number of SIS Disabled Volumes: 7
 Number of SIS Change Log Disabled Volumes: 23

If we want to see the amount of space saved with all efficiencies, use the following:

cluster::*> vol show -vserver SVM1 xcp_hardlinks_source -fields sis-space-saved,sis-space-saved-percent
vserver volume               sis-space-saved sis-space-saved-percent
------- -------------------- --------------- -----------------------
SVM1    xcp_hardlinks_source 6.92GB          44%

If you want that number broken down into specific storage efficiencies, use this:

cluster::*> vol show -vserver SVM1 xcp_hardlinks_source -fields dedupe-space-saved,dedupe-space-saved-percent,compression-space-saved,compression-space-saved-percent
vserver volume               dedupe-space-saved dedupe-space-saved-percent compression-space-saved compression-space-saved-percent
------- -------------------- ------------------ -------------------------- ----------------------- -------------------------------
SVM1    xcp_hardlinks_source 2.14GB             14%                        4.77GB                  30%

Or, for a more basic view, use System Manager!

You can see storage efficiencies when you drill down into the actual volumes (Storage -> Volumes on the left menu) and then click on the desired volume and view the “Space Allocation” section:

space-allocation-ocsm.png

If you have questions, comment below!

Advertisements

2 thoughts on “Logical vs. Used Space in ONTAP 9.4

  1. Does this take the place of default quotas that don’t set a soft or hard limit? I would normally create a tree default quota as it would go off logical used instead of actual. This just gives me a nice assurance that the volume would rarely if ever hit the actual used size with dedup or compression enabled.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s