In storage, the race to zero usually takes on two different faces:
- Performance/latency
- Space used/storage efficiencies
While it may seem counter-intuitive for a company that sells data storage to want people to have to use less of it, this is what people want. They don’t want to feel like they’re being double-charged for inefficiencies in how the storage filesystem operates or for duplicate files. It’s now gone from a “nice to have” feature to a necessity that every vendor must have.
NetApp has been one of the leaders in storage efficiency since the ONTAP 7.3 days, but they’ve recently stepped up their game by shaving even more off the top for space savings without having to sacrifice storage performance to do it.
Included in the storage efficiencies offered by ONTAP:
- Volume Deduplication (inline and post-process)
- Aggregate-level/cross volume deduplication (inline and post-process)
- Data compaction
- Compression (inline and post-process)
- FabricPool
Sometimes, things like clones, snapshots and space guarantees are also factored in to storage efficiency, but that depends on your definition.
But I’m not here to tout the values of storage efficiency. I’m here to tell you about a new, relatively unknown feature added in ONTAP 9.4 that was brought to my attention by one of NetApp’s Sales Engineers, Riley Johnson.
So, what is it?
One of the main challenges we’ve seen is that while you like to save space in your cluster as a storage administrator, you may not necessarily like your end users/customers *knowing* you’re saving space, because that eats into your overall bottom line. Service providers, for example, might want to charge on data ingested rather than data that has been reduced, since storage efficiencies can make the actual amount of data being written to a volume wildly unpredictable and hard to track and if you ever needed to unpack that data, you might not have room.
For example… I can provision a 1TB volume and write 1.2TB to it, but with storage efficiencies, my storage system might report only 800GB used. If that data ever needed to inflate for whatever reason, then I could find myself painted into a corner. Before ONTAP 9.4, if my storage efficiencies saved me 400GB in space, then the client would see only 800GB used despite actually writing 1.2TB. ONTAP 9.4, however, introduced a volume-level option to change how space is reported back to clients when storage efficiencies are in use. (FlexVol only)
[-is-space-reporting-logical {true|false}] - Logical Space Reporting
If this parameter is specified, the command displays information only about the volumes that have logical space reporting enabled or disabled as specified. When space is reported logically, ONTAP reports the volume space such that all the physical space saved by the storage efficiency features are also as reported as used. This parameter is not supported on FlexGroups or Infinite Volumes.
By default, this option is off. But when you enable it, ONTAP will hide the fact that efficiencies have saved space from clients that run a df or other space reporting utility, and will instead report back the actual data amount that was written to the volume.
There are also several other options added to the volume level to help report space used vs. logical space used.
[-logical-used {<integer>[KB|MB|GB|TB|PB]}] - Logical Used Size
If this parameter is specified, the command displays information only about the volume or volumes that have the specified logical used size. This value includes all the space saved by the storage efficiency features along with the physically used space. This does not include Snapshot reserve but does consider Snapshot spill. This parameter is not supported on FlexGroups or Infinite Volumes.
[-logical-used-percent <percent_no_limit>] - Logical Used Percentage
If this parameter is specified, the command displays information only about the volume or volumes that have the specified logical used percentage. This parameter is not supported on FlexGroups or Infinite Volumes.
[-logical-available {<integer>[KB|MB|GB|TB|PB]}] - Logical Available Size
If this parameter is specified, the command displays information only about the volume or volumes that have the specified logical available size. This value is the amount of free space currently available considering space saved by the storage efficiency features as being used. This does not include Snapshot reserve. This parameter is not supported on FlexGroups or Infinite Volumes.
[-logical-used-by-afs {<integer>[KB|MB|GB|TB|PB]}] - Logical Size Used by Active Filesystem
If this parameter is specified, the command displays information only about the volume or volumes that have the specified logical size used by the active file system. This value differs from logical-used by the amount of Snapshot spill that exceeds Snapshot reserve. This parameter is not supported on FlexGroups or Infinite Volumes.
[-logical-used-by-snapshots {<integer>[KB|MB|GB|TB|PB]}] - Logical Size Used by All Snapshots
If this parameter is specified, the command displays information only about the volume or volumes that have the specified logical size used across all Snapshot copies. This value differs from size-used-by-snapshots by the space saved by the storage efficiency features across the Snapshot copies. This parameter is not supported on FlexGroups or Infinite Volumes.
I proved this out on one of my volumes in a cluster running ONTAP 9.4. As you can see, this volume has a discrepancy in used vs. logical space:
vserver volume used logical-used is-space-reporting-logical
------- -------------------- ------ ------------ --------------------------
SVM1 xcp_hardlinks_source 8.97GB 15.89GB false
From a NFS client, this is what I see with logical space reporting set to false:
[root@XCP /]# df -h
Filesystem Size Used Avail Use% Mounted on
10.x.x.x:/xcp_hardlinks_source 9.5G 9.0G 542M 95% /xcphardlink
As you can see, the client is reporting the “used” field from ONTAP – not “logical-used.” That means it’s not taking storage efficiency savings into account. If I’m an end user and I see that, I think “oh goodie! I have more space left!” even though I’ve actually used 15.9GB in logical space.
When “-is-space-reporting-logical” is set to true, the client will see the space as if no storage efficiencies were applied:
cluster ::*> vol modify -vserver SVM1 -volume xcp_hardlinks_source -is-space-reporting-logical true
Volume modify successful on volume xcp_hardlinks_source of Vserver SVM1.
[root@XCP /]# df -h | grep xcphardlink
Filesystem Size Used Avail Use% Mounted on
10.x.x.x:/xcp_hardlinks_source 17G 16G 542M 97% /xcphardlink
Fuzzy math
What we also see is that the “size” reported back is essentially logical used + available, so if you provisioned 10GB in a volume and have enabled logical space reporting, your clients will see a larger size than what the volume actually has. In the above, the size reporting is 17GB, which is roughly 16GB (used) + 542MB (available), rounded up.
As a result, reporting the logical space needs to be a decision made based on what you want your end users to see.
So where did all that space go?
On this volume, I had all available storage efficiencies enabled. As a result, there is roughly a 7GB discrepancy in logical space and used space after efficiencies are applied. We can see the difference with the following command:
vol show -fields size,logical-used,used,available,logical-available -is-flexgroup false -is-constituent false -vserver SVM -volume volname
Or with:
cluster::*> volume show-space -vserver SVM1 -volume xcp_hardlinks_source
Vserver: SVM1
Volume Name: xcp_hardlinks_source
Volume MSID: 2163226536
Volume DSID: 1932
Vserver UUID: 05e7ab78-2d84-11e6-a796-00a098696ec7
Aggregate Name: aggr1_node1
Aggregate UUID: 4b7f701e-ee9a-43db-980f-ba6f1ea7a8cb
Hostname: ontap9-tme-8040-01
User Data: 8.96GB
User Data Percent: 90%
Deduplication: 12KB
Deduplication Percent: 0%
Temporary Deduplication: -
Temporary Deduplication Percent: -
Filesystem Metadata: 7.02MB
Filesystem Metadata Percent: 0%
SnapMirror Metadata: -
SnapMirror Metadata Percent: -
Tape Backup Metadata: -
Tape Backup Metadata Percent: -
Quota Metadata: -
Quota Metadata Percent: -
Inodes: 3.88MB
Inodes Percent: 0%
Inodes Upgrade: -
Inodes Upgrade Percent: -
Snapshot Reserve: 512MB
Snapshot Reserve Percent: 5%
Snapshot Reserve Unusable: -
Snapshot Reserve Unusable Percent: -
Snapshot Spill: -
Snapshot Spill Percent: -
Performance Metadata: 1.74MB
Performance Metadata Percent: 0%
Total Used: 9.47GB
Total Used Percent: 95%
Total Physical Used Size: 9.03GB
Physical Used Percentage: 90%
Logical Used Size: 15.89GB
Logical Used Percent: 159%
Logical Available: 542.1MB
We can also see efficiencies at the aggregate level:
cluster::*> aggr show-efficiency -aggregate aggr1_node1
Name of the Aggregate: aggr1_node1
Node where Aggregate Resides: node1
Logical Size Used by Volumes, Clones, Snapshot Copies in the Aggregate: 3.19TB
Total Physical Used: 140.9GB
Total Storage Efficiency Ratio: 23.20:1
Total Data Reduction Logical Used: 268.0GB
Total Data Reduction Physical Used: 137.0GB
Total Data Reduction Efficiency Ratio: 1.95:1
Logical Space Used for All Volumes: 269.8GB
Physical Space Used for All Volumes: 139.3GB
Space Saved by Volume Deduplication: 58.33GB
Space Saved by Volume Deduplication and pattern detection: 122.7GB
Volume Deduplication Savings ratio: 1.83:1
Space Saved by Volume Compression: 7.85GB
Volume Compression Savings ratio: 1.03:1
Space Saved by Inline Zero Pattern Detection: 64.33GB
Volume Data Reduction SE Ratio: 1.94:1
Logical Space Used by the Aggregate: 149.1GB
Physical Space Used by the Aggregate: 140.9GB
Space Saved by Aggregate Data Reduction: 8.21GB
Aggregate Data Reduction SE Ratio: 1.06:1
Logical Size Used by Snapshot Copies: 2.93TB
Physical Size Used by Snapshot Copies: 3.48GB
Logical Size Used by FlexClone Volumes: 1.77GB
Physical Sized Used by FlexClone Volumes: 364.5MB
Snapshot And FlexClone Volume Data Reduction SE Ratio: 781.17:1
Snapshot Volume Data Reduction Ratio: 860.45:1
FlexClone Volume Data Reduction Ratio: 4.97:1
Number of Volumes Offline: -
Number of SIS Disabled Volumes: 7
Number of SIS Change Log Disabled Volumes: 23
If we want to see the amount of space saved with all efficiencies, use the following:
cluster::*> vol show -vserver SVM1 xcp_hardlinks_source -fields sis-space-saved,sis-space-saved-percent
vserver volume sis-space-saved sis-space-saved-percent
------- -------------------- --------------- -----------------------
SVM1 xcp_hardlinks_source 6.92GB 44%
If you want that number broken down into specific storage efficiencies, use this:
cluster::*> vol show -vserver SVM1 xcp_hardlinks_source -fields dedupe-space-saved,dedupe-space-saved-percent,compression-space-saved,compression-space-saved-percent
vserver volume dedupe-space-saved dedupe-space-saved-percent compression-space-saved compression-space-saved-percent
------- -------------------- ------------------ -------------------------- ----------------------- -------------------------------
SVM1 xcp_hardlinks_source 2.14GB 14% 4.77GB 30%
Or, for a more basic view, use System Manager!
You can see storage efficiencies when you drill down into the actual volumes (Storage -> Volumes on the left menu) and then click on the desired volume and view the “Space Allocation” section:
If you have questions, comment below!
