ONTAP 9.4RC1 is now available!

/ Justin Parisi

Hear ye! Hear ye! All ye storage admins! ONTAP 9.4RC1 is announced today!

sully-hearye

That’s right! Every 6 months, without fail, a new ONTAP version with a payload of new features is released.

You can find ONTAP 9.4RC1 here:

http://mysupport.netapp.com/NOW/download/software/ontap/9.4RC1

For info on what a release candidate is, see:

http://mysupport.netapp.com/NOW/products/ontap_releasemodel/

Also, check out the documentation center:

docs.netapp.com/ontap-9/index.jsp

NetApp published a general overview blog on NVMe with Joel Reich here:

https://blog.netapp.com/the-future-is-here-ai-ready-cloud-connected-all-flash-storage-with-nvme/

Stay tuned for a more general ONTAP 9.4 overview blog on the official site. Also, I recorded a brief 5-minute teaser/trailer for ONTAP 9.4 features and podcasts coming soon. Find that here:

Also a new lightboard video! Watch me write… BACKWARDS???

This blog is intended to go a little deeper into the main features available in ONTAP 9.4. We’ll break them down as follows:

  • Cloud
  • Performance
  • Efficiency
  • Security
  • General ONTAP Goodness

Without further ado…

Cloud!

FabricPools were introduced in ONTAP 9.2 as a way to tier blocks from your performance tier solution to a capacity tier, such as cloud or StorageGrid.

We covered FabricPools in detail in episode 92 of the Tech ONTAP Podcast, which you can find here:

In ONTAP 9.4, the first major updates to the feature have been released! FabricPools in ONTAP 9.4 bring the following…

Tiering cold data from the active file system

Prior to ONTAP 9.4, FabricPools only tiered cold data from snapshots on primary systems and data protection volumes on secondary systems. This allowed ONTAP to free up valuable real estate on flash systems for data actively being used. In ONTAP 9.4, inactive blocks can now be tiered off to cloud or StorageGrid from the active file system. ONTAP does this automatically by way of a new “auto” tiering policy, which has a configurable cooling period of 2-63 days (-tiering-minimum-cooling-days option in CLI). This cooling period determines how long ONTAP will wait before tiering off data considered “cool” by the policy to the FabricPool tiering destination. The tiering destination choices used to be only Amazon S3 and StorageGrid, but ONTAP 9.4 brings us…

Tiering to Azure Blob Storage

Support for Azure Blob storage was added to ONTAP 9.4 for FabricPools, which gives storage administrators more options for cloud providers. In addition, other cloud providers (such as Google Cloud, IBM Cloud Object Storage, etc) can be added via product variance requests (PVR) to your NetApp Sales reps. Keep in mind that only one cloud provider per FabricPool aggregate can be used.

fabricpools-afs

But how do you know if FabricPools will be of any value to you?

Inactive Data Reporting

Inactive Data Reporting is new in ONTAP 9.4 and can offer insight from OnCommand System Manager into whether there’s enough inactive data in your system for FabricPools to make a difference.

fabricpools-inactive-report.png

By default, this feature is enabled for aggregates participating in FabricPools, but you can also enable it via the CLI for non-FabricPool aggregates to predict space savings with the following command:

storage aggregate modify -aggregate <name> -is-inactive-data-reporting-enabled true

You can also test the performance of your FabricPool target with…

Object Store Profiler

Also new in ONTAP 9.4, the Object Store Profiler provides a way to evaluate the performance (via throughput and latency) to your desired FabricPool target. From the CLI, start the profiler using:

storage aggregate object-store profiler start -object-store-name <name> -node <name>

Then show the results with:

storage aggregate object-store profiler show

This gives a general idea of how FabricPools will work for you before you implement them.

object-profiler

But that’s not the only object store enhancements. FabricPools in ONTAP 9.4 also offers….

Better efficiency for object storage

Prior to ONTAP 9.4, there was really no concept of freeing up space on the object store once the data blocks that had been tiered off were deleted on the source. ONTAP would see the free space, but the capacity tier would not. ONTAP 9.4 offers object defragmentation for the FabricPool destination to free up deleted blocks on the destination. This is done without any admin interaction at a specific % of free space by default for different providers. The default settings are:

  • 15%Microsoft Azure Blob Storage
  • 20% Amazon S3
  • 40% StorageGRID Webscale

These percentages are adjustable via the CLI with the following command in advanced privilege:

storage aggregate object-store modify –aggregate <name> -object-store-name <name> –unreclaimed-space-threshold <%> (0%-99%)

ONTAP 9.4 also brings support for the data compaction functionality to FabricPool aggregates to provide even more storage efficiency. For more information on data compaction, see TR-4476.

What’s great about ONTAP 9.4 is that FabricPool can now be used on any ONTAP deployment (other than MCC) with…

Support for ONTAP Select and ONTAP Cloud

FabricPools can now tier from a cloud instance to a cloud tier. This is especially useful now that we have NetApp Cloud Volumes, which run on a performance tier.

Additionally, you can use FabricPools on all versions of ONTAP Select, whether standard or Premium. This means you can tier from ONTAP Select, even if it has spinning media running under the covers. This support for spinning media does not extend into FAS systems, however – just ONTAP Select. The concern there is performance; FabricPools won’t perform well on FAS systems with spinning media.

So that’s all for the FabricPool section. Now let’s talk…

Performance!

ONTAP 9.4’s biggest news is the introduction of support for NVMe over fibre channel, as well as the NVMe attached SSDs in the new AFF A800 platform. This gives NetApp the industry’s first end-to-end NVMe platform. If you’re interested in a deep dive into what NVMe is, this podcast covered it:

Early testing numbers on the new platform show sub-200 micro-second latencies, with 1.3 million IOPS per HA pair at sub-500 micro-second latencies and 34GB/s throughput. It’s a pretty beastly system.

NVMe is integral to implementaion of workloads such as machine learning and AI, which powers tech like self-driving cars, IoT devices and other budding tech.

nvme-ai.png

If you’re a NetApp employee or partner, check out the recording of the Solutions Insight Webcast from May 9 that covers NVMe in more detail.

Another performance enhancement in ONTAP 9.4 is SMB multichannel, which provides a way for SMB3 connections to leverage more TCP streams and CPU cores on the ONTAP system to increase throughput. This especially benefits SQL server workloads.

smb-multichannel.png

The new platform and ONTAP 9.4 update doesn’t just add performance, however. It also adds…

More efficiency!

The new AFF A800 platform chassis offers efficiency in the form of both power/cooling and rack space savings with >2.5PB of storage (based on a 4:72 storage efficiency ratio) in a 4U footprint. Later, when the platform supports larger NVMe attached drives, we’ll see even more density. ONTAP 9.4 also brings support for 30TB SAS attached SSDs.

But ONTAP 9.4 also brings some additional efficiencies, such as…

Snapshot block sharing

snapshot-block-share

 

Prior to ONTAP 9.4, deduplication did not take blocks locked in a snapshot under consideration for storage efficiencies. In ONTAP 9.4, if a file is locked in a snapshot *and* it exists in the active file system, deduplication will reduce the blocks needed for the file in the active file system to save even more space. ONTAP 9.4 is also adding support for up to 1,023 snapshots per FlexVol.

Background Aggregate Level Deduplication

background-aggr-dedupe

Deduplication at the aggregate level was added in ONTAP 9.2 and provides storage efficiencies when identical blocks exist across volumes in the same aggregate. This was all done inline. In ONTAP 9.4, you can now deduplicate at the aggregate level on data that’s already been placed.

Automatic Efficiency Enablement on Data Protection Volumes

auto-dedupe-schedule.png

ONTAP 9.4 also automatically enables all storage efficiencies on data protection volumes to help simplify the role of storage administrators and save space on secondary systems.

Decreased Node Root Aggregate Sizes

Every node in an ONTAP cluster has a node root aggregate, which hosts a node root volume. The node root volume holds logs, system critical files and any core files that might get generated in the event of a crash. The core file size is based on the size of system memory. As platforms add memory to systems, these core files get larger, which was causing the core files to increase, which made root volume sizes increase… wait. This is getting confusing. Here’s a diagram:

root-vol-size-equation

Advanced Disk Partitioning (or root-data partitioning) helped save some space by spreading the volume across disk partitions, but we took steps to save even more space. For example, the 1TB root aggregate that would have been needed on the A800 node gets reduced down to just 150GB!

Long story short – ONTAP 9.4 with newer systems moved the ever-increasing core files from disk media to the local flash boot storage. This applies only to newer systems (such as the A800, FAS2700 and beyond) that have large enough boot devices to hold 2 core files and cannot be retroactively applied to older systems.

ONTAP 9.4 is also bringing…

More Security!

One of the areas of ONTAP that I feel has seen some of the most significant enhancements over the past several years  has been security (credit to Juan Mojica for making it happen).

Starting with the onboard key manager, which grew into NetApp Volume Encryption and evolved into off-box key manager support and multi-factor authentication, security has grown leaps and bounds in ONTAP. This is necessary in today’s hyper-focused security minded IT organizations, as hacks, breaches and ransomware attacks are all very fresh in their minds.

ONTAP 9.4 is bringing several more security features that don’t just help guard against external threats, but also help cover internal threats (or user mistakes) from hurting a business’s bottom line.

First of all, admins can upgrade to…

Validated ONTAP Images!

validated-ontap ONTAP is now a validated image, which gives administrators peace of mind that they’re not accidentally installing some hacked version of ONTAP that can compromise their systems. In addition, it prevents engineering builds of ONTAP (which can expose clusters to undiscovered bugs or disruptions) from being used to upgrade on clusters in the field. This helps minimize the risk and exposure of running unverified builds of ONTAP.

But we’re not just protecting against upgrading to unverified installations. ONTAP 9.4 also provides…

Key-based boot technology

secure-boot

Onboard Key Manager can be leveraged to prevent reboots without a passphrase. This protects against nefarious attempts to change the admin password on a system (which can be done with console/service processor access to the boot menu of a node), as well as against physical theft of systems. In addition to the onboard key manager, you can also enable protected boot with a USB key – but you’d need a product variance request (PVR). Check with your NetApp sales rep for details. Next generation platform (yet to be released) will also provide the ability to use UEFI Secure Boot, which works in conjunction with validated ONTAP images to not only prevent upgrades to unverified ONTAP images, but from running them at all.

These provide security against external and internal threats alike, but what do you do when someone accidentally writes a classified document to a public, unclassified share

Securely purge it!

secure-purge

ONTAP 9.4 provides the ability to cryptographically shred individual files from the drive while the system remains online, and the rest of the files remain intact. This can be helpful for data spillage – e.g. when a classified document ends up in an unclassified location. This is also particularly timely and useful for the upcoming GDPR regulations’ “Right to Erasure” rules.

Security is playing a big part in the new release of ONTAP. In addition, here’s some more…

General ONTAP goodness

ONTAP 9.4 also brings several other valuable features, such as:

  • Rapid disk zeroing technology – initialize disks near-instantaneously in newer platforms!
  • 3-step, 1-click ONTAP upgrades – even easier to update your cluster non-disruptively
  • Install ONTAP without needing a separate web or FTP server
  • SQL Server support for Application Data Management in System Manager

So, there you are! A thorough rundown of the new features in ONTAP 9.4. If you feel I missed something, feel free to reach out in the comments with input!

Check out these brief videos for some lightboard action on new ONTAP 9.4 stuff:

Some other information on the launch can be found as follows:

GCP Cloud Volumes for NFS with native access to the GCP tool suite (Google Cloud)
https://blog.netapp.com/sweet-new-storage-service-from-netapp-for-google-cloud-platform/ 

Storage Grid Update 11.1
https://blog.netapp.com/storagegrid-11-1-and-netapp-hci-the-perfect-one-two-punch-for-scaling-your-environment/ 

A800 and the A220
https://blog.netapp.com/the-future-is-here-ai-ready-cloud-connected-all-flash-storage-with-nvme/ 

ONTAP 9.4 with first to market NVMe/FC support
http://www.demartek.com/Demartek_NetApp_Broadcom_NVMe_over_Fibre_Channel_Evaluation_2018-05.html

11 thoughts on “ONTAP 9.4RC1 is now available!”

  1. Hey Justin, great post. Two quick Fabric Pool questions:

    1. The threshold for Fabric Pool defrag is by % unreclaimed on the cloud side right? How is this determined? Some sort of object defrag scanner? Is it run on demand, or on a schedule? From the controller, or in the cloud somehow?

    2. Why the difference in % defaults between S3, Azure, and StorageGrid? SG @ 40% seems high, I’m sure there’s a good reason….

    Like

    Reply

    • Hi Eric, Thanks for the questions

      1. In case of FabricPool, ONTAP still maintains the filesystem even when the WAFL blocks are in cloud. When this number of referenced WAFL blocks within a 4MB object reaches a certain threshold, then the defrag kicks in. This is automatically done by ONTAP
      2. All the hyperscalers charge for the data that is read, the % differences is based on the calculation, whether it is economical to leave the unreferenced blocks and pay storage costs or read all the referenced blocks to form a new object and pay the data transfer costs. With on-prem there is no cost associated with it.

      Hope it helps

      Thanks
      Arun

      Like

      Reply

  2. A few questions, which I cannot get a solid answer from your article or the release notes:
    Are the Snapshot Block Sharing and Automatic Efficiency Enablement on Data Protection Volumes also available for non-AFF systems? The release notes talk about a new DP_Optimized (DPO) license to have these features enabled, is that license also available for existing systems?

    Like

    Reply

    • DPO licenses are apparently only on new systems that are sold as DPO systems. Non-AFF systems apply to auto-efficiency enablement. Still waiting on word about snapshot block sharing from the PM team, but I suspect it is.

      Like

      Reply

      • Is it certain this is only available for specific DPO bundles?

        The documentation really speaks of a license, without noting the requirement of a specific personality of the system.

        “Starting with ONTAP 9.3, a new DP_Optimized (DPO) license is available that supports an increased number of volumes and peer relationships. A SnapMirror license is still required on both the source and destination.”

        If this is only available on systems with this specific DPO personality the documentation is at least ambiguous on this point.

        Like

  5. Pingback: New and updated FlexGroup Technical Reports now available for ONTAP 9.4! | Why Is The Internet Broken?

  6. Pingback: ONTAP 9.4 is now GA! | Why Is The Internet Broken?

Leave a comment