Behind the Scenes: Episode 140 Quarterly Security Update: ONTAP 9.4 and GDPR

Welcome to the Episode 140, part of the continuing series called “Behind the Scenes of the NetApp Tech ONTAP Podcast.”

tot-gopher

This week on the podcast, we bring in Security PM Juan Mojica (@Juan_M_Mojica) and Security TMEs Andrae Middleton and Dan Tulledge to get ready for GDPR by discussing ONTAP 9.4’s newest security enhancements and what they mean for the new European regulation as the grace period ends. We also discuss best practices and how to best protect your storage systems from breaches.

For our GDPR landing page: https://www.netapp.com/us/info/gdpr.aspx

For the latest ONTAP 9.4 Security blog: https://blog.netapp.com/new-data-security-and-privacy-features-in-ontap-9-4

Finding the Podcast

The podcast is all finished and up for listening. You can find it on iTunes or SoundCloud or by going to techontappodcast.com.

This week’s episode is here:

Also, if you don’t like using iTunes or SoundCloud, we just added the podcast to Stitcher.

http://www.stitcher.com/podcast/tech-ontap-podcast?refid=stpr

I also recently got asked how to leverage RSS for the podcast. You can do that here:

http://feeds.soundcloud.com/users/soundcloud:users:164421460/sounds.rss

Our YouTube channel (episodes uploaded sporadically) is here:

Advertisements

New and updated FlexGroup Technical Reports now available for ONTAP 9.4!

ONTAP 9.4 is now available, so that means the TRs need to get a refresh.

161212-westworld-news

Here’s what I’ve done for FlexGroup in ONTAP 9.4…

New Tech Report!

First, I moved the data protection section of the best practices TR (TR-4571) into its own dedicated backup and data protection TR, which can be found here:

TR-4678: Data Protection and Backup – FlexGroup volumes

Why? Well, that section is going to grow larger and larger as we add more data protection and backup functionality, so it made sense to proactively create a new one.

Updated TRs!

TR-4557 got an update of mostly just what’s new in ONTAP 9.4. That TR is a technical overview, which is intended just to give information on how FlexGroups work. The new feature payload for FlexGroup volumes in ONTAP 9.4 included:

  • QoS minimums and Adaptive QoS
  • FPolicy and file audit
  • SnapDiff support

TR-4571 is the best practices TR and got a brunt of the updates. Included in the TR (aside from details about new features), I added:

  • More detailed information about high file count environments and directory structure
  • More information about maxdirsize limits
  • Information on effects of drive failures
  • Workarounds for lack of NFSv4.x ACL support
  • Member volume count considerations when dealing with small and large files
  • Considerations when deleting FlexGroup volumes (and the volume recovery queue)
  • Clarifications on requirements for available space in an aggregate
  • System Manager support updates

Most of these updates came from feedback and questions I received. If you have something you want to see added to the TRs, let me know!

Behind the Scenes: Episode 139 – NVMe and New Hardware in ONTAP 9.4

Welcome to the Episode 139, part of the continuing series called “Behind the Scenes of the NetApp Tech ONTAP Podcast.”

tot-gopher

Finding the Podcast

The podcast is all finished and up for listening. You can find it on iTunes or SoundCloud or by going to techontappodcast.com.

This week’s episode is here:

Also, if you don’t like using iTunes or SoundCloud, we just added the podcast to Stitcher.

http://www.stitcher.com/podcast/tech-ontap-podcast?refid=stpr

I also recently got asked how to leverage RSS for the podcast. You can do that here:

http://feeds.soundcloud.com/users/soundcloud:users:164421460/sounds.rss

Our YouTube channel (episodes uploaded sporadically) is here:

Behind the Scenes: Episode 138 – ONTAP 9.4 General Overview

Welcome to the Episode 138, part of the continuing series called “Behind the Scenes of the NetApp Tech ONTAP Podcast.”

tot-gopher

This week on the podcast, ONTAP 9.4 is here! I managed to snag ONTAP Senior Vice President Octavian Tanase (@octav) and ONTAP Chief Evangelist Jeff Baxter (@baxontap) to discuss what went into some of the decisions we made regarding the new feature payload, the future vision for ONTAP and what new stuff you can expect in this release.

To download the new release:

http://mysupport.netapp.com/NOW/download/software/ontap/9.4RC1

Check out these videos for some lightboard action on new ONTAP 9.4 stuff:

Finding the Podcast

The podcast is all finished and up for listening. You can find it on iTunes or SoundCloud or by going to techontappodcast.com.

This week’s episode is here:

Also, if you don’t like using iTunes or SoundCloud, we just added the podcast to Stitcher.

http://www.stitcher.com/podcast/tech-ontap-podcast?refid=stpr

I also recently got asked how to leverage RSS for the podcast. You can do that here:

http://feeds.soundcloud.com/users/soundcloud:users:164421460/sounds.rss

Our YouTube channel (episodes uploaded sporadically) is here:

ONTAP 9.4RC1 is now available!

Hear ye! Hear ye! All ye storage admins! ONTAP 9.4RC1 is announced today!

sully-hearye

That’s right! Every 6 months, without fail, a new ONTAP version with a payload of new features is released.

You can find ONTAP 9.4RC1 here:

http://mysupport.netapp.com/NOW/download/software/ontap/9.4RC1

For info on what a release candidate is, see:

http://mysupport.netapp.com/NOW/products/ontap_releasemodel/

Also, check out the documentation center:

docs.netapp.com/ontap-9/index.jsp

NetApp published a general overview blog on NVMe with Joel Reich here:

https://blog.netapp.com/the-future-is-here-ai-ready-cloud-connected-all-flash-storage-with-nvme/

Stay tuned for a more general ONTAP 9.4 overview blog on the official site. Also, I recorded a brief 5-minute teaser/trailer for ONTAP 9.4 features and podcasts coming soon. Find that here:

Also a new lightboard video! Watch me write… BACKWARDS???

This blog is intended to go a little deeper into the main features available in ONTAP 9.4. We’ll break them down as follows:

  • Cloud
  • Performance
  • Efficiency
  • Security
  • General ONTAP Goodness

Without further ado…

Cloud!

FabricPools were introduced in ONTAP 9.2 as a way to tier blocks from your performance tier solution to a capacity tier, such as cloud or StorageGrid.

We covered FabricPools in detail in episode 92 of the Tech ONTAP Podcast, which you can find here:

In ONTAP 9.4, the first major updates to the feature have been released! FabricPools in ONTAP 9.4 bring the following…

Tiering cold data from the active file system

Prior to ONTAP 9.4, FabricPools only tiered cold data from snapshots on primary systems and data protection volumes on secondary systems. This allowed ONTAP to free up valuable real estate on flash systems for data actively being used. In ONTAP 9.4, inactive blocks can now be tiered off to cloud or StorageGrid from the active file system. ONTAP does this automatically by way of a new “auto” tiering policy, which has a configurable cooling period of 2-63 days (-tiering-minimum-cooling-days option in CLI). This cooling period determines how long ONTAP will wait before tiering off data considered “cool” by the policy to the FabricPool tiering destination. The tiering destination choices used to be only Amazon S3 and StorageGrid, but ONTAP 9.4 brings us…

Tiering to Azure Blob Storage

Support for Azure Blob storage was added to ONTAP 9.4 for FabricPools, which gives storage administrators more options for cloud providers. In addition, other cloud providers (such as Google Cloud, IBM Cloud Object Storage, etc) can be added via product variance requests (PVR) to your NetApp Sales reps. Keep in mind that only one cloud provider per FabricPool aggregate can be used.

fabricpools-afs

But how do you know if FabricPools will be of any value to you?

Inactive Data Reporting

Inactive Data Reporting is new in ONTAP 9.4 and can offer insight from OnCommand System Manager into whether there’s enough inactive data in your system for FabricPools to make a difference.

fabricpools-inactive-report.png

By default, this feature is enabled for aggregates participating in FabricPools, but you can also enable it via the CLI for non-FabricPool aggregates to predict space savings with the following command:

storage aggregate modify -aggregate <name> -is-inactive-data-reporting-enabled true

You can also test the performance of your FabricPool target with…

Object Store Profiler

Also new in ONTAP 9.4, the Object Store Profiler provides a way to evaluate the performance (via throughput and latency) to your desired FabricPool target. From the CLI, start the profiler using:

storage aggregate object-store profiler start -object-store-name <name> -node <name>

Then show the results with:

storage aggregate object-store profiler show

This gives a general idea of how FabricPools will work for you before you implement them.

object-profiler

But that’s not the only object store enhancements. FabricPools in ONTAP 9.4 also offers….

Better efficiency for object storage

Prior to ONTAP 9.4, there was really no concept of freeing up space on the object store once the data blocks that had been tiered off were deleted on the source. ONTAP would see the free space, but the capacity tier would not. ONTAP 9.4 offers object defragmentation for the FabricPool destination to free up deleted blocks on the destination. This is done without any admin interaction at a specific % of free space by default for different providers. The default settings are:

  • 15%Microsoft Azure Blob Storage
  • 20% Amazon S3
  • 40% StorageGRID Webscale

These percentages are adjustable via the CLI with the following command in advanced privilege:

storage aggregate object-store modify –aggregate <name> -object-store-name <name> –unreclaimed-space-threshold <%> (0%-99%)

ONTAP 9.4 also brings support for the data compaction functionality to FabricPool aggregates to provide even more storage efficiency. For more information on data compaction, see TR-4476.

What’s great about ONTAP 9.4 is that FabricPool can now be used on any ONTAP deployment (other than MCC) with…

Support for ONTAP Select and ONTAP Cloud

FabricPools can now tier from a cloud instance to a cloud tier. This is especially useful now that we have NetApp Cloud Volumes, which run on a performance tier.

Additionally, you can use FabricPools on all versions of ONTAP Select, whether standard or Premium. This means you can tier from ONTAP Select, even if it has spinning media running under the covers. This support for spinning media does not extend into FAS systems, however – just ONTAP Select. The concern there is performance; FabricPools won’t perform well on FAS systems with spinning media.

So that’s all for the FabricPool section. Now let’s talk…

Performance!

ONTAP 9.4’s biggest news is the introduction of support for NVMe over fibre channel, as well as the NVMe attached SSDs in the new AFF A800 platform. This gives NetApp the industry’s first end-to-end NVMe platform. If you’re interested in a deep dive into what NVMe is, this podcast covered it:

Early testing numbers on the new platform show sub-200 micro-second latencies, with 1.3 million IOPS per HA pair at sub-500 micro-second latencies and 34GB/s throughput. It’s a pretty beastly system.

NVMe is integral to implementaion of workloads such as machine learning and AI, which powers tech like self-driving cars, IoT devices and other budding tech.

nvme-ai.png

If you’re a NetApp employee or partner, check out the recording of the Solutions Insight Webcast from May 9 that covers NVMe in more detail.

Another performance enhancement in ONTAP 9.4 is SMB multichannel, which provides a way for SMB3 connections to leverage more TCP streams and CPU cores on the ONTAP system to increase throughput. This especially benefits SQL server workloads.

smb-multichannel.png

The new platform and ONTAP 9.4 update doesn’t just add performance, however. It also adds…

More efficiency!

The new AFF A800 platform chassis offers efficiency in the form of both power/cooling and rack space savings with >2.5PB of storage (based on a 4:72 storage efficiency ratio) in a 4U footprint. Later, when the platform supports larger NVMe attached drives, we’ll see even more density. ONTAP 9.4 also brings support for 30TB SAS attached SSDs.

But ONTAP 9.4 also brings some additional efficiencies, such as…

Snapshot block sharing

snapshot-block-share

 

Prior to ONTAP 9.4, deduplication did not take blocks locked in a snapshot under consideration for storage efficiencies. In ONTAP 9.4, if a file is locked in a snapshot *and* it exists in the active file system, deduplication will reduce the blocks needed for the file in the active file system to save even more space. ONTAP 9.4 is also adding support for up to 1,023 snapshots per FlexVol.

Background Aggregate Level Deduplication

background-aggr-dedupe

Deduplication at the aggregate level was added in ONTAP 9.2 and provides storage efficiencies when identical blocks exist across volumes in the same aggregate. This was all done inline. In ONTAP 9.4, you can now deduplicate at the aggregate level on data that’s already been placed.

Automatic Efficiency Enablement on Data Protection Volumes

auto-dedupe-schedule.png

ONTAP 9.4 also automatically enables all storage efficiencies on data protection volumes to help simplify the role of storage administrators and save space on secondary systems.

Decreased Node Root Aggregate Sizes

Every node in an ONTAP cluster has a node root aggregate, which hosts a node root volume. The node root volume holds logs, system critical files and any core files that might get generated in the event of a crash. The core file size is based on the size of system memory. As platforms add memory to systems, these core files get larger, which was causing the core files to increase, which made root volume sizes increase… wait. This is getting confusing. Here’s a diagram:

root-vol-size-equation

Advanced Disk Partitioning (or root-data partitioning) helped save some space by spreading the volume across disk partitions, but we took steps to save even more space. For example, the 1TB root aggregate that would have been needed on the A800 node gets reduced down to just 150GB!

Long story short – ONTAP 9.4 with newer systems moved the ever-increasing core files from disk media to the local flash boot storage. This applies only to newer systems (such as the A800, FAS2700 and beyond) that have large enough boot devices to hold 2 core files and cannot be retroactively applied to older systems.

ONTAP 9.4 is also bringing…

More Security!

One of the areas of ONTAP that I feel has seen some of the most significant enhancements over the past several years  has been security (credit to Juan Mojica for making it happen).

Starting with the onboard key manager, which grew into NetApp Volume Encryption and evolved into off-box key manager support and multi-factor authentication, security has grown leaps and bounds in ONTAP. This is necessary in today’s hyper-focused security minded IT organizations, as hacks, breaches and ransomware attacks are all very fresh in their minds.

ONTAP 9.4 is bringing several more security features that don’t just help guard against external threats, but also help cover internal threats (or user mistakes) from hurting a business’s bottom line.

First of all, admins can upgrade to…

Validated ONTAP Images!

validated-ontap ONTAP is now a validated image, which gives administrators peace of mind that they’re not accidentally installing some hacked version of ONTAP that can compromise their systems. In addition, it prevents engineering builds of ONTAP (which can expose clusters to undiscovered bugs or disruptions) from being used to upgrade on clusters in the field. This helps minimize the risk and exposure of running unverified builds of ONTAP.

But we’re not just protecting against upgrading to unverified installations. ONTAP 9.4 also provides…

Key-based boot technology

secure-boot

Onboard Key Manager can be leveraged to prevent reboots without a passphrase. This protects against nefarious attempts to change the admin password on a system (which can be done with console/service processor access to the boot menu of a node), as well as against physical theft of systems. In addition to the onboard key manager, you can also enable protected boot with a USB key – but you’d need a product variance request (PVR). Check with your NetApp sales rep for details. Next generation platform (yet to be released) will also provide the ability to use UEFI Secure Boot, which works in conjunction with validated ONTAP images to not only prevent upgrades to unverified ONTAP images, but from running them at all.

These provide security against external and internal threats alike, but what do you do when someone accidentally writes a classified document to a public, unclassified share

Securely purge it!

secure-purge

ONTAP 9.4 provides the ability to cryptographically shred individual files from the drive while the system remains online, and the rest of the files remain intact. This can be helpful for data spillage – e.g. when a classified document ends up in an unclassified location. This is also particularly timely and useful for the upcoming GDPR regulations’ “Right to Erasure” rules.

Security is playing a big part in the new release of ONTAP. In addition, here’s some more…

General ONTAP goodness

ONTAP 9.4 also brings several other valuable features, such as:

  • Rapid disk zeroing technology – initialize disks near-instantaneously in newer platforms!
  • 3-step, 1-click ONTAP upgrades – even easier to update your cluster non-disruptively
  • Install ONTAP without needing a separate web or FTP server
  • SQL Server support for Application Data Management in System Manager

So, there you are! A thorough rundown of the new features in ONTAP 9.4. If you feel I missed something, feel free to reach out in the comments with input!

Check out these brief videos for some lightboard action on new ONTAP 9.4 stuff:

Some other information on the launch can be found as follows:

GCP Cloud Volumes for NFS with native access to the GCP tool suite (Google Cloud)
https://blog.netapp.com/sweet-new-storage-service-from-netapp-for-google-cloud-platform/ 

Storage Grid Update 11.1
https://blog.netapp.com/storagegrid-11-1-and-netapp-hci-the-perfect-one-two-punch-for-scaling-your-environment/ 

A800 and the A220
https://blog.netapp.com/the-future-is-here-ai-ready-cloud-connected-all-flash-storage-with-nvme/ 

ONTAP 9.4 with first to market NVMe/FC support
http://www.demartek.com/Demartek_NetApp_Broadcom_NVMe_over_Fibre_Channel_Evaluation_2018-05.html