When I started this site back in October of 2014, it was mainly to drive traffic to my NetApp Insight sessions -and it worked.
(By the way… stay tuned for a blog on this year’s new Insight sessions by yours truly. Now with more lab!)
As I continued writing, my goal was to keep creating content – don’t be the guy who just shows up during conference season.
So far, so good.
But since I create so much content, it gets hard to find for new visitors to this site, The WordPress archives/table of contents is lacking. So, what I’ve done is create my own table of contents of the top 5 most visited posts.
You can also search on the site or click through the archives, if you choose. Or, subscribe to the RSS feed. If you have questions or want to see something changed or added to the site, follow me on Twitter @NFSDudeAbides or comment on one of the posts here!
Welcome to the Episode 140, part of the continuing series called “Behind the Scenes of the NetApp Tech ONTAP Podcast.”
This week on the podcast, we bring in Security PM Juan Mojica (@Juan_M_Mojica) and Security TMEs Andrae Middleton and Dan Tulledge to get ready for GDPR by discussing ONTAP 9.4’s newest security enhancements and what they mean for the new European regulation as the grace period ends. We also discuss best practices and how to best protect your storage systems from breaches.
First, I moved the data protection section of the best practices TR (TR-4571) into its own dedicated backup and data protection TR, which can be found here:
Why? Well, that section is going to grow larger and larger as we add more data protection and backup functionality, so it made sense to proactively create a new one.
Updated TRs!
TR-4557 got an update of mostly just what’s new in ONTAP 9.4. That TR is a technical overview, which is intended just to give information on how FlexGroups work. The new feature payload for FlexGroup volumes in ONTAP 9.4 included:
QoS minimums and Adaptive QoS
FPolicy and file audit
SnapDiff support
TR-4571 is the best practices TR and got a brunt of the updates. Included in the TR (aside from details about new features), I added:
More detailed information about high file count environments and directory structure
More information about maxdirsize limits
Information on effects of drive failures
Workarounds for lack of NFSv4.x ACL support
Member volume count considerations when dealing with small and large files
Welcome to the Episode 138, part of the continuing series called “Behind the Scenes of the NetApp Tech ONTAP Podcast.”
This week on the podcast, ONTAP 9.4 is here! I managed to snag ONTAP Senior Vice President Octavian Tanase (@octav) and ONTAP Chief Evangelist Jeff Baxter (@baxontap) to discuss what went into some of the decisions we made regarding the new feature payload, the future vision for ONTAP and what new stuff you can expect in this release.
Stay tuned for a more general ONTAP 9.4 overview blog on the official site. Also, I recorded a brief 5-minute teaser/trailer for ONTAP 9.4 features and podcasts coming soon. Find that here:
Also a new lightboard video! Watch me write… BACKWARDS???
This blog is intended to go a little deeper into the main features available in ONTAP 9.4. We’ll break them down as follows:
Cloud
Performance
Efficiency
Security
General ONTAP Goodness
Without further ado…
Cloud!
FabricPools were introduced in ONTAP 9.2 as a way to tier blocks from your performance tier solution to a capacity tier, such as cloud or StorageGrid.
We covered FabricPools in detail in episode 92 of the Tech ONTAP Podcast, which you can find here:
In ONTAP 9.4, the first major updates to the feature have been released! FabricPools in ONTAP 9.4 bring the following…
Tiering cold data from the active file system
Prior to ONTAP 9.4, FabricPools only tiered cold data from snapshots on primary systems and data protection volumes on secondary systems. This allowed ONTAP to free up valuable real estate on flash systems for data actively being used. In ONTAP 9.4, inactive blocks can now be tiered off to cloud or StorageGrid from the active file system. ONTAP does this automatically by way of a new “auto” tiering policy, which has a configurable cooling period of 2-63 days (-tiering-minimum-cooling-days option in CLI). This cooling period determines how long ONTAP will wait before tiering off data considered “cool” by the policy to the FabricPool tiering destination. The tiering destination choices used to be only Amazon S3 and StorageGrid, but ONTAP 9.4 brings us…
Tiering to Azure Blob Storage
Support for Azure Blob storage was added to ONTAP 9.4 for FabricPools, which gives storage administrators more options for cloud providers. In addition, other cloud providers (such as Google Cloud, IBM Cloud Object Storage, etc) can be added via product variance requests (PVR) to your NetApp Sales reps. Keep in mind that only one cloud provider per FabricPool aggregate can be used.
But how do you know if FabricPools will be of any value to you?
Inactive Data Reporting
Inactive Data Reporting is new in ONTAP 9.4 and can offer insight from OnCommand System Manager into whether there’s enough inactive data in your system for FabricPools to make a difference.
By default, this feature is enabled for aggregates participating in FabricPools, but you can also enable it via the CLI for non-FabricPool aggregates to predict space savings with the following command:
You can also test the performance of your FabricPool target with…
Object Store Profiler
Also new in ONTAP 9.4, the Object Store Profiler provides a way to evaluate the performance (via throughput and latency) to your desired FabricPool target. From the CLI, start the profiler using:
This gives a general idea of how FabricPools will work for you before you implement them.
But that’s not the only object store enhancements. FabricPools in ONTAP 9.4 also offers….
Better efficiency for object storage
Prior to ONTAP 9.4, there was really no concept of freeing up space on the object store once the data blocks that had been tiered off were deleted on the source. ONTAP would see the free space, but the capacity tier would not. ONTAP 9.4 offers object defragmentation for the FabricPool destination to free up deleted blocks on the destination. This is done without any admin interaction at a specific % of free space by default for different providers. The default settings are:
15%Microsoft Azure Blob Storage
20% Amazon S3
40% StorageGRID Webscale
These percentages are adjustable via the CLI with the following command in advanced privilege:
ONTAP 9.4 also brings support for the data compaction functionality to FabricPool aggregates to provide even more storage efficiency. For more information on data compaction, see TR-4476.
What’s great about ONTAP 9.4 is that FabricPool can now be used on any ONTAP deployment (other than MCC) with…
Support for ONTAP Select and ONTAP Cloud
FabricPools can now tier from a cloud instance to a cloud tier. This is especially useful now that we have NetApp Cloud Volumes, which run on a performance tier.
Additionally, you can use FabricPools on all versions of ONTAP Select, whether standard or Premium. This means you can tier from ONTAP Select, even if it has spinning media running under the covers. This support for spinning media does not extend into FAS systems, however – just ONTAP Select. The concern there is performance; FabricPools won’t perform well on FAS systems with spinning media.
So that’s all for the FabricPool section. Now let’s talk…
Performance!
ONTAP 9.4’s biggest news is the introduction of support for NVMe over fibre channel, as well as the NVMe attached SSDs in the new AFF A800 platform. This gives NetApp the industry’s first end-to-end NVMe platform. If you’re interested in a deep dive into what NVMe is, this podcast covered it:
Early testing numbers on the new platform show sub-200 micro-second latencies, with 1.3 million IOPS per HA pair at sub-500 micro-second latencies and 34GB/s throughput. It’s a pretty beastly system.
NVMe is integral to implementaion of workloads such as machine learning and AI, which powers tech like self-driving cars, IoT devices and other budding tech.
If you’re a NetApp employee or partner, check out the recording of the Solutions Insight Webcast from May 9 that covers NVMe in more detail.
Another performance enhancement in ONTAP 9.4 is SMB multichannel, which provides a way for SMB3 connections to leverage more TCP streams and CPU cores on the ONTAP system to increase throughput. This especially benefits SQL server workloads.
The new platform and ONTAP 9.4 update doesn’t just add performance, however. It also adds…
More efficiency!
The new AFF A800 platform chassis offers efficiency in the form of both power/cooling and rack space savings with >2.5PB of storage (based on a 4:72 storage efficiency ratio) in a 4U footprint. Later, when the platform supports larger NVMe attached drives, we’ll see even more density. ONTAP 9.4 also brings support for 30TB SAS attached SSDs.
But ONTAP 9.4 also brings some additional efficiencies, such as…
Snapshot block sharing
Prior to ONTAP 9.4, deduplication did not take blocks locked in a snapshot under consideration for storage efficiencies. In ONTAP 9.4, if a file is locked in a snapshot *and* it exists in the active file system, deduplication will reduce the blocks needed for the file in the active file system to save even more space. ONTAP 9.4 is also adding support for up to 1,023 snapshots per FlexVol.
Background Aggregate Level Deduplication
Deduplication at the aggregate level was added in ONTAP 9.2 and provides storage efficiencies when identical blocks exist across volumes in the same aggregate. This was all done inline. In ONTAP 9.4, you can now deduplicate at the aggregate level on data that’s already been placed.
Automatic Efficiency Enablement on Data Protection Volumes
ONTAP 9.4 also automatically enables all storage efficiencies on data protection volumes to help simplify the role of storage administrators and save space on secondary systems.
Decreased Node Root Aggregate Sizes
Every node in an ONTAP cluster has a node root aggregate, which hosts a node root volume. The node root volume holds logs, system critical files and any core files that might get generated in the event of a crash. The core file size is based on the size of system memory. As platforms add memory to systems, these core files get larger, which was causing the core files to increase, which made root volume sizes increase… wait. This is getting confusing. Here’s a diagram:
Advanced Disk Partitioning (or root-data partitioning) helped save some space by spreading the volume across disk partitions, but we took steps to save even more space. For example, the 1TB root aggregate that would have been needed on the A800 node gets reduced down to just 150GB!
Long story short – ONTAP 9.4 with newer systems moved the ever-increasing core files from disk media to the local flash boot storage. This applies only to newer systems (such as the A800, FAS2700 and beyond) that have large enough boot devices to hold 2 core files and cannot be retroactively applied to older systems.
ONTAP 9.4 is also bringing…
More Security!
One of the areas of ONTAP that I feel has seen some of the most significant enhancements over the past several years has been security (credit to Juan Mojica for making it happen).
Starting with the onboard key manager, which grew into NetApp Volume Encryption and evolved into off-box key manager support and multi-factor authentication, security has grown leaps and bounds in ONTAP. This is necessary in today’s hyper-focused security minded IT organizations, as hacks, breaches and ransomware attacks are all very fresh in their minds.
ONTAP 9.4 is bringing several more security features that don’t just help guard against external threats, but also help cover internal threats (or user mistakes) from hurting a business’s bottom line.
First of all, admins can upgrade to…
Validated ONTAP Images!
ONTAP is now a validated image, which gives administrators peace of mind that they’re not accidentally installing some hacked version of ONTAP that can compromise their systems. In addition, it prevents engineering builds of ONTAP (which can expose clusters to undiscovered bugs or disruptions) from being used to upgrade on clusters in the field. This helps minimize the risk and exposure of running unverified builds of ONTAP.
But we’re not just protecting against upgrading to unverified installations. ONTAP 9.4 also provides…
Key-based boot technology
Onboard Key Manager can be leveraged to prevent reboots without a passphrase. This protects against nefarious attempts to change the admin password on a system (which can be done with console/service processor access to the boot menu of a node), as well as against physical theft of systems. In addition to the onboard key manager, you can also enable protected boot with a USB key – but you’d need a product variance request (PVR). Check with your NetApp sales rep for details. Next generation platform (yet to be released) will also provide the ability to use UEFI Secure Boot, which works in conjunction with validated ONTAP images to not only prevent upgrades to unverified ONTAP images, but from running them at all.
These provide security against external and internal threats alike, but what do you do when someone accidentally writes a classified document to a public, unclassified share
Securely purge it!
ONTAP 9.4 provides the ability to cryptographically shred individual files from the drive while the system remains online, and the rest of the files remain intact. This can be helpful for data spillage – e.g. when a classified document ends up in an unclassified location. This is also particularly timely and useful for the upcoming GDPR regulations’ “Right to Erasure” rules.
Security is playing a big part in the new release of ONTAP. In addition, here’s some more…
General ONTAP goodness
ONTAP 9.4 also brings several other valuable features, such as:
Rapid disk zeroing technology – initialize disks near-instantaneously in newer platforms!
3-step, 1-click ONTAP upgrades – even easier to update your cluster non-disruptively
Install ONTAP without needing a separate web or FTP server
SQL Server support for Application Data Management in System Manager
So, there you are! A thorough rundown of the new features in ONTAP 9.4. If you feel I missed something, feel free to reach out in the comments with input!
Check out these brief videos for some lightboard action on new ONTAP 9.4 stuff:
Welcome to the Episode 137, part of the continuing series called “Behind the Scenes of the NetApp Tech ONTAP Podcast.”
This week on the podcast, we talk Name Services in ONTAP and the introduction of the new global name services cache in ONTAP 9.3 with NAS TME, Chris Hurley (@averageguyX)!
We’ll be taking next week off as we record and prepare for some big announcements coming soon!
Finding the Podcast
The podcast is all finished and up for listening. You can find it on iTunes or SoundCloud or by going to techontappodcast.com.
This week’s episode is here:
Also, if you don’t like using iTunes or SoundCloud, we just added the podcast to Stitcher.
This week on the podcast, we invited a couple of NetApp’s own dev and QA members – David Pressley and Mekka Williams – to discuss how NetApp is modernizing and streamlining their processes and how other businesses can take lessons from how NetApp is leveraging automation, dev/ops and the cloud to make development life fitter, happier and more productive!
Finding the Podcast
The podcast is all finished and up for listening. You can find it on iTunes or SoundCloud or by going to techontappodcast.com.
This week’s episode is here:
Also, if you don’t like using iTunes or SoundCloud, we just added the podcast to Stitcher.
This week on the podcast, NetApp Director of Technology and Strategy in EMEA, Matt Watts (@mtjwatts), joins us, to talk about his latest blog and how current technology is simply repeating the patterns in nature to better understand how people are using data and leveraging applications.
For Matt’s latest blog titled “Pets, Cattle, Insects and the Fabric of life,” see:
This week on the podcast, Active IQ Technical Director Shankar Pasupathy joins us and tells us how AutoSupport’s infrastructure and backend evolved into Active IQ’s multicloud data pipeline. Learn how NetApp is using big data analytics and machine learning on ONTAP to improve the overall customer experience
Finding the Podcast
The podcast is all finished and up for listening. You can find it on iTunes or SoundCloud or by going to techontappodcast.com.
This week’s episode is here:
Also, if you don’t like using iTunes or SoundCloud, we just added the podcast to Stitcher.
I happened to be in Sunnyvale at NetApp headquarters this week and was pleasantly surprised to find out that Cloud Field Day 3 was going to also be in town.
You can also read this blog along with the recording to re-enact the live-ness. Kinda like watching the Wizard of Oz while listening to Dark Side of the Moon.
4:18PM PST – The start
Foskett kicks it off and we begin with Erik Hrafnsson, co-founder of Greenqloud, which NetApp acquired back in 2017. He looks a bit like Thor, which is awesome, because Thor is the god of thunder. And lightning. Which comes from… clouds.
He’s also Icelandic, so I’m probably stereotyping a bit here… (Yes I know Thor is Norse)
4:22PM PST
Erik hates corporations. But he likes NetApp. He’s a fan of the data fabric and where they’re taking cloud.
4:27PM PST
Erik is now flying the Jolly Roger on his slide deck. I believe this is him and Greenqloud taking over.
4:28PM PST
Greeqloud is a way to make the Data Fabric tangible and real, by way of orchestrating multi-cloud environments.
Also, simplification. Lots of features and components (threads?) in the fabric to tie together. Pun intended.
4:31PM PST
Erik just named dropped Cloud Control. There’s a podcast for that. 🙂
4:33PM PST
Now we’re going into ONTAP Cloud and Cloud Volumes. Guess what? There’s a podcast for that, too! (I promise I won’t keep plugging the podcast, but as Keith Townsend just mentioned, not everyone knows what ONTAP is)
4:37PM PST
Erik pointing out that ONTAP in the cloud out-performs AWS in some cases, and can be more cost effective.
Plus, you get cool stuff like instant snapshots/restores.
4:40PM PST
What’s in a name? Erik explains why cloud volumes, ONTAP cloud, etc.
Common names mean NetApp will be… multi-cloud!
4:43PM PST
Demo time!
Fun fact: Demo is short for “demonstration.” Demonstration contains the word “demon.” Demo can also be a way to say “demolition.” When dealing with live demos, these words are all interchangeable.
I believe Erik is tempting the live demo fates here. Godspeed.
4:48PM PST
So far, so good. Live demo going smoothly. Erik must actually be Thor.
*checks over shoulder for lightning strike*
4:51PM PST
Multiprotocol NAS volumes in the cloud! Erik did commit the cardinal sin of saying “CIFS” in the presence of Stephen Foskett, but because Erik is Thor, Foskett has no recourse.
4:55PM PST
Question was posed about NPS (NetApp Private Storage) and whether it still exists and if this replaces it. Yes, it still exists. No, it doesn’t replace it. Find out more about it here:
Roadmap alert! Erik says “with high confidence” that you’re going to be able to know where in the world your data is. It’s like Waldo, for data.
5:05PM PST
Erik is mounting via NFS. I like this guy.
The cool part is that the orchestration will formulate the NFS mount options to best optimize performance based on the application.
5:07PM PST
Michelle Laverick asks how this isn’t just a storage array in the cloud. I’m sad to say…
5:16PM PST
Part of simplicity has to do with not having to think. Application-based storage provisioning in the cloud removes the thought process behind having to decide how to provision.
5:21PM PST
FIO RACE!
Cloud Volumes took 52s to complete the same test that EFS took 16.5 minutes to finish. I guess calling it a “race” was being generous.
5:32PM PST
Uh oh. Another live demo. This time, CLOUD SYNC!
Current status: clutching rosary beads.
5:34PM PST
You wouldn’t put all your eggs in one basket. Why would you put all your data in one bucket? NetApp Cloud Sync allows you to move freely between clouds.
5:43PM PST
Erik is building a GitLab cluster right now. Looking forward to having FlexGroup volumes being a part of this story in the future.
5:47PM PST
Ooh… container orchestration in the cloud using Kubernetes! Fancy schmancy!
5:50PM EST
FOR THE LOVE OF ALL THAT’S HOLY WE ARE A DATA MANAGEMENT COMPANY!
5:56PM EST
All done here! The CFD delegates don’t seem to be breaking out the pitchforks, so we’ll call that a win!
Thoughts on Industrial Computing. Or sometimes economics or music or trail runnning or carpentry. Employee of Datto. Proud New Englander and Schaghticoke. Site is my own.
ONTAP is now a validated image, which gives administrators peace of mind that they’re not accidentally installing some hacked version of ONTAP that can compromise their systems. In addition, it prevents engineering builds of ONTAP (which can expose clusters to undiscovered bugs or disruptions) from being used to upgrade on clusters in the field. This helps minimize the risk and exposure of running unverified builds of ONTAP.
