ONTAP 9.6RC1 is here!

Spring has barely sprung and already we’re seeing the latest ONTAP release!

ONTAP 9.6 has been announced and it’s chock full of goodness.

For the short overview podcast, check this link out:

For the longer overview, go here:

This time, the prevailing themes are simplicity, security, data protection and storage efficiency. We’ll have a few podcasts centered around some of the new features, as well. They will be available to download in a few weeks, but for now, let’s discuss what’s in it.

What’s new?

Generally speaking, new stuff in ONTAP comes in the following forms:

• New features
• Enhanced features

With the 6 month cadence, features are often phased in, with new features being released with stability as the top priority. Feature parity comes in chunks in later releases. Bug fixes are a part of every ONTAP release.

In ONTAP 9.6, there’s actually a new change that doesn’t really fall into the three categories above. Instead, it’s a departure from how ONTAP handled support for releases.

Long Term vs. Short Term

Around ONTAP 9.0, ONTAP adopted the 6 month cadence for releases. A decision was made to make odd numbered/Fall releases “long term” support releases (LTS) and even numbered/Spring releases “short term” support releases (STS). Short term releases would get a shorter runway of official software support than long term releases.

There was an unintended consequence, however – fewer people wanted to use short term releases, even though the short term releases had the same rigorous quality testing and stability as long term releases. “Short term” had negative connotations, and customers didn’t want to face the prospect of being forced to upgrade sooner, even if that upgrade cycle was years after the release.

So, ONTAP 9.6 scraps the short term release. All new ONTAP releases will be considered long term support releases and will have:

• 3 years full support (technical support, root cause analysis, SW and documentation available online, frequent service updates in year 1)
• 2 years limited support (technical support, root cause analysis, SW and documentation available online, no service updates)
• 3 years of self-service support (documentation available online, no technical support from NetApp, no service updates)

New Features

ONTAP 9.6 focuses on upping the game on simplicity in order to help consumers of ONTAP – new and old – provision storage and move in and out of the cloud much easier.

Most of the stuff in ONTAP 9.6 centers around feature enhancements, but there is one main new feature that I think merits discussion…

REST APIs

I cover REST APIs in another blog post, but essentially, ONTAP is adding support for a standard interface that can query or make changes to an ONTAP cluster via normal HTTP GET and PUT requests.

This allows storage administrators to more easily automate routine tasks without needing to use a proprietary model like ZAPI or ONTAP-specific PowerShell cmdlets. REST APIs also offer a performance improvement over ZAPI calls.

Aggregate level encryption

Another new feature in ONTAP 9.6 centers around the ability to encrypt data at rest at the aggregate level. Previously, you could either encrypt data at rest using drive-enabled hardare encryption on NetApp Storage Encryption (NSE) systems or software-based volume level encryption.

With NSE drives, you can encrypt an entire HA pair, but that means everything on that HA pair is encrypted. That provides less flexibility in management and limits your options with regards to which drives you can use in a system. NSE also means a common key for all volumes on the cluster, and thus no separation for tenants on different SVMs.

NetApp Volume Encryption (NVE) gives more granularity in controlling secure data access, but because each volume has its own encryption key, you lose some simplicity (having to generate a key for every new volume) and storage efficiency (unique keys for each volume means aggregate level deduplication doesn’t work with NVE).

Aggregate level encryption provides simplicity by way of setting an encryption once when the aggregate is created and all volumes in the aggregate sharing the same encryption key, and it allows common blocks across all volumes in the aggregate to be deduplicated.

Feature Enhancements – Simplicity and Productivity

This is where most of the ONTAP 9.6 update does its magic – feature enhancements. One of the prevailing enhancements in ONTAP 9.6 is the focus on simplicity and productivity.

System Manager Enhancements – System Manager is getting a bit of an overhaul. For starters, it’s no longer called OnCommand – instead, it has been re-branded as ONTAP System Manager.

ONTAP System Manager also features:

• Redesigned page views and simpler workflows optimized for use by IT generalists
• Visualizations of network topology and storage hardware
• Intelligent capacity reporting of on-premises and cloud usage at a glance
• Preview “view” version only in ONTAP 9.6 – feedback welcomed, write to: voc@netapp.com

The preview version can be viewed by clicking on “Try the new experience” in the top left corner:

There are also some improvements added with the new System Manager view.

For capacity, you see combined capacity and efficiency views as soon as you open ONTAP System Manager.

For example:

Performance views in System Manager are getting an update as well. Previously, if you wanted to see system performance in System Manager, you only were given a real-time view of performance that did not capture historical data.

In the new view, we store up to a year of perf data in System Manager for historical views that can help narrow down hard-to-pinpoint performance issues in your ONTAP cluster.

ONTAP System Manager has also streamlined how protocols are managed. ONTAP is one of the only storage systems in the industry that can support SAN and NAS protocols, but managing them in previous System Manager iterations required more clicks.

In the new System Manager view, we have a single splash page:

There’s also a network diagram in the dashboard that gives a breakdown of available ports, LIFs and SVMs.

Expect more to come for System Manager enhancements in future ONTAP releases!

Simplicity doesn’t just mean “make System Manager better,” however. There are a number of other feature enhancements that improve ONTAP simplicity, as well as overall productivity.

FabricPool simplicity – Automatic inactive data reporting and new tiering policy

In ONTAP 9.4, a feature enhancement was added where you could have ONTAP analyze your cold data and report back how much space savings you’d get from enabling FabricPool. In ONTAP 9.6, this functionality is now on by default.

Additionally, a new tiering policy has been added called “all,” which will allow storage administrators to easily tier all data from a volume instantly, without having to wait for the cold data policy to kick in. This comes with some usage guidelines:

• Use “all” policy only when importing already cold data
  • Or data that will be read sequentially only
• Don’t use the “all” policy with random read or random read/write workloads
  • Random reads from the cloud tier are slow, and
  • Write updates (overwrites) fragment object storage and increase physical storage consumpition
• User data is never inserted into the performance tier
  • Reads always come from the cloud tier
• Storage efficiency savings result from inline data reduction operations only
  • Automatic background deduplication and background scans are not available
• Backup and DR processes read data from cloud tier

Other FabricPool enhancements include:

• A FabricPool aggregate can store twice as much data
  • 1:20 performance-to-cloud tier ratio limit removed
• Subscription licenses are replacing perpetual license for cloud tiering
  • 1- and 3-year license terms offered
  • Term-based licenses provide more customer flexibility and enable OPEX accounting
  • Perpetual license option will be removed from June quote tool
• SVM-DR support
• Volume move support without needing to re-tier

FlexGroup simplicity – Rename, shrink, elastic sizing

FlexGroup volumes also got some simplicity enhancements. Starting in ONTAP 9.6, you can rename a FlexGroup without having to re-create it and shrink a FlexGroup volume at the FlexGroup level.

Additionally, elastic sizing provides a way for file writes to complete when a member volume fills up by borrowing space from other member volumes without admin intervention.

For more information, see Sneak Peek! Elastic Sizing for FlexGroup Volumes in ONTAP 9.6.

Feature Enhancements – Data Protection and Security

In addition to simplicity and productivity, ONTAP 9.6 also brings some enhancements to data protection and security.

In-flight encryption support

Both SnapMirror (synchronous and asynchronous) and FlexCache now support TLS 1.2 encryption for communication in-flight. This can help prevent man-in-the-middle attacks and protect data contents from being viewed while being transferred.

Per-tenant/SVM key management

Previously in ONTAP, only one key manager could be used for a cluster. This was not ideal for admins who wanted to host multiple tenants on the same cluster, but provide different key management options to each. In ONTAP 9.6, each SVM can have its own unique key management system (off-box only).

SnapMirror Sync – Additional enhancements

SnapMirror Sync was re-introduced in ONTAP 9.5 and supported only NFSv3 and SAN protocols.  In ONTAP 9.6, support for SMB 2/3.x and NFSv4.x has been added.

Also added:

• Quota enforcement
• fpolicy support
• Synchronous to asynchronous SnapMirror cascades (“cascade” is multiple SnapMirrors of the same volume chained together)
  • Synchronous replication from primary to secondary for DR
  • Asynchronous replication from secondary to tertiary for backup/DR

FlexCache Enhancements

FlexCache was also re-introduced in ONTAP 9.5 and offers a way for storage admins to create site-local caches of volumes to accelerate workloads at remote sites or within the same cluster.

In addition to the aforementioned in-flight encryption, the feature was expanded to add:

• Qtree and quota enforcement support
• 100 cache volumes per node
• Cloud Volumes ONTAP support

MetroCluster over IP Enhancements

MetroCluster over IP allows ONTAP clusters to stretch over a distance of 700km via standard ethernet networks. ONTAP 9.6 adds support for entry level platforms like the A220 and FAS2750, as well as support for shared inter-site links.

Advertisements

Behind the Scenes: Episode 189 – ONTAP 9.6 Overview

Welcome to the Episode 189, part of the continuing series called “Behind the Scenes of the NetApp Tech ONTAP Podcast.”

This week on the podcast, we give you the lowdown on the latest ONTAP 9.6 release with ONTAP Systems Group Vice President Octavian Tanase (@octav), Senior Director of Product Management Jeff Baxter (@baxontap), and Technical Product Marketing Manager Skip Shapiro (skip.shapiro@netapp.com)! 

Join us as we talk about how ONTAP 9.6 brings more simplicity, productivity, customer use cases, data protection and security to your datacenter. 

Finding the Podcast

You can find this week’s episode here:

Also, if you don’t like using iTunes or SoundCloud, we just added the podcast to Stitcher.

http://www.stitcher.com/podcast/tech-ontap-podcast?refid=stpr

I also recently got asked how to leverage RSS for the podcast. You can do that here:

http://feeds.soundcloud.com/users/soundcloud:users:164421460/sounds.rss

Our YouTube channel (episodes uploaded sporadically) is here:

Sneak Peek! Elastic Sizing for FlexGroup Volumes in ONTAP 9.6

ONTAP 9.6 is coming soon and I recently posted a sneak peek for REST API support. But REST APIs aren’t the only new feature coming with the release. FlexGroup volumes are getting some new enhancements as well.

These include:

• Ability to rename a FlexGroup volume
• Ability to shrink a FlexGroup volume
• Support for MetroCluster with FlexGroup volumes
• SMB CA share support

One of the bigger features (albeit more under the radar) is a way for ONTAP to help FlexGroup volumes avoid failed writes to volumes due to being out of space – elastic sizing!

Prior to ONTAP 9.6, storage administrators had to be a bit more cognizant of member volume capacity, because if a member volume ran out of space in a FlexGroup volume, the file write would fail. Since files do not stripe across member volumes, a single file could grow over time to cause issues with space allocation.

There are a few reasons a member volume in a FlexGroup might fill up.

• A single file that exceeds the available space of a member volume is attempted to be written. For example, a 10GB file is written to a member volume with just 9GB available.
• A file is appended/written to over time and eventually fills up a member volume. For example, if a database resides in a member volume.
• Snapshots eat into the active file system space available.

FlexGroup volumes do a generally good job at allocating space across member volumes, but if a workload anomaly occurs, it can throw things off. (Like if your volume is mostly a bunch of 4K files but then you zip a lot of them up and create a giant single file).

Remediation of this problem is generally growing volumes or deleting data. But usually, admins won’t notice the issue until it’s too late and “out of space” errors have occurred. That’s where Elastic Sizing comes in handy.

Elastic Sizing – An Airbag for your Data

One of our FlexGroup volume developers refers to elastic sizing as an “airbag” in that it’s not designed to stop you from getting into an accident, but it does help soften the landing when it happens.

In other words, it’s not going to prevent you from writing large files or from running out of space, but it is going to provide a way for those writes to complete.

Here’s how it works…

1. When a file is written to ONTAP, the system has no idea how large that file will become. The client doesn’t know. The application usually doesn’t know. All that’s known is “hey, I want to write a file.”
2. When a FlexGroup volume receives a write request, it will get placed in the best available member based on a variety of factors – such as available capacity, inode count, time since last file creation, member volume performance (new in ONTAP 9.6), etc…
3. When a file is placed, since ONTAP doesn’t know how big a file will get, it also doesn’t know if the file is going to grow to a size that’s larger than the available space. So, the write is allowed as long as we have space to allow it.
4. If/when the member volume runs out of space, right before ONTAP sends an error to the client that we’ve run out of space, it will query the other member volumes in the FlexGroup to see if there’s any available space to borrow. If there is, ONTAP will add 1% of the volume’s total capacity (in a range of 10MB to 10GB) to the volume that is full (while taking the same amount from another member volume in the same FlexGroup volume) and then the file write will continue.
5. During the time ONTAP is looking for space to borrow, that file write is paused – this will appear to the client as a performance issue. But the overall goal isn’t to finish the write fast – it’s to allow the write to finish at all. In most cases, a member volume will be large enough to provide the 10GB increment (1% of 1TB is 10GB), which is often more than enough to allow a file creation to complete. In smaller member volumes, the performance impact could be greater, as the system will need to query to borrow space more often.
6. The capacity borrowing will maintain the overall size of the FlexGroup – for example, if your FlexGroup is 40TB in size, it will remain 40TB.

Once files are deleted/volumes are grown and space is available in that member volume again, ONTAP will re-adjust the member volumes back to their original sizes to maintain an evenness in space.

Ultimately, elastic sizing helps remove the admin overhead of managing space, as well as worrying so much about the initial sizing/deployment of a FlexGroup. You can spend less time thinking about how many member volumes you need, what size they should be, etc.

When you combine elastic sizing in ONTAP 9.6 with features like autogrow/shrink, then ONTAP can pretty much manage your capacity in most cases and help avoid emergency space issues.

Elastic sizing = new FlexGroup use cases?

Traditionally, FlexGroup volume use cases have mainly been for unstructured NAS data, high file count environments, small files, etc. and I’ve cautioned people against putting larger files into FlexGroup volumes because of the aforementioned issues with large files/files that grow potentially filling up a member volume.

But now, with elastic sizing to mitigate those issues, along with volume autogrow/shrink, the FlexGroup use cases get a bit more expanded and interesting.

Why not put a workload with large files/files that grow on a FlexGroup now? In fact, with SMB support for Continuously Available shares for Hyper-V and SQL server, there is further proof that FlexGroup volumes are becoming more viable solutions for a variety of workloads.

You can find the latest podcast for FlexGroup volumes here:

Behind the Scenes: Episode 188 – FlexGroup Volumes Update

Welcome to the Episode 188, part of the continuing series called “Behind the Scenes of the NetApp Tech ONTAP Podcast.”

This week on the podcast, we deliver a long overdue update to Episode 46 of the Tech ONTAP podcast, where we first covered FlexGroup volumes.

We bring back lead developer Richard Jernigan – as well as Technical Director Dan Tennant – to discuss what’s new, what’s changed and what’s coming down the line for FlexGroup volumes.

Finding the Podcast

You can find this week’s episode here:

Also, if you don’t like using iTunes or SoundCloud, we just added the podcast to Stitcher.

http://www.stitcher.com/podcast/tech-ontap-podcast?refid=stpr

I also recently got asked how to leverage RSS for the podcast. You can do that here:

http://feeds.soundcloud.com/users/soundcloud:users:164421460/sounds.rss

Our YouTube channel (episodes uploaded sporadically) is here:

New White Paper! Media and Entertainment Workloads using NetApp ONTAP! #NAB2019

Every year, the National Association of Broadcasters puts on a show to deliver the latest and greatest in media and entertainment content and technology solutions.

This year, I decided to try to piggyback on the show and put out a new white paper about how NetApp ONTAP works with media and entertainment workloads. Included in this whitepaper:

• DreamWorks Animation case study on NetApp ONTAP
• Media/entertainment benchmark numbers on NetApp FlexGroup volumes
• Why you’d want to use NetApp ONTAP

You can find the white paper here:

https://www.netapp.com/us/media/wp-7301.pdf

Leave your feedback in the comments!

Behind the Scenes: Episode 182 – NetApp on NetApp: FlexGroup Volumes and ActiveIQ

Welcome to the Episode 182, part of the continuing series called “Behind the Scenes of the NetApp Tech ONTAP Podcast.”

This week on the podcast, we invite in the guys from Customer One, who operate the NetApp on NetApp program. NetApp on NetApp is a program where we leverage the latest NetApp technologies within our own organizations. Eduardo Rivera (@mredrivera) and Faisal Salaam (https://www.linkedin.com/in/faisal-salam-754a13104/) as we discuss how NetApp is using FlexGroup volumes to power Active IQ. 

Finding the Podcast

You can find this week’s episode here:

Also, if you don’t like using iTunes or SoundCloud, we just added the podcast to Stitcher.

http://www.stitcher.com/podcast/tech-ontap-podcast?refid=stpr

I also recently got asked how to leverage RSS for the podcast. You can do that here:

http://feeds.soundcloud.com/users/soundcloud:users:164421460/sounds.rss

Our YouTube channel (episodes uploaded sporadically) is here:

New ONTAP Release = Updated Technical reports!

ONTAP 9.5 is finally available, which means technical reports are in the process of being updated. For me, that means FlexGroup volumes!

You can find the latest updates to the FlexGroup volume documentation here:

https://www.netapp.com/us/media/tr-4571.pdf

https://www.netapp.com/us/media/tr-4557.pdf

https://www.netapp.com/us/media/tr-4571-a.pdf

https://www.netapp.com/us/media/tr-4678.pdf

Keep an eye out for a new high file count Technical Report as well, currently in progress.

ONTAP 9.5RC1 is available!

There are a few things in life that are certain… death, taxes and a new ONTAP release every 6 months!

ONTAP 9.5 was just officially announced at Insight 2018, and this blog will give you the technical breakdown of all the new goodness. We’ll have a new podcast up soon to cover it as well.

If you want to download and install it, you can find it here:

https://mysupport.netapp.com/products/ontap9/9.5RC1/index.html

Also, check out the following podcast episodes for more information:

• Behind the Scenes: Episode 165 – Accelerate your NAS Data with FlexCache
• Behind the Scenes: Episode 164 – NEW with ONTAP 9.5 : NetApp Data Availability Services for the Hybrid Cloud
• Behind the Scenes: Episode 163 – ONTAP 9.5

I also created a shorter, more digestible feature recap here:

If you’re going to be at Insight (Las Vegas or Barcelona), or if you want to review sessions after the event, you can check out the following session:

1214-2 – What’s On Tap in the Next Major Release of NetApp ONTAP

What’s new?

Generally speaking, new stuff in ONTAP comes in the following forms:

• New features
• Enhanced features
• Bug fixes

With the 6 month cadence, features are often phased in, with new features being released with stability as the top priority. Feature parity comes in chunks in later releases. Bug fixes are a part of every ONTAP release.

So, let’s start with…

New Features

ONTAP 9.5 continues the emphasis on the “modern datacenter” with a slew of new features that help enable higher performance and better resiliency, as well as extending your storage stack beyond on-premises and into a true global architecture.

SnapMirror Synchronous

SnapMirror Synchronous adds the ability to replicate – at a volume level – data cross a WAN connection (RTT <10ms – distance of ~150km) with zero Recovery Point Objective (RPO) and near-zero Recovery Time Objective (RTO). This helps address regulatory and industry mandated needs for synchronous replication.

SnapMirror Synchronous will have two different modes available in the initial release.

Full Synchronous

This is the default mode and guarantees zero application data loss between sites by disallowing writes if the SnapMirror Synchronous replication fails for any reason. This provides the “zero RPO” guarantee.

Relaxed Synchronous

Alternatively, relaxed mode allows application writes to continue to a primary site if the SnapMirror Synchronous relationship fails. Once the relationship is able to resume, resync will automatically occur.

In the initial release of SnapMirror Synchronous, NFSv3, iSCSI and FCP will be supported. Licensing will be capacity-based, in addition to the base SnapMirror license.

FlexCache Volumes

One thing I’ve heard fairly often is “how can I serve NAS data across multiple sites while still honoring locking mechanisms?” Previously, the only way to accomplish this was by way of a 3^rd party NAS lock orchestrator. Now, in ONTAP 9.5, NAS data can be shared across multiple global sites with performance as if the NAS data was local with FlexCache volumes and provides a true global namespace for ONTAP.

FlexCache volumes are sparsely populated volumes that can be cached on the same cluster or a different cluster as the origin volumes to accelerate data access. FlexCache volumes are created on FlexGroup volumes and can cache reads, writes and metadata.  Writes are currently using write-around for locking orchestration at the origin. FlexCache volumes can also help offload mount points to avoid hot spots. Initially, NFSv3 will be the only supported protocol, but future releases will enable more data protocol support.

BGP routing support

The networking stack in ONTAP is getting a bit of a makeover in ONTAP 9.5 as well. Previously, data LIFs in ONTAP were hosted on a single physical port, which lived on a single physical node. Load balancing was done via layer 2 (L2) hashing, which wasn’t super efficient, as hash collisions would leave ports underutilized or even completely unused! When storage nodes have 40GB and 100GB ports, that can be an expensive waste of resources. Additionally, the L2 architecture meant that additional layer 3 (l3) switches needed to be in place to provide proper network traffic distribution.

ONTAP 9.5 introduces support for L3 routing via the Border Gateway Protocol (BGP), which allows ONTAP to automatically load balance traffic based on routing metrics, rather than L2 hashes. Additionally, this allows data LIFs to become Virtual IPs (VIPs) that can live anywhere in the network, which adds better redundancy for IP failover events, and avoids inactive links. This also eliminates the need for L3 switching infrastructure, which reduces overall CapEx and OpEx networking costs.

ONTAP 9.5 is further modernizing the datacenter by modernizing its networking stack.

Logical Space Accounting

ONTAP 9.4 introduced a way to report storage efficiency savings to storage administrators, but to mask those savings to users. For example, if a user is writing to a 10TB volume and 6TB of data has been written to the volume, but storage efficiencies have saved 2TB, then ONTAP can report the actual 6TB of capacity back to users, rather than the 4TB used by way of space savings. This provides storage administrators a way to charge back properly to end users and helps prevent overruns of storage capacity.

ONTAP 9.5 ups the game by integrating logical space accounting into quota enforcement, which not only displays the logical space used, but also prevents new writes once a quota has been reached based on the logical space used.

MAX Data

While this was announced a couple months ago, MAX Data officially makes its debut alongside ONTAP 9.5. This is a server-side software product that lives outside of ONTAP. We covered it on the Tech ONTAP Podcast in Episode 154.

MAX Data offers ultra-low latency (think sub 10 microsecond) and more Ops/second with server-side software-based memory acceleration that leverages persistent memory such as NVDIMM and Optane Memory as they become available. Based on the Plexistor technology that NetApp acquired last year, MAX Data also offers enterprise-class data resiliency with MAX Recovery technology, for high availability and faster data recovery.

MAX Data can help accelerate database applications like Oracle, Cassandra, MongoDB and a variety of other Linux-based applications.

NetApp Data Availability Services (NDAS)

While not technically an ONTAP feature (though there are ONTAP elements such as the NDAS proxy and copy-to-cloud APIs), NetApp Data Availability Services is an integral part of the NetApp Data Fabric. It’s a cloud-resident orchestration app that simplifies hybrid cloud data protection workflows behind a single pane of glass. It’s also an intuitive search catalog for easy file and volume restores and leverages intelligent S3 object storage in AWS for lower cost solutions for backing up your ONTAP data. For more information, see https://www.netapp.com/ndas.

Feature Enhancements

NVMe over FC – Industry’s only HA failover story for NVMeoFC namespaces via asymmetric namespace access (ANA), which is a NVMe standard that NetApp helped develop.

Storage efficiencies – Up to 15% more storage efficiencies seen with compression improvements.

FlexGroup volumes – New functionality such as FabricPool support, quota enforcement and qtree statistics open up a whole new set of workloads that can leverage FlexGroup volumes, such as home directories.

SnapLock – SnapLock adds feature enhancements such as Unified SnapMirror engine support, resync without data loss, clock synchronization in software defined ONTAP and 1,023 snapshot support.

MetroCluster (MCC) – ONTAP 9.5 adds support for SVM-DR and ONTAP Select with MetroCluster, increases the supported distance for MCC IP to 700km(!), and expands the platforms supported for use with MCC IP to the A300 and FAS8200 series.

Where can you find me at #NetAppInsight 2018?

NetApp Insight 2018 in Las Vegas is just a few weeks away and I’m currently in “get ready” mode. Between building sessions, updating docs for the upcoming ONTAP release and putting together podcast promos and episodes, I’ve been a busy little gopher.

For the general NetApp Insight preview, check it out here:

Behind the Scenes: Episode 158 – NetApp Insight 2018 Preview

Speaking of gophers, we’re unveiling a new shirt and sticker design this year, with a variation of the gopher, which I built on gopherize.me!

The week before Insight, I’ll also be doing my annual “get in touch with nature before dealing with Las Vegas” side quest. The last 2 years, I did Zion and covered it here:

• #NextStopInsight
• One down, one to go. #NetAppInsight
• Ready for Insight 2017?
• Behind the Scenes: Episode 108 – #NetAppInsight 2017: Las Vegas Recap (with the #NetAppATeam)

This year, I plan on driving from Sunnyvale through Yosemite, Sequoia and Death Valley over the course of a few days. Road trip!

I’ll be in Las Vegas on Sunday for a bootcamp on ONTAP AI, which I will be picking up on a more regular basis after Insight, which ties in nicely to my core competencies of NAS and FlexGroup volumes!

Insight 2018 – The Main Event

Starting Monday, I’ll be all over the place – booth, customer meetings and sessions. At this point, I only know what my session schedule looks like, so feel free to register for a spot in one of them!

Here are the sessions I’ll be presenting…

1214-2 – What’s On Tap in the Next Major Release of NetApp ONTAP

The next major release of NetApp ONTAP is upon us, and it is chock-full of data management goodness. Come and learn about the latest features in the new release, as well as what features came in the spring release, and how those features are enabling storage administrators to harness the power of their ONTAP systems.

Dates/times offered:

• Monday Oct 22, 10AM PST (currently full – waitlist!)
• Monday Oct 22, 3PM PST – Japanese translated session (119 seats remain!)
• Tuesday Oct 23, 3:15PM PST (23 seats remain!)

1255-2 – FlexGroup: The Foundation of the Next-Generation NetApp Scale-Out NAS

Workloads are growing–both in capacity and performance needs. As data becomes the primary factor driving enterprise businesses, storage administrators need a simple and efficient way to store a lot of it in a single place. This session covers NetApp FlexGroup volumes and how their use expands beyond the high-file-count, large-capacity use cases and into others such as home directories, backup and archive, big data, and media/entertainment. Come learn how FlexGroup, the NetApp scale-out NAS solution, accelerates your enterprise file service performance, simplifies this traditional on-premises workload, and shifts it into the cloud-enabled paradigm.

Dates/times offered:

• Monday Oct 22, 4:30PM PST (47 seats remain!)
• Wednesday Oct 24, 2PM PST (75 seats remain!)

You can also find a bunch of other sessions in the Insight Event Catalog! If you’re a customer, you can also schedule 1:1 meetings through our EBC. Contact your sales rep for details. You can also tweet me @NFSDudeAbides or email me at whyistheinternetbroken@gmail.com if you just want to sync up or get free stickers!

See you at NetApp Insight 2018!

Docker + NFS + FlexGroup volumes = Magic!

A couple of years ago, I wrote up a blog on using NFS with Docker as I was tooling around with containers, in an attempt to wrap my head around them. Then, I never really touched them again and that blog got a bit… stale.

Why stale?

Well, in that blog, I had to create a bunch of kludgy hacks to get NFS to work with Docker, and honestly, it likely wasn’t even the best way to do it, given my lack of overall Docker knowledge. More recently, I wrote up a way to Kerberize NFS mounts in Docker containers that is a little better effort.

Luckily, realizing that I’m not the only one who wants to use Docker but may not know all the ins and outs, NetApp developers created a NetApp plugin to use with Docker that will do all the volume creation, removal, etc for you. Then, you can leverage the Docker volume options to mount via NFS. That plugin is named “Trident.”

Trident + NFS

Trident is an open source storage provisioner and orchestrator for the NetApp portfolio.

You can read more about it here:

https://netapp.io/2016/12/23/introducing-trident-dynamic-persistent-volume-provisioner-kubernetes/

You can also read about how we use it for AI/ML here:

https://www.theregister.co.uk/2018/08/03/netapp_a800_pure_airi_flashblade/

When you’re using the Trident plugin, you can create Docker-ready NFS exported volumes in ONTAP to provide storage to all of your containers just by specifying the -v option during your “docker run” commands.

For example, here’s a NFS exported volume created using the Trident plugin:

# docker volume create -d netapp --name=foo_justin
foo_justin
# docker volume ls
DRIVER VOLUME NAME
netapp:latest foo_justin

Here’s what shows up on the ONTAP system:

::*> vol show -vserver DEMO -volume netappdvp_foo_justin -fields policy
vserver volume               policy
------- -------------------- -------
DEMO    netappdvp_foo_justin default

Then, I can just start up the container using that volume:

# docker run --rm -it -v foo_justin:/foo alpine ash
/ # mount | grep justin
10.x.x.x:/netappdvp_foo_justin on /foo type nfs (rw,relatime,vers=3,rsize=65536,wsize=65536,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.193.67.237,mountvers=3,mountport=635,mountproto=udp,local_lock=none,addr=10.x.x.x)

Having a centralized NFS storage volume for your containers to rely on has a vast number of use cases, providing access for reading and writing to the same location across a network on a high-performing storage system with all sorts of data protection capabilities to ensure high availability and resiliency.

Customization of Volumes

With the Trident plugin, you have the ability to modify the config files to change attributes from the defaults, such as custom names, size, export policies and others. See the full list here:

http://netapp-trident.readthedocs.io/en/latest/docker/install/ndvp_ontap_config.html

Trident + NFS + FlexGroup Volumes

Starting in Trident 18.07, a new Trident NAS driver was added that supports creation of FlexGroup volumes with Docker.

To change the plugin, change the /etc/netappdvp/config.json file to use the FlexGroup driver.

{
"version": 1,
"storageDriverName": "ontap-nas-flexgroup",
"managementLIF": "10.x.x.x",
"dataLIF": "10.x.x.x.",
"svm": "DEMO",
"username": "admin",
"password": "********",
"aggregate": "aggr1_node1",
}

Then, create your FlexGroup volume. That simple!

A word of advice, though. The FlexGroup driver defaults to 1GB and creates 8 member volumes across your aggregates, which creates 128MB member volumes. That’s problematic for a couple reasons:

• FlexGroup volumes should have members that are no less than 100GB in size (as per TR-4571) – small members will affect performance due to member volumes doing more remote allocation than normal
• Files that get written to the FlexGroup will fill up 128MB pretty fast, causing the FlexGroup to appear to be out of space.

You can fix this either by setting the config.json file to use larger sizes, or specifying the size up front in the Docker volume command. I’d recommend using the config file and overriding the defaults.

To set this in the config file, just specify “size” as a variable (full list of options can be found here: https://netapp-trident.readthedocs.io/en/latest/kubernetes/operations/tasks/backends/ontap.html:

{
    "version": 1,
    "storageDriverName": "ontap-nas-flexgroup",
    "managementLIF": "10.0.0.1",
    "dataLIF": "10.0.0.2",
    "svm": "svm_nfs",
    "username": "vsadmin",
    "password": "secret",
    "defaults": {
      "size": "800G",
      "spaceReserve": "volume",
      "exportPolicy": "myk8scluster"
    }}

Since the volumes default to thin provisioned, you shouldn’t worry too much about storage space, unless you think your clients will fill up 800GB. If that’s the case, you can apply quotas to the volumes if needed to limit how much space can be used. (For FlexGroups, quota enforcement will be available in an upcoming release; FlexVols can currently use quota enforcement)

# docker volume create -d netapp --name=foo_justin_fg -o size=1t
foo_justin_fg

And this is what the volume looks like in ONTAP:

::*> vol show -vserver DEMO -volume netappdvp_foo_justin* -fields policy,is-flexgroup,aggr-list,size,space-guarantee 
vserver volume                  aggr-list               size policy  space-guarantee is-flexgroup
------- ----------------------- ----------------------- ---- ------- --------------- ------------
DEMO netappdvp_foo_justin_fg    aggr1_node1,aggr1_node2 1TB  default none            true

Since the FlexGroup is 1TB in size, the member volumes will be 128GB, which fulfills the 100GB minimum. Future releases will enforce this without you having to worry about it.

::*> vol show -vserver DEMO -volume netappdvp_foo_justin_fg_* -fields aggr-list,size -sort-by aggr-list
vserver volume                        aggr-list   size
------- ----------------------------- ----------- -----
DEMO    netappdvp_foo_justin_fg__0001 aggr1_node1 128GB
DEMO    netappdvp_foo_justin_fg__0003 aggr1_node1 128GB
DEMO    netappdvp_foo_justin_fg__0005 aggr1_node1 128GB
DEMO    netappdvp_foo_justin_fg__0007 aggr1_node1 128GB
DEMO    netappdvp_foo_justin_fg__0002 aggr1_node2 128GB
DEMO    netappdvp_foo_justin_fg__0004 aggr1_node2 128GB
DEMO    netappdvp_foo_justin_fg__0006 aggr1_node2 128GB
DEMO    netappdvp_foo_justin_fg__0008 aggr1_node2 128GB
8 entries were displayed.

Practical uses for FlexGroups with containers

It’s cool that we *can* provision FlexGroup volumes with Trident for use with containers, but does that mean we should?

Well, consider this…

In an ONTAP cluster that uses FlexVol volumes for NFS storage presented to containers, I am going to be bound to a single node’s resources, as per the design of a FlexVol. This means that even though I bought a 4 node cluster, I can only use 1 node’s RAM, CPU, network, capacity, etc. If I have a use case where thousands of containers spin up at any given moment and attach themselves to a NFS volume, then I might see some performance bottlenecks due to the increased load. In most cases, that’s fine – but if you could get more out of your storage, wouldn’t you want to do that?

You could add layers of automation into the mix to add more FlexVols to the solution, but then you have new mount points/folders. And what if those containers all need to access the same data?

With a FlexGroup volume that gets presented to those same Docker instances, the containers now can leverage all nodes in the cluster, use a single namespace and simplify the overall automation structure.

The benefits become even more evident when those containers are constantly writing new files to the NFS mount, such as in an Artificial Intelligence/Machine Learning use case. FlexGroups were designed to handle massive amounts of file creations and can provide 2-6x the performance over a FlexVol in use cases where we’re constantly creating new files.

Stay tuned for some more information on how FlexGroups and Trident can bring even more capability to the table to AI/ML workloads. In the meantime, you can learn more about NetApp solutions for AI/ML here:

https://www.netapp.com/us/solutions/applications/ai-deep-learning.aspx