Behind the Scenes: Episode 187 – MAX Data and Intel Optane DCPMM

Welcome to the Episode 187, part of the continuing series called “Behind the Scenes of the NetApp Tech ONTAP Podcast.”

This week on the podcast, we discuss MAX Data, NetApp’s partnership with Intel and how Optane Datacenter Persistent Memory is changing the game for application performance.

ep187-maxdata-graphs

Featured in the podcast this week: 

For more information, see the following:

Finding the Podcast

You can find this week’s episode here:

Also, if you don’t like using iTunes or SoundCloud, we just added the podcast to Stitcher.

http://www.stitcher.com/podcast/tech-ontap-podcast?refid=stpr

I also recently got asked how to leverage RSS for the podcast. You can do that here:

http://feeds.soundcloud.com/users/soundcloud:users:164421460/sounds.rss

Our YouTube channel (episodes uploaded sporadically) is here:

Advertisements

Behind the Scenes: Episode 157 – Performance Analysis Using OnCommand Unified Manager

Welcome to the Episode 157, part of the continuing series called “Behind the Scenes of the NetApp Tech ONTAP Podcast.”

tot-gopher

This week on the podcast, we welcome Mr. Performance himself, Tony Gaddis (gaddis@netapp.com) to give us a tutorial on easily finding performance issues using OnCommand Unified Manager, as well as some common “rules of thumb” when it comes to how much latency and node utilization is too much.

Also, check out Tony’s NetApp Insight 2018 session in Las Vegas and Barcelona:

1181-1 – ONTAP Storage Performance Design Considerations for Emerging Technologies

Podcast listener Mick Landry was kind enough to document the “rules of thumb” that I forgot to add to the blog in the comments. Here they are:

  1. Performance utilization on a node > 85% points to latency issue on the node (broad latency for volumes on the node)
  2. Performance capacity used on a node > 100% points one or more volumes on the node that have latency due to CPU resources running out.
    • This is not an indicator of CPU headroom.
    • 100% is “optimal” – below is wiggle room.
  3. Spinning disk
    • Aggregate performance utilization – not capacity.
    • > 50% relates to disk latency impact will increase.
    • When queueing starts will double or triple latency on slow platters.
    • Performance utilization of the disk drive.
  4. Fragmented free space on spinning disk
    • Increases CP processing time
    • 85% utilization of capacity of aggregate, this will become a problem.
    • > 90% will impact heavy workloads
  5. Node utilization from an HA point of view
    • Keep the sum on the node utilizations less than 100% and will be okay.
    • For “user hours”, on “revenue generating systems”
  6. Disk
    • Spinning disk utilization < 50%
  7. Aggregate latency expectations
    • SATA latency < 12ms
    • SAS latency < 8ms
    • SSD latency < 2ms

Finding the Podcast

You can find this week’s episode here:

Also, if you don’t like using iTunes or SoundCloud, we just added the podcast to Stitcher.

http://www.stitcher.com/podcast/tech-ontap-podcast?refid=stpr

I also recently got asked how to leverage RSS for the podcast. You can do that here:

http://feeds.soundcloud.com/users/soundcloud:users:164421460/sounds.rss

Our YouTube channel (episodes uploaded sporadically) is here:

Behind the Scenes: Episode 154 – Max DATA

Welcome to the Episode 154, part of the continuing series called “Behind the Scenes of the NetApp Tech ONTAP Podcast.”

tot-gopher

This week on the podcast, we talk about the latest and greatest NetApp product, MAXData! Join the MAXData evangelists as they give the rundown on what it is, where you’d use it and how screaming fast it is. Featured in this episode:

Finding the Podcast

You can find this week’s episode here:

Also, if you don’t like using iTunes or SoundCloud, we just added the podcast to Stitcher.

http://www.stitcher.com/podcast/tech-ontap-podcast?refid=stpr

I also recently got asked how to leverage RSS for the podcast. You can do that here:

http://feeds.soundcloud.com/users/soundcloud:users:164421460/sounds.rss

Our YouTube channel (episodes uploaded sporadically) is here:

Behind the Scenes: Episode 149 – Cloud Volume Services Performance with Oracle Databases

Welcome to the Episode 149, part of the continuing series called “Behind the Scenes of the NetApp Tech ONTAP Podcast.”

tot-gopher

This week on the podcast, TME Chad Morgenstern (@sockpupets) joins us to discuss how performance looks in Cloud Volume Services for Oracle Database workloads.

Interested in Cloud Volume Services? You can investigate on your own here:

https://cloud.netapp.com/cloud-volumes

You can also check out Eiki Hrafnsson’s Cloud Field Day presentation on Cloud Volume Services here:

http://techfieldday.com/appearance/netapp-presents-at-cloud-field-day-3/

Finding the Podcast

The podcast is all finished and up for listening. You can find it on iTunes or SoundCloud or by going to techontappodcast.com.

This week’s episode is here:

Also, if you don’t like using iTunes or SoundCloud, we just added the podcast to Stitcher.

http://www.stitcher.com/podcast/tech-ontap-podcast?refid=stpr

I also recently got asked how to leverage RSS for the podcast. You can do that here:

http://feeds.soundcloud.com/users/soundcloud:users:164421460/sounds.rss

Our YouTube channel (episodes uploaded sporadically) is here:

Behind the Scenes: Episode 147 – SPC-1v3 Results – NetApp AFF A800

Welcome to the Episode 147, part of the continuing series called “Behind the Scenes of the NetApp Tech ONTAP Podcast.”

tot-gopher

This week on the podcast, we find out how the new NetApp A800 system fared in the rigorous SPC-1 v3 storage benchmarks. Can the NVMe attached SSDs truly help reduce latency while maintaining high number of IOPs? Performance TME Dan Isaacs (@danisaacs) and the workload engineering team of Scott Lane, Jim Laing and Joe Scott join us to discuss! 

Check out the published results here: 

http://spcresults.org/benchmarks/results/spc1-spc1e#A32007

And the official NetApp blog:

https://blog.netapp.com/nvme-benchmark-spc-1-testing-validates-breakthrough-performance-aff/

Finding the Podcast

The podcast is all finished and up for listening. You can find it on iTunes or SoundCloud or by going to techontappodcast.com.

This week’s episode is here:

Also, if you don’t like using iTunes or SoundCloud, we just added the podcast to Stitcher.

http://www.stitcher.com/podcast/tech-ontap-podcast?refid=stpr

I also recently got asked how to leverage RSS for the podcast. You can do that here:

http://feeds.soundcloud.com/users/soundcloud:users:164421460/sounds.rss

Our YouTube channel (episodes uploaded sporadically) is here:

ONTAP 9.4RC1 is now available!

Hear ye! Hear ye! All ye storage admins! ONTAP 9.4RC1 is announced today!

sully-hearye

That’s right! Every 6 months, without fail, a new ONTAP version with a payload of new features is released.

You can find ONTAP 9.4RC1 here:

http://mysupport.netapp.com/NOW/download/software/ontap/9.4RC1

For info on what a release candidate is, see:

http://mysupport.netapp.com/NOW/products/ontap_releasemodel/

Also, check out the documentation center:

docs.netapp.com/ontap-9/index.jsp

NetApp published a general overview blog on NVMe with Joel Reich here:

https://blog.netapp.com/the-future-is-here-ai-ready-cloud-connected-all-flash-storage-with-nvme/

Stay tuned for a more general ONTAP 9.4 overview blog on the official site. Also, I recorded a brief 5-minute teaser/trailer for ONTAP 9.4 features and podcasts coming soon. Find that here:

Also a new lightboard video! Watch me write… BACKWARDS???

This blog is intended to go a little deeper into the main features available in ONTAP 9.4. We’ll break them down as follows:

  • Cloud
  • Performance
  • Efficiency
  • Security
  • General ONTAP Goodness

Without further ado…

Cloud!

FabricPools were introduced in ONTAP 9.2 as a way to tier blocks from your performance tier solution to a capacity tier, such as cloud or StorageGrid.

We covered FabricPools in detail in episode 92 of the Tech ONTAP Podcast, which you can find here:

In ONTAP 9.4, the first major updates to the feature have been released! FabricPools in ONTAP 9.4 bring the following…

Tiering cold data from the active file system

Prior to ONTAP 9.4, FabricPools only tiered cold data from snapshots on primary systems and data protection volumes on secondary systems. This allowed ONTAP to free up valuable real estate on flash systems for data actively being used. In ONTAP 9.4, inactive blocks can now be tiered off to cloud or StorageGrid from the active file system. ONTAP does this automatically by way of a new “auto” tiering policy, which has a configurable cooling period of 2-63 days (-tiering-minimum-cooling-days option in CLI). This cooling period determines how long ONTAP will wait before tiering off data considered “cool” by the policy to the FabricPool tiering destination. The tiering destination choices used to be only Amazon S3 and StorageGrid, but ONTAP 9.4 brings us…

Tiering to Azure Blob Storage

Support for Azure Blob storage was added to ONTAP 9.4 for FabricPools, which gives storage administrators more options for cloud providers. In addition, other cloud providers (such as Google Cloud, IBM Cloud Object Storage, etc) can be added via product variance requests (PVR) to your NetApp Sales reps. Keep in mind that only one cloud provider per FabricPool aggregate can be used.

fabricpools-afs

But how do you know if FabricPools will be of any value to you?

Inactive Data Reporting

Inactive Data Reporting is new in ONTAP 9.4 and can offer insight from OnCommand System Manager into whether there’s enough inactive data in your system for FabricPools to make a difference.

fabricpools-inactive-report.png

By default, this feature is enabled for aggregates participating in FabricPools, but you can also enable it via the CLI for non-FabricPool aggregates to predict space savings with the following command:

storage aggregate modify -aggregate <name> -is-inactive-data-reporting-enabled true

You can also test the performance of your FabricPool target with…

Object Store Profiler

Also new in ONTAP 9.4, the Object Store Profiler provides a way to evaluate the performance (via throughput and latency) to your desired FabricPool target. From the CLI, start the profiler using:

storage aggregate object-store profiler start -object-store-name <name> -node <name>

Then show the results with:

storage aggregate object-store profiler show

This gives a general idea of how FabricPools will work for you before you implement them.

object-profiler

But that’s not the only object store enhancements. FabricPools in ONTAP 9.4 also offers….

Better efficiency for object storage

Prior to ONTAP 9.4, there was really no concept of freeing up space on the object store once the data blocks that had been tiered off were deleted on the source. ONTAP would see the free space, but the capacity tier would not. ONTAP 9.4 offers object defragmentation for the FabricPool destination to free up deleted blocks on the destination. This is done without any admin interaction at a specific % of free space by default for different providers. The default settings are:

  • 15%Microsoft Azure Blob Storage
  • 20% Amazon S3
  • 40% StorageGRID Webscale

These percentages are adjustable via the CLI with the following command in advanced privilege:

storage aggregate object-store modify –aggregate <name> -object-store-name <name> –unreclaimed-space-threshold <%> (0%-99%)

ONTAP 9.4 also brings support for the data compaction functionality to FabricPool aggregates to provide even more storage efficiency. For more information on data compaction, see TR-4476.

What’s great about ONTAP 9.4 is that FabricPool can now be used on any ONTAP deployment (other than MCC) with…

Support for ONTAP Select and ONTAP Cloud

FabricPools can now tier from a cloud instance to a cloud tier. This is especially useful now that we have NetApp Cloud Volumes, which run on a performance tier.

Additionally, you can use FabricPools on all versions of ONTAP Select, whether standard or Premium. This means you can tier from ONTAP Select, even if it has spinning media running under the covers. This support for spinning media does not extend into FAS systems, however – just ONTAP Select. The concern there is performance; FabricPools won’t perform well on FAS systems with spinning media.

So that’s all for the FabricPool section. Now let’s talk…

Performance!

ONTAP 9.4’s biggest news is the introduction of support for NVMe over fibre channel, as well as the NVMe attached SSDs in the new AFF A800 platform. This gives NetApp the industry’s first end-to-end NVMe platform. If you’re interested in a deep dive into what NVMe is, this podcast covered it:

Early testing numbers on the new platform show sub-200 micro-second latencies, with 1.3 million IOPS per HA pair at sub-500 micro-second latencies and 34GB/s throughput. It’s a pretty beastly system.

NVMe is integral to implementaion of workloads such as machine learning and AI, which powers tech like self-driving cars, IoT devices and other budding tech.

nvme-ai.png

If you’re a NetApp employee or partner, check out the recording of the Solutions Insight Webcast from May 9 that covers NVMe in more detail.

Another performance enhancement in ONTAP 9.4 is SMB multichannel, which provides a way for SMB3 connections to leverage more TCP streams and CPU cores on the ONTAP system to increase throughput. This especially benefits SQL server workloads.

smb-multichannel.png

The new platform and ONTAP 9.4 update doesn’t just add performance, however. It also adds…

More efficiency!

The new AFF A800 platform chassis offers efficiency in the form of both power/cooling and rack space savings with >2.5PB of storage (based on a 4:72 storage efficiency ratio) in a 4U footprint. Later, when the platform supports larger NVMe attached drives, we’ll see even more density. ONTAP 9.4 also brings support for 30TB SAS attached SSDs.

But ONTAP 9.4 also brings some additional efficiencies, such as…

Snapshot block sharing

snapshot-block-share

 

Prior to ONTAP 9.4, deduplication did not take blocks locked in a snapshot under consideration for storage efficiencies. In ONTAP 9.4, if a file is locked in a snapshot *and* it exists in the active file system, deduplication will reduce the blocks needed for the file in the active file system to save even more space. ONTAP 9.4 is also adding support for up to 1,023 snapshots per FlexVol.

Background Aggregate Level Deduplication

background-aggr-dedupe

Deduplication at the aggregate level was added in ONTAP 9.2 and provides storage efficiencies when identical blocks exist across volumes in the same aggregate. This was all done inline. In ONTAP 9.4, you can now deduplicate at the aggregate level on data that’s already been placed.

Automatic Efficiency Enablement on Data Protection Volumes

auto-dedupe-schedule.png

ONTAP 9.4 also automatically enables all storage efficiencies on data protection volumes to help simplify the role of storage administrators and save space on secondary systems.

Decreased Node Root Aggregate Sizes

Every node in an ONTAP cluster has a node root aggregate, which hosts a node root volume. The node root volume holds logs, system critical files and any core files that might get generated in the event of a crash. The core file size is based on the size of system memory. As platforms add memory to systems, these core files get larger, which was causing the core files to increase, which made root volume sizes increase… wait. This is getting confusing. Here’s a diagram:

root-vol-size-equation

Advanced Disk Partitioning (or root-data partitioning) helped save some space by spreading the volume across disk partitions, but we took steps to save even more space. For example, the 1TB root aggregate that would have been needed on the A800 node gets reduced down to just 150GB!

Long story short – ONTAP 9.4 with newer systems moved the ever-increasing core files from disk media to the local flash boot storage. This applies only to newer systems (such as the A800, FAS2700 and beyond) that have large enough boot devices to hold 2 core files and cannot be retroactively applied to older systems.

ONTAP 9.4 is also bringing…

More Security!

One of the areas of ONTAP that I feel has seen some of the most significant enhancements over the past several years  has been security (credit to Juan Mojica for making it happen).

Starting with the onboard key manager, which grew into NetApp Volume Encryption and evolved into off-box key manager support and multi-factor authentication, security has grown leaps and bounds in ONTAP. This is necessary in today’s hyper-focused security minded IT organizations, as hacks, breaches and ransomware attacks are all very fresh in their minds.

ONTAP 9.4 is bringing several more security features that don’t just help guard against external threats, but also help cover internal threats (or user mistakes) from hurting a business’s bottom line.

First of all, admins can upgrade to…

Validated ONTAP Images!

validated-ontap ONTAP is now a validated image, which gives administrators peace of mind that they’re not accidentally installing some hacked version of ONTAP that can compromise their systems. In addition, it prevents engineering builds of ONTAP (which can expose clusters to undiscovered bugs or disruptions) from being used to upgrade on clusters in the field. This helps minimize the risk and exposure of running unverified builds of ONTAP.

But we’re not just protecting against upgrading to unverified installations. ONTAP 9.4 also provides…

Key-based boot technology

secure-boot

Onboard Key Manager can be leveraged to prevent reboots without a passphrase. This protects against nefarious attempts to change the admin password on a system (which can be done with console/service processor access to the boot menu of a node), as well as against physical theft of systems. In addition to the onboard key manager, you can also enable protected boot with a USB key – but you’d need a product variance request (PVR). Check with your NetApp sales rep for details. Next generation platform (yet to be released) will also provide the ability to use UEFI Secure Boot, which works in conjunction with validated ONTAP images to not only prevent upgrades to unverified ONTAP images, but from running them at all.

These provide security against external and internal threats alike, but what do you do when someone accidentally writes a classified document to a public, unclassified share

Securely purge it!

secure-purge

ONTAP 9.4 provides the ability to cryptographically shred individual files from the drive while the system remains online, and the rest of the files remain intact. This can be helpful for data spillage – e.g. when a classified document ends up in an unclassified location. This is also particularly timely and useful for the upcoming GDPR regulations’ “Right to Erasure” rules.

Security is playing a big part in the new release of ONTAP. In addition, here’s some more…

General ONTAP goodness

ONTAP 9.4 also brings several other valuable features, such as:

  • Rapid disk zeroing technology – initialize disks near-instantaneously in newer platforms!
  • 3-step, 1-click ONTAP upgrades – even easier to update your cluster non-disruptively
  • Install ONTAP without needing a separate web or FTP server
  • SQL Server support for Application Data Management in System Manager

So, there you are! A thorough rundown of the new features in ONTAP 9.4. If you feel I missed something, feel free to reach out in the comments with input!

Check out these brief videos for some lightboard action on new ONTAP 9.4 stuff:

Some other information on the launch can be found as follows:

GCP Cloud Volumes for NFS with native access to the GCP tool suite (Google Cloud)
https://blog.netapp.com/sweet-new-storage-service-from-netapp-for-google-cloud-platform/ 

Storage Grid Update 11.1
https://blog.netapp.com/storagegrid-11-1-and-netapp-hci-the-perfect-one-two-punch-for-scaling-your-environment/ 

A800 and the A220
https://blog.netapp.com/the-future-is-here-ai-ready-cloud-connected-all-flash-storage-with-nvme/ 

ONTAP 9.4 with first to market NVMe/FC support
http://www.demartek.com/Demartek_NetApp_Broadcom_NVMe_over_Fibre_Channel_Evaluation_2018-05.html

ONTAP 9.3 is now GA!

ONTAP 9 is on a new cadence model, which brings a new release every 6 months.

Today, ONTAP 9.3GA is available here!

http://mysupport.netapp.com/NOW/download/software/ontap/9.3

ONTAP 9.3 was announced at NetApp Insight 2017 in Las Vegas and was covered at a high level by Jeff Baxter in the following blog:

Announcing NetApp ONTAP 9.3: The Next Step in Modernizing Your Data Management

Jeff has a follow-up infographic here:

https://blog.netapp.com/10-good-reasons-to-upgrade-to-ontap-9-3-infographic/

I also did a brief video summary here:

We also did a podcast with ONTAP Chief Evangelist Jeff Baxter (@baxontap) and ONTAP SVP Octavian Tanase (@octav) here:

For info on what GA means, see:

http://mysupport.netapp.com/NOW/products/ontap_releasemodel/

Also, check out the documentation center:

docs.netapp.com/ontap-9/index.jsp

The general theme around ONTAP 9.3 is modernization of the data center. Here’s a high level list of features, with more detail on some of them later in this blog.

Security enhancements

Simplicity innovations

  • MongoDB support added to application provisioning
  • Simplified data protection flows in System Manager
  • Guided cluster setup and expansion
  • Adaptive QoS

Performance and efficiency improvements

  • Up to 30% performance improvement for specific workloads via WAFL improvements, parallelization and flash optimizations
  • Automatic schedules for deduplication
  • Backgroup inline aggregate deduplication (AFF only; automatic schedule only)

NetApp FlexGroup volume features

This is covered in more detail in What’s New for NetApp FlexGroup Volumes in ONTAP 9.3?

  • Qtrees
  • Antivirus
  • Volume autogrow
  • SnapVault/Unified SnapMirror
  • SMB Change/notify
  • QoS Maximums
  • Improved automated load balancing logic

Data Fabric additions

  • SolidFire to ONTAP SnapMirror
  • MetroCluster over IP

Now, let’s look at a few of the features in a bit more detail. If you have things you want covered more, leave a comment.

Multifactor Authentication (MFA)

Traditionally, to log in to an ONTAP system as an admin, all you needed was a username and password and you’d get root-level access to all storage virtual machines in a cluster. If you’re the benevolent storage admin, that’s great! If you’re a hostile actor, great!* (*unless you’re the benevolent storage admin… then, not so great)

ONTAP 9.3 introduces the ability to configure an external Identity Provider (IdP) server to interact with OnCommand System Manager and Unified Manager to require a key to be passed in addition to a username and password. Initial support for IdP will include Microsoft Active Directory Federation Services and Shibboleth.

MFA

For the command line, the multifactor portion would be passed by way of SSH keys currently. We cover MFA in the following Tech ONTAP podcast:

SnapLock Enhancements

SnapLock is a NetApp ONTAP feature that provides data compliance for businesses that need to preserve data for regulatory reasons, such as HIPAA standards (SnapLock compliance) or for internal requirements, such as needing to preserve records (SnapLock enterprise).

ONTAP 9.3 provides a few enhancements to SnapLock, including one that isn’t available from any storage provider currently.

legal-hold.png

Legal hold is useful in the event that a court has ordered specific documents to be preserved for an ongoing case or investigation. This can be applied to multiple files and remains in effect until you choose to remove it.

event-based

Event-based retention allows storage administrators to set protections on data based on defined events, such as an employee leaving the company (to avoid disgruntled deletions), or for insurance use cases (such as death of a policy holder).

vol-append.png

Volume append mode is the SnapLock feature I alluded to, where no one else can currently accomplish this. Essentially, it’s for media workloads (audio and video) and will write-protect the portion of the files that have already been streamed and allow appending to those files after they’ve been protected. It’s kind of like having a CD-R on  your storage system.

Performance improvements

improve-perf

Every release of ONTAP strives to improve performance in some way. ONTAP 9.3 introduces performance enhancements (mostly for SAN)/block via the following changes:

  • Read latency reductions via WAFL optimizations for All Flash FAS SAN (block) systems
  • Better parallelization for all workloads on mid-range and high-end systems (FAS and AFF) to deliver more throughput/IOPS at lower latencies
  • Parallelization of the iSCSI layer to allow iSCSI to use more cores (best results on 20 core or higher systems)

The following graphs show some examples of that performance improvement versus ONTAP 9.2.

a700-fcp

a700-iscsi

Adaptive Quality of Service (QoS)

Adaptive QoS is a way for storage administrators to allow ONTAP to manage the number of IOPS per TB of volume space without the need to intervene. You simply set a service level class and let ONTAP control the rest.

The graphic below shows how it works.

adaptive-qos

We cover QoS minimums and performance enhancements in the following Tech ONTAP podcast:

MetroCluster over IP

MetroCluster is a way for clusters to operate in a high availability manner over long distances. (hundreds of kilometers) Traditionally, MetroCluster has been done over fiber channel networks due to low latency requirements needed to guarantee writes can be committed to both sites.

However, now that IP networks are getting more robust, ONTAP is able to support MetroCluster over IP, which provides the following benefits:

  • Reduced CapEx and OpEx (no more dedicated fiber channel networks, cards, bridges)
  • Simplicty of management (use existing IP networks)

mcc-ip.png

The ONTAP 9.3 release is going to be a limited release for this feature, with the following caveats:

  • A700, FAS9000 only
  • 100km limit
  • Dedicated ISL with extended VLAN currently required
  • 1 iWARP card per node

We cover MetroCluster over IP in this podcast:

SolidFire to ONTAP SnapMirror

A few years back, the concept of a data fabric (where all of your data can be moved anywhere with the click of a button) was introduced.

That vision continued this year with the inclusion of SnapMirror from SolidFire (and NetApp HCI systems) to ONTAP.

sf-snapmirror.png

ONTAP 9.3 will allow storage administrators to implement a disaster recovery plan for their SolidFire systems.

This includes the following:

  • Baseline and incremental replication using NetApp SnapMirror from SolidFire to ONTAP
  • Failover storage to ONTAP for disaster recovery
  • Failback storage from ONTAP to SolidFire
    • Only for LUNs replicated from SolidFire
    • Replication from ONTAP to SolidFire only for failback

That covers a deeper look at some of the new ONTAP 9.3 features. Feel free to comment if you want to learn more about these features, or any not listed in the overview.

Behind the Scenes: Episode 117 – Storage QoS in ONTAP 9.3

Welcome to the Episode 117, part of the continuing series called “Behind the Scenes of the NetApp Tech ONTAP Podcast.”

tot-gopher

This week on the podcast, we invited the NTAPFLIGuy, Mike Peppers, to talk about QoS and performance in ONTAP 9.3. Listen for a general overview of QoS maximums and minimums, as well as the new Adaptive QoS feature!

Finding the Podcast

The podcast is all finished and up for listening. You can find it on iTunes or SoundCloud or by going to techontappodcast.com.

This week’s episode is here:

Also, if you don’t like using iTunes or SoundCloud, we just added the podcast to Stitcher.

http://www.stitcher.com/podcast/tech-ontap-podcast?refid=stpr

I also recently got asked how to leverage RSS for the podcast. You can do that here:

http://feeds.soundcloud.com/users/soundcloud:users:164421460/sounds.rss

Our YouTube channel (episodes uploaded sporadically) is here:

ONTAP 9.3RC1 is now available!

ONTAP 9.3 was announced at NetApp Insight 2017 in Las Vegas and was covered at a high level by Jeff Baxter in the following blog:

Announcing NetApp ONTAP 9.3: The Next Step in Modernizing Your Data Management

I also did a brief video summary here:

We also did a podcast with ONTAP Chief Evangelist Jeff Baxter (@baxontap) and ONTAP SVP Octavian Tanase (@octav) here:

ONTAP releases are delivered every 6 months, with the odd numbered releases landing around time for Insight. Now, the first release candidate for 9.3 is available here:

http://mysupport.netapp.com/NOW/download/software/ontap/9.3RC1

For info on what a release candidate is, see:

http://mysupport.netapp.com/NOW/products/ontap_releasemodel/

Also, check out the documentation center:

docs.netapp.com/ontap-9/index.jsp

The general theme around ONTAP 9.3 is modernization of the data center. I cover this at Insight in session 30682-2, which is available as a recording from Las Vegas for those with a login. If you’re going to Insight in Berlin, feel free to add it to your schedule builder. Here’s a high level list of features, with more detail on some of them later in this blog.

Security enhancements

Simplicity innovations

  • MongoDB support added to application provisioning
  • Simplified data protection flows in System Manager
  • Guided cluster setup and expansion
  • Adaptive QoS

Performance and efficiency improvements

  • Up to 30% performance improvement for specific workloads via WAFL improvements, parallelization and flash optimizations
  • Automatic schedules for deduplication
  • Backgroup inline aggregate deduplication (AFF only; automatic schedule only)

NetApp FlexGroup volume features

This is covered in more detail in What’s New for NetApp FlexGroup Volumes in ONTAP 9.3?

  • Qtrees
  • Antivirus
  • Volume autogrow
  • SnapVault/Unified SnapMirror
  • SMB Change/notify
  • QoS Maximums
  • Improved automated load balancing logic

Data Fabric additions

  • SolidFire to ONTAP SnapMirror
  • MetroCluster over IP

Now, let’s look at a few of the features in a bit more detail. If you have things you want covered more, leave a comment.

Multifactor Authentication (MFA)

Traditionally, to log in to an ONTAP system as an admin, all you needed was a username and password and you’d get root-level access to all storage virtual machines in a cluster. If you’re the benevolent storage admin, that’s great! If you’re a hostile actor, great!* (*unless you’re the benevolent storage admin… then, not so great)

ONTAP 9.3 introduces the ability to configure an external Identity Provider (IdP) server to interact with OnCommand System Manager and Unified Manager to require a key to be passed in addition to a username and password. Initial support for IdP will include Microsoft Active Directory Federation Services and Shibboleth.

MFA

For the command line, the multifactor portion would be passed by way of SSH keys currently.

SnapLock Enhancements

SnapLock is a NetApp ONTAP feature that provides data compliance for businesses that need to preserve data for regulatory reasons, such as HIPAA standards (SnapLock compliance) or for internal requirements, such as needing to preserve records (SnapLock enterprise).

ONTAP 9.3 provides a few enhancements to SnapLock, including one that isn’t available from any storage provider currently.

legal-hold.png

Legal hold is useful in the event that a court has ordered specific documents to be preserved for an ongoing case or investigation. This can be applied to multiple files and remains in effect until you choose to remove it.

event-based

Event-based retention allows storage administrators to set protections on data based on defined events, such as an employee leaving the company (to avoid disgruntled deletions), or for insurance use cases (such as death of a policy holder).

vol-append.png

Volume append mode is the SnapLock feature I alluded to, where no one else can currently accomplish this. Essentially, it’s for media workloads (audio and video) and will write-protect the portion of the files that have already been streamed and allow appending to those files after they’ve been protected. It’s kind of like having a CD-R on  your storage system.

Performance improvements

improve-perf

Every release of ONTAP strives to improve performance in some way. ONTAP 9.3 introduces performance enhancements (mostly for SAN)/block via the following changes:

  • Read latency reductions via WAFL optimizations for All Flash FAS SAN (block) systems
  • Better parallelization for all workloads on mid-range and high-end systems (FAS and AFF) to deliver more throughput/IOPS at lower latencies
  • Parallelization of the iSCSI layer to allow iSCSI to use more cores (best results on 20 core or higher systems)

The following graphs show some examples of that performance improvement versus ONTAP 9.2.

a700-fcp

a700-iscsi

Adaptive Quality of Service (QoS)

Adaptive QoS is a way for storage administrators to allow ONTAP to manage the number of IOPS per TB of volume space without the need to intervene. You simply set a service level class and let ONTAP control the rest.

The graphic below shows how it works.

adaptive-qos

MetroCluster over IP

MetroCluster is a way for clusters to operate in a high availability manner over long distances. (hundreds of kilometers) Traditionally, MetroCluster has been done over fiber channel networks due to low latency requirements needed to guarantee writes can be committed to both sites.

However, now that IP networks are getting more robust, ONTAP is able to support MetroCluster over IP, which provides the following benefits:

  • Reduced CapEx and OpEx (no more dedicated fiber channel networks, cards, bridges)
  • Simplicty of management (use existing IP networks)

mcc-ip.png

The ONTAP 9.3 release is going to be a limited release for this feature, with the following caveats:

  • A700, FAS9000 only
  • 100km limit
  • Dedicated ISL with extended VLAN currently required
  • 1 iWARP card per node

SolidFire to ONTAP SnapMirror

A few years back, the concept of a data fabric (where all of your data can be moved anywhere with the click of a button) was introduced.

That vision continued this year with the inclusion of SnapMirror from SolidFire (and NetApp HCI systems) to ONTAP.

sf-snapmirror.png

ONTAP 9.3 will allow storage administrators to implement a disaster recovery plan for their SolidFire systems.

This includes the following:

  • Baseline and incremental replication using NetApp SnapMirror from SolidFire to ONTAP
  • Failover storage to ONTAP for disaster recovery
  • Failback storage from ONTAP to SolidFire
    • Only for LUNs replicated from SolidFire
    • Replication from ONTAP to SolidFire only for failback

That covers a deeper look at some of the new ONTAP 9.3 features. Feel free to comment if you want to learn more about these features, or any not listed in the overview.

Behind the Scenes: Episode 97 – ONTAP Analytics and Telemetry Service (OATS)

Welcome to the Episode 97, part of the continuing series called “Behind the Scenes of the NetApp Tech ONTAP Podcast.”

group-4-2016

This week on the podcast, we’re talking ONTAP Analytics in the cloud and the new OATS product with a crew of performance folks including Greg Keller (@kellergc, Sr. Director, Performance Engineering), Matt Hambrick (Director of Performance Engineering) and Fred Peiffer (Performance Characterization Engineer/Architect). Find out how OATS is using existing performance data and machine learning to teach a system how to analyze and resolve performance issues, as well as how you can use OATS for your environment.

You can find OATS in the AWS Marketplace at: https://aws.amazon.com/marketplace/pp/B072JRLP87?qid=1497912376310&sr=0-10&ref_=srh_res_product_title

For an example of how to use OATS, see the following ONTAP recipe: https://community.netapp.com/t5/Data-ONTAP-Discussions/ONTAP-Recipes-Correlate-EMS-Messages-and-Performance-Metrics-with-NetApp-OATS/td-p/132184

For a short video, see:

To contact the team, email ng-oats-info@netapp.com.

Finding the Podcast

The podcast is all finished and up for listening. You can find it on iTunes or SoundCloud or by going to techontappodcast.com.

Also, if you don’t like using iTunes or SoundCloud, we just added the podcast to Stitcher.

http://www.stitcher.com/podcast/tech-ontap-podcast?refid=stpr

I also recently got asked how to leverage RSS for the podcast. You can do that here:

http://feeds.soundcloud.com/users/soundcloud:users:164421460/sounds.rss

You can listen here: