Why Is the Internet Broken: Greatest Hits

When I started this site back in October of 2014, it was mainly to drive traffic to my NetApp Insight sessions -and it worked.

(By the way… stay tuned for a blog on this year’s new Insight sessions by yours truly. Now with more lab!)

As I continued writing, my goal was to keep creating content – don’t be the guy who just shows up during conference season.

blogfieldofdreams

So far, so good.

But since I create so much content, it gets hard to find for new visitors to this site, The WordPress archives/table of contents is lacking. So, what I’ve done is create my own table of contents of the top 15 most visited posts and the last 5-10 newest. I will keep it up as the main landing page. The list will change on occasion to keep up changing stats.

Newest posts (excluding “Behind the Scenes posts”)

FlexGroups: An evolution of NAS

Setting up BIND to be as insecure as possible in Centos/RHEL7

ONTAP 9 RC1 is now available!

ONTAP 9 Feature: Volume rehosting

Migrating to ONTAP – Ludicrous speed!

 

Top 5 Blogs (by number of visits)

TECH::Using NFS with Docker – Where does it fit in?

TECH:: NetApp is kicking some flash!

TECH::Clustered Data ONTAP 8.3.1 is now in general availability (GA)!

TECH::Data LIF best practices for NAS in cDOT 8.3

TECH::Become a clustered Data ONTAP CLI Ninja

DataCenterDude

I also write for datacenterdude.com on occasion. To read those, go to this link:

My DataCenterDude stuff

How else do I find stuff?

You can also search on the site or click through the archives, if you choose. If you have questions or want to see something changed or added to the site, follow me on Twitter @NFSDudeAbides or comment on one of the posts here!

You can also email me at whyistheinternetbroken@gmail.com.

#NextStopInsight

NetApp Insight 2016 in Las Vegas is finally here. Months of preparation and tons of Powerpoint slides are coming to a close and next week, I’ll be able to deliver on the work I’ve put in. If you’re interested in what sessions I’m doing, I list them in Will I see you at #NetAppInsight 2016?

#NextStopInsight

This year, the hashtag #NextStopInsight is being promoted to track the exploits of the various speakers and attendees making their way there.

I flew out on the 23rd and got a rental car. I lucked out – I got a nice Fiat 500 SUV, with a sunroof!

citylook_misurelato_500x

The drive to St. George was about 2 hours, so I made sure to fuel up on some In N Out burger before I left. Because health.

ctehhgxumaascli

The speed limit between Vegas and St. George is 75 most of the way, which is nice. There’s also a really cool segment of highway that winds through some immense mountains. Makes you feel super small and insignificant. I’m going to try to set up the camera to get some video during the drive on the way back if I can.

Once I arrived, I made my way out to take a short hike in Pioneer Park. Climbed some big ol’ rocks.

ctfazdhueaqfpj1

Tomorrow, I head out to Zion National Park to do some hiking. Hopefully, I don’t end up like this guy:

127hours-48

Check out a photographic account of my travel exploits on my way to Insight:

Photos from Zion

Behind the Scenes: Episode 57 – Scale Out Networking in ONTAP

Welcome to the Episode 57, part of the continuing series called “Behind the Scenes of the NetApp Tech ONTAP Podcast.”

This week on the podcast, we invite Juan Mojica (@juan_m_mojica), Product Manager at NetApp, for a technical discussion about scale out networking in ONTAP. We cover IP Spaces, broadcast domains and subnets, as well as some other tidbits to help you understand how the network stack works in your cluster.

We originally had plans for another podcast on a new feature in ONTAP 9.1, but then we found out we couldn’t publish it until the week of Insight. So…. stay tuned!😉

Finding the Podcast

The podcast is all finished and up for listening. You can find it on iTunes or SoundCloud or by going to techontappodcast.com.

Also, if you don’t like using iTunes or SoundCloud, we just added the podcast to Stitcher.

http://www.stitcher.com/podcast/tech-ontap-podcast?refid=stpr

I also recently got asked how to leverage RSS for the podcast. You can do that here:

http://feeds.soundcloud.com/users/soundcloud:users:164421460/sounds.rss

You can listen here:

Behind the Scenes: Episode 56 – VMworld 2016 Recap

Welcome to the Episode 56, part of the continuing series called “Behind the Scenes of the NetApp Tech ONTAP Podcast.”

group-4-2016

This week, we invite Amy Lewis (@CommsNinja), Josh Atwell (@josh_atwell)and Chris Gebhardt (@chrisgeb) to discuss VMworld 2016 in Vegas and how NetApp fared.

We had a full house in the studio – as well as on the phone – and as such, we got lots of “bonus” content. Be sure to stick around after the outro for a beat box session.

Finding the Podcast

The podcast is all finished and up for listening. You can find it on iTunes or SoundCloud or by going to techontappodcast.com.

Also, if you don’t like using iTunes or SoundCloud, we just added the podcast to Stitcher.

http://www.stitcher.com/podcast/tech-ontap-podcast?refid=stpr

I also recently got asked how to leverage RSS for the podcast. You can do that here:

http://feeds.soundcloud.com/users/soundcloud:users:164421460/sounds.rss

You can listen here:

Behind the Scenes: Episode 55 –Foreign LUN Import

Welcome to the Episode 55, part of the continuing series called “Behind the Scenes of the NetApp Tech ONTAP Podcast.”

ep55.jpg

This week, we talk Foreign LUN Import with the @NTAPFLIGuy himself, NetApp SAN TME Mike Peppers! Find out how you can easily migrate LUNs from competitor storage systems to ONTAP with minimal disruption.

Getting Mike into the studio was pretty easy, even though we asked him at the last minute. That was a side effect of a ton of people being at VMworld that week. Be sure to check out his session at Insight:  54370-2 – AFF Business Processing ONTAP SAN.

Mike was also gracious enough to give us some additional info for a blog on the new interoperability matrix for FLI.

New IMT solution for FLI backend interop is live now!

NetApp is pleased to announce that the New IMT solution for FLI Back-end Interoperability is live now. This new solution offers much greater source array matrix for FLI compare to FlexArray.

Here are some of the highlights of the new IMT solution:

  • Significant Increase of source arrays: Offline FLI Array support increased by 192% and Online FLI by 108%
  • Number of arrays for Offline FLI with new IMT is 228 vs previously supported 78 (192% increase)
  • Number of arrays for Online FLI is 162 vs previously supported 78 (108%)
  • New source arrays include Legacy, All Flash and NetApp controllers
    • Simplified matrix
    • 7 to C Migration
    • Ready for AFF support

To check for a supported configuration for FLI follow the steps below:

  1. Go to http://mysupport.netapp.com/matrix/#search
  2. Search for the array model
  3. Select the solution “Foreign LUN Import (FLI) Back-end Interoperability”
  4. Select the FAS model, ONTAP version to determine the supported configurations
  5. For front end supported host configuration click on build end to end view with “FAS SAN host”
  6. For switch supported configuration click on build end to end view for “SAN-Switch” from “FAS SAN host” tab.

Info can also be found in TR-4380: SAN Migration Using Foreign LUN Import.

Finding the Podcast

The podcast is all finished and up for listening. You can find it on iTunes or SoundCloud or by going to techontappodcast.com.

Also, if you don’t like using iTunes or SoundCloud, we just added the podcast to Stitcher.

http://www.stitcher.com/podcast/tech-ontap-podcast?refid=stpr

I also recently got asked how to leverage RSS for the podcast. You can do that here:

http://feeds.soundcloud.com/users/soundcloud:users:164421460/sounds.rss

You can listen here:

ONTAP 9.0 is now generally available (GA)!

meaning-of-vault-boy-thumbs-up-jpg

Back in June, ONTAP 9RC1 was released. Tons of new features were added, which I covered in ONTAP 9 RC1 is now available! as well as during the Tech ONTAP podcast’s ONTAP 9 month.

Now, ONTAP 9.0 is officially GA. For information on what GA means:

http://mysupport.netapp.com/NOW/products/ontap_releasemodel/

You can find it here:

http://mysupport.netapp.com/NOW/download/software/ontap/9.0/

Also, check out the documentation center:

docs.netapp.com/ontap-9/index.jsp

Happy upgrading!

Will I see you at #NetAppInsight 2016?

netapp-logo-v4-284x96

NetApp Insight 2016 in Las Vegas is approaching quickly (too quickly if you have to present sessions). Last year, I created blogs to highlight the sessions I’ll be presenting, in case any of you are attending Insight and want to pop in.

This year, more of the same!

I’ll be floating around Insight, possibly donning the Flash costume:

flash-netapp

Tech ONTAP

I’ll also be doing some Tech ONTAP podcast stuff. Want to be interviewed? Let me know or email podcast@netapp.com with the subject “Insight 2016.” We’ll roll them into our daily Insight recaps or the post-Insight show.

Last year, we also did some video interviews:

My Sessions

I have 3 sessions this year. You can register for them via the Insight app or Schedule Builder. My sessions are:

60831-2 – How Customers and Partners Use Network File System for Virtualization

I’ll be presenting this one with Dave Morera (@greatwhitetec) of HA Storage in Las Vegas and Mark Carlton (@mcarlton1983) of Arrow in Berlin. Both are members of the NetApp A-Team.

This session will present partner and customer use cases, real-world experience and lessons learned in network file system deployments and implementation for virtualized environments. Learn who prefers network file system over iSCSI for VMware® deployments, what Microsoft® applications are supported on network file system in virtual machine disks. Find out how to make sure your network file system virtual machine deployments are following the best practices from people who implement and architect on a day-to-day basis.

60411-2 – Do More With Less: A FlexGroup Technical Primer

You may or may not have heard of FlexGroups. If so, come to this session to get a deep dive into the inner workings of the new scale-out NAS file system. If you haven’t, check out Sunitha Rao’s FlexGroup Overview session (61054-1) and then come to mine anyway.

NetApp® ONTAP® 9 FlexGroup technology provides a new way to increase scapacity to near infinite levels while increasing performance for concurrent workloads. This technical deep dive will take a look at what is FlexGroup, how they work under the covers, as well as how FlexGroup’s stack up to other scale out NAS solutions on the market today. Come learn about the next generation of scale-out NAS solutions and find out how to do more with less!

60191-3-TT – Authentication Deep Dive: NetApp ONTAP

This is a reprise of last year’s popular session (rated nearly a 5.0!) where we go into a deep dive of ONTAP’s authentication mechanisms for NAS. This is a level 3 session and internal to partners and NetApp employees only.

Authentication in NetApp® ONTAP® uses the security daemon (SecD) for connectivity to external dependencies for NAS protocols such as active directory, LDAP, NIS and DNS. This session will cover how the security daemon works in ONTAP and beyond, as well as how to troubleshoot issues and what kind of statistics are available for the security daemon and external name services.

Also, check out Melissa Wright’s (@vMiss33) session list here!

 

Behind the Scenes: Episode 54 –VVols and SolidFire

Welcome to the Episode 54, part of the continuing series called “Behind the Scenes of the NetApp Tech ONTAP Podcast.”

ep54

This week, we bring in the Storage Janitorhimself, Andy Banta (@andybanta) to do a deep dive into VVols on SolidFire. Andy will also be at VMWorld to answer your VVol questions, so be sure to visit him at the NetApp booth!

Andy likes to work in dirty word phrases to his podcasts, so see if you can find the hidden gem in this one. Hint: He intersperses it in several sections.

We also did a VVols episode with VMware’s VVols guy, Pete Flecha. You may remember him from previous roles, such as “Tech ONTAP podcast host.”

Finding the Podcast

The podcast is all finished and up for listening. You can find it on iTunes or SoundCloud or by going to techontappodcast.com.

Also, if you don’t like using iTunes or SoundCloud, we just added the podcast to Stitcher.

http://www.stitcher.com/podcast/tech-ontap-podcast?refid=stpr

I also recently got asked how to leverage RSS for the podcast. You can do that here:

http://feeds.soundcloud.com/users/soundcloud:users:164421460/sounds.rss

You can listen here:

Behind the Scenes: Episode 53 –Developer Advocacy and Kubernetes

Welcome to the Episode 53, part of the continuing series called “Behind the Scenes of the NetApp Tech ONTAP Podcast.”

ep53

This week, we welcome a very special guest – developer advocate rockstar Kelsey Hightower (@kelseyhightower) from Google/Kubernetes! Kelsey gives us the run down on his views on advocacy vs. evangelism and where Kubernetes fits in with a changing IT landscape. I was able to convince Kelsey to join us by simply asking. Super accessible!

If that’s not enough, we also bring in NetApp SolidFire’s developer advocate, Josh Atwell (@josh_atwell). Josh echoes some of the same feelings as Kelsey, and gives us his Monty Python pitch for Puppet.

Finding the Podcast

The podcast is all finished and up for listening. You can find it on iTunes or SoundCloud or by going to techontappodcast.com.

Also, if you don’t like using iTunes or SoundCloud, we just added the podcast to Stitcher.

http://www.stitcher.com/podcast/tech-ontap-podcast?refid=stpr

I also recently got asked how to leverage RSS for the podcast. You can do that here:

http://feeds.soundcloud.com/users/soundcloud:users:164421460/sounds.rss

You can listen here:

Behind the Scenes: Episode 52 –SolidFire’s Position in NetApp’s Portfolio

Welcome to the Episode 52, part of the continuing series called “Behind the Scenes of the NetApp Tech ONTAP Podcast.”

ep52

This week, we welcomed NetApp/SolidFire’s Business Development Manager Keith Norbie (@keithnorbie) and Product Marketing Manager Kelly Boeckman (@kellyboeckman) to discuss the SolidFire positioning in NetApp’s portfolio, as well as playing a rousing game of “You might be SolidFire if…” (Spoiler: Keith lost)

Keith also made use of the guest beard/neckbeard/mullet.

Finding the Podcast

The podcast is all finished and up for listening. You can find it on iTunes or SoundCloud or by going to techontappodcast.com.

Also, if you don’t like using iTunes or SoundCloud, we just added the podcast to Stitcher.

http://www.stitcher.com/podcast/tech-ontap-podcast?refid=stpr

I also recently got asked how to leverage RSS for the podcast. You can do that here:

http://feeds.soundcloud.com/users/soundcloud:users:164421460/sounds.rss

You can listen here:

What’s the deal with remote I/O in ONTAP?

cropped-jerry-seinfeld-stand-up-comedy-seinfeld1

I’m sure most of you have seen Seinfeld, so be sure to read the title in your head as if Seinfeld is delivering it.

I used a comedian as a starter because this post is about a question that I get asked – a lot – that is kind of a running joke by now.

The set up…

When Clustered Data ONTAP first came out, there was a pretty big kerfuffle (love that word) about the architecture of the OS. After all, wasn’t it just a bunch of 7-Mode systems stitched together with duct tape?

Actually, no.

It’s a complete re-write of the ONTAP operating system, for one. The NAS stack from 7-Mode was gutted and became a new architecture built for clustering.

Then, in 8.1, the SAN concepts in 7-Mode were re-done for clustering.

So, while a clustered Data ONTAP cluster is, at the hardware level, a series of HA pairs stitched together with a 10GB network, the operating system has been turned into essentially what I like to call a storage blade center. Your storage systems span clusters of up to 24 physical hardware nodes, effectively obfuscating the hardware and allowing a single management plane for the entire subsystem.

Every node in a cluster is aware of every other node, as well as every other storage object. If a volume lives on node 1, then node 20 knows about it and where it lives via the concept of a replicated database (RDB).

Additionally, the cluster also has a clustered networking stack, where an IP address or WWPN is presented via a logical interface (a LIF). While SAN LIFs have to stay put and leverage host-side pathing for data locality, NAS LIFs have the ability to migrate across any node and any port in the cluster.

However, volumes are still located on physical disks and owned by physical nodes, even though you can move them around via volume move or vol rehost. LIFs are still located on physical ports and nodes, even though you can move them around and load balance connections on them. This raises the question…

What is the deal with remote I/O in ONTAP?

Since you can have multiple nodes in a cluster and a volume can only exist on one node (well, unless you want to check out FlexGroups), and since data LIFs live on single or aggregated ports on a single node, you are bound to run into scenarios where you end up traversing the backend cluster network for data operations unless you want to take on the headache of ensuring every client mounts to a specific IP address to ensure data locality, or you want to leverage one of the data locality features in NAS, such as pNFS or node referrals on initial connection (available for NFSv4.x and CIFS/SMB). I cover some of the NFS-related data locality features in TR-4067, and CIFS autolocation is covered in TR-4191.

In SAN, we have ALUA to manage that locality (or optimized paths), but even adding an extra layer of protection in the form of protocol locality can’t avoid scenarios where interfaces go down or volumes move around after a TCP connection has been established.

That backend network? Why, it’s a 10GB dedicated network with 2-4 dedicated ports per node. No other data is allowed on the network other than cluster operations. Data I/O traverses the network in a proprietary protocol known as SpinNP, which leverages TCP to guarantee the arrival of packets. And, with the advent of 40GB ethernet and other speedier methods of data transfer, I’d be shocked if we didn’t see that backend network improve over the next 5-10 years. The types of operations that traverse the cluster network include:

  • SpinNP for data/local snapmirror
  • ZAPI calls

That’s pretty much it. It’s a beefy, robust backend network that is *extremely* hard to saturate. You’re more likely to bottleneck somewhere else (like your client) before you overload a cluster network.

So now that we’ve established that remote I/O will likely happen, let’s talk about if that matters…

The punchline

simpson_krusty_il_clown

Remote I/O absolutely adds overhead to operations. There’s no technical way around saying it. Suggesting there is no penalty would be dishonest. The amount of penalty, however, varies, depending on protocol. This is especially true when  you consider that NAS operations will leverage a fast path when you localize data.

But the question wasn’t “is there a penalty?” The question is “does it matter?”

I’ll answer with some anecdotal evidence – I spent 5 years in support, working on escalations for clustered Data ONTAP for 3 of those years. I closed thousands of cases over that time period. In that time, I *never* fixed a performance issue by making sure a customer used a local data path.  And believe me, it wasn’t for lack of effort. I *wanted* remote traffic to be the root cause, because that was the easy answer.

Sure, it could help when dealing with really low latency applications, such as Oracle. But in those cases, you architect the solution with data locality in mind. In the other vast majority of scenarios, the “remote I/O” penalty is pretty much irrelevant and causes more hand wringing than necessary.

The design of clustered Data ONTAP was intended to help storage administrators stop worrying about the layout of the data. Let’s start allowing it to do its job!