TECH::What Super Bowl 49 Taught Us About Transitioning from NFSv3 to NFSv4.x

I’m sure most everyone either watched Super Bowl 49 or at least read about it somewhere. The Seattle Seahawks and New England Patriots clashed down to the wire for one of the most thrilling games in recent history. Both sides played fairly well, but not without mistakes. And as most of us know, it all came down to a final decision – do we run or pass from the 1 yard line?

The Seahawks chose to pass and the rest was history.

So what lessons can storage administrators learn about NFS from football (or as some call it, sportball)? In this case, we’ll call NFSv3 “the run” and NFSv4.x? The forward pass.

The forward pass: A little history

American football was not always the game it is today. The sport evolved from rugby and the first American football game was played in 1869. Like rugby, it was a run-heavy, violent game. Keep in mind that these used to be the helmets, and helmets weren’t even worn in the early games:

Yeah… that’s not gonna hurt, right? Source: AntiqueAthlete.com

In 1906, a rule change was added to try to help make the game less… bloody.

Mutant League Football – Source: Game Informer

As a result, the game opened up and became more exciting with higher scores and it added a facet of strategy to the game. Now it wasn’t just about simple brute force – you had to try to guess what the other team was going to do based on the situation.

NFS: Not Football Specific

NFS actually stands for Network File System and is used in enterprise environments across the world for file storage. Everything from music to movies to medical images to seismic data to virtual machines are hosted on NFS storage (preferably NetApp :-P). There are plenty of resources on what it is out there, starting with the Request For Comments (RFC) standards. As far as NFS on NetApp, I cover it in some detail in the following technical reports (Light reading, especially if you’re having trouble sleeping.):

TR-3580: NFSv4 Enhancements and Best Practices Guide: Data ONTAP Implementation

TR-4063: Parallel Network File System Configuration and Best Practices for Clustered Data ONTAP 8.2 and Later

TR-4067: Clustered Data ONTAP NFS Best Practice and Implementation Guide

TR-4073: Secure Unified Authentication

TR-4379: Name Services Best Practices

But NFS, and NAS storage administration, has some things in common with football – you have to decide on what to do based on every situation.

NFSv4.x and the forward pass

If we look at the decision made last night not to run  the ball on 2nd and goal from the 1 yard line with a player nicknamed “Beast Mode” (how do you NOT run it??) in terms of transitioning from the generally rock-solid NFSv3 to NFSv4.x, we can make the following analogies.

  • Do I give the ball to my best player (NFSv3) that has been dependable for years and see what happens?
    I know it could backfire on me, and he does have his flaws…
  • Do I throw a forward pass (NFSv4.x) without really thinking about the immediate consequences of just jumping into that decision?
    I could misconfigure the pass and that would be it for my enterprise data storage…
  • Do I call a timeout and re-think the play design?
    We are running out of time. We need to make the best decision for our business!

Planning the transition to NFSv4.x

Like any good coach or storage administrator, we need to come into every game prepared. We need to think out all possible scenarios and plan for them accordingly, but also be flexible enough to adjust on the fly if unforeseen circumstances occur. After all, unforeseen circumstances are, by definition, unforeseen.

In TR-4067, I cover some considerations that need to be made when deciding to move from NFSv3 to NFSv4.x. The following is a sneak preview. Keep in mind that this section may change in the final release of the TR, but the general concepts remain.

Transitioning from NFSv3 to NFSv4.x: Considerations

The following section covers some considerations that need to be addressed when migrating from NFSv3 to NFSv4.x. When choosing to use NFSv4.x after using NFSv3, you cannot simply turn it on and have it work as expected. There are specific items to address, such as:

  • Domain strings/ID mapping
  • Storage failover considerations
  • Name services
  • Firewall considerations
  • Export policy rule considerations
  • Client support
  • NFSv4.x features and functionality

ID Domain Mapping

While customers prepare to migrate their existing setup and infrastructure from NFSv3 to NFSv4, some environmental changes must be made before moving to NFSv4. One of them is “id domain mapping.”

In clustered Data ONTAP 8.1, a new option called v4-id-numerics was added. With this option enabled, even if the client does not have access to the name mappings, numeric IDs can be sent in the user name and group name fields and the server accepts them and treats them as representing the same user as would be represented by a v2/v3 UID or GID having the corresponding numeric value.

Essentially, this makes NFSv4.x behave more like NFSv3. This also removes the security enhancement of forcing ID domain resolution for NFSv4.x name strings, so whenever possible, keep this option as the default of disabled. If a name mapping for the user is present, however, the name string will be sent across the wire rather than the UID/GID. The intent of this option is to ensure the server never sends “nobody” as a response to credential queries in NFS requests.

To access this command prior to clustered Data ONTAP 8.3, you must be in diag mode. Commands related to diag mode should be used with caution.

Some production environments have the challenge to build new naming service infrastructures like NIS or LDAP for string-based name mapping to be functional in order to move to NFSv4. With the new “numeric_id” option, setting name services does not become an absolute requirement. The “numeric_id” feature must be supported and enabled on the server as well as on the client. With this option enabled, the user and groups exchange UIDs/GIDs between the client and server just as with NFSv3. However, for this option to be enabled and functional, NetApp recommends having a supported version of the client and the server. For client versions that support numeric IDs with NFSv4, please contact the OS vendor.

Note that -v4-id-numerics should be enabled only if the client supports it.

Storage failover considerations

NFSv4.x uses a completely different locking model than NFSv3. Locking in NFSv4.x is a lease-based model that is integrated into the protocol itself rather than separated out as it is in NFSv3 (NLM). From the ONTAP documentation:

In accordance with RFC 3530, Data ONTAP “defines a single lease period for all state held by an NFS client. If the client does not renew its lease within the defined period, all states associated with the client’s lease may be released by the server.” The client can renew its lease explicitly or implicitly by performing an operation, such as reading a file.

Furthermore, Data ONTAP defines a grace period, which is a period of special processing in which clients attempt to reclaim their locking state during a server recovery.

Term Definition (as per RFC-3530)
Lease The time period in which Data ONTAP irrevocably grants a lock to a client.
Grace period The time period in which clients attempt to reclaim their locking state from Data ONTAP during server recovery.
Lock Refers to both record (byte-range) locks as well as file (share) locks unless specifically stated otherwise.

For more information on locking, see the section in this document on NFSv4.x locking. Because of this new locking methodology, as well as the statefulness of the NFSv4.x protocol, storage failover operates differently as compared to NFSv3. For more information, see the section in this document on storage failover and in clustered Data ONTAP.

Name services

If deciding to use NFSv4.x, it is a best practice to centralize the NFSv4.x users in name services such as LDAP or NIS. This allows all clients and clustered Data ONTAP NFS servers to leverage the same resources and guarantees that all names, UID and GIDs will be consistent across the implementation. For more information on name services, see TR-4073: Secure Unified Authentication and TR-4379: Name Services Best Practices.

Firewall considerations

NFSv3 required several ports to be opened for ancillary protocols such as NLM, NSM, etc. in addition to port 2049. NFSv4.x requires only port 2049. If wishing to use NFSv3 and NFSv4.x in the same environment, all relevant NFS ports should be opened. These ports are referenced in this document.

Volume language considerations

In NetApp’s Data ONTAP, volumes can have specific languages set. This is intended to be used for internationalization of file names for languages that use characters not common to English, such as Japanese, Chinese, German, etc. When using NFSv4.x, RFC-3530 states that UTF-8 is recommended.

11.  Internationalization

   The primary issue in which NFS version 4 needs to deal with
   internationalization, or I18N, is with respect to file names and
   other strings as used within the protocol.  The choice of string
   representation must allow reasonable name/string access to clients
   which use various languages.  The UTF-8 encoding of the UCS as
   defined by [ISO10646] allows for this type of access and follows the
   policy described in "IETF Policy on Character Sets and Languages",
   [RFC2277].

If you intend to migrate to clustered Data ONTAP from a 7-mode system and use NFSv4.x, you should be using some form of UTF-8 language support, such as C.UTF-8 (which is the default language of volumes in clustered Data ONTAP). If the 7-mode system is not already using a UTF-8 language, then it should be converted before transitioning to cDOT or when intending to transition from NFSv3 to NFSv4. The exact UTF-8 language specified will depend on the specific requirements of the native language to ensure proper display of character sets.

7-mode allowed volumes that hosted NFSv4.x data to use C language types. Clustered Data ONTAP does not, as it honors the RFC standard recommendation of UTF-8. TR-4160: Secure Multi-tenancy Considerations covers language recommendations in cDOT. When changing a volume’s language, every file in the volume must be accessed after the change to ensure they all reflect the language change. This can be done via a simple “ls -lR” to do a recursive listing of files.

Export policy rules

In clustered Data ONTAP, it is possible to specify which version of NFS is supported for an exported filesystem. If an environment was configured for NFSv3 and the export policy rule option –protocol was limited to allow NFSv3 only, then it would need to be modified to allow NFSv4. Additionally, policy rules could be configured to allow only NFSv4.x clients access.

Example:

cluster::> export-policy rule modify -policy default -vserver NAS -protocol nfs4For more information, consult the product documentation for your specific version of clustered Data ONTAP.

Client considerations

When using NFSv4.x, clients are as important to consider as the NFS server. The following client considerations should be followed when implementing NFSv4.x. Other considerations may be necessary. Please contact the OS vendor for specific questions about NFSv4.x configuration.

  • x is supported.
  • The fstab file and NFS configuration files are configured properly. When mounting, the client will negotiate the highest NFS version available with the NFS server. If NFSv4.x is not allowed by the client or fstab specifies NFSv3, then NFSv4.x will not be used at mount.
  • The idmapd.conf file is configured with the proper settings.
  • The client either contains identical users and UID/GID (including case sensitivity) or is using the same name service server as the NFS server/clustered Data ONTAP SVM.
  • If using name services on the client, the client is configured properly for name services (nsswitch.conf, ldap.conf, sssd.conf, etc.) and the appropriate services are started, running and configured to start at boot.
  • The NFSv4.x service is started, running and configured to start at boot.

NFSv4.x Features and Functionality

NFSv4.x is the next evolution of the NFS protocol and enhances NFSv3 with new features and functionality, such as referrals, delegations, pNFS, etc. These features are covered throughout this document and should be factored in to any design decisions for NFSv4.x implementations.

If done right and you make the right play calls, NFSv4.x could bring home the championship for  your NAS storage environment!

Advertisements

2 thoughts on “TECH::What Super Bowl 49 Taught Us About Transitioning from NFSv3 to NFSv4.x

  1. Pingback: Amazon announces Elastic File System for File Services - Datacenter Dude

  2. Pingback: Why Is the Internet Broken: Greatest Hits | Why Is The Internet Broken?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s