NOTE: Some of the documents linked in this blog are in the process of being updated for 8.3, so they may contain older information about clustered Data ONTAP 8.2.x. Check back on occasion to these links to get the most up to date content.
Clustered Data ONTAP 8.3 allows storage administrators to provide the following benefits:
- Seamless scale-out storage
- Multiprotocol Unified Access (NFS, CIFS and SAN)
- Non-disruptive operations
This is done by way of secure multi-tenant architecture with Storage Virtual Machines. This blog will cover logical interface (LIF) considerations for clustered Data ONTAP 8.3 as they pertain to NAS storage operations. These considerations will eventually be added to TR-4067. For a complete list of networking best practices, see TR-4182: Ethernet Storage Best Practices for cDOT Configurations.
Storage Virtual Machines (SVMs)
SVMs are logical storage containers that own storage resources such as flexible volumes, logical interfaces (LIFs), exports, CIFS shares, etc. Think of them as a storage “blade center” in your cluster. These SVMs share physical hardware resources in the cluster with one another, such as network ports/VLANs, aggregates with physical disk, CPU, RAM, switches, etc. As a result, load for SVMs can be balanced across a cluster for maximum performance and efficiency, or to leverage SaaS functionality, among other benefits.
A cluster can be comprised of several HA pairs of nodes (4 HA pairs/8 nodes with SAN, 12 HA pairs/24 nodes with NAS). Each node in the cluster has its own copy of a replicated database with the cluster and SVM configuration information. Additionally, each node has its own set of user space applications that handle cluster operations and node-specific caches, not to mention its own set of RAM, CPU, disks, etc. So while a cluster operates as a single entity, it does have the underlying concept of individualized components. As a result, it makes sense to take the physical hardware in a cluster under consideration when implementing and designing.
Data LIF considerations
Data LIFs can live on any physical port in a cluster that is added to a valid broadcast domain. These data LIFs are configured with SVM-aware routing mechanisms that allow for the correct pathing of ethernet traffic in a SVM, regardless of where a valid data LIF lives in the cluster. Prior to 8.3, SVMs routed at a node level, so traffic could only travel via the node that owned a data LIF. In cDOT 8.3, traffic will route from the data LIF even if it is a non-local path.
However, despite this enhancement in cDOT 8.3, it is still worth considering the original best practice recommendation for data LIFs participating in NAS operations…
One data LIF per node, per SVM
With the introduction of IP Spaces in clustered Data ONTAP, this recommendation is more of a reality, as storage administrators no longer have to use unique IP addresses in the cluster for SVMs. With IP Spaces, IP addresses can be duplicated in the cluster on a per-SVM basis to allow for true secure multi-tenancy architecture. For more information on IP Spaces, see TR-4182.
Thus, if using a 24 node cluster, using 24 data LIFs per SVM would be ideal for the following reasons:
- Ability to leverage data locality features – Features such as NFS referrals, CIFS auto location and pNFS ensure data locality for NAS traffic regardless of where the volumes live in a cluster. These help balance load better, but also make use of local caches and fastpath mechanisms for NAS traffic. For more information, see TR-4067.
- Ability to reduce cluster network traffic – While cluster network traffic is generally not an issue (you’re more likely to peg the CPU or disk before you saturate a 10gb network), it is better to limit the amount of traffic on a cluster network as much as possible.
- Ability to ensure data locality in the event of a volume move – If you move a volume to another node, you can ensure you still have a local path to the data if every node has a data LIF for the SVM.
- Ability to spread the load out across nodes and leverage all the available hardware (CPU, RAM, etc) – If you load up all your NAS traffic on one node via one data LIF, you aren’t realizing the value of the other nodes in the cluster. Spreading network traffic ensures all available physical entities are being utilized. Why pay for hardware you aren’t using?
- Ability to balance network connections across multiple cluster nodes – Clusters are single entities, as are SVMs. But they do have underlying hardware that have their own maximums, such as number of connections, etc. For info on hardware maximums in cDOT, see the configuration information for your version of ONTAP.
- Ability to reduce impact of storage failover/givebacks (SFO). – Fewer clients impacted when SFO events happen, whether they are planned or unplanned.
- Ability to leverage features such as on-box DNS – On-box DNS allows data LIFs to act as DNS servers and honor forwarded zone requests. Once a zone request is received, the cluster will determine the ideal node to service that request based on that node’s CPU and throughput, providing intelligent DNS load balancing (as opposed to round robin DNS, which is a serial process). For more information regarding on-box DNS (and how to configure it), see TR-4182 and TR-4073.
Keep in mind that the above are merely recommendations and not requirements unless using data locality features such as pNFS.
Any questions? Comments? Suggestions for blog topics? Feel free to comment or contact me on Twitter.