SMB1 Vulnerabilities: How do they affect NetApp’s Data ONTAP?

Google SMBv1 vulnerability, and you’ll get a ton of hits. There’s a reason for this.

SMB1 is the devil!

waterboy-smb1.jpg

But seriously, there are some major security holes in the protocol.

For a good rundown, check out the new NetApp CIFS/SMB TME Chris Hurley’s blog:

http://averageguyx.blogspot.com/2017/03/smb1-is-baaaaaad.html

This is in addition to the limitations of SMB1, such as lack of resiliency for network loss, lack of durable handles and overall performance and chattiness. There are many good reasons why Microsoft has decided to deprecate SMB1 in favor of newer protocols. SMB owner at Microsoft, Ned Pyle (@NerdPyle), gives a plethora of impassioned reasoning in his TechNet blog “Stop using SMB1!

So, there we are. SMB1 is bad, mmkay?

How does SMB1’s devil status affect NetApp’s ONTAP operating systems?

For the official NetApp statement, see this KB:

https://kb.netapp.com/support/s/article/NTAP-20170515-0001

This question comes up a bit here at NetApp, since security scanners will throw bells, whistles and alarms whenever SMB1 is detected in an environment. What follows is:

  • Does SMB1 in ONTAP have the same vulnerabilities?
  • Can I disable SMB1 in ONTAP?
  • If I can’t disable it, can I block it?

The good news is, the main security vulnerabilities that plague SMB1 in Windows generally don’t affect ONTAP (such as 0-day), because ONTAP isn’t a Windows client. It’s using a proprietary, custom built CIFS/SMB stack (akin to Samba). Thus, the vulnerabilities that impact Windows don’t impact ONTAP.

Note: I can’t take all the credit for the information in this blog. That credit goes to John Lantz (CIFS TME at NetApp), as well as various CIFS/SMB engineering resources here.

Can I disable SMB1 in ONTAP?

While the vulnerabilities don’t necessarily affect ONTAP, the security scanners still are triggering alarms and managers are still wanting the red X’s to go away.

scan

As a result, people want to just turn it off in ONTAP, especially since they aren’t currently using it in their environments (hopefully).

The good news is that ONTAP is in the process of deprecating SMB1. The bad news? It’s still there.

However, in ONTAP 9.2, NetApp introduced a new CIFS option to disable SMB1 in advanced privilege!

cluster::> set advanced
cluster::*> cifs options modify -vserver DEMO -smb1-enabled false
 [-smb1-enabled {true|false}] - Enable SMB1 Protocol (privilege: advanced)
 This optional parameter specifies whether the CIFS server negotiates the SMB 1.0 version of the CIFS protocol. The default value for this parameter is true.

If you need to disable SMBv1 now in ONTAP, you’d need to be on ONTAP 9.2.

We also have the ability to control what SMB version is used with domain controllers for authentication. In systems running ONTAP 7-mode, use the following option to enable SMB2.

cifs.smb2.client.enable

In systems running clustered ONTAP, starting in ONTAP 8.3.2P5, you can disable SMB1 connections to the DC, as well as enabling SMB2.

[-smb1-enabled-for-dc-connections {false|true|system-default}] - SMB1 Enabled for DC Connections
 This parameter specifies whether SMB1 is enabled for use with connections to domain controllers. If you do not specify this parameter, the default is system-default.

SMB1 Enabled For DC Connections can be one of the following:
o false - SMB1 is not enabled.
o true - SMB1 is enabled.
o system-default - This sets the option to whatever is the default for the release of Data ONTAP that is running. For this release it is: SMB1 is enabled.

[-smb2-enabled-for-dc-connections {false|true|system-default}] - SMB2 Enabled for DC Connections
 This parameter specifies whether SMB2 is enabled for use with connections to domain controllers. If you do not specify this parameter, the default is system-default.

SMB2 Enabled For DC Connections can be one of the following:
o false - SMB2 is not enabled.
o true - SMB2 is enabled.
o system-default - This sets the option to whatever is the default for the release of Data ONTAP that is running. For this release it is: SMB2 is not enabled.

Use the following command to do that:

cifs security modify -vserver DEMO -smb1-enabled-for-dc-connections false -smb2-enabled-for-dc-connections true

If I can’t disable it in ONTAP, can I block it?

Technically, you *could* block the SMB1 ports. However, if you block ports that SMB2 also needs (such as 445), you’d be in trouble.

The official recommendation from Microsoft is a combination of disabling SMB1 on clients (you could handle this via Group Policy), as well as blocking ports on *external* facing interfaces. In other words, don’t allow SMB outside of the firewall.

Here’s the official link:

https://technet.microsoft.com/en-us/library/cc766392%28v=ws.10%29.aspx?f=255&MSPPError=-2147217396

To disable SMB1 on the client:

https://support.microsoft.com/en-us/kb/2696547

Inside your firewall, you shouldn’t need the following ports, so block away:

  • UDP/137 (NetBIOS name service)
  • UDP/138 (NetBIOS datagram service)
  • TCP/139 (NetBIOS session service)

In some cases, you won’t be able to rid yourself entirely of SMB1. Remember that $30k printer/copier/scanner that you bought 10 years ago that was cool because you could scan directly to a SMB share? Yeah…. that’s probably still using SMB1. Check with your scanner/copier vendor to see if they have any software updates. Otherwise, you may need to disable SMB1 on the copier/scanner, or budget for a new one.

copier

For official NetApp statement on SMB1, check out this TR, starting on page 4:

http://www.netapp.com/us/media/tr-4543.pdf

Also, check out Episode 95 of the Tech ONTAP podcast, where we discuss WannaCry and Petya!

https://m.soundcloud.com/techontap_podcast/episode-95-quarterly-security-update-wannacry-and-petya

Advertisements

Managing ACLs via the ONTAP Command Line

In a previous post, I covered multiprotocol NAS in ONTAP, as well as mixed security styles. The following post covers how to manage permissions from the ONTAP CLI, as well as how to centralize permission management from a single Linux client. Some of the following was moved from the previous post to this one to make it easier to read and digest.

Viewing permissions in multiprotocol NAS

There are options to display permissions from both types of clients. For viewing UNIX permissions from Windows property tabs, use the cifs option is-unix-nt-acl-enabled.

cluster::*> cifs option show -vserver parisi-fields is-unix-nt-acl-enabled
vserver is-unix-nt-acl-enabled
----------- ----------------------
parisi     true

When using this option, the Windows clients will show a security tab entry that approximates the UNIX mode bits into ACLs. It will show the owner, group and “other” permissions. It will also attempt to convert the UNIX UID into a Windows-friendly SID so the client can display it. The Windows user will look like this:

unix-windows-acl1

That user is a “fake SID” that is tied to the cluster’s Storage Virtual Machine. It translates to a SID that ONTAP creates based on the numeric ID of the user or group. The Windows client uses that SID to translate into a name.

For example:

cluster::*> diag secd authentication translate -node node1 -vserver SVM -win-name UNIXPermUid\root
S-1-5-21-2038298172-1297133386-11111-0

cluster::*> diag secd authentication translate -node node1 -vserver SVM -unix-user-name root
0

cluster::*> diag secd authentication translate -node node1 -vserver SVM -win-name UNIXPermUid\user3
S-1-5-21-2038298172-1297133386-11111-703

cluster::*> diag secd authentication translate -node node1 -vserver SVM -unix-user-name user3
703

cluster::*> diag secd authentication translate -node node1 -vserver SVM -win-name UNIXPermGid\homedirs
S-1-5-21-2038298172-1297133386-22222-1002

cluster::*> diag secd authentication translate -node node1 -vserver SVM -unix-group-name homedirs
1002

From Windows, we can see the level of access for the users from the “Change Permissions” window:

unix-windows-acl2

On the NFS side, mode bits have no clue how to translate NTFS permission concepts like extended attributes. Instead, the clients only know Read, Write, Execute, Traverse, etc. It’s possible to show an approximation of those mode bits in UNIX for NTFS security style volumes with this option:

cluster::*> nfs server show -fields ntacl-display-permissive-perms
vserver ntacl-display-permissive-perms
----------- ------------------------------
parisi     disabled

When that option is disabled, NTFS ACLs show up as closely to UNIX permissions as they can. In the following example, I have an NTFS security style folder that allowed only the owner to have full control, but allows read to “Everyone.” With the option mentioned, we see that reflected as “755” in permissions:

unix-windows-acl3

drwxr-xr-x 3 user1 homedirs 4096 Nov 8 14:15 user1

Translating NTFS style DACLs

As previously mentioned, in ONTAP we can view the Windows ACLs on a file, folder or volume using vserver security file-directory show.

cluster::*> vserver security file-directory show -vserver SVM-path /homedir1/user1

Vserver: SVM
 File Path: /homedir1/user1
 File Inode Number: 10363
 Security Style: mixed
 Effective Style: ntfs
 DOS Attributes: 10
 DOS Attributes in Text: ----D---
Expanded Dos Attributes: -
 UNIX User Id: 701
 UNIX Group Id: 1002
 UNIX Mode Bits: 777
 UNIX Mode Bits in Text: rwxrwxrwx
 ACLs: NTFS Security Descriptor
 Control:0x8004
 Owner:CPOC\user1
 Group:CPOC\Domain Users
 DACL - ACEs
 ALLOW-CPOC\Administrator-0xe0000040-OI|IO
 ALLOW-CPOC\Administrator-0x1201ff-CI
 ALLOW-CPOC\user1-0x10000000-OI|IO
 ALLOW-CPOC\user1-0x1f01ff-CI
 ALLOW-Everyone-0xa0000000-OI|IO
 ALLOW-Everyone-0x1200a9-CI

However, as you can see, those ACLs don’t make a ton of sense unless you can read hexadecimal. (If you can, more power to ya.)

Let’s break down the ACLs a bit to understand them better.

  • First, DACL means “Discretionary Access Control List.” From MSDN:
  • An access control list that is controlled by the owner of an object and that specifies the access particular users or groups can have to the object.
  • In the DACLs above, we can see whether the DACL is an ALLOW or a DENY ACL. (Deny ACLs override ALLOWS.) We can also see the user or group being allowed access. After that, the information isn’t really in a “human readable” format.
  • The CI, IO, OI values are “ACE strings” and tell us whether the ACL was inherited by the owner or container. MSDN has a handy list of those here: ACE Strings

The rest of the ACLs are hexadecimal values and translate into what the actual permissions that were set were.

Expanding ACLs

Rather than try to decode all of those, ONTAP has an option on the file-directory show command that allows you to expand the ACL mask from the CLI (-expand-mask). This actually cracks open the DACLs and shows an expanded view of what actual permissions are allowed.

For example:

cluster::> vserver security file-directory show -vserver parisi -path /cifs -expand-mask true

Vserver: parisi
 File Path: /cifs
 File Inode Number: 64
 Security Style: ntfs
 Effective Style: ntfs
 DOS Attributes: 10
 DOS Attributes in Text: ----D---
Expanded Dos Attributes: 0x10
 ...0 .... .... .... = Offline
 .... ..0. .... .... = Sparse
 .... .... 0... .... = Normal
 .... .... ..0. .... = Archive
 .... .... ...1 .... = Directory
 .... .... .... .0.. = System
 .... .... .... ..0. = Hidden
 .... .... .... ...0 = Read Only
 UNIX User Id: 0
 UNIX Group Id: 0
 UNIX Mode Bits: 777
 UNIX Mode Bits in Text: rwxrwxrwx
 ACLs: NTFS Security Descriptor
 Control:0x8004

1... .... .... .... = Self Relative
 .0.. .... .... .... = RM Control Valid
 ..0. .... .... .... = SACL Protected
 ...0 .... .... .... = DACL Protected
 .... 0... .... .... = SACL Inherited
 .... .0.. .... .... = DACL Inherited
 .... ..0. .... .... = SACL Inherit Required
 .... ...0 .... .... = DACL Inherit Required
 .... .... ..0. .... = SACL Defaulted
 .... .... ...0 .... = SACL Present
 .... .... .... 0... = DACL Defaulted
 .... .... .... .1.. = DACL Present
 .... .... .... ..0. = Group Defaulted
 .... .... .... ...0 = Owner Defaulted

Owner:BUILTIN\Administrators
 Group:BUILTIN\Administrators
 DACL - ACEs
 ALLOW-Everyone-0x1f01ff
 0... .... .... .... .... .... .... .... = Generic Read
 .0.. .... .... .... .... .... .... .... = Generic Write
 ..0. .... .... .... .... .... .... .... = Generic Execute
 ...0 .... .... .... .... .... .... .... = Generic All
 .... ...0 .... .... .... .... .... .... = System Security
 .... .... ...1 .... .... .... .... .... = Synchronize
 .... .... .... 1... .... .... .... .... = Write Owner
 .... .... .... .1.. .... .... .... .... = Write DAC
 .... .... .... ..1. .... .... .... .... = Read Control
 .... .... .... ...1 .... .... .... .... = Delete
 .... .... .... .... .... ...1 .... .... = Write Attributes
 .... .... .... .... .... .... 1... .... = Read Attributes
 .... .... .... .... .... .... .1.. .... = Delete Child
 .... .... .... .... .... .... ..1. .... = Execute
 .... .... .... .... .... .... ...1 .... = Write EA
 .... .... .... .... .... .... .... 1... = Read EA
 .... .... .... .... .... .... .... .1.. = Append
 .... .... .... .... .... .... .... ..1. = Write
 .... .... .... .... .... .... .... ...1 = Read

ALLOW-Everyone-0x10000000-OI|CI|IO
 0... .... .... .... .... .... .... .... = Generic Read
 .0.. .... .... .... .... .... .... .... = Generic Write
 ..0. .... .... .... .... .... .... .... = Generic Execute
 ...1 .... .... .... .... .... .... .... = Generic All
 .... ...0 .... .... .... .... .... .... = System Security
 .... .... ...0 .... .... .... .... .... = Synchronize
 .... .... .... 0... .... .... .... .... = Write Owner
 .... .... .... .0.. .... .... .... .... = Write DAC
 .... .... .... ..0. .... .... .... .... = Read Control
 .... .... .... ...0 .... .... .... .... = Delete
 .... .... .... .... .... ...0 .... .... = Write Attributes
 .... .... .... .... .... .... 0... .... = Read Attributes
 .... .... .... .... .... .... .0.. .... = Delete Child
 .... .... .... .... .... .... ..0. .... = Execute
 .... .... .... .... .... .... ...0 .... = Write EA
 .... .... .... .... .... .... .... 0... = Read EA
 .... .... .... .... .... .... .... .0.. = Append
 .... .... .... .... .... .... .... ..0. = Write
 .... .... .... .... .... .... .... ...0 = Read

This also works with NFSv4 ACLs:

cluster::*> vserver security file-directory show -vserver DEMO -path /shared/unix -expand-mask true

                Vserver: DEMO
              File Path: /shared/unix
      File Inode Number: 20034
         Security Style: unix
        Effective Style: unix
         DOS Attributes: 10
 DOS Attributes in Text: ----D---
Expanded Dos Attributes: 0x10
     ...0 .... .... .... = Offline
     .... ..0. .... .... = Sparse
     .... .... 0... .... = Normal
     .... .... ..0. .... = Archive
     .... .... ...1 .... = Directory
     .... .... .... .0.. = System
     .... .... .... ..0. = Hidden
     .... .... .... ...0 = Read Only
           UNIX User Id: 1100
          UNIX Group Id: 1101
         UNIX Mode Bits: 770
 UNIX Mode Bits in Text: rwxrwx---
                   ACLs: NFSV4 Security Descriptor
                         Control:0x8014
                              1... .... .... .... = Self Relative
                              .0.. .... .... .... = RM Control Valid
                              ..0. .... .... .... = SACL Protected
                              ...0 .... .... .... = DACL Protected
                              .... 0... .... .... = SACL Inherited
                              .... .0.. .... .... = DACL Inherited
                              .... ..0. .... .... = SACL Inherit Required
                              .... ...0 .... .... = DACL Inherit Required
                              .... .... ..0. .... = SACL Defaulted
                              .... .... ...1 .... = SACL Present
                              .... .... .... 0... = DACL Defaulted
                              .... .... .... .1.. = DACL Present
                              .... .... .... ..0. = Group Defaulted
                              .... .... .... ...0 = Owner Defaulted

                         DACL - ACEs
                           ALLOW-OWNER@-0x1601ff
                              0... .... .... .... .... .... .... .... = Generic Read
                              .0.. .... .... .... .... .... .... .... = Generic Write
                              ..0. .... .... .... .... .... .... .... = Generic Execute
                              ...0 .... .... .... .... .... .... .... = Generic All
                              .... ...0 .... .... .... .... .... .... = System Security
                              .... .... ...1 .... .... .... .... .... = Synchronize
                              .... .... .... 0... .... .... .... .... = Write Owner
                              .... .... .... .1.. .... .... .... .... = Write DAC
                              .... .... .... ..1. .... .... .... .... = Read Control
                              .... .... .... ...0 .... .... .... .... = Delete
                              .... .... .... .... .... ...1 .... .... = Write Attributes
                              .... .... .... .... .... .... 1... .... = Read Attributes
                              .... .... .... .... .... .... .1.. .... = Delete Child
                              .... .... .... .... .... .... ..1. .... = Execute
                              .... .... .... .... .... .... ...1 .... = Write EA
                              .... .... .... .... .... .... .... 1... = Read EA
                              .... .... .... .... .... .... .... .1.. = Append
                              .... .... .... .... .... .... .... ..1. = Write
                              .... .... .... .... .... .... .... ...1 = Read

                           ALLOW-user-prof1-0x1601ff
                              0... .... .... .... .... .... .... .... = Generic Read
                              .0.. .... .... .... .... .... .... .... = Generic Write
                              ..0. .... .... .... .... .... .... .... = Generic Execute
                              ...0 .... .... .... .... .... .... .... = Generic All
                              .... ...0 .... .... .... .... .... .... = System Security
                              .... .... ...1 .... .... .... .... .... = Synchronize
                              .... .... .... 0... .... .... .... .... = Write Owner
                              .... .... .... .1.. .... .... .... .... = Write DAC
                              .... .... .... ..1. .... .... .... .... = Read Control
                              .... .... .... ...0 .... .... .... .... = Delete
                              .... .... .... .... .... ...1 .... .... = Write Attributes
                              .... .... .... .... .... .... 1... .... = Read Attributes
                              .... .... .... .... .... .... .1.. .... = Delete Child
                              .... .... .... .... .... .... ..1. .... = Execute
                              .... .... .... .... .... .... ...1 .... = Write EA
                              .... .... .... .... .... .... .... 1... = Read EA
                              .... .... .... .... .... .... .... .1.. = Append
                              .... .... .... .... .... .... .... ..1. = Write
                              .... .... .... .... .... .... .... ...1 = Read

                           ALLOW-GROUP@-0x1201ff-IG
                              0... .... .... .... .... .... .... .... = Generic Read
                              .0.. .... .... .... .... .... .... .... = Generic Write
                              ..0. .... .... .... .... .... .... .... = Generic Execute
                              ...0 .... .... .... .... .... .... .... = Generic All
                              .... ...0 .... .... .... .... .... .... = System Security
                              .... .... ...1 .... .... .... .... .... = Synchronize
                              .... .... .... 0... .... .... .... .... = Write Owner
                              .... .... .... .0.. .... .... .... .... = Write DAC
                              .... .... .... ..1. .... .... .... .... = Read Control
                              .... .... .... ...0 .... .... .... .... = Delete
                              .... .... .... .... .... ...1 .... .... = Write Attributes
                              .... .... .... .... .... .... 1... .... = Read Attributes
                              .... .... .... .... .... .... .1.. .... = Delete Child
                              .... .... .... .... .... .... ..1. .... = Execute
                              .... .... .... .... .... .... ...1 .... = Write EA
                              .... .... .... .... .... .... .... 1... = Read EA
                              .... .... .... .... .... .... .... .1.. = Append
                              .... .... .... .... .... .... .... ..1. = Write
                              .... .... .... .... .... .... .... ...1 = Read

                           ALLOW-EVERYONE@-0x120080
                              0... .... .... .... .... .... .... .... = Generic Read
                              .0.. .... .... .... .... .... .... .... = Generic Write
                              ..0. .... .... .... .... .... .... .... = Generic Execute
                              ...0 .... .... .... .... .... .... .... = Generic All
                              .... ...0 .... .... .... .... .... .... = System Security
                              .... .... ...1 .... .... .... .... .... = Synchronize
                              .... .... .... 0... .... .... .... .... = Write Owner
                              .... .... .... .0.. .... .... .... .... = Write DAC
                              .... .... .... ..1. .... .... .... .... = Read Control
                              .... .... .... ...0 .... .... .... .... = Delete
                              .... .... .... .... .... ...0 .... .... = Write Attributes
                              .... .... .... .... .... .... 1... .... = Read Attributes
                              .... .... .... .... .... .... .0.. .... = Delete Child
                              .... .... .... .... .... .... ..0. .... = Execute
                              .... .... .... .... .... .... ...0 .... = Write EA
                              .... .... .... .... .... .... .... 0... = Read EA
                              .... .... .... .... .... .... .... .0.. = Append
                              .... .... .... .... .... .... .... ..0. = Write
                              .... .... .... .... .... .... .... ...0 = Read

However, with a ton of ACLs on an object, this could get a bit overwhelming. So, translating the hex might be better overall. This blog covers it in a bit more detail:

About the ACCESS_MASK structure

In the above ACL, we see 0x1f01ff for Everyone. That’s Full Control. In addition, 0x10000000 is considered GENERIC_ALL.

Applying ACLs to objects from the storage

In addition to displaying ACLs, vserver security file-directory commands can be used to apply SACLs and DACLs to objects from the cluster’s CLI.

The general steps are covered in this KB article:

https://kb.netapp.com/support/s/article/how-to-modify-permissions-on-files-and-folders-in-clustered-data-ontap-when-there-is-no-permission-to-take-ownership?t=1484836401866

The following shows an example of doing this on a single qtree in ONTAP.

This is a qtree called “mixed.” It has an effective security style of UNIX, unix permissions 770 and root:sharedgroup as the owners.

cluster::*> vserver security file-directory show -vserver DEMO -path /shared/mixed

                Vserver: DEMO
              File Path: /shared/mixed
      File Inode Number: 20035
         Security Style: mixed
        Effective Style: unix
         DOS Attributes: 10
 DOS Attributes in Text: ----D---
Expanded Dos Attributes: -
           UNIX User Id: 0
          UNIX Group Id: 1206
         UNIX Mode Bits: 770
 UNIX Mode Bits in Text: rwxrwx---
                   ACLs: -

To change permissions on this object (or other objects, if desired), first create a security policy:

cluster::*> file-directory policy create -vserver DEMO -policy-name Policy1
  (vserver security file-directory policy create)
 
cluster::*> vserver security file-directory policy show -vserver DEMO -instance
    Vserver: DEMO
Policy Name: Policy1

Then, create a security descriptor, which allows a storage admin to add access control entries (ACEs) to the discretionary access control list (DACL) and the system access control list (SACL). This provides the ability to add, in bulk, an owner, group or control flags in raw hex:

cluster::*> vserver security file-directory ntfs create -vserver DEMO -ntfs-sd sdname 
 -owner ntfsonly

cluster::*> vserver security file-directory ntfs show -instance
                      Vserver: DEMO
NTFS Security Descriptor Name: sdname
                        Owner: NTAP\ntfsonly
                Primary Group: -
            Raw Control Flags: -

Next, create one or more DACLs or SACLs. In this case, I’ve created 2 DACLs. This command allows the following:

cluster::*> vserver security file-directory ntfs dacl add ?
    -vserver                                                   Vserver
   [-ntfs-sd]                                             NTFS Security Descriptor Name
   [-access-type] {deny|allow}                                               Allow or Deny
   [-account]                                                   Account Name or SID
  { [[-rights] {no-access|full-control|modify|read-and-execute|read|write}]  DACL ACE's Access Rights
  | [ -advanced-rights , ... ]                        DACL ACE's Advanced Access Rights
  | [ -rights-raw  ] }                                          *DACL ACE's Raw Access Rights
  [ -apply-to {this-folder|sub-folders|files}, ... ]                         Apply DACL Entry

The users I’m adding are ntfsonly and student1. Ntfsonly gets full control; student1 gets readonly access. I’m applying the DACL to all objects (this-folder, sub-folders, files).

NOTE: If you don’t apply the DACL to the top level folder, you run the risk of denying access to everyone because the owner doesn’t get set properly.

ontap9-tme-8040::*> vserver security file-directory ntfs dacl add -vserver DEMO -ntfs-sd sdname -access-type allow -account ntfsonly -apply-to this-folder,sub-folders,files -advanced-rights full-control

ontap9-tme-8040::*> vserver security file-directory ntfs dacl add -vserver DEMO -ntfs-sd sdname -access-type allow -account student1 -rights read -apply-to this-folder,sub-folders,files

In addition to the ACLs we define, we also get default built-in DACLs. Feel free to delete those as needed.

ontap9-tme-8040::*> vserver security file-directory ntfs dacl show -vserver DEMO -instance


                      Vserver: DEMO
NTFS Security Descriptor Name: sdname
                Allow or Deny: allow
          Account Name or SID: BUILTIN\Administrators
                Access Rights: full-control
            Raw Access Rights: -
       Advanced Access Rights: -
             Apply DACL Entry: this-folder, sub-folders, files
                Access Rights: full-control

                      Vserver: DEMO
NTFS Security Descriptor Name: sdname
                Allow or Deny: allow
          Account Name or SID: BUILTIN\Users
                Access Rights: full-control
            Raw Access Rights: -
       Advanced Access Rights: -
             Apply DACL Entry: this-folder, sub-folders, files
                Access Rights: full-control

                      Vserver: DEMO
NTFS Security Descriptor Name: sdname
                Allow or Deny: allow
          Account Name or SID: CREATOR OWNER
                Access Rights: full-control
            Raw Access Rights: -
       Advanced Access Rights: -
             Apply DACL Entry: this-folder, sub-folders, files
                Access Rights: full-control

                      Vserver: DEMO
NTFS Security Descriptor Name: sdname
                Allow or Deny: allow
          Account Name or SID: NT AUTHORITY\SYSTEM
                Access Rights: full-control
            Raw Access Rights: -
       Advanced Access Rights: -
             Apply DACL Entry: this-folder, sub-folders, files
                Access Rights: full-control

                      Vserver: DEMO
NTFS Security Descriptor Name: sdname
                Allow or Deny: allow
          Account Name or SID: NTAP\ntfsonly
                Access Rights: -
            Raw Access Rights: -
       Advanced Access Rights: full-control
             Apply DACL Entry: this-folder, sub-folders, files
                Access Rights: full-control

                      Vserver: DEMO
NTFS Security Descriptor Name: sdname
                Allow or Deny: allow
          Account Name or SID: NTAP\student1
                Access Rights: read
            Raw Access Rights: -
       Advanced Access Rights: -
             Apply DACL Entry: this-folder, sub-folders, files
                Access Rights: read
6 entries were displayed.

Now that the policy is created and I have the desired DACLs and SACLs, I can apply them to whatever paths I want. In the above, I’ve set the DACLs to only apply to the specific folder. To apply the policy, create a new task and define the path you want to re-ACL. The task will “propogate” by default. You can also specify “replace” if desired.

cluster::*> file-directory policy task add -vserver DEMO -policy-name Policy1 -path /shared/mixed -ntfs-sd sdname
  (vserver security file-directory policy task add)

cluster::*> file-directory policy task show
  (vserver security file-directory policy task show)

Vserver: DEMO
  Policy: Policy1

   Index  File/Folder  Access           Security  NTFS       NTFS Security
          Path         Control          Type      Mode       Descriptor Name
   -----  -----------  ---------------  --------  ---------- ---------------
   1      /shared/mixed
                       file-directory   ntfs      propagate  sdname

Once everything appears in order, apply the policy:

cluster::*> file-directory apply -vserver DEMO -policy-name Policy1
  (vserver security file-directory apply)

[Job 3229] Job is queued: Fsecurity Apply. Use the "job show -id 3229" command to view the status of this operation.

If you want status of the progress, use job show. If you want detailed progress, use job show -instance.

cluster::*> job show -id 3229
                            Owning
Job ID Name                 Vserver    Node           State
------ -------------------- ---------- -------------- ----------
3229   Fsecurity Apply      cluster
                                       cluster2
                                                      Success
       Description: File Directory Security Apply Job

Then, check your ACLs. Note how the effective style of the mixed qtree has changed from UNIX to NTFS:

cluster::*> vserver security file-directory show -vserver DEMO -path /shared/mixed

                Vserver: DEMO
              File Path: /shared/mixed
      File Inode Number: 20035
         Security Style: mixed
        Effective Style: ntfs
         DOS Attributes: 10
 DOS Attributes in Text: ----D---
Expanded Dos Attributes: -
           UNIX User Id: 0
          UNIX Group Id: 0
         UNIX Mode Bits: 777
 UNIX Mode Bits in Text: rwxrwxrwx
                   ACLs: NTFS Security Descriptor
                         Control:0x8014
                         Owner:NTAP\ntfsonly
                         Group:BUILTIN\Administrators
                         DACL - ACEs
                           ALLOW-BUILTIN\Administrators-0x1f01ff-OI|CI
                           ALLOW-BUILTIN\Users-0x1f01ff-OI|CI
                           ALLOW-CREATOR OWNER-0x1f01ff-OI|CI
                           ALLOW-NT AUTHORITY\SYSTEM-0x1f01ff-OI|CI
                           ALLOW-NTAP\ntfsonly-0x1f01ff
                           ALLOW-NTAP\student1-0x120089     

If you want to apply the policy to other paths (or multiple paths at once), create new tasks:

cluster::*> vserver security file-directory show -vserver DEMO -path /shared/security
                Vserver: DEMO
              File Path: /shared/security
      File Inode Number: 96
         Security Style: mixed
        Effective Style: unix
         DOS Attributes: 10
 DOS Attributes in Text: ----D---
Expanded Dos Attributes: -
           UNIX User Id: 0
          UNIX Group Id: 0
         UNIX Mode Bits: 770
 UNIX Mode Bits in Text: rwxrwx---
                   ACLs: -

cluster::*> file-directory policy task add -vserver DEMO -policy-name Policy1 -path /shared/security -ntfs-sd sdname
  (vserver security file-directory policy task add)

cluster::*> file-directory policy task show
  (vserver security file-directory policy task show)
Vserver: DEMO
  Policy: Policy1
   Index  File/Folder  Access           Security  NTFS       NTFS Security
          Path         Control          Type      Mode       Descriptor Name
   -----  -----------  ---------------  --------  ---------- ---------------
   1      /shared/mixed
                       file-directory   ntfs      propagate  sdname
   2      /shared/security
                       file-directory   ntfs      propagate  sdname
2 entries were displayed.

cluster::*> file-directory apply -vserver DEMO -policy-name Policy1
  (vserver security file-directory apply)

[Job 3232] Job is queued: Fsecurity Apply. Use the "job show -id 3232" command to view the status of this operation.

cluster::*> vserver security file-directory show -vserver DEMO -path /shared/security
                Vserver: DEMO
              File Path: /shared/security
      File Inode Number: 96
         Security Style: mixed
        Effective Style: ntfs
         DOS Attributes: 10
 DOS Attributes in Text: ----D---
Expanded Dos Attributes: -
           UNIX User Id: 0
          UNIX Group Id: 0
         UNIX Mode Bits: 777
 UNIX Mode Bits in Text: rwxrwxrwx
                   ACLs: NTFS Security Descriptor
                         Control:0x8014
                         Owner:NTAP\ntfsonly
                         Group:BUILTIN\Administrators
                         DACL - ACEs
                           ALLOW-BUILTIN\Administrators-0x1f01ff-OI|CI
                           ALLOW-BUILTIN\Users-0x1f01ff-OI|CI
                           ALLOW-CREATOR OWNER-0x1f01ff-OI|CI
                           ALLOW-NT AUTHORITY\SYSTEM-0x1f01ff-OI|CI
                           ALLOW-NTAP\ntfsonly-0x1f01ff
                           ALLOW-NTAP\student1-0x120089

Example of a running job with more information:

cluster::*> job show -id 3317 -instance
                      Job ID: 3317
              Owning Vserver: cluster
                        Name: Fsecurity Apply
                 Description: File Directory Security Apply Job
                    Priority: Low
                        Node: cluster02
                    Affinity: Cluster
                    Schedule: @now
                  Queue Time: 01/24 09:45:19
                  Start Time: 01/24 09:45:19
                    End Time: -
              Drop-dead Time: -
                  Restarted?: false
                       State: Running
                 Status Code: 0
           Completion String:
                    Job Type: FSEC_APPLY
                Job Category: FSECURITY
                        UUID: b9e7bf61-e243-11e6-a40c-00a0986b1210
          Execution Progress: Fsecurity Apply processed 46766 files/dirs. Last Processed: /shared/security/files/topdir_77/subdir_81
                   User Name: admin
                     Process: mgwd
  Restart Is or Was Delayed?: false
Restart Is Delayed by Module: -

Centralizing permission management

With multiprotocol NAS, it’s possible to view and manage ACLs from multiple clients, as well as the storage. The way I did this was to set up passwordless SSH on a Linux client and then create simple shell scripts that call SSH commands to the cluster. Another way to do this would be to leverage the ONTAP SDK. I’ll write up a post on the SDK at some point in the future, but for now, we’ll focus on the bash scripts.

To set up passwordless SSH to the cluster, do the following (from TR-4073):

Create the SSH Keypair

In the following example, ssh-keygen is used on a Linux box.

  • If a ssh key pair already exists, there is no need to generate one using ssh-keygen.
monitor@linux:/$ ssh-keygen -q -f ~/.ssh/id_rsa -t rsa
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
monitor@linux:/$ ls -lsa ~/.ssh
total 16
4 drwx------ 2 monitor monitor 4096 2008-08-26 11:47 .
4 drwxr-xr-x 3 monitor monitor 4096 2008-08-26 11:47 ..
4 -rw------- 1 monitor monitor 1679 2008-08-26 11:47 id_rsa
4 -rw-r--r-- 1 monitor monitor 401 2008-08-26 11:47 id_rsa.pub

Create the User with a Public Key Authentication Method

cluster::> security login create -username monitor -application ssh -authmethod publickey -profile admin

Create the Public Key on the Cluster

Copy the public key contents of the id_rsa.pub file and place it between quotes in the security login public key create command. Take caution not to add carriage returns or other data that modifies the keystring; leave it in one line.

netapp::> security login publickey create -username monitor -index 1 -publickey “ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5s4vVbwEO1sOsq7r64V5KYBRXBDb2I5mtGmt0+3p1jjPJrXx4/IPHFLalXAQkG7LhV5Dyc5jyQiGKVawBYwxxSZ3GqXJNv1aORZHJEuCd0zvSTBGGZ09vra5uCfxkpz8nwaTeiAT232LS2lZ6RJ4dsCz+GAj2eidpPYMldi2z6RVoxpZ5Zq68MvNzz8b15BS9T7bvdHkC2OpXFXu2jndhgGxPHvfO2zGwgYv4wwv2nQw4tuqMp8e+z0YP73Jg0T3jV8NYraXO951Rr5/9ZT8KPUqLEgPZxiSNkLnPC5dnmfTyswlofPGud+qmciYYr+cUZIvcFaYRG+Z6DM/HInX7w==  monitor@linux”

Alternatively, you can use the load-from-uri function to bring the public key from another source.

cluster::> security login publickey load-from-uri -username monitor -uri http://linux/id_rsa.pub

Verify Creation

netapp::> security login publickey show -username monitor

UserName: monitor Index: 1

Public Key:

ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5s4vVbwEO1sOsq7r64V5KYBRXBDb2I5mtGmt0+3p1jjPJrXx4/IPHFLalXAQkG7LhV5Dyc5jyQiGKVawBYwxxSZ3GqXJNv1aORZHJEuCd0zvSTBGGZ09vra5uCfxkpz8nwaTeiAT232LS2lZ6RJ4dsCz+GAj2eidpPYMldi2z6RVoxpZ5Zq68MvNzz8b15BS9T7bvdHkC2OpXFXu2jndhgGxPHvfO2zGwgYv4wwv2nQw4tuqMp8e+z0YP73Jg0T3jV8NYraXO951Rr5/9ZT8KPUqLEgPZxiSNkLnPC5dnmfTyswlofPGud+qmciYYr+cUZIvcFaYRG+Z6DM/HInX7w==monitor@linux

Test Access from the Host

monitor@linux:~$ ssh 10.61.64.150
The authenticity of host '10.61.64.150 (10.61.64.150)' can't be established.
DSA key fingerprint is d9:15:cf:4b:d1:7b:a9:67:4d:b0:a9:20:e4:fa:f4:69.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.61.64.150' (DSA) to the list of known hosts.

Once that’s done, you can set up scripts to make SSH calls without having to interact.

Sample scripts

I’ve posted some sample bash scripts on GitHub to allow for open-sourcing of the tasks. But, essentially, the scripts I created can:

  • Show ACLs for specified paths
  • Change ACLs en masse for a specified object
  • Clean up policies and DACLs created
  • Be used as a wrapper

Creating a wrapper command

In addition to the scripts above, it’s also possible to create a simple wrapper command in Linux that will call a script to make life easier for an administrator. To do this, modify the .bashrc file in the user’s home directory. In the following example, I created a command called ONTAP_ACL and pointed it to my script.

# cat ~/.bashrc
# .bashrc

# User specific aliases and functions

alias rm='rm -i'
alias cp='cp -i'
alias mv='mv -i'
alias ONTAP_ACL='/scripts/./show-ACL.sh'

Once this is done, you can restart the shell session and the command can be run to execute the script. In the above, the show-ACL script simply takes the path input and asks a yes/no question and dumps the output.

# ONTAP_ACL /home
Do you want to expand the ACL masks to show all fields? (enter 1 or 2)
CAUTION: Output may be lengthy

1) Yes
2) No
#? 2


 Vserver: DEMO
 File Path: /home
 File Inode Number: 64
 Security Style: mixed
 Effective Style: ntfs
 DOS Attributes: 10
 DOS Attributes in Text: ----D---
Expanded Dos Attributes: -
 UNIX User Id: 0
 UNIX Group Id: 1
 UNIX Mode Bits: 711
 UNIX Mode Bits in Text: rwx--x--x
 ACLs: NTFS Security Descriptor
 Control:0x9504
 Owner:NTAP\Administrator
 Group:NTAP\Domain Users
 DACL - ACEs
 ALLOW-NTAP\Administrator-0x1f01ff-OI|CI
 ALLOW-Everyone-0x100020-OI|CI

I could also apply a wrapper to other scripts, such as the script to modify ACLs if I chose. This allows a simple way to centrally manage your file and folder ACLs  rather than having to jump between clients and storage.

Testing the scripts

Be sure to test the scripts only in protected environments, rather than on your production data. Make generous use of ONTAP features such as FlexClone, SnapMirror (to new Storage Virtual Machines) and Snapshots.

Questions? Leave them in the comments!

 

Mixed perceptions with NetApp multiprotocol NAS access

EDIT: As the original post for this was super long, I’ve since broken it up into a 2 part post. I moved the vserver security information to the following post:

Managing ACLs via the ONTAP Command Line

NetApp’s ONTAP operating system is one of the few storage operating systems out there that supports data access from both CIFS/SMB and NFS clients. NetApp’s been doing this for a long time – longer than I’ve been there, and I’m going on 10 years!

Despite the fact that it’s been around so long and is one of *the* core competencies in ONTAP, it’s one of the most frequently misunderstood configurations I see. When I was in support, it was one of the biggest case generators. As the NFS TME, it’s one of the most common emails I get that customers need assistance on.

I can tell you what it’s not….

Multiprotocol NAS is NOT “Mixed Mode”

Many people use this terminology for describing access from multiple clients. Unfortunately, it only adds to the confusion, because there is also a security style called “mixed” (see below) and that makes people associate the two and then they start setting mixed security styles when they don’t need to…

So, call it what it is – Multiprotocol NAS. 🙂

What’s so hard about it?

The reason it seems to confound so many people is two fold:

  • Windows administrators are generally not UNIX-savvy
  • UNIX administrators are generally not Windows-savvy

To truly understand multiprotocol NAS, you either have to know both Windows and UNIX file systems/security sematics pretty well, or be open to the fact that Windows and UNIX have similarities and differences.

That said, when you do understand how it works and get it configured properly, it’s a pretty powerful tool for serving data for multiple client types.

There’s currently a Multiprotocol TR in the works, but will be a ways out. However, I just dealt with a recent multiprotocol NAS issue and wanted to do a brain dump before the information got stale and I had to revisit it. This blog is intended to be a quick hit guide to multiprotocol NAS in ONTAP. Some of the ideas will make their way into official TR format.

What makes multiprotocol NAS possible in ONTAP?

ONTAP is fairly agnostic when it comes to file systems and ACL styles. SMB and NFS clients use different security semantics, but the general concepts of those are the same.

Users, groups, permissions.

From there, things tend to skew a bit. Windows uses NTFS security concepts. NFS clients use mode bits for NFSv3/NFSv4.x or ACLs for NFSv4.x. NFSv3 had the concept of POSIX ACLs, but ONTAP doesn’t support those.

The issue is that NTFS ACLs are more complex than mode bits, but match up pretty nicely with NFSv4.x ACLs. Mode bits only do Read, Write, eXecute (RWX), so Windows ACLs don’t match up 1 to 1, especially when you have “special permissions” in the mix. As a result, when dealing with ONTAP file systems, we have the concept of a security style that helps us choose the style of ACL we want to implement. The choices we have:

  • NTFS – NTFS ACLs only
  • UNIX – UNIX style permissions only
  • Mixed – UNIX or NTFS permissions, depending on who last changed permissions
  • Unified (Infinite Volume only)

To properly address permissions, ONTAP has to pick one security style over the other. This allows the storage system to decide which direction a user will map to determine the correct permissions. After all, what’s the point of permissions if they don’t work properly?

User mapping

ONTAP is not unique in the concept of user mapping, but it is still a concept that gets people confused on occasion.

Essentially, to get the proper permissions on a NetApp storage system, a client must first pass a “test” in the form of initial authentication.

The initial test is “Who are you?”

The storage system needs to know that the user you are claiming to be is actually you. There are varying degrees of how secure this test is, mostly dependent on the protocol you’re using, but the bottom line is this: authentication helps us get a user name. That user name allows us to map to another user name, depending on the volume security style.

In general:

  • SMB clients always map to a UNIX user because ONTAP is UNIX-based, even if NTFS security style is in use
  • If no name mapping rules or 1:1 name mappings exist, SMB users map to a default UNIX user set in CIFS options (pcuser/65534 by default)
  • 65534 is “nobody” or “nfsnobody” in most UNIX clients
  • NFS clients only map to Windows users when the security style is NTFS
  • NFS clients cannot chmod or chown on NTFS style volumes; SMB clients cannot take ownership or change ACLs on UNIX style volumes

Once a user has authenticated, the permissions can be discerned based on access control lists. One can see those ACLs via the CLI of the storage system with “vserver security file-directory show.”

cluster::*> vserver security file-directory show -vserver parisi -path /cifs

                 Vserver: parisi
               File Path: /cifs
       File Inode Number: 64
               Security Style: ntfs
         Effective Style: ntfs
          DOS Attributes: 10
  DOS Attributes in Text: ----D---
 Expanded Dos Attributes: -
            UNIX User Id: 0
           UNIX Group Id: 0
          UNIX Mode Bits: 777
  UNIX Mode Bits in Text: rwxrwxrwx
                    ACLs: NTFS Security Descriptor
                          Control:0x8004
                            Owner:BUILTIN\Administrators
                            Group:BUILTIN\Administrators
                            DACL - ACEs
                             ALLOW-Everyone-0x1f01ff
                             ALLOW-Everyone-0x10000000-OI|CI|IO

User/name mapping is one of the most important pieces of the multiprotocol NAS puzzle. Get that part right and most everything else is easy.

Name mapping can be done either locally (via name mapping rules) or with LDAP. TR-4073 covers this sort of thing in pretty finite detail.

Name services/LDAP

The easiest way to handle name mapping in ONTAP for multiprotocol NAS is to leverage a name service server like LDAP. When dealing with both SMB and NFS, the most logical choice is to use the existing Active Directory infrastructure to host UNIX identities. While you can host name mapping rules for users that don’t have the same UNIX and Windows names, it’s best to try to have UNIX and Windows user names match 1:1. (I.e., DOMAIN\nfsdudeabides == nfsdudeabides in UNIX).

TR-4073 covers LDAP and TR-4379 covers name service best practices, for your reference.

Mixed Security Style

Fun fact – Mixed security style isn’t truly “mixed.” When you use mixed security style, it’s always either NTFS or UNIX security style at any given moment. This is known as the “effective” security style, which can be seen in “vserver security file-directory show.”

cluster::*> vserver security file-directory show -vserver parisi -path /cifs

                 Vserver: parisi
               File Path: /cifs
       File Inode Number: 64
               Security Style: ntfs
         Effective Style: ntfs

The “effective” style changes based on  the last permission change. If an NFS client does a chmod or chown, the mixed security style volume changes to effective UNIX security style. If an SMB client changes owner or sets an ACL, the effective security style changes to NTFS. When these effective styles change, how the storage does name mapping changes (ie; win-unix to unix-win, etc).

Is mixed security style recommended?

Generally speaking, you don’t want file systems changing something behind the scenes without the knowledge of the storage administrators. Plus, these changes can affect functionality, and even access. As a result, mixed security style is generally not recommended. The only time you’d want to use mixed security style is if your environment requires the ability for clients or applications to change permissions from both NFS and SMB. And even then, if you do set up mixed security style, consider limiting the ability for regular users to take ownership or change permissions on folders and files via NTFS ACLs.

Otherwise, I personally recommend picking either NTFS or UNIX and sticking with it. That choice would be based on how you want your users to manage their ACLs, as well as how granular you want control to be on those file systems. For example, mode bits in UNIX only allow setting an owner, group and everyone else. There’s no way to set multiple groups with different access on the object unless you use NFSv4 or NTFS ACLs.

I usually prefer NTFS because you get the granularity, as well as the GUI functionality many users are accustomed to.

If you do decide to use mixed security style, keep the following in mind:

  • If a volume is using mixed security style and the effective style gets flipped from NTFS to UNIX and then back to NTFS by way of the clients, the previous NTFS ACLs are lost.
  • When a volume flips from UNIX effective to NTFS effective, you get the mode bit translation. For example, if the UNIX volume was 755, you get “Owner – Full Control” and “Everyone – Read/Execute” as Windows ACLs. 700 gives “Owner – Full Control” only.
  • Administrator always gets added onto the ACL with Read/Write access when we flip to NTFS from UNIX.
  • With mixed security style, there are two types of owners – UNIX owner and Windows owner. When Windows “takes ownership,” the UNIX owner does not change.
  • When the effective style of the volume is NTFS, UNIX clients will see permissions as 777 unless the NFS server option ntacl-display-permissive-perms is set to “disabled.”

For information on how to manage permissions in ONTAP, see the following post:

Managing ACLs via the ONTAP Command Line

Be on the lookout for a multiprotocol TR in the future that covers this and more!

Got any questions? Feel free to post in the comments!

TECH::Register for #NetAppInsight 2015 today!

Every year, NetApp holds a technical conference for storage and data management professionals called NetApp Insight. For the past 2 years, I’ve presented at these conferences for my technical field (NFS/multiprotocol). This year, I will be presenting again in Las Vegas and Berlin!

(If you register by July 31, you get a “super saver” discount.)

What sessions am I doing?

This year, I’ll be focusing solely on multiprotocol in clustered Data ONTAP in an attempt to help people better understand the myriad questions surrounding the use SMB and NFS on the same unified data storage. I’ll be doing a session on general multiprotocol best practices and operation called:

1884 –  Unlocking the Mysteries of Multiprotocol NAS in Data ONTAP

Having trouble understanding multiprotocol? Can’t figure out the difference between authentication and authorization? Do you think a name mapping rule is something doctors provide to new parents at the hospital? Wondering what a “CIFS/SMB” and “NFS” is anyway? This session can help! We cover all things NAS in clustered Data ONTAP, from the basics to the best practices, and try to help make multiprotocol a little clearer for storage administrators.

For a deeper understanding of the inner workings of authentication in clustered Data ONTAP (which is integral to multiprotocol), I’ll be doing a session called:

1881 –  Authentication in Clustered Data ONTAP: SecD Deep Dive

Authentication in clustered Data ONTAP uses the security daemon (SecD) for connectivity to external dependencies for NAS protocols such as Active Directory, LDAP, NIS and DNS. This session will cover how SecD works in clustered Data ONTAP 8.3 and beyond, as well as how to troubleshoot issues and what kind of statistics are available for SecD and external name services.

In addition, I will be offering a Hands On Lab for Multiprotocol, where you can play with LDAP and NFS/SMB access on clustered Data ONTAP in real time!

Schedules will be available soon, so sign up early to ensure you get a spot in one of my sessions and the lab!

Check back here periodically for any updates and links to the sessions.

TECH::Sneak preview! Converting 7-Mode NAS Troubleshooting Commands to clustered Data ONTAP Commands

It’s Technical Report revision time again, as we await the next release of clustered Data ONTAP – 8.3.1.

While writing my new revision of TR-4073: Secure Unified Authentication, I was making a table of commands for NAS troubleshooting that I converted from 7-Mode to cDOT. The TR won’t be available for a bit, but I can at least post the table here for you to peruse/use. Check back on occasion for updates. If you have things you want me to add, comment here or follow me on Twitter @NFSDudeAbides!

7-Mode Command Clustered Data ONTAP 8.3.x Command What it does
exportfs –c vserver export-policy check-access Verifies if a specific host has access to a specific mount and what level of access it has.
exportfs –f vserver export-policy cache flush Flushes the exports cache on a local node. Use with caution, as flushing cache causes it to need to be repopulated.
fsecurity show vserver security file-directory show Show the file owner, ACLs, permissions, etc. from ONTAP CLI.
getXXbyYY getXXbyYY Allows simulation of external name service queries from ONTAP CLI. (ie, LDAP, NIS)
ifstat node run ifstat Shows physical network port statistics.
lock status/break vserver locks show/break Shows and breaks NFS or CIFS locks on files. Use with caution.
nbtstat vserver cifs nbtstat Shows NetBIOS information.
netstat node run netstat

network connections active show

network connections listening show

Shows network status, port information (listening, open, etc).
nfs diag show

access_cache

nsdb_cache

diag exports nblade access-cache show

diag nblade credentials show

Show diagnostic level NFS information. Use with caution.
nfs nsdb flush diag nblade credentials flush Flush the nsdb (Name Services Database) cache.
nfs_hist N/A Shows nfs histogram.
nfsstat statistics start -object nfs*/statistics stop Shows NFS related statistics.
nis info nis show-statistics

nis show-bound-debug

Shows NIS information.
options nfs.mountd.trace logger mgwd log modify -node -module mgwd::exports -level debug Shows trace output of mount requests. Use with caution and always disable after use. Logs to /mroot/etc/mlog/mgwd.log.
options cifs.trace_login diag secd trace set -node -trace-all yes

diag secd log set -node -level debug -enter-exit on

Shows trace output of name mapping, CIFS logins,etc. Use with caution and always disable after use. Logs to /mroot/etc/mlog/secd.log.
ping network ping Runs ping.
pktt node run [nodename] pktt Collects a packet capture. For details on packet traces in clustered Data ONTAP, see the following knowledge base article.

How to collect packet traces in clustered Data ONTAP

route network route Create/show/delete routes.
sectrace vserver security trace filter Used to troubleshoot security/permissions issues. For more information, see the following knowledge base article:

How to troubleshoot Microsoft Client permission issues on a NetApp Vserver running Clustered Data ONTAP

showfh node run [nodename where file lives] “priv set diag; showfh” Shows the file handle, FSID, etc for files and folders.
traceroute traceroute Trace routes between devices.
wcc diag nblade credentials

diag secd authentication show-creds

Show/flush credential stores on local nodes. Use with caution.
ypcat N/A Dumps contents of entire NIS map.
ypmatch diag secd authentication show-ontap-admin-unix-creds Fetches NIS object’s attributes.
ypwhich diag secd connections show -node [nodename] -vserver [SVM] -type nis Shows the NIS map server/master.

In addition, check out the Command Map for 7-Mode administrators, as well as How 7-Mode Configuration Files Map to cDOT commands and How 7-Mode Options Map to cDOT commands.

TECH::Multiprotocol NAS, Locking and You

Bank_Vault_3D_Wallpaper-HD

One question I got today, and have gotten a few times in my years as a NAS guy, was “how does locking work when you are sharing files between CIFS/SMB and NFS?”

Essentially, “can you guarantee my data will be safe?”

Well, the first answer to that question is a question: What is a file lock?

File locks explained

Essentially, a file lock is exactly what it sounds like. We all know what locks are; we have them on our doors. They keep unwanted people and things out. A file lock is no different; it’s a way to prevent unwanted people and applications from accessing files while they are in use to prevent the “c” word – CORRUPTION!

ohfudge

File locks are always issued by the requesting client or application – a NAS will only honor or deny the lock. If the client or application does not issue a lock to the file, all bets are off. This is similar to our door locking analogy – a door will not lock unless you turn the key.

Similarly, a lock will only be safely broken or released when the application or client is done with it. If the client or application dies, the lock needs to be cleaned up. Depending on the NAS protocol/protocol version, this may or may not require manual intervention. For example, if NFSv3 locks are left over by an application crash (such as an Oracle database), then the locks must be manually cleared on the server before the application can be restarted to use the same files. This is because locking in NFSv3 is handled via the Network Lock Manager (NLM), which is an ancillary protocol to NFS and doesn’t always play well with others.

Conversely, NFSv4.x locking is integrated with the protocol and is lease-based. With leases, the locks will live for a pre-determined amount of time. If the client doesn’t renew the lock in that amount of time, the locks will expire. If the client or application crashes, the locks will release on their own after the lease period expires. If the NFS server restarts, the locks will remain intact until either a client reclaims them or the lease expires.

From RFC-5661:

Lease: A lease is an interval of time defined by the server for which the client is irrevocably granted locks. At the end of a lease period, locks may be revoked if the lease has not been 
extended. A lock must be revoked if a conflicting lock has been granted after the lease interval

A server grants a client a single lease for all state.

Simple enough. But what many people don’t know is that there are also different types of file locks.

File Locking in CIFS/SMB

Special thanks to NetApp CIFS/SMB TME Marc Waldrop (@CIFSorSMB on Twitter) for the CIFS/SMB file lock sanity check.

Locks in CIFS/SMB are done either at a share level or a file level. A share lock will dictate what level of access is allowed to the open file while the original opener of the file has the file opened. The share level lock, commonly known as share access mode, will dictate whether additional openers of the file can read or write to the file. The share access mode can lock the entire file from additional clients doing specific read or write operations. A file level lock is what most know as byte-range lock.

File locking in CIFS/SMB is done via oplocks and share locks. When a file is opened for editing, an oplock is applied and the share-level lock is modified to control what access a client or application has to the file. In some cases, classic byte-range locks are used when portions of a file need to be locked.

CIFS/SMB uses three types of “opportunistic locks,” or “oplocks.”

  1. Batch locks
    Created for batch files to help performance issues with files that are opened and closed many times.
  2. Exclusive locks
    Used when an application opens a file in “shared” mode; for example, Microsoft Office. Exclusive locks allow a client to assume they are the only ones using a file and will cache all changes locally. This is where that weird looking ~FILE.doc comes from in Word. If another application requests an exclusive lock on the file, the server will invalidate the original lock and the application will flush the cached changes to the file.
  3. Level 2 Oplocks
    These get issued when multiple clients want to access the same file. When an application gets issued a Level 2 Oplock, multiple clients can read/cache reads of a file. If any client attempts a write, that client gets issued an exclusive lock.

File Locking in NFS

Unlike CIFS/SMB, NFS doesn’t do share-level locking. It only does file locking. NFS locking comes in two flavors:

  1. Shared locks
    Shared locks can be used by multiple processes at the same time and can only be issued if there are no exclusive locks on a file. These are intended for read-only work, but can be used for writes (such as with a database). This is similar to the Level 2 Oplock in CIFS/SMB.
  2. Exclusive locks
    These operate the same as exclusive locks in CIFS/SMB – only one process can use the file when there is an exclusive lock. If any other processes have locked the file, an exclusive lock cannot be issued, unless that process was “forked.” However, unlike CIFS/SMB, there isn’t a notion of “opportunistic” locking, where a file will allow access without outside intervention.

One of the best analogies I’ve seen for this is a real-world example on stackoverflow:

I wrote this answer down because I thought this would be a fun (and fitting) analogy:

Think of a lockable object as a blackboard (lockable) in a class room containing a teacher (writer) and many students (readers).

While a teacher is writing something (exclusive lock) on the board:

1. Nobody can read it, because it’s still being written, and she’s blocking your view => If an object is exclusively
          locked, shared locks cannot be obtained.

2. Other teachers won’t come up and start writing either, or the board becomes unreadable, and confuses
students => If an object is exclusively locked, other exclusive locks cannot be obtained.

When the students are reading (shared locks) what is on the board:

1. They all can read what is on it, together => Multiple shared locks can co-exist.

2. The teacher waits for them to finish reading before she clears the board to write more => If one or more
          shared locks already exist, exclusive locks cannot be obtained.

The reason I prefer the CIFS/SMB notion of locking is that it feels a lot less messy in general and there is less overhead/management involved with the locking. This is especially true with NFSv3, where locking is not integrated into the protocol, but instead uses an ancillary process called NLM. (as mentioned above)

NFSv4.x and later realized this folly and have integrated locking into the protocol. Locking is better now in NFS, and applications are starting to adopt NFSv4.x as a standard because of the improved locking mechanisms.

Locking in multiprotocol environments

Multiprotocol in NAS simply means “ability to access the same datasets from multiple protocols.” Thus, CIFS/SMB and NFS can read and write to the same files.

This can be problematic for a few reasons:

  • CIFS/SMB and NFS permissions are not the same
    NFS, especially NFSv3, uses mode bit permissions (such as 777, 755, etc). NFSv4.x implements ACLs, which look and feel an awful lot like Windows NTFS ACLs. They’re so close, in fact, that it solves a lot of the old permissioning issues you would see in multiprotocol environments. But they don’t solve all of our problems.
  • CIFS/SMB and NFS user/group concepts are not the same
    Windows users and groups use a super long Security Identifier (SID) for unique identifiers, which is constructed by leveraging the domain SID and unique user/group Relative ID (RID). NFS users and groups use a numeric UID/GID and/or NFSv4 ID domain string. In order to get a user to leverage the correct security permission structure, a name mapping has to take place, depending on the originating request + the type of ACL on the file or folder. Unified name services, such as LDAP, are often useful in alleviating the pain of this scenario.
  • CIFS/SMB and NFS file locks are not the same
    As described above. Because of this, the underlying file system has to be able to negotiate the file locking. As it so happens, NetApp invented integrated locking in the 1990s.

So, how do NAS vendors (like NetApp) get this to work?

In ONTAP (and, by proxy, WAFL), the file system owns the locks and gets the final say as to who gets what access. When a CIFS/SMB client grants an exclusive lock to a file, a NFS client that tries to get a file lock to that same file would not be granted access until the lock has been released. The same goes for NFS clients that have an exclusive lock on a file – CIFS/SMB can’t do anything with that file until the locks is broken/released. These locks are only released when the original client releases them, either voluntarily or by lease expiration.

The general idea here is, protocols don’t matter – protect the files at all cost. If I buy a deadbolt lock, I should be able to use it on any door I choose and it should keep my house safe.

In ONTAP, you can check to see if a file has a lock via the command line or API calls.

In Data ONTAP operating in 7-Mode, the commands are:

7mode> lock status
lock status -f [file] [-p protocol] [-n]
lock status -h [host [-o owner]] [-f file] [-p protocol] [-n]
lock status -o [-f file] [-p protocol] [-n]
lock status -o owner [-f file] [-p protocol] [-n] (valid for CIFS only)
lock status -p protocol [-n]
lock status -n

In clustered Data ONTAP, the command is:

cluster::> vserver locks show ?
 [ -instance | -smb-attrs | -fields , ... ]
 { [ -vserver  ] Vserver
 [[-volume] ] Volume
 [[-lif] ] Logical Interface
 [[-path] ] Object Path
 | [ -lockid  ] } Lock UUID
 [ -protocol  ] Lock Protocol
 [ -type {byte-range|share-level|op-lock|delegation} ] Lock Type
 [ -node  ] Node Holding Lock State
 [ -lock-state  ] Lock State
 [ -bytelock-offset  ] Bytelock Starting Offset
 [ -bytelock-length  ] Number of Bytes Locked
 [ -bytelock-mandatory {true|false} ] Bytelock is Mandatory
 [ -bytelock-exclusive {true|false} ] Bytelock is Exclusive
 [ -bytelock-super {true|false} ] Bytelock is Superlock
 [ -bytelock-soft {true|false} ] Bytelock is Soft
 [ -oplock-level {exclusive|level2|batch|null|read-batch} ] Oplock Level
 [ -sharelock-mode  ] Shared Lock Access Mode
 [ -sharelock-soft {true|false} ] Shared Lock is Soft
 [ -delegation-type {read|write} ] Delegation Type
 [ -client-address  ] Client Address
 [ -smb-open-type {none|durable|persistent} ] SMB Open Type
 [ -smb-connect-state  ] SMB Connect State
 [ -smb-expiration-time  ] SMB Expiration Time (Secs)
 [ -smb-open-group-id  ] SMB Open Group ID

Testing file locking between protocols

This can be tricky. I’ve seen a lot of people try to test multiprotocol file locking in the following manner:

  • Open a file in a CIFS/SMB share with notepad in Windows.
  • Go to the NFS client. Open the file with vi.
  • Wonder why the heck writes are allowed on both. Bang head repeatedly on desk. Destroy a copier.

office-space-copier

The reason why writes are allowed on both clients in this scenario is because NEITHER APPLICATION HAS ISSUED A LOCK. File locks are the responsibility of the client and/or application, not the server. Vi and notepad don’t lock files!

Testing locks from NFS when CIFS/SMB owns the lock

The easiest way to lock a file in Windows? Microsoft Office.

When I open a MS Word file in a CIFS/SMB share on cDOT, I get some share locks and a batch oplock:

cluster::> vserver locks show -vserver NAS -volume unix 

Vserver: NAS
Volume   Object Path               LIF         Protocol  Lock Type   Client
-------- ------------------------- ----------- --------- ----------- ----------
unix     /unix/                    data1       cifs      share-level 10.228.225.120
               Sharelock Mode: read-deny_none
               Sharelock Mode: read-deny_none
                                                                     10.62.194.166
               Sharelock Mode: read-deny_none
         /unix/office.docx                               op-lock     10.62.194.166
               Oplock Level: batch
                                                         share-level 10.62.194.166
               Sharelock Mode: read_write-deny_write_delete

In some cases, I may see an exclusive oplock, especially during edits. I can generate an exclusive oplock if another CIFS/SMB client attempts to access the doc with something like WordPad and gets denied access:

Vserver: NAS
Volume   Object Path               LIF         Protocol  Lock Type   Client
-------- ------------------------- ----------- --------- ----------- ----------
unix     /unix/                    data1       cifs      share-level 10.228.225.120
               Sharelock Mode: read-deny_none
               Sharelock Mode: read-deny_none
                                                                     10.62.194.166
               Sharelock Mode: read-deny_none
         /unix/office.docx                               op-lock     10.62.194.166
               Oplock Level: exclusive
                                                         share-level 10.62.194.166
               Sharelock Mode: read_write-deny_write_delete

If I use another instance of MS Word on a separate client, I can get a level 2 oplock:

Vserver: NAS
Volume   Object Path               LIF         Protocol  Lock Type   Client
-------- ------------------------- ----------- --------- ----------- ----------
unix     /unix/                    data1       cifs      share-level 10.228.225.120
               Sharelock Mode: read-deny_none
               Sharelock Mode: read-deny_none
                                                                     10.62.194.166
               Sharelock Mode: read-deny_none
         /unix/office.docx                               op-lock     10.62.194.166
               Oplock Level: level2
                                                         share-level 10.62.194.166
               Sharelock Mode: read_write-deny_write_delete

With this Office file open, I want to test to see if my NFS client can access/write to the file. When I look at the share with “ls,” I can see that funny ~filename listed.

# ls | grep office
~$office.docx
office.docx

As I mentioned, vi is a terrible way to test locks. However, Linux clients have utilities to test file locking in NFS. In CentOS/RHEL, one utility to use is flock. With flock, I can run a command to lock a file. To be cute, I use vi. 😉

When I try to get an exclusive lock on that file, it hangs:

# flock --exclusive office.docx vi

Since I’m doing NFSv3, I can check to see if any NLM locks have been issued on my cDOT Storage Virtual Machine. I can see that I have been issued a byte-range lock, but since the command hung, I can’t do any damage to the file:

cluster::> vserver locks show -vserver NAS -volume unix -protocol nlm

Vserver: NAS
Volume   Object Path               LIF         Protocol  Lock Type   Client
-------- ------------------------- ----------- --------- ----------- ----------
unix     /unix/office.docx         data1       nlm       byte-range  10.228.225.140
                Bytelock Offset(Length): 0 (18446744073709551615)

When I try to get a shared lock to that file, it allows me in:

~ VIM - Vi IMproved 
~ 
~ version 7.2.411 
~ by Bram Moolenaar et al. 
~ Modified by <bugzilla@redhat.com> 
~ Vim is open source and freely distributable 
~ 
~ Become a registered Vim user! 
~ type :help register for information 
~ 
~ type :q to exit 
~ type :help or  for on-line help 
~ type :help version7 for version info

And my SVM grants a byte-range lock:

cluster::> vserver locks show -vserver NAS -volume unix -protocol nlm

Vserver: NAS
Volume   Object Path               LIF         Protocol  Lock Type   Client
-------- ------------------------- ----------- --------- ----------- ----------
unix     /unix/office.docx         data1       nlm       byte-range  10.228.225.140
                Bytelock Offset(Length): 0 (18446744073709551615)

But when I try to write, it fails.

E32: No file name

So, that’s good, right? I close the Word doc out and all the file locks are gone. Only share locks remain:

cluster::> vserver locks show -vserver NAS -volume unix
Vserver: NAS
Volume   Object Path               LIF         Protocol  Lock Type   Client
-------- ------------------------- ----------- --------- ----------- ----------
unix     /unix/                    data1       cifs      share-level 10.228.225.120
               Sharelock Mode: read-deny_none
               Sharelock Mode: read-deny_none
                                                                     10.62.194.166
               Sharelock Mode: read-deny_delete
3 entries were displayed.

Testing locks from CIFS/SMB when NFS owns the lock

From my NFS client, I lock one of the files in the share with an exclusive lock. In this case, I use “newfile,” because vi has no idea what to do with an Office doc.

# flock --exclusive newfile vi

And I can see the new byte-range lock on the SVM:

cluster::> vserver locks show -vserver NAS -volume unix
Vserver: NAS
Volume   Object Path               LIF         Protocol  Lock Type   Client
-------- ------------------------- ----------- --------- ----------- ----------
unix     /unix/                    data1       cifs      share-level 10.228.225.120
               Sharelock Mode: read-deny_none
               Sharelock Mode: read-deny_none
                                                                     10.62.194.166
               Sharelock Mode: read-deny_delete
unix     /unix/office.docx         data1       nlm       byte-range  10.228.225.140
                Bytelock Offset(Length): 0 (18446744073709551615)

The expected behavior here? The file will only allow reads. And I am not disappointed!

Screen Shot 2015-05-20 at 12.34.12 AM

What happens if I have a stale lock?

If my NFS client dies, for whatever reason, and that byte-range lock is still lingering, I can break it from the storage in advanced privilege (complete with scary/legit warning):

cluster::*> vserver locks break -vserver NAS -volume unix -lif data1 -path /unix/newfile

Warning: Breaking file locks can cause applications to become unsynchronized and may lead to data corruption.
Do you want to continue? {y|n}: y
1 entry was acted on.
cluster::> vserver locks show -vserver NAS -volume unix
Vserver: NAS
Volume   Object Path               LIF         Protocol  Lock Type   Client
-------- ------------------------- ----------- --------- ----------- ----------
unix     /unix/                    data1       cifs      share-level 10.228.225.120
               Sharelock Mode: read-deny_none
               Sharelock Mode: read-deny_none
                                                                     10.62.194.166
               Sharelock Mode: read-deny_delete
3 entries were displayed.

Once that happens, I can issue new locks from other clients, whether they are CIFS/SMB or NFS. Doesn’t matter – multiprotocol locking in ONTAP just works!

Where can I find out more?

For more information on NFS in clustered Data ONTAP, see TR-4067: NFS Best Practice and Implementation Guide.

For more information on assorted aspects of multiprotocol NAS access in clustered Data ONTAP, see TR-4073: Secure Unified Authentication.

Additionally, subscribe to Why is the Internet Broken and follow @NFSDudeAbides on Twitter for more NAS-related information!

TECH::Docker + CIFS/SMB? That’s unpossible!

docker-smb-ralph

Recently, I’ve been playing with Docker quite a bit more, trying to educate myself on what it can and cannot do and where it fits in to NetApp and file services/NAS.

I wrote a blog on setting up a PaaS container that can do Firefox over VNC (for Twitter, of all things), as well as one on using NFS in Docker. People have asked me (and I have wondered), what about CIFS/SMB? Now, we could totally do this via the Linux container I created via mount -t cifs or Samba. But I’m talking about Windows-based CIFS/SMB.

Microsoft supports Docker?

Recently, Microsoft issued an announcement that it will be integrating Docker into Windows Server and Windows Azure, as well as adding Server container images in Docker hub. In fact, you can find Microsoft containers in GitHub today. But the content is a bit sparse, as far as I could see. This could be due to new-ness, or worse, apathy. Time will tell.

As far as Server containers, it seems that Windows containers won’t support RDP, nor local login. Only PowerShell and WMI, as per this Infoworld article on Microsoft doing a Docker demo. And when I look for PowerShell images, I found just one:

# docker search powershell
NAME DESCRIPTION STARS OFFICIAL AUTOMATED
docker.io: docker.io/solarkennedy/powershell

It would be totally valid to connect to a CIFS/SMB share via PowerShell, but it looks like there’s a bit of work to do to get this image running – namely, running it on a Windows server rather than Linux:

# docker run -t -i --privileged docker.io/solarkennedy/powershell:latest
Application tried to create a window, but no driver could be loaded.
Make sure that your X server is running and that $DISPLAY is set correctly.
Encountered a problem reading the registry. Cannot find registry key SOFTWARE\Microsoft\PowerShell.

Registry errors? That sure looks familiar… 🙂

What about Azure?

Microsoft also has Azure containers out there. I installed one of the Azure CLI containers, just to see if we could do anything with it. No dice. The base OS for Azure appears to be Linux:

# docker run -t -i --privileged docker.io/microsoft/azure-cli:latest
root@b23878ec46c4:/# uname -a
Linux b23878ec46c4 3.10.0-229.1.2.el7.x86_64 #1 SMP Fri Mar 27 03:04:26 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

This is the set of commands I get:

# help
GNU bash, version 4.3.11(1)-release (x86_64-pc-linux-gnu)
These shell commands are defined internally. Type `help' to see this list.
Type `help name' to find out more about the function `name'.
Use `info bash' to find out more about the shell in general.
Use `man -k' or `info' to find out more about commands not in this list.
A star (*) next to a name means that the command is disabled.
job_spec [&] history [-c] [-d offset] [n] or history -anrw [filename] or history -ps arg [arg..>
 (( expression )) if COMMANDS; then COMMANDS; [ elif COMMANDS; then COMMANDS; ]... [ else COMMANDS; >
 . filename [arguments] jobs [-lnprs] [jobspec ...] or jobs -x command [args]
 : kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
 [ arg... ] let arg [arg ...]
 [[ expression ]] local [option] name[=value] ...
 alias [-p] [name[=value] ... ] logout [n]
 bg [job_spec ...] mapfile [-n count] [-O origin] [-s count] [-t] [-u fd] [-C callback] [-c quantum] >
 bind [-lpsvPSVX] [-m keymap] [-f filename] [-q name] [-u name] [-r keyseq] [-x keys> popd [-n] [+N | -N]
 break [n] printf [-v var] format [arguments]
 builtin [shell-builtin [arg ...]] pushd [-n] [+N | -N | dir]
 caller [expr] pwd [-LP]
 case WORD in [PATTERN [| PATTERN]...) COMMANDS ;;]... esac read [-ers] [-a array] [-d delim] [-i text] [-n nchars] [-N nchars] [-p prompt] [->
 cd [-L|[-P [-e]] [-@]] [dir] readarray [-n count] [-O origin] [-s count] [-t] [-u fd] [-C callback] [-c quantum>
 command [-pVv] command [arg ...] readonly [-aAf] [name[=value] ...] or readonly -p
 compgen [-abcdefgjksuv] [-o option] [-A action] [-G globpat] [-W wordlist] [-F fu> return [n]
 complete [-abcdefgjksuv] [-pr] [-DE] [-o option] [-A action] [-G globpat] [-W wordl> select NAME [in WORDS ... ;] do COMMANDS; done
 compopt [-o|+o option] [-DE] [name ...] set [-abefhkmnptuvxBCHP] [-o option-name] [--] [arg ...]
 continue [n] shift [n]
 coproc [NAME] command [redirections] shopt [-pqsu] [-o] [optname ...]
 declare [-aAfFgilnrtux] [-p] [name[=value] ...] source filename [arguments]
 dirs [-clpv] [+N] [-N] suspend [-f]
 disown [-h] [-ar] [jobspec ...] test [expr]
 echo [-neE] [arg ...] time [-p] pipeline
 enable [-a] [-dnps] [-f filename] [name ...] times
 eval [arg ...] trap [-lp] [[arg] signal_spec ...]
 exec [-cl] [-a name] [command [arguments ...]] [redirection ...] true
 exit [n] type [-afptP] name [name ...]
 export [-fn] [name[=value] ...] or export -p typeset [-aAfFgilrtux] [-p] name[=value] ...
 false ulimit [-SHabcdefilmnpqrstuvxT] [limit]
 fc [-e ename] [-lnr] [first] [last] or fc -s [pat=rep] [command] umask [-p] [-S] [mode]
 fg [job_spec] unalias [-a] name [name ...]
 for NAME [in WORDS ... ] ; do COMMANDS; done unset [-f] [-v] [-n] [name ...]
 for (( exp1; exp2; exp3 )); do COMMANDS; done until COMMANDS; do COMMANDS; done
 function name { COMMANDS ; } or name () { COMMANDS ; } variables - Names and meanings of some shell variables
 getopts optstring name [arg] wait [-n] [id ...]
 hash [-lr] [-p pathname] [-dt] [name ...] while COMMANDS; do COMMANDS; done
 help [-dms] [pattern ...] { COMMANDS ; }

There is an Azure command set also, but that seems to connect directly to an Azure cloud instance, which requires an account, etc. I suspect I’d have to pay to use commands like “azure storage,” which is why I haven’t set one up yet. (I’m cheap)

azure-cli

root@b23878ec46c4:/# azure storage share show
info: Executing command storage share show
error: Please set the storage account parameters or one of the following two environment variables to use storage command. 1.AZURE_STORAGE_CONNECTION_STRING, 2. AZURE_STORAGE_ACCOUNT and AZURE_STORAGE_ACCESS_KEY
info: Error information has been recorded to /root/.azure/azure.err
error: storage share show command failed

Whither Windows file services?

The preliminary results of using Docker to connect to CIFS/SMB shares aren’t promising. That isn’t to say it won’t be possible. I still need to install Docker on a Windows server and try that PowerShell container again. Once I do that, I’ll update this blog, so stay tuned!

Plus, it’s entirely possible that more containers will pop up as the Microsoft repository grows. However, I do hope this works or is at least in the plans for Microsoft. While it’s cool to connect to a cloud share via CIFS/SMB and Azure, I’d like to be able to have control over connecting to shares on my private storage, such as NetApp.

TECH::Using PowerShell to back up and restore CIFS shares/NFS exports in NetApp’s clustered Data ONTAP

NOTE: This post covers DR for NAS objects prior to 8.3.1. After 8.3.1, use the new SVM DR functionality if possible.

macrovs-sharepoint-powershell-script-2013-backup-delete[1]

NetApp’s Data ONTAP operating in 7-mode kept all relevant configuration files in its root volume under /etc. These files get read at boot and are used to set up the filer. This included stuff like DNS configuration (resolv.conf), name service switches (nsswitch.conf), initial config (rc file), hosts and other various configuration files.

Another file that is stored in /etc in 7-mode is the file that builds the filer’s CIFS shares each time it is booted – cifsconfig_share.cfg.

This file is essentially a list of CIFS share and access commands that gets sourced each time the system boots. This is what one of those files looks like in 7-mode:

#Generated automatically by cifs commands
cifs shares -add "ETC$" "/etc" -comment "Remote Administration"
cifs access "ETC$" S-1-5-32-544 Full Control
cifs shares -add "HOME" "/vol/vol0/home" -comment "Default Share"
cifs access "HOME" S-NONE "nosd"
cifs shares -add "C$" "/" -comment "Remote Administration"
cifs access "C$" S-1-5-32-544 Full Control
cifs shares -add "CIFS" "/vol/cifs" -comment "CIFS"
cifs access "CIFS" S-NONE "nosd"
cifs shares -add "mixed" "/vol/mixed" -comment ""
cifs access "mixed" S-NONE "nosd"

7mode> cifs shares
Name Mount Point      Description
---- -----------      -----------
ETC$ /etc             Remote Administration
                 BUILTIN\Administrators / Full Control
HOME /vol/vol0/home   Default Share
                 everyone / Full Control
C$ /                  Remote Administration
                 BUILTIN\Administrators / Full Control
CIFS /vol/cifs        CIFS
                 everyone / Full Control
mixed /vol/mixed
                 everyone / Full Control

One benefit of this file in 7-mode was the ability to copy this file off somewhere to back up and possibly restore the shares at a later date, or even retrieve the file from snapshot.

However, with the newer clustered Data ONTAP, the concept of flat files is gone. Everything gets stored in a replicated database, which helps the cluster act like a cluster. I cover that in some detail in a previous post on DataCenterDude.com, NetApp cDOT, RDB, & Epsilon.

Additionally, in clustered Data ONTAP, if a CIFS server gets deleted (such as when removing it from the domain/re-adding it), the CIFS shares get blown away and would need to get re-created one by one.

So what do the people who relied on the old 7-mode CIFS share files do?

Script it out, of course! For more information, including where to find pre-written scripts, see the post on DataCenterDude.com!

Requires powershell module for Data ONTAP, which can be found here: http://mysupport.netapp.com/NOW/download/tools/powershell_toolkit/download.shtml

UPDATE #1:

Recently, a consultant named Scott Harney was inspired by the CIFS share script and not only made some improvements to it, but also created one for NFS exports and rules!

Check it out at his blog:

http://scottharney.com/powershell-scripts-for-backup-of-cdot-nfs-exports/

http://www.datacenterdude.com/storage/backup-restore-cifs-shares-netapp-clustered-data-ontap-powershell/

UPDATE #2 (7/6/15):

Tested the scripts with both 8.2.4 and 8.3.1. Had to work out a few kinks/make some improvements. There is an issue in 8.3.1 with Add-NcCifsShare.

The following changes were made:

  • Tested with 8.2.4 and 8.3.1 cDOT releases
  • Change Import-Module to generic “DataONTAP” to avoid path issues
  • Added link to DataONTAP PS module download in comments
  • Changed PS commands to replace “-Name” with “-Share”
  • Changed output file of ACLs to $aclFile (was $shareFile)

These changes are up on the github repository now. Feel free to notify me if anything else is broken or needs improvement!

https://github.com/DatacenterDudes/cDOT-CIFS-share-backup-restore

If you’re looking for a way to backup Snapmirror schedules, see this link: http://mysupport.netapp.com/NOW/download/tools/smtk/