Welcome to the Episode 348, part of the continuing series called “Behind the Scenes of the NetApp Tech ONTAP Podcast.”
This week on the podcast, NetApp TME John Lantz (ng-ontap-s3-info@netapp.com) joins us to discuss the latest ONTAP S3 updates for ONTAP 9.12.1, including S3/NAS interoperability.
And much more!
We also now have a presence on the NetApp Communities page. You can subscribe there to get emails when we have new episodes.
Finding the Podcast
You can find this week’s episode here:
I’ve also resurrected the YouTube playlist. You can find this week’s episode here:
You can also find the Tech ONTAP Podcast on:
I also recently got asked how to leverage RSS for the podcast. You can do that here:
http://feeds.soundcloud.com/users/soundcloud:users:164421460/sounds.rss
Transcription
The following transcript was generated using Descript’s speech to text service and then further edited. As it is AI generated, YMMV.
Episode 348: NetApp ONTAP 9.12.1 – S3 Updates – Transcript
Justin Parisi: I’m here in the basement of my house and with me today on the phone, John Lantz is here. So John, what do you do here at NetApp and how do I reach you?
John Lantz: Hey Justin, my name’s John Lantz. I am a TME here at NetApp. I pretty much do all things objects, so you might know me from FabricPool, which uses object in the backend and now, ONTAP S3 as well. Best way to reach me is by email. I try to stay off the socials as much as I can. So, sorry. No Twitter, Twitch for you.
Justin Parisi: That’s all right. So would you say that people often treat you like an object?
John Lantz: Frequently. I have been objectified.
Justin Parisi: It’s terrible. It’s terrible to be objectified. So, we’re here to talk about S3 and object storage with ONTAP, and before we get into what’s new in the latest release of ONTAP, I’d like to just kind of talk about S3 and what it is and why people are using it and kind of some of the background in ONTAP with it. So take it away John.
John Lantz: Sure. So I guess S3 in general has been around for a long time, so I guess – reset – is ONTAP, when we think of ONTAP, we think of NAS and probably SAN. So NAS is gonna be files and you’re using file-based protocols: NFS, SMB, SAN, iSCSI, fiber channel. And it’s all block based with, with SAN. Objects are something totally different. And so what we’ve seen in the last couple years, it’s really good for reading more than writing.
And it’s not really known for performance, but it is known for scalability. And so in the olden days, I think people go cheap and deep and they just use like archival storage and they target, object there because it’s really good at just, scaling out at scales, like way beyond what we could do, even with ONTAP in a single cluster.
We’re seeing a lot of applications nowadays. Kind of with the growth of Amazon S3, lots of client applications are doing basically additional workloads with existing data sets. So it’s not just an archival kind of use case. Actually that’s kind of gone by the wayside.
We still see that for sure. But it’s more of like active archives where it’s like, maybe it’s not in my screaming NFS, or fiber channel workloads. But it’s still something that I want to do work with. So that could be like analytics, A!, ML, lots of use cases there.
And so that’s what we’re seeing a lot with object storage or really S3 just because customers got comfortable doing that in Amazon. And I guess technically, Microsoft blob storage similar kind of use cases with cold data sets that you could get more work out of. And then those kind of migrated into not so cold data sets.
And nowadays we just see straight up it was built as a S3 client first and they weren’t even thinking about file or block kind of protocols. And so that’s kind of where ONTAP S3 kind of came from, is we had, the ability to do hybrid systems where it’s like I’m doing file and block in the same system, or multi-protocol in the same volume where it’s like I’m using NFS and SMB in the same volume.
We had a customer saying, Hey, everybody does S3 or a lot of folks are doing S3, or specifically me – the customer – today wants to do S3. How come I can’t do that on ONTAP? I can do everything else except for S3. And so that changed back in 9.8. And so now you can do S3 inside of ONTAP?
Just going to where the customers were, where they’re saying, "Hey, I have native S3 client applications. I want to use it to do that." Likewise, FabricPool, completely different product. That’s our tiering product inside of ONTAP where it’s just like, "Hey, I have cold data.
Why would I keep it on my expensive SSDs if I can move it somewhere else, but still having some access to it?" Customers started using ONTAP S3 as an endpoint for that as well. And so those are kind of our big use cases right now with ONTAP. So that was back in 9.8 was when we launched. We’re talking about 9.12.1 in today’s podcast.
And so it’s been around for a while. And the big use cases today are just using it as a FabricPool endpoint where it’s a very dumb implementation, but it’s extremely high performance. And it’s, massively reduces total cost of ownership. Or, more recently, just straight up native S3 client applications ’cause there’s a ton of those out there nowadays where they were just built, natively in the cloud kind of thing.
And , what’s even faster than the cloud is OnPrem. And so we’re having customers using S3 on, ONTAP now.
Justin Parisi: So we already have an S3 provider with StorageGRID at NetApp. So when would I choose something like an ONTAP versus something like a StorageGRID? Where do they kind of converge and where do they differ?
John Lantz: That’s an excellent question. So like, and to be clear, so StorageGRID is predates ONTAP S3 or, running S3 inside of ONTAP as our kind of premier object storage. And honestly, that is not changing. If anything, we’re gonna see these kind of converge and get better in terms of how they work together.
But StorageGRID remains like that’s our flagship object store. And the scale of StorageGRID is phenomenal certainly greater than the scale. What we see in ONTAP clustering ONTAP, honestly, most customers are like, a two node HA pair. And then we have a lot of a smaller subset of customers that are using 24 nodes.
But that’s where we max out, is a 24 node cluster. And the grid, we have customers using StorageGRID where literally the sun never sets on their grid. It’s so large. 680, I think petabytes is what they can kind of scale out to versus, you can’t fit that under 24 nodes.
And so those are I think the big differences of, they have a larger capacity, a more unified space instead of just being cluster based. Like that grid goes across so many different nodes and their ILM policies or, information lifecycle management, is significantly better.
I mean, it, it was built its object store from the ground up. It remains our flagship object store. For new customers, I think that aren’t coming as, object first. If you’re coming object first, you’re probably gonna look at StorageGRID.
But if you’re looking at, Hey, I already have an ONTAP system and I wanna tinker around or run some smaller scale things, ONTAP S3 is really good. The flip side of having this ultimate capacity out in StorageGRID, 200 node grids and stuff is, even the smallest grid, I think is like three nodes, and then you need some, some admin.
So you probably have like a, SG100 load balancer, something like that. So there’s a not insignificant footprint that you have to start with, with StorageGRID. And that’s not the case with ONTAP, where it’s just like, "Hey, I want to create a bucket inside my existing ONTAP system."
No one creates, a 20 terabyte StorageGRID because it’s way too small. You can a hundred percent do that, at the snap of your fingers with ONTAP S3. So I think that’s where they are today. One is like massive at scale, ultimate object store versus the other’s just, all ONTAP all the time, runs anywhere ONTAP, does using the same GUI CLI, et cetera that you’re used to just now, it speaks the S3 protocol. I think, roadmap kind of things is, we’re looking at those working better together as we move forward. They already kind of work together today in terms of, some of our data protection inside of ONTAP S3 in terms of, basically mirroring like a bucket that’s in ONTAP, out to the grid.
And we just like to improve that over time. So yeah, they are different. But I think they’re actually gonna work closer together as we move forward in the future. Rather than being, two kind of separate entities.
So it sounds
Justin Parisi: like, with S3, with StorageGRID, you’re looking at if you’re massive scale, native S3 not really doing anything else. With ONTAP, it’s if you already have it in place or you want to use other protocols in addition to S3, that would be more like what you’re looking for, especially if you’re not looking for that same kind of scale.
John Lantz: Exactly. Exactly. All the benefits of ONTAP are already there. So it’s not just the scale, it’s just sometimes familiarity counts for a lot. Single pane of glass kind of counts as well. So we have customers that just say, " 20 terabytes. I just wanna spin that up right now.
I wanna look at it all in System Manager or maybe in like Blue XP or, whatever their GUI of choice is they wanna play with, it that way. And then it’s just ONTAP. So, they know what a FlexGroup is. They’re expecting, dedupe, compression, compaction, all the storage efficiencies.
They expect everything that ONTAP brings, to bring that to S3 as well. And so that’s the benefit to ONTAP is, yeah, it doesn’t have the scale or the ILM and management capabilities that StorageGRID does. But it has all the other things that ONTAP brings to the table.
Justin Parisi: And what about the snapshot protection? Is that pretty similar in S3 for StorageGRID and ONTAP, or is there a vast difference there?
John Lantz: There’s kind of a big difference. So objects are not files. So they really don’t have the, I don’t know, what do we do with snapshots? It’s like a delta and it’s like a point in time that says between this version and this version, things changed and I only need to know what changed.
And even that, I can make extremely storage efficient. Objects don’t do that. Objects do have versioning, but it’s basically, the entire object gets copied and appended onto the object. So it’s not super storage efficient in terms of versioning. So it’s not like that sliver delta.
Data protection just does work differently. We do have a thing called S3 SnapMirror inside of ONTAP S3. Honestly, if you’re coming from the world of, SnapMirror, SnapMirror in ONTAP, it’s not gonna be the same. It’s not those like, extremely efficient deltas. It’s really just like a copy of that data that we can spin off and use it as a backup copy or even use it as a DR copy. If you, disaster event takes place, my DR just became my primary, that works too. But it is different. It’s basically coming in from an object kind of perspective rather than a file based perspective.
Justin Parisi: With object storage, you have an entirely separate copy of an object, once there’s a new version. I would imagine that lends itself pretty well to something like deduplication. Is it pretty space efficient when you turn those storage efficiencies on? Does that work pretty well?
John Lantz: Yeah, so again, dedupe compression, compaction, the basic efficiencies that you see in ONTAP, they apply to your objects as well. You’re totally right that the appended object that lands on top of the other object, that becomes a single object with versioning on it.
It’s still subject to all the storage efficiency inside of ONTAP. It’s just not that little sliver of delta that you get with a classic SnapMirror snapshot.
Justin Parisi: Right. So, our replication is not as efficient, but our storage efficiencies are still pretty good. If not in some cases.
John Lantz: Yeah. Oh yeah. Way better than you would see in like a cloud provider just doing versioning where it’s just like, that starts eating away at your capacity, rapidly. Yeah. You don’t see that in ONTAP just because of the storage efficiencies that are cooked into ONTAP.
Justin Parisi: So what about the S3 protocol itself? I mean, are we pretty much up to speed with all the supported features with the S3 protocol as compared to, say StorageGRID or any other S3 provider? Or do we still have a little ways to go in ONTAP with that?
John Lantz: I say we still have a ways to go, but we’re getting really close. Truth be told, it really depends on what version you’re on.
And in 9.12.1, which is our latest version, most of what customer clients that speak S3, most of what they want to do, is there now. Multipart upload, versioning, tagging , et cetera, et cetera. That’s there nowadays in 9.12.1. , if not even, 9.11.1 kind of thing.
Upload part copy is new in 9.12.1. , copying in general. The truthful story is, it just gets better every single release. But honestly, I’m pretty comfortable on 9.12.1 in terms of its capabilities. I’ll be frank, StorageGRID is superior in terms of, list of the API kind of operations that it can do.
But in terms of, other folks in kind of the ONTAP space and what customer client applications are trying to do, we’re pretty close to being there today. I’d say that probably wasn’t the case, like back in, 9.8, 9.9.1. ‘Cause it was like brand new to ONTAP and we were trying to just hit kind of like the 80/20 rule.
What we were sticking on for a while, like versioning was a big one. And that got solved in our last release where, lots of data protection applications, a hundred percent need to use versioning. And you couldn’t until 9.11.1. So, it does get better every release,
but I’ll be honest, if you need some edge S3 API operation, you’re probably gonna want to look at StorageGRID instead. But for the vast majority of client apps out there it’s pretty robust now in terms of what we can do with with ONTAP S3.
You mentioned S3 SnapMirror and, and what’s great about that is if you do start on something like an ONTAP and then you realize that, hey, maybe this isn’t quite what I needed, I can S3 SnapMirror that to StorageGRID and then start using it there. Is that accurate?
Yes. Yeah, it’s a copy of all the data in that bucket and so you can move it anywhere you want. So, DR that’s for like, backup and you can, break that relationship and now start running data in the grid or anywhere else. The DR function really only works from ONTAP to ONTAP.
So say I was using on-prem ONTAP and my DR location was a Cloud Volumes ONTAP instance I could actually have a Dr, capability there where flood hits my on-prem. I can start hosting data straight out of the cloud. You can’t do that, at least today with other providers.
But we expect to see that hopefully with StorageGRID , probably over the horizon don’t wanna promise, timeframes. We’re looking at that sort of capability in terms of, those working better together. But yeah, just quick answer is like, can you copy everything that’s in your bucket and move it and do additional work with it somewhere else? Absolutely.
Justin Parisi: It’s good. You mentioned 9.12.1 having new feature functionality and you mentioned the stuff about S3 protocol. Now there is a pretty major piece of the 9.12.1 release and it’s something that I think that people have been really asking for quite a bit and it’s not really available on StorageGRID.
You mentioned StorageGRID being more robust for a lot of things, but this is one area where it doesn’t compete. So tell me about the new feature that’s available in 9.12.1 with ONTAP S3.
Sure. So
John Lantz: you’ll like this, ’cause in my mind you’re the multi-protocol guru or certainly were for a very long point in time in terms of NFS and SMB.
And so the exciting thing that comes out in 9.12.1 for, for ONTAP is basically bringing S3 into that multi-protocol kind of support. So it’s not a hundred percent file/object duality yet, cause it, it really does have to be NAS first today. But yeah, that’s, brand new in 9.12.1. And then the ask was, we had customers, that maybe they, or they’ve absolutely had data that brought in from NFS and they wanted to work with it doing backend analytic tasks and their favorite analytics tool was a S3 client. So bad news, 9.8 through 9.11.1, the previous ONTAP S3 releases is you couldn’t do that. You could, but you basically have to copy all your data out.
Put it into a bucket, so it leaves a volume or a directory, moves over to a bucket. Now you’re basically replicating data. It’s becomes inefficient. It’s more labor for the storage admin, et cetera. So, it was not multi-protocol support at all. Is basically data leaves a volume,
moves over to a bucket and then you can work with it as normal. So what changed in 9.12.1 is now we brought in the ability to use the S3 protocol in your existing NAS volumes. So now, those analytics operations, whether it’s, AI, ML, business intelligence, whatever.
Those are what we’re seeing in terms of requests right now, and they’re native S3. You can point your native S3 clients at that volume. You basically present it as a bucket. And yes, it’s still files underneath but just like those files can speak NFS or SMB depending on the clients,
now they can speak S3 as well, depending on that client. And really all we’re doing is doing the same kind of mapping, that you’re familiar with, is again, it has to be NAS first. You need to have the SMB users or the UNIX users that are already there, ’cause we’re gonna map that bucket and that S3 user to an existing set of users.
But assuming you have that, It’s read/write. It’s not just read only, it’s read/write. It’s everything into the data that’s, it’s really a file under the hood, but it’s presenting itself as an object. So you can have classic applications that are doing data collection today.
Say I have like satellite imagery. Or medical imagery. I see it a lot with imagery, at least in this first release, is they bring in their images using classic NFS or SMB and then they want to work with it and basically mine that for additional data using their modern S3 analytics tools. And now they can.
So they basically mount that volume. It can even be a directory. They mount that and looks like a bucket to the S3 client. And now the S3 client can do all its rewrites that it wants to do. So you’re basically getting double labor out of your existing data sets. It’s kind of the cool thing.
So it already was doing double duty in terms of NFS and SMB and now you can point S3 at it as well. So yeah, that’s brand new. Just again, to be clear, it’s NAS first. So, we do have native S3 with ONTAP, but native S3 is basically S3 only. You can’t convert a bucket into a volume that presents files. What you can do is take a volume that presents files or has literal files in it and present that as objects, present that as a bucket. That’s something, it’s brand new, in 9.12.1. And we’re, at least from my perspective, obviously I’m in the world of object and stuff, but it’s pretty hot right now and lots of folks are looking into that kinda capability.
Justin Parisi: It takes the legacy stuff that you’ve always been using for your applications and then modernizes it in a way because now you can leverage an S3 application against stuff that’s already writing using NFS or SMB. And you see that a lot I think with medical imaging, right.
Taking radiology images and being able to have your analytics, compare it to other radiology images to figure out if there’s a cancer present or adding data to the database for that particular use case.
John Lantz: Yep. So imaging is probably the number one thing I’m seeing now.
But I’m expecting lots of tasks that work with metadata as well, is folks will start looking at that, where it’s like, "Hey, I don’t need my NFS or SMB clients to do that." Maybe they’re the data gathering side of the workload versus, " I’m just gonna crunch analytics in the background and get more data out of data sets that I already have just using the latest and greatest S3 clients." It doesn’t have to be something fancy like curing cancer with MRIs, except we a hundred percent do see that or amazing satellite data and mining that it can be something as simple as, "Hey, I signed my mortgage and they’re gonna run OCR on it and grab additional data out of that. So lots of use cases that, some of them are super cool and some of them are super helpful to go to that actual customer.
Justin Parisi: Yeah, absolutely. So, you, you were talking about writing to a volume and then presenting it as a bucket and how this particular feature is only one way right now.
So that got me thinking… if I’m presenting it as a bucket, can I write to it later as a file? Or once it’s a bucket, is it an object and I can’t do anything with it? How is that gonna work there?
John Lantz: So Justin, it’s awesome. You can read/write from both ends.
So obviously, there’s no locks really. It is different than having handles and locks and what we’re used to in NAS environments. But, if there’s a conflict, there’s conflict resolution that kicks off and stuff. But yeah, you can have active clients from your NFS or SMB side of the house, as well as active clients from S3 all touching the same data.
The difference is, when a NAS client touches it, it’s gonna treat it like a file. Cause it really is a file. And so you can like, make individual changes to a file, which is like unheard of in the world of object, where, if the S3 client touches it, it blows out the old file and creates a brand new one.
It’s still object in terms of how the puts are working versus how a file is doing, read, writes, deletes, et cetera. So it is different depending on who, or I guess what client is touching it. But in terms of, is the data set active to everybody at the same time? Yes.
Justin Parisi: So John, this is a relatively new feature functionality for S3, the multi-protocol piece. So I imagine there’s some things that are gonna be limited here, like things that aren’t supported, things that maybe you should or shouldn’t do. So can you kind of run down that list for me just to kinda gimme an idea so I don’t walk into any pits filled with snakes?
John Lantz: Sure, sure. So there are a couple snakes. Let’s just be honest about what this is. It’s files, so it’s still a file-based system. Again, we’re starting with a NAS volume that presents itself as S3 and you can read/write to it and do all sorts of things, but it’s still files under the hood.
So some things that just make sense in the object world and don’t make sense in the file world. And so there’s some that just don’t work and the things I’m about to list are like, these are things that work in native S3 in ONTAP. So ONTAP S3 totally supports these things. But the multi-protocol version of presenting a NAS volume as a bucket, these aren’t supported yet.
We’re working hard on these, but , I’ll just be frank in 9.12.1, versioning isn’t supported. A file doesn’t really have the same kind of concept as an object version. Multipart upload is kind of a big one. Tags and the user defined metadata is also not supported. So a couple things that aren’t supported.
Again, that’s exclusive to the multi-protocol version of S3. If you’re doing native S3 applications that have nothing to do with a file so – there’s no NFS, no SMB involved whatsoever. All those work. So that’s kind of the snakes. Here’s the ladder that gets you out of the pit where just, something interesting is snakes; versioning, multipart upload, tags, et cetera, latter.
The things that work in file that didn’t work in S3, like native S3, they do work in the multi-protocol version. So things like FabricPool or SnapMirror, not S3 SnapMirror, but real SnapMirror. How about FlexC lone? Something that everybody does.
So those things all work. Just the same way they always do because it’s still a file under the covers. So we have lots of customers say, "Hey, I want to tier my S3 data. I’m doing like real hot workloads on native S3. But I can’t use FabricPool here because, you don’t allow FabricPool with a native S3 bucket."
Well there’s a way around that. If you build it as a native NAS volume and then presented it as S3. Now you can tier your S3 data out of there. So just like, there are some snakes, there are some ladders as well. And again, it comes back to, this starts as a file based system that presents as S3.
It’s not as a native S3 system. Honestly that’s just like file and block is different, file and object is different too. So there’s some things that just don’t transfer over. If it makes you feel better. Engineering hard at work on some of these. So like that list I just gave, like that’s not gonna be an eternal list. That’s just where we are on 9.12.1, which again, very first release for multi-protocol file/object duality.
Justin Parisi: Yeah. And I think what happens is people have ONTAP and they look at it and they’re like, "Oh, this is ONTAP 9.12.1. It’s all good." But in reality, you’re dealing with ONTAP and feature 1.0.
So feature goes to 2.0, 3.0, and then we start to see things that we want in there more often and fewer snakes to worry about. But, it is important to know that they exist.
John Lantz: Correct. And I think that’s fair.
Justin Parisi: So, you mentioned FabricPool and FlexClone and all that. What about FlexCache? Like, can I use FlexCache to present to a client as an object store as a cache, or do I not have that ability yet?
John Lantz: Yeah, FlexCache is in there as well. So if it’s coming from the file-based world it’s still there. So the catch with FlexCache is, you’re presenting the files out as the cache, so you can basically still have FlexCache out front serving up your file-based systems, but your S3 clients aren’t coming into the cache volume.
They’re coming into the classic NAS volume. So you can run ’em both at the same time, but it’s not caching for your S3 clients at least today.
Justin Parisi: Yeah, and I guess that the point of my question was, can I touch S3 with the cache? And I guess you’ve answered that for me. It’s, it’s still a NAS cache and not an S3 cache.
John Lantz: Correct, yeah.
Justin Parisi: Okay, cool. So sounds like we’ve got some cool functionality here and it’s only gonna get better throughout the future releases. Do you have anything else to add about multi-protocol for S3 or, anything that we haven’t covered here yet?
John Lantz: No, I think these are the big things. I think the 9.11.1, 9.12.1, every single release brings on additional S3 actions, but we kind of expect those and it’s not super exciting. It is if your client needs versioning, it’s exciting when we support it. But this one’s kind of groundbreaking. File/object duality in general is kind of like a holy grail in the world of object.
Lots of folks try it. Not very many people, implement it well. That’s another thing. Performance is really good here. We haven’t even mentioned performance. Again, it’s technical. We’re talking with a RC release right around the corner. But, I’ve seen it in the lab.
It’s pretty good. There is gonna be a performance hit compared to straight up NFS or even straight up S3. There is some translation and mapping and that’s gonna add a little bit of latency, but it’s really good performance. A lot of tools that say they do file/object duality kind of on the box.
When you actually try to run them in a production enterprise environment they don’t always meet expectations. And in terms of performance, we’re meeting, if not surpassing our initial expectations right now. So and again, like you were saying, this is the first release, it’s 9.12.1, but it’s like the first release of multi-protocol support for S3. So we anticipate that will only improve moving forward.
Justin Parisi: So most of these features they get like an early access program where they get some customers in there, try it out, and I imagine this feature is no different. So that said, I mean, did you have one of those, and if so, what were your customers saying about the feature?
John Lantz: We did. Feedback has been universally positive, so can’t name names. And to be fair, when we do the early access program, we kind of set an expectation ahead of time. So we’re saying, "Hey, you need to have a NAS volume. Your initial data has to be populated, users, et cetera."
We’re gonna be working with a file-based system that’s gonna present so your S3 clients can then do the mining jobs that they’re doing. And universally happy customers here. Really the problem that it solves especially for the EAP customers who are gonna be implementing it, as soon as we go RC if not GA, is it’s that problem of, I want to use S3 clients with my existing data sets and it’s a real hassle, if not just a bad hit in terms of capacity to have to copy that out to some other location, or even if it’s the same location now, it just becomes inefficient. And so, it solves that. And it might not sound like a big problem if you’re just thinking, "Oh, I have a directory with a couple gigs of data," but with customers at petabyte scales now, you don’t wanna have to try to copy a petabyte of data out to a bucket in the cloud somewhere when you can just let the volume where that petabyte already lives start speaking S3. And, that’s the neat part about multi-protocol in 9.12.1.
Justin Parisi: Yeah. And I mean, you’re kinda short selling it here. It is about capacity. But it’s also about speed. It takes time to copy that data. It’s also about cost, egress and ingress charges in the cloud. It’s also about management like manageability, being able to do it time efficiently and like not having to worry about it as a storage administrator. It’s just there. You don’t have to do anything but just present the data.
John Lantz: Yeah.
Justin Parisi: Cool. Well, it sounds like it’s a really promising new feature. I think a lot of people are gonna really want to, take it out for a ride and try it out and kick the tires and that sort of thing. Anything else new in S3 in ONTAP 9.12.1 or did we kind of recap it all here?
John Lantz: Again, we got a couple new actions, so not super exciting and honestly there are like some things that we could already do. Again, it’s kind of a new feature. So some of these actions where it’s like, oh, like, delete a bucket or something like that. Where, "what?
You couldn’t do that before?" Well, this is not true for 9.12.1. This is just an example of something where it’s like, yeah, you could always do that in ONTAP, and maybe we didn’t have, the restful APIs, so you could do it via a call. So some new things that came in S3 is mostly about policies and copying.
So, DELETE, PUT, GET for bucket policies is all brand new in terms of being able to use like an actual, restful S3 action. Likewise for objects. We got object copy and, upload part copy so we could, do some of these things before. Not necessarily on, on the copy side.
So again, it’s just every single release gets new functionality. So that’s kind of what we’re looking at for 9.12.1 in terms of new APIs or support for S3 operations. Yeah, the big one though, we spend a lot of time is multi-protocol. I think that’s where the excitement is coming from.
Justin Parisi: Yeah, absolutely. So I did think of one other thing and, and I can’t believe we haven’t already covered it, but , what’s the setup like? Like what do I have to do to get this working in the environment?
John Lantz: Yeah, so it’s so simple. You go into System Manager and basically you’re going to go in and find that volume and say, " I’m gonna basically mount that volume" or even like a folder, directory underneath that and say, "Hey, I want to present this as a bucket." And, ta-da! you’re good to go.
What we’re finding right now for customers that are initially touching it is they’re coming in, they’re saying, "Hey, you got a new toy. I can’t wait to play with it." And so they go in, they create a volume, and then they try to present it as a bucket. And, they’re like, "Hey, what’s happening? My user can’t access this." And, again, it’s NAS first. So, you have to have users to map it to . So if you don’t have your UNIX users or SMB, if there’s nothing to map that S3 user to, we’re gonna have problems. So that’s the gotcha right now. Make sure it’s NAS and can present NAS data normally, and then you’re good to go. The actual implementing the S3 protocol on top of it is super simple. I do wanna say for customers that are new to just S3 in general, especially if you’re coming from an older release, the S3 license is just like any kind of protocol license. You gotta install it if it didn’t come on that release.
So that didn’t happen until 9.8. So if you’re coming over from earlier releases, you don’t have the S3 license, you gotta go out to the support site and get it. It’s totally free, but it is something that you gotta put on your system.
Justin Parisi: All right. So licensing… I would imagine I’d have to set up the S3 server configuration as well, right? Like all the different keys and that sort of thing to allow access and authentication. Is that correct?
John Lantz: Correct. You’re still gonna set up a data LIF to present your data, et cetera. So absolutely. Thank you Justin.
Justin Parisi: And that data LIF, is it a special data LIF for S3 or can I use the S3 data LIF with NFS and SMB as well?
John Lantz: Yep. Same LIF. You’re, you’re good.
Justin Parisi: Okay. And then the name mapping rules. I know we have, UNIX to Windows, Windows to UNIX. Do we now have a new S3 to UNIX or S3 to Windows, or is that just all using the same name mapping logic we’ve always had?
John Lantz: So if you’re familiar with how mapping works, NFS, SMB we’re just adding S3 protocol into the mix.
Justin Parisi: Right. And I guess my question more of was, if I already have a multi-protocol environment, I’ve already got maybe name mappings in there, or maybe I’m using an LDAP server for implicit mapping.
So I’ve already got UNIX users. So when that S3 user comes in, it should map one to one if the name’s the same, but aren’t the users a little differently formulated than, say, a UNIX user? Don’t they have like a UUID type of tag with them? How does that work?
John Lantz: It’s not gonna map same to same, just based on the name. You do have to do that when the client comes in during the initial config. So you are gonna give your permissions on the S3 side to say, "Hey, these are my clients" or "these are my clients and the policies and groups that are associated with, the permissions that they have," then the NFS side gets mapped on top of that. But you do have to say, "This is my client this is the authentication, et cetera," that I want to map over to this other one.
Justin Parisi: Okay. And I imagine this will be all covered in the technical report, and which one is that gonna be in?
John Lantz: Actually, yeah. Great. So for the first release the TR is gonna be 4814. That’s our S3 TR. So good news, bad news. I know we have some folks that work in the field, or partners and have access to the field portal. This information as soon as we go RC is gonna be out there on the field portal. And that’s the technical presentations, the FAQs, et cetera.
For the public facing data, we normally wait to go GA. But that’ll show up in TR-4814. That’s ONTAP S3 Best Practices and that shows up as soon as we go GA.
Justin Parisi: Okay, cool. And we’ll include that link in the blog as well. So , it sounds like we’ve got a lot of promising stuff here with S3 and multi-protocol. Again, John, if we wanted to reach you, how do we do that?
John Lantz: Another good place to do that is gonna be ng-ontap-s3-info@netapp.com. Lot of dashes in that one. A lot of dashes. Netapp.com. , I think it’ll be better to just like put a link on the page.
Yeah, Yeah. We’ll put a link here. You nobody’s gonna write that down, man. Nobody’s gonna do. If you’re internal to NetApp, it’ll auto populate, where if you’re external to NetApp, go to the blog. You’ll find the email address there that NG will have multiple people on there ready and willing to answer all your S3 questions.
Why Is The Internet Broken? 
Pingback: A Year in Review: 2022 Highlights | Why Is The Internet Broken?