Monday Jul 28, 2008

Setting Up an OpenSolaris NAS Box: Father-Son Bonding - The Video

Updated!  Video transcript added below.

I just posted the write up on how my son and I built an OpenSolaris based NAS box. Now here is the video of our presentation at FROSUG.

21 mins.

Also available in iPod video format

My thanks to all who helped make this possible! 

Chase: You did an awesome job buddy! I'm so proud of you/ You took a huge step of faith to do this video and you did great. It is kinda weird to see yourself on video, but it is way cool. Great job, my friend!

Thanks to Paul, Damon, Lynn and Deirdre for all of their help making this video and the previous write up possible. "None of us is as smart as all of us." - Ken Blanchard

[NOTE: I'm reposting this entry to correct the mp4 version of the file. The original file disappeared some time yesterday. Sorry for any who tried to d/l and received a 404. --jc]

Video Transcript 

Jeff: Thank you very much, John. And thank you FROSUG for allowing us to come and show off what we've been up to for the last couple months. I’m Jeff Cheeney.

Chase: And I'm Chase.

Jeff: Outstanding. Well, we'll give a little bit of introduction on who we are.

Chase: Okay, so, I'm Chase.

Chase: I'm 11, and I'm in 5th grade, and this is my dad.

Jeff: And I am not in 5th grade, nor am I 11.

My job at Sun is the OpenSolaris community manager for the storage community. So I think about things about storage and I think about things like how to build a community and who would be interested in participating -- you know, contributing code or helping out with projects that we've got running, so that's part of what I do for Sun. And therefore, "the storage box" was an idea of ours in order to -- go ahead and advance that slide, Chase – [We] wanted to do something different, and we wanted to try some present technology and I wanted to see if we could do this as my son and I. So what else did we do ?

Chase: Um, we wanted a way to pass winter break and to learn about computers.

Jeff: Chase has this understanding that computers are for web browsing, playing games, occasionally doing spelling words-- you know, typing out spelling words a few times. And I decided that that was a little bit of limited knowledge, that we needed to teach him a little bit more. Show him a little bit of hardware, as well as a little bit of software. So that's what we're up to.

So, Chase, why don't you run us through the list of ingredients ? We could-- we could put 'em up over there as well as on the screen.

Chase: Okay.

Chase: So we have one Sun W2100z, which is Fred, big box. We have a SCSI HBA, 0ne 70-gig internal drive, one 140-gig internal drive, and we have one SCSI multipack. The multipack has six 40-gigabyte drives.

Jeff: So the thing I should mention about this is we decided we wanted to do something that we could just-- we're trying to do a demo here of what you might be able to find in your house-- Well, you may not find the multipack in your house-- but you'd find old hardware, find some stuff that you can cobble together. So what I did is I went in and raided the lab. So I went in upstairs, went into the Solaris engineering lab and said, "Okay, what's not being used ? What's on the racks that's not being used ?" So found an old 2100, had a gig of memory in it, scavenged out of some other ones that weren't being used, a couple of other DIMMs, so it could beef up the memory a little bit. Found an HBA that I could attach the multipack to, you know, so we've got a total of 200 gig on that multipack. Boy, that's real exciting. I got more hard drive space in the one computer at home than that multipack that weighs like 40 pounds and sounds like a jet engine. But nonetheless, we can do it all and we'll show you how it all works with what we've got.

The other components that were in our kit here, we're using the SXDE distribution of Solaris, so that's the Solaris Express Developer Edition. We're using the January '08 version, because that's what we had available at spring-- at winter break. And then the other thing I wanted to point out, and we'll mention this again-- there's a "getting started" guide that's available off of the Genunix wiki, and that wiki is where we do a lot of open source-- or, excuse me, the OpenSolaris collaboration. So that "getting started" guide was critical For our success and I wanna make sure we point that out. That's one of the key ingredients in our home NAS.

The other thing, I guess I should have mentioned a little sooner is, we were doing this as a recipe. We decided that people like recipes, they can identify with that format, there's ingredients, there's time, there's how you mix it all together, so we utilized that-- that methodology of this presentation.

All right, Chase.

Chase: Okay, so, we installed the hardwares first-- okay. And that took about 20 minutes or so, and then, um, we installed the SXDE, and that took forever. I was in bed when he finished it.

Jeff: Yeah, we got that started, it was about bedtime, we walked through the screens-- We'll talk about that in a minute-- But the full install takes at least a couple hours and it was getting a little past bedtime, so I finished that one up with him in bed and he got to check it out in the morning.

Chase: Then we configured Solaris, which took 40, 45 minutes.

Jeff: You know, the hope is with this presentation, as well as with that "getting started" guide, it won't take any of you 40 minutes to do the config. I was doing a lot of constructing 'cause we were going through this. I was also utilizing some of the different tools that we have in Solaris. If you're familiar with OpenSolaris, you're familiar with Solaris, you know the command line, it'll be a whole lot faster. But I was using graphical interfaces just so Chase had something interesting to look at 'cause it's kind of like a web browser, as well as just trying out some of the technologies that I hadn't utilized myself, so that's what we did.

Chase: And then we had Windows share the files from our computer to--

Jeff: The other way around, we shared it to Windows.

Chase: Yeah, nevermind.

Jeff: So we had to talk-- we had to configure Windows and yeah-- we'll get to that one. That one took a little while. All right, move on. All right, so here's the pictures of the hardware. Obviously, we-- like I said before, we used-- we used about 5 gig of memory. One of the big reasons we used that much memory is ZFS loves memory. When ZFS is active, it keeps a lot of pages in memory, so it-- that's what gives it some of its speed, but as well as it does its transactions in memory. So, because of that, we gave it a little extra memory. Okay, we gotta do your animations. Here we go, all right. All right, then what else did we do ?

Chase: We installed the SCSI HBA.

Jeff: Okay, why did we do that ?

Chase: Um...

Jeff: It's the HBA part, all right. Why did we put the HBA in ?

Chase: Um...

Jeff: So the HBA is how we connect it, the multipack, right ?

Chase: Right, yeah.

Jeff: So we had to add one of those in because we didn't have external SCSI on the 2100. Okay.

One more. You've got the flying connecter... And then we powered the whole thing up. And fortunately for us, we did all the installs of all the boards correctly and it worked ! And the thing to point out here was Chase was doing all this for the very first time. He'd never opened up a system before, he'd never seen a motherboard, never had seen a DIMM, um, never had seen an HBA. So it was a great opportunity to use grounding strips and teach him a little bit of electrical safety that we had-- at least I learned when I was in the university. Okay. So, tell us about OpenSolaris ? How hard was it to install ?

Chase: Not. All we did was follow the screens.

Jeff: So, like, what kind of questions did it ask you ? What did you have to do while you were installing ?

Chase: You made the users--

Jeff: Yeah, and before that ?

Chase: Passwords.

Jeff: Made some passwords, time zones...

Chase: Yeah.

Jeff: Did you have any trouble doing that ?

Chase: Nope.

Jeff: And that took how long ?

Chase: I don't know, five minutes ?

Jeff: Five, ten minutes, that was it.

Chase: Yeah.

Jeff: That's the point. Following the graphical installer was pretty trivial. We just walked right through it, didn't have any issues at all. So that was the easy part. The long part was getting it installed, and you didn't have to do that. All right, so now, we've got the machine built, we've got an operating system on it, what are we gonna do now ?

Chase: Configure Solaris.

Jeff: There you go, and what do we have to do ?

Chase: We have to read the "getting started" book.

Jeff: Again. Teaching-- read the directions. We're trying at least, to learn to read the directions. So, that's what we did. We read the directions, and then what did we have to do ?

Chase: Then we created the local users.

Jeff: Okay, what did we create ?

Chase: We created media.

Jeff: Right, why did we create media ?

Chase: So we could have all our files from our own computer on that one.

Jeff: Okay, so that's who's gonna own the files on the-- on fred, okay.

Chase: Then we have edmund ?

Jeff: We have edmund, right.

Chase: And root.

Jeff: Yeah, root gets created by default, so we already got that one. All right, so we got our users, and we got everything installed. Now what's up ?

Chase: Now you have to combine all the disks on the SCSI to make a ZFS pool.

Jeff: Right, so what does that mean ? What did we do ?

Chase: It means, um, you follow the screens again

Chase: To get to the place where you can combine the six disks in the SCSI to make a ZFS.

Jeff: Yep, so we took-- and the thing that was-- That was fun for me, was walking through the graphical interface that I knew existed, and the only reason I knew it existed was because I managed the engineer who wrote it at the time, but I had never played with it. And I kept hearing about how wonderful it was. So I gave it a shot, and you bring up a web browser, point it to a specific port and you've got a graphical interface. You can pick what kind of configuration you want for ZFS. It automatically defines all of the drives on the system, whether they're being used for something or not. You collect-- you select what kind of pool we wanted, so in our case, we created a single-parity raid-z. Added the drives, clicked "done," showed us the command that we could've typed, which is pretty straightforward as well. And how fast did it create this ?

Chase: Pretty quick.

Jeff: How fast ?

Chase: I don't know.

Jeff: Like-- that fast.

Chase: Yeah.

Jeff: You didn't even have to think. It really-- it is snap-a-finger fast. All right, so we created the pool…

Chase: Right.

Jeff: And then what are we gonna have to do ?

Chase: The file systems.

Jeff: Okay, what are the file systems for ?

Chase: So we could share.

Jeff: Right, and that's where we can put different kinds of files, right ?

Chase: Right.

Jeff: So what we did is we went through and created three different file systems, all in our pool. We created-- we'll show these in a couple minutes-- se created one for our movies, one for our music and one for our pictures, xo that's what they are. And we set some different properties on those as well, so knowing that our music might be compressible, we turned compression on on that file system, but we didn't do it on the movies because movies are typically pretty well-compressed and so it would be just wasted cycles to try to recompress the music. And so we did some tweaking like that of the different file systems. And we did-- again, we did all that through the browser interface. That was the time where you actually had to do some typing.

Chase: Yes.

Jeff: Chase is still learning how to type, so this one took us a little while. So what did we do ?

Chase: We configured CIFS-- c-i-f-s ?

Jeff: CIFS, right.

And what is CIFS used for ?

Chase: Isn't it used for sharing ?

Jeff: That's right, it shares our files from our OpenSolaris box to...

Chase: Our regular computer.

Jeff: Our regular computer-- In this case, it's a Windows box. Don't you love this ? So, yes, we shared it to our regular computer. All right, and so how do we do that, we turned it on...

Chase: Yeah.

Jeff: So we used what command ?

Chase: Um...

Jeff: You can read it off the slide.

Chase: "S-v-c-a-d-m enable."

Jeff: Right, so we enabled that service, the svc service, we enabled it, and then we did the join. So the join step right in there is-- at our house, i've created a workgroup for all of our computers and our workgroup is called "central," that was all that we had, so I used “smb join,” which is the-- Smb is the file-sharing protocol administration, I'm joining the centralized-- the central workgroup. So that was a critical step in that whole process, To make sure that we make-- that we could share our files in that workgroup and that we set up the CIFS server to be in workgroup mode versus another mode, like active directory, okay ? So then what did we do ?

Chase: We shared the file systems ?

Jeff: Mm-hmm. And how did we tell ZFS that we wanted to share them ?

Chase: Um, we did "sharesmb" and then we did the name of the file.

Jeff: Right. So this is one of the tricks that we had to pull out of the guide that I wouldn't have figured out before and honestly, this is one of the places where I really struggled with the Windows configuration and getting Solaris to work. This is now in the "getting started guide" as well as on this presentation and it'll be on my blog in a couple days. but-- as well as Doug McCallum put this on his blog. So "sharesmb" is the command-- is the subcommand to ZFS to say, "I wanna share this file with the CIFS protocol, I wanna share it as an smb share versus an nfs share." And then what you can do is you can name that share So that when Windows is browsing your OpenSolaris box, you'll have a name associated with that share. What was happening is, without that specific name in there-- so in our case, the very first line, "movies"-- the share was being-- it was being exported as the "zpool" name. So in our case, "mediapool," with an underscore and then the file system name. So the "mediapool, underscore, movies." And it wasn't obvious because when you went into the Windows share browser, it didn't show up that way, and so I really struggled with, "How do I configure this ?" It took me a while to read through all of the discussion lists, which are, you know, as you guys all probably know, it's very rich with information.

So found it in there, dug it up, It's now in the-- excuse me-- "getting started" guide. That was one of those critical pieces of information that took a while to figure out. So we share our three different file systems, gave 'em unique names, And we've got some mixed case in there as well, so we can share the different case sensitivities. And then we had to make our files writable to... Media, our local user. We had created these all as root so they, by default, get mounted and shared as root so we had to go ahead and change the permissions on those. So we did that.

Jim: So Jeff, is that--

Jeff: Yeah, just one second, Jim. If we could get a mic to you so you can ask a question so we can get it on tape, that would be great.

Jim: So, jeff, that command, I kind of missed it. So it's not case-sensitive, I guess ? The ZFS share command ?Yeah, because if you have the movies with a capital "m" and then you have a lowercase--

Jeff: It is case-sensitive. So you can share it with different cases.

Jim: Oh, okay.

Jeff: And actually, you bring up a great point there, Jim, and I should have put this on the slides and I'll make sure I update my blog with this. There is another ZFS command that you need to use to tell ZFS to accept case sensitivity. So ZFS said "case-sensitive equals on," with a pool, and then it'll be a-- that property will be inherited across all of the-- across all the file systems. Thanks for that reminder, jim.

Great, now like I said, the tricky part, getting Windows-- getting the “regular” computer to talk to the OpenSolaris computer. We're gonna change that, Chase. OpenSolaris is the regular one, that's the irregular.

Chase: I like the regular one.

Jeff: So it's fred and not fred. So, again, this was tricky. This took-- once we figured it out, once we got the guide, it all goes fairly quickly, but learning, this was a big learning curve for me. I'm not a Windows guy. I'm a Solaris guy. I've been doing Solaris for 20 plus years, and-- Or, right at 20 years. So what we had to do is, in the Solaris developer express build that we had, our CIFS server on OpenSolaris didn't export things in a browsable mode, so as some of us are accustomed to in Windows, you can go to a machine and browse that machine and you can see all of the shares that it's offering out.

In the build that we're currently running on this machine, that didn't work, so you had to explicitly-- you had to explicitly mount a drive. So you had to say, "well, I mount-- in this case, drive "z"-- and I want it to be this explicit share," and you have to name it. In subsequent builds of OpenSolaris, the builds that are available now that you can go out and download, browsing is available, so you can browse and mount those shares with just a single click.

So in this-- this is the case where you'll see, Fred is our machine name and our share, just as we talked about before, it's capital, It's Movies with a capital "m." The other big thing, it took us a while-- It took me a while to figure out because I didn't read-- I didn't read the guide as thoroughly as Chase did and he was in bed at that point.

When you go to mount that share, the access privileges have to match the OpenSolaris box, not your Windows box. So in this case, I was trying to mount these shares as "owner," and "owner" doesn't exist on my OpenSolaris box. So the critical piece here is that you'll notice on that mapped network drive-- this is Windows XP -- I connect as a different user. You click that different user and then you put the credentials in, login and password, for the OpenSolaris machine.

So in our case, we had created an account media, that's what we're gonna be mounting the shares as. So that's another little trick. It escaped me for the longest time and it's an absolute critical step. And with that, we're sharing files. We're reading and writing files back and forth between the machines, and we're up and running. And that really-- I try to distill all of my learning down into these simple, four, five slides. And that's how straightforward it is once you get through the learning curve. So hopefully, i've done some things to help you all get through the learning curve and you can try this out as well.

So just like any good recipe, you gotta look for variations. How do you vary this recipe ? How do you make it tailored to your needs ? So one of the things we can do--

Chase: Get bigger disks.

Jeff: Get a lot bigger disks. So we each have two half-terabyte drives now that we're gonna be putting in the box, we're gonna try sata connections. We're gonna see if we can get the onboard sata on the w2100 to talk to those-- to talk to those half-terabyte drives, so that's gonna be our next project.

One of the other things we wanna do is, OpenSolaris has embedded in it the protocol, "icap" and in conjunction with icap, we have the support to be able to catch file events. So why do you wanna do this ? Well, antivirus is a great example of this. So there's a program out there, “clamav"-- and it's an open-source program and you can find it at this link-- that will scan files on OpenSolaris, it'll scan Windows files on OpenSolaris for known viruses. You can download virus definitions and it will be automatically scanning. The really cool thing that happens is, every time ZFS gets a write or it gets a new file create, ZFS stops the operation, hands it over to the scanning engine and lets the scanning engine proceed first. And it won't-- it won't commit that file to disk until it knows that it's clean. So it's a really cool thing-- or cool feature, because now I can run virus-scanning as files are being created on my OpenSolaris box, which has a whole lot more processing power than my "irregular" computer. And I don't have to worry as much about the viruses that somebody might be downloading onto the computer.

Chase: It's called e-mail.

Jeff: Yeah, e-mailing viruses, that's right. One of the other things we wanna do is we wanna take this box and instead of having it sitting in a closet somewhere, I'm gonna put it somewhere else in the house. And with sunray technology as well as with VirtualBox, I want to have an “irregular” box inside of Solaris. So if the kids wanted to play their games on Windows, we could do that, we could use our VirtualBox software, which, if you guys don't know about VirtualBox, we probably should come back and have a fun talk about that because it's just a lot of fun. It's the vmware for-- that actually runs on OpenSolaris, OpenSolaris is the host.

So those are a couple of things that we're thinking about doing, and obviously any other things that might be in your imaginations as well. All right, and we've gotta give credit where credit is due.

Jeff & Chase: The bibliographies.

Jeff: He was the one who told me, "you gotta have a bibliography." He's been writing too many research reports. So here's our sources, these are the places that we have referenced heavily to get this working.

The CIFS server project page on OpenSolaris was a great resource for us. Doug McCallum, I'll mention Doug again, he has a wonderful blog that is very detailed, full of rich reference material on how to use the "share adm" command. Just a fabulous reference,

And then those two wiki pages that we referenced as well. And I probably should have put on there the OpenSolaris storage discuss forum. That's where all of the discussions, the tips and tricks for this project are going on, so that's OpenSolaris. The storage discuss on OpenSolaris.

So that's it ! We're done. In all, it was a fun thing. Did you have fun ?

Chase: Yeah.

Jeff: You learned a couple things ?

Chase: Yeah.

Jeff: You might wanna play with it a little bit more ?

Chase: Mm-hmm.

Jeff: All right, all right. So with that, if you guys have got any questions, please do ask for a microphone since we are trying to record this, it is live. But that's what we did, hope you enjoyed it ! Great ! Thank you.

Thursday May 08, 2008

OpenSolaris community - an introduction

Deirdre just posted some great videos from the OpenSolaris developer summit. This was a great weekend and it was fun getting to see the faces behind the messages on opensolaris-discuss.

Thanks for the great videos Deirdre!



Here I am writing about the OS I love. Trying to show other people how cool the innovations are in [Open]Solaris.


« July 2016

No bookmarks in folder

OpenSolaris Storage Links