Jail time for SSH

Long time ago in the galaxy not so far away when you had to upload a file to your friend you would call him up and have a ZMODEM session. With this new fad called Internet it is all about SSH of course. But is the default configuration secure enough? Is creating an account for a friend who needs to upload a file to you all you need? How do you communicate credentials?

In fact, if you don't want to futz with GnuPG and send your buddy a an encrypted email with a login name and a password to go with it, your choice is a rather limited one: either call him up or assume that this information could be leaked. In fact, assume that you're building a public upload site and build your security model based on that assumption. In particular it makes sense to build your solution so that:

  1. The only subset of your filesystem visible to anybody coming from the outside is strictly limited
  2. The only thing that a user from the outside can do is to upload a file to a specific location

How hard can it be to build a rig that would satisfy the requirements outlined above? My initial estimate was about 15 minutes. After all, the only thing you have to do to satisfy visibility aspect is to get creative with chroot and the second item is a simple matter of a proper shell entry in the /etc/passwd. Right? Wrong! It took me about 4 hours to make it all work and I blame at least 3 of those hours on GNU libc being a bloated complicated mess.

But I'm getting a bit ahead of myself. Lets talk about user model first. What I wanted my friend to be able to do was to issue a simple:

    $ scp fileIneed publicscp@my.comcast.ip.address: 
on his end and be done. Of course, to properly support that command on a server side with all the additional security requirements you have to know exactly what it translates to on the receiving end. How do you figure that out? Simple, you create a user called test and put the following into the /etc/passwd:
Now, you can see what sshd actually does when it establishes a connection for the scp session by simply trying to establish that session in the first place and looking in the log files (I guess you can also read a bit of OpenSSH docs, but manuals are for wimps ;-)). Curious? Well, here's what actually gets executed whenever somebody does scp:
    $ scp dummy.txt test@localhost:/tmp
    -c /usr/bin/scp -t /tmp 
See that '-c' option? Of course sshd expects /bin/echo to be a full fledged shell and the way you tell shell what to execute is via the -c option. Which means that even though we only want scp functionality to be provided sshd is still capable of running anything that the remote user might request (provided, of course, that what is specified in /etc/passwd is, indeed a shell). But what if the only thing that our pseudo-shell was capable of were to do an scp transfer? What if, in fact, our pseudo shell was an /usr/bin/scp? And not just a regular /usr/bin/scp, but a statically linked one. So that it can be run in a chrooted environment so isolated that the only file in it would be scp itself (after all, if it is statically linked we don't even need /lib/libc.so). Would such an architecture be pretty secure compared to the default install? I thought it would (and I would love to hear real security experts to prove me wrong in the comments!).

But how do we rig it that way? First of all our publicscp user can not really have /usr/bin/scp as a shell in his /etc/passwd entry. That is because of the extra '-t /tmp' option -- you can't specify options in /etc/passwd. This calls for a tiny C wrapper: $ cat scp-wrapper.c #include <unistd.h> int main() { return execl("/home/publicscp/scp-static", "scp", "-t", "/home/publicscp", NULL); } $ cc -static -o scp-wrapper scp-wrapper.c And speaking of scp-static, it should be nothing more than a statically built version of the /usr/bin/scp. You can pull this trick off by downloading source code for the OpenSSH, building it the normal way, removing scp binary, saying make once again, capturing the link step and finally doing it manually with an extra '-static' option added at the very begging of the gcc command line options. Rename scp into scp-static to reflect its static nature and copy it along with scp-wrapper to the /home/publicscp. The only step left is to add the following to /etc/passwd:


Now we've got everything ready to also make /home/publicscp directory be the chrooted jail for publicscp. Our goal is to make /scp-static and /scp-wrapper the only visible files (note the leading slashes -- even though we've copied these files to /home/publicscp after you do chroot("/home/publicscp"); they become visible as though they were copied to the root of the filesystem AND nothing outside of /home/publicscp can be ever accessed). But wait! Look how our wrapper passes '-t /home/publicscp' to the scp commad; once chrooted, that path actually has to exist. Which means we would have to create /home/publicscp/home/publicscp. Ok, the next question is -- how do we actually do the chroot'ing on every login session of publicscp? Turns out that on Linux there is a PAM module called pam_chroot that does exactly what is need. The only thing that is required is to add it to the /etc/pam.d/sshd and specify /home/publicscp as a publicscp's chrooted jail in /etc/security/chroot.conf. Oh, and since it will be a chrooted environment it might make sense to duplicate scp-wrapper and scp-static at /home/publicscp/home/publicscp level. Just in case ;-)

Ready! Set! scp! Fail! What? Well, this joke is on me. I should've known that most non trivial applications require at least /dev/null to be created and that seems to be exactly what scp is complaining about. mknod /home/publicscp/dev/null to the rescue!

Take two! Failure again! This time the message is rather cryptic: unknown user 30000. Good news is that a quick search in the scp.c source code turns this fragment as the culprit: if ((pwd = getpwuid(0)) == NULL) printf("unknown user %u", (u_int) userid); so that at least we can isolate it into test.c, compile statically and try to execute in a chrooted environment: # cat test.c #include <stdio.h> #include <sys/types.h> #include <pwd.h> int main() { uid_t userid; struct passwd \*pwd; if ((pwd = getpwuid(0)) == NULL) printf("unknown user %u", (u_int) userid); return 0; } # cc -static -o /home/publicscp/test test.c # chroot /home/publicscp /test unknown user 0

At this point a more UNIX literate reader would exclaim: "Of course! You forgot to provide /home/publicscp/etc/nsswitch.conf and /home/publicscp/etc/passwd for getpwuid to work properly" and if I only could hear that voice it would have saved me 45 minutes right there (I knew that passwd was needed after about 3 minutes, the rest 42 of them was occupied by not remembering nsswitch.conf). After an hour here's the environment I had: $ cd /home/publicscp $ find \\! -type d ./scp-wrapper ./scp-static ./dev/null ./etc/passwd ./etc/nsswitch.conf ./home/publicscp/scp-static ./home/publicscp/scp-wrapper The only problem with it was -- it still didn't work.

I don't know why GNU libc has to be that way, but reading its source code is like pulling teeth. You can't understand anything! It is the worst kind of software obfuscation that I have seen since sendmail.cf. Add to it the fact that you can't really debug anything inside the chrooted environment because, you know what, debugger is a dynamically linked binary and you can easily see how I spent next 3 hours reverse engineering nss in glibc only to uncover the most hideous crime of all -- GNU libc dlopens itself from within the code linked into my static scp-static! I think Ulrich Drepper has now officially crossed the line. It is one thing to spread false rumors about static linking. But it is bordering on illegal behavior to actually subvert static linking like that. That, plus he now officially owes me 3 hours of quality time. Can a person be sued for something like that? Oh well...

With that final mystery solved (and yet another atrocity committed by GNU libc) I had an environment that looked like this: # grep publicscp /etc/passwd publicscp:x:30000:30000::/home/publicscp:/home/publicscp/scp-wrapper # cat /etc/security/chroot.conf publicscp /home/publicscp # tail -1 /etc/pam.d/sshd session required pam_chroot.so # cd /home/publicscp # find \\! -type d ./lib/ld-linux.so.2 ./lib/libnss_files.so.2 ./lib/libc.so.6 ./scp-wrapper ./scp-static ./dev/null ./etc/passwd ./etc/nsswitch.conf ./home/publicscp/scp-static ./home/publicscp/scp-wrapper # cat etc/passwd publicscp:x:30000:30000::/home/publicscp:/home/publicscp/scp-wrapper # cat etc/nsswitch.conf passwd: files And what's even more important overnight I had /home/publicscp/home/publicscp/fileIneed.tgz uploaded to me.

P.S. Thank you Leha! ;-)


Maybe not a budget solution, but the commercial ssh from ssh.com handles this easily.

Posted by Mike on August 25, 2007 at 06:52 AM PDT #

If you are using Solaris 10 or later, you can make all this happen with Zones. Just cretate a zone, configure using a script that any Solaris admin should be used too, add user accounts for each of your friends or possibly just one anonymous account. forward port of your choice on the firewall/router to port 22 on the zone. When file is transfered, shut down the zone. retrieve file by accessing the file in the zone, or any other method you like. When the zone is not running no security issues, no complex setup. Just a few commands once the zone exists. You can recreate the whole thing when ever you want. You can even start and end zone access in a simple cron entry or two if recurring use is expected.

Posted by James Dickens on August 27, 2007 at 01:49 AM PDT #

That all sounded like a lot of fun, but I try to avoid this type of fun if at all possible. The last time I needed to do this I found the script at the following location, which worked quite nicely on Debian.


I enjoyed your article in Linux Journal. Thanks.

Posted by Sarel Botha on October 27, 2007 at 12:01 PM PDT #

Post a Comment:
  • HTML Syntax: NOT allowed



Top Tags
« July 2016