Wednesday Mar 16, 2011

machine that goes ping

The default mode of ping(1M) on Solaris is to send one packet, then report "foo is alive" after getting a response. But many people, especially those more used to the various BSD flavors of Unix, prefer the "statistics" mode of ping (i.e., "ping -s"), where a packet is sent once per second, and the round-trip-time statistics are presented, until the user hits control-c. So in build 33 of Solaris 11 I added a check: if the variable MACHINE_THAT_GOES_PING is set in the environment to a non-null value, then ping will go into "statistics" mode as if the -s flag had been provided. I neglected to update the man page for some reason, however; perhaps Easter was on my mind.

Thursday Feb 18, 2010

managing across upgrades

There have been problems in the past caused by sendmail upgrades, where custom configurations have been moved aside and mail started bouncing. This is because long ago there were some sendmail upgrades that included security fixes, and it was deemed that the new default /etc/mail/ should be installed on upgrade, taking precedence over the old customized version of that file, which was instead moved aside to /etc/mail/ . Some suggested that the packaging for should be changed from type renameold to renamenew, but that had down sides as well. So in Nevada build 90 I introduced a solution to this problem:
  • First, you need the .mc file from which your gets built. This .mc file is typically stored under /etc/mail/cf/cf/ (no, that is not a typo: the first cf is for the configuration hierarchy; the second is where the individual files are stored). You need to know the exact name of your .mc file; we recommend /etc/mail/cf/cf/`hostname`.mc .
  • As root or some other user/role with sufficient privilege, do the following, where X is replaced by the name of the file from above:
      # svccfg -s sendmail
      svc:/network/smtp:sendmail> setprop config/path_to_sendmail_mc=astring: X
      svc:/network/smtp:sendmail> \^D
      # svcadm refresh svc:/network/smtp:sendmail
      # svcadm restart svc:/network/smtp:sendmail
  • If you ever do a fresh install, be sure to save the file (X) beforehand and restore it afterward, then repeat the above setprop/refresh/restart dance after you restore the file.
  • On service start, the specified .mc file will be expanded into a .cf file and copied to /etc/mail/ before sendmail is started.
Well, that new feature in build 90 was delivered with sendmail version 8.14.3, which went a very long time before it was superseded, when version 8.14.4 was introduced in build 132. So for the first time in 42 builds (almost two years), this upgrade issue has raised its head again. In particular, doing a pkg image-update from build 131 or earlier to build 132 or later will result in any customizations to being moved aside as described above, and mail may start bouncing unless you deploy the new service property as noted above.
In addition to this generally useful tidbit, there is a particular issue with doing a pkg image-update from build 132 or earlier to build 133 or later:
14570 file install logic discommoded by excessive cleverness if preserve=rename\*
To work around this problem, do the following, again as root or a user/role with sufficient privilege:
# svcadm disable -t sendmail
# cp /etc/mail/cf/cf/ /etc/mail/
before the pkg image-update. This will prevent the packaging system from being confused by the customized and allow the upgrade to proceed smoothly. After rebooting to the new boot environment, sendmail will re-generate its configuration per the above method and mail should continue to be served properly.

Thursday Mar 06, 2008

OGB Positions & Bio


I believe in open communication and open development.
  • Communication
    Our (Sun's) communication with the OpenSolaris community has been poor. We need to be clearer about both making unilateral decisions (explaining why we decided whatever) and sharing information with the community in general. People who have been to any number of Solaris all-hands meetings in the last 11+ years can attest that I am one of the chief questioners, always trying to keep the lines of communication with management as open as possible.
  • Development
    Solaris has been open source since 2005, but for it truly to flourish, it needs to have open development as well. The sponsor program has been a band-aid while we worked on opening up our tools and processes and moving our source repositories outside the firewall, all prerequisites for open development, but the community is thirsting for more, and we need to provide it. This in my view should be our #1 priority to both keep current community members interested and attract new community members. As ON tech lead, I have actively sought (and found) people to work on migrating our tools from being Teamware-based to being Mercurial-based, and I plan to continue driving both this migration, then the move of the ON source repository outside the firewall, then whatever else will be needed so we can have true open development.

Employment history

  • Sun, December 1996 - present
    • ON C-team tech lead, August 2007 - present: lead engineer for the ON consolidation, responsible for overseeing all changes made to the source base
    • Network Auto-Magic (NWAM) project lead, February 2005 - August 2007: led story-boarding, requirements, architecture, design and implementation of Phase 0 and initial implementation of Phase 1.
    • Zones development, April 2002 - February 2005: wrote zonecfg, libzonecfg, most of zoneadm and large parts of zoneadmd, plus worked on the booting sub-system; brought libtecla to Solaris for use in zonecfg.
    • ON gatekeeper, October 1999 - April 2002: lead administrator of gate machines, also core C-team member, responsible for keeping up the quality of the ON consolidation
    • Sendmail, December 1996 - October 1999 (still have responsibility for it, but since late 1999 it has been ¼ time or less): upgraded ancient Solaris version, kept Solaris up-to-date, eventually deprecated all old Sun hacks or got them integrated into the version. Contributed many enhancements, including anti-spam features and got Solaris to a unified config file. FAQ maintainer since 1997.
  • InReference, August 1995 - November 1996
    Wrote a Usenet search engine (code: Python and C).
  • HP, June 1989 - August 1995
    Supported various network applications and protocols on MPE and HP-UX. Added 8BITMIME support to sendmail, offered patch to Eric Allman.



  • Married since 1992, 10-year-old daughter, (almost) 7-year-old son
  • Physical fitness enthusiast: running, ultimate frisbee, cycling, hiking
  • I serve as an usher at our church most Sundays
  • We live in San José, CA (USA)

Thursday Mar 01, 2007

How to set up Sendmail certificates

This entry describes how to set up certificates to enable sendmail to use TLS (transport-layer security). Note that for Solaris 10, the command /usr/sfw/bin/openssl and the configuration file /etc/sfw/openssl/openssl.cnf are provided by the SUNWopenssl-commands and SUNWopensslr packages respectively, which are included in the End User (and greater) package meta-clusters. So if your system is installed with the Core (or lesser) package meta-cluster, you will need to install these two packages.
  1. Set up certificates. Read this section in its entirety before trying any of the steps, as there is an important note at the end of the section regarding certificate expiration. (Note that this section was derived from another page written by Greg Shapiro of Sendmail.)
    To create a certificate authority:
    1# cd /etc/mail
    2# mkdir -p certs/CA
    3# cd certs/CA
    4# mkdir certs crl newcerts private
    5# echo "01" > serial
    6# cp /dev/null index.txt
    7# cp /etc/sfw/openssl/openssl.cnf .
    Note that the above source path is for Solaris 10; if anyone happens to be doing the same on say, Mac OS X, the source path is /System/Library/OpenSSL/openssl.cnf; FreeBSD 4.11 puts the file in /etc/ssl/openssl.cnf.
    8# vi openssl.cnf
    Set values: I changed the dir value from /etc/sfw/openssl to /etc/mail/certs/CA and the stateOrProvinceName_default value from Some-State to California; the former was important but the latter was not.)
    9# openssl req -new -x509 -keyout private/cakey.pem -out cacert.pem -days 365 -c
    onfig openssl.cnf
    Notes on how to answer the questions: for Organization Name, I answer Sun Microsystems, for Organizational Unit Name, I answer Solaris, for Common Name, I answer with the fully-qualified host-name (FQHN) of the machine in question (see check-hostname(1M) for help on this), and for Email Address, I answer with my e-mail address.
    To make a new certificate:
    10# openssl req -nodes -new -x509 -keyout newreq.pem -out newreq.pem -days 365 -
    config openssl.cnf
    (The certificate and private key are in the file newreq.pem.) To sign the new certificate with the certificate authority:
    11# openssl x509 -x509toreq -in newreq.pem -signkey newreq.pem -out tmp.pem
    12# openssl ca -config openssl.cnf -policy policy_anything -days 365 -out newcer
    t.pem -infiles tmp.pem
    13# rm -f tmp.pem
    (The file newcert.pem contains the signed certificate; newreq.pem still contains the unsigned certificate and private key.)

    Note that commands 9, 10 and 12 have a -days 365 argument, which specifies how long until the certificate expires. There does not appear to be a reliable way to extend certificates, and going back to every machine and reapplying this process a year later is a major pain, as I found out the hard way. So the second time thru, I calculated how many days it would be until January 18, 2038, and used that value. :-)

  2. Tell about the certificates. (Note that this section was derived from another page written by Claus Aßmann, also of Sendmail.) Add:
    define(`confCACERT_PATH', `/etc/mail/certs')dnl
    define(`confCACERT', `/etc/mail/certs/CAcert.pem')dnl
    define(`confSERVER_CERT', `/etc/mail/certs/MYcert.pem')dnl
    define(`confSERVER_KEY', `/etc/mail/certs/MYkey.pem')dnl
    define(`confCLIENT_CERT', `/etc/mail/certs/MYcert.pem')dnl
    define(`confCLIENT_KEY', `/etc/mail/certs/MYkey.pem')dnl
    to your .mc file and rebuild your file and install it in /etc/mail.
  3. General sym-links:
    # cd /etc/mail/certs
    # ln -s CA/cacert.pem CAcert.pem
    # ln -s CA/newcert.pem MYcert.pem
    # ln -s CA/newreq.pem MYkey.pem
  4. Fix permissions:
    # chmod go-r MYkey.pem
  5. Specific sym-link (note that this section was also derived from Claus' STARTTLS page):
    # C=CAcert.pem
    # ln -s $C `openssl x509 -noout -hash < $C`.0
  6. Install other host(s) certificates: For any hosts you wish to exchange secure mail with, grab that host's copy of /etc/mail/certs/CAcert.pem (or whatever file's CACertFile option points to on that host) and copy it to /etc/mail/certs/host.domain.cert.pem on your host (where host.domain is the other host's FQHN), and repeat step 5, except using C=host.domain.cert.pem.
By doing the above, I am able to send mail between hosts which are both running a version of sendmail compiled with STARTTLS defined and linked with both libssl and libcrypto, and the Received: header shows e.g.:
Received: from ([IPv6:2002:8192:56bb:9259::8192:5932])
        by (8.13.4+Sun/8.13.4) with ESMTP id j2TNUB8i242496
        (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK)
        for <>; Tue, 29 Mar 2005 15:30:11 -0800 (PST
Received: from (opal.SFBay.Sun.COM [])
        by (8.13.4+Sun/8.13.4) with ESMTP id j2TNU7cl571102
        (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK)
        for <>; Tue, 29 Mar 2005 15:30:07 -0800 
where the verify=OK is the key part (see Claus' STARTTLS page for an explanation of what the various verify= values correspond to).

Thursday Aug 31, 2006

using ZFS to migrate data across boxes

My old desktop (with SCSI disks) died, and I'd been limping along on a loaner until the new one (with SATA disks) arrived a few days ago. The steps needed to migrate the data from one to the other turned out to be remarkably simple, so I thought I would share them in case anyone else ever runs into this. The old box had a single pool:
NAME                    SIZE    USED   AVAIL    CAP  HEALTH     ALTROOT
space                    68G   16.6G   51.4G    24%  ONLINE     -
with two file-systems therein:
space                 16.6G  50.3G    99K  /export/home
space/jbeck           12.2G  50.3G  12.2G  /export/home/jbeck
space/local           4.25G  50.3G  4.25G  /usr/local
First, on the new box, I ran:
% zpool create space mirror c2d0 c3d0
  • I temporarily tweaked /etc/ssh/sshd_config and /.ssh/authorized_keys to allow root from the old box to ssh to the new without being prompted for a password (not sure if this step was needed or not)
  • Since my desktop is also my mail server, I disabled sendmail to make sure that no mail would come in during the migration
  • On the old box I ran:
% zfs snapshot -r space@today
% zfs send space/jbeck@today | ssh newbox zfs recv -d space
% zfs send space/local@today | ssh newbox zfs recv -d space
The two send/recv runs took roughly 30 and 10 minutes respectively, much less than the almost two hours that the back-to-back tar method used to take to do this. Finally, on the new box, I ran:
% zfs set mountpoint=/export/home space
% zfs set mountpoint=/usr/local space/local
% zfs set sharenfs=on space/jbeck space/local
And that was it! I'm a happy ZFS user, but note that Matt told me that he is working on an RFE to make the above even simpler:
I'm working on some enhancements to zfs send/recv that will simplify this even further, especially in cases where you have many filesystems, snapshots, or changed properties. In particular, you'll be able to simply do:
# zfs snapshot -r space@today
# zfs send -r -b space@today | ssh newbox zfs recv -p -d newpool
The "send -b" flag means to send from the beginning. This will send a full stream of the oldest snapshot, and incrementals up to the named snapshot (eg, from @a to @b, from @b to @c, ... @j to @today). This way your new pool will have all of the snapshots from your old pool.

The "send -r" flag means to do this for all the filesystem's descendants as well (in this case, space/jbeck and space/local).

The "recv -p" flag means to preserve locally set properties (in this case, the mountpoint and sharenfs settings).

For more information, see RFEs 6421959 and 6421958, and watch for a forthcoming formal interface proposal.

Nice work, ZFS team.

Thursday Jun 08, 2006

Network Auto-Magic

For the last year or so, I've been leading a project called Network Auto-Magic to simplify and automate network configuration on Solaris. See the project page on OpenSolaris for full details about story boards, requirements, architecture, etc., but the exciting news is that we just released a prototype yesterday.

It monitors network interfaces and brings them up and down as availability changes, including the ability to activate user-specified hooks (such as a VPN) automatically. It will scan wireless networks, then offer the user a list of each, including signal strength and encryption type, plus an Other choice to allow the user to select a WLAN which does not broadcast its ESSID. When the user makes a selection, it will remember the selected WLAN, and will connect to it automatically thereafter. It will also prompt for WEP keys as needed and remember them, so such prompting need occur only once. So if you have a recent build of Solaris, try it out and let us know what you think!

Tuesday Oct 18, 2005

Stand up for what's Right

People a little older than me (I was 3 at the time) will remember the "black power salute", one of the most famous photographs in sports history. In the 1968 Olympic Games in Mexico City, while the national anthem played during the awards ceremony of the men's 200-meter run, gold medalist Tommie Smith and bronze medalist John Carlos stood with their heads bowed, each with a black-gloved fist held straight up. Smith raised his right fist to represent black power, Carlos his left to represent black unity, and both had taken off their shoes and just wore black socks to represent black poverty. The silver medalist, Peter Norman from Australia, wore a pin on his sweat jacket as a show of unity. Smith and Carlos were sent home from the games as a result of their stand, and were harassed and threatened when they got back to America.

Well, that was 1968. As years went by, more and more people saw their stand in a more positive light, understanding that they were saying, in a dignified way, that it was not right for Black Americans to be given equal treatment only when convenient for the country, such as in war or in sports, but treated as second class citizens otherwise. So by 2005, Smith and Carlos tend to be viewed as heroes more than anything else, and heroes overdue for recognition at that.

As it turns out, both Smith and Carlos (and Lee Evans, who won gold in the 400-meter run that year, and participated with them in their planning of "the stand") went to San Jose State University, and a few years ago a student there learned about their story and thought the university should do something to commemorate their stand. It took a few years, but last night (37 years and 1 day after their courageous stand), there was a ceremony on campus at San Jose State, with Smith, Carlos, Norman and Evans all in attendance, where a 20-foot statue was unveiled. I was on hand for the event, and took some pictures.

award ceremony

In the cropped picture above, Carlos is on the left in the bluish suit with the white goatee, Smith is on the right in the greenish suit, Norman is standing behind Carlos' left shoulder, and Evans is standing behind Smith's left shoulder; San Jose Vice Mayor Cindy Chavez is presenting them with an award. In the uncropped version of the picture, you can see how big the (still veiled) statue is.


The statue, seen unveiled, is magnificent. The skin and hair and socks are bronze, while the sweat suits are ceramic tiles. Norman's spot on the dais was intentionally left empty, to make the statue interactive, so people could pose "taking a stand".

I think I was about 7 when I first learned about "the stand", probably during the 1972 Olympic Games in Munich. I remember thinking that they must have been very brave. As I got older, and eventually ran track in high school and college myself, I came to think of them more as heroes, the ultimate in track: winning a gold medal and doing something noble with one's moment of glory. It is hard to put into words how uplifting it was to see these heroes in the flesh, and to hear them speak. Smith and Carlos both credited God for helping them get thru their trying times and both urged the young people present at the ceremony to take a stand and do what's right. Amen, brothers.

Tuesday Jun 14, 2005

Adding command-line editing/history to an application

As soon as Zones integrated into Solaris 10, we started getting some RFEs for ways to make it better. One of them was to add command history to zonecfg; now that OpenSolaris has been launched, I thought I would share how I did it.

Some existing libraries were suggested for enabling this; after studying them, I recommended libtecla, written by Martin Shepherd, a staff scientist / programmer in the Astronomy department at Caltech, because of its feature set (command-line editing and history, highly customizable, etc.), quality, and license.

Getting libtecla ported to Solaris was trivial, as it already compiled and linked fine. I just had to massage it into the Solaris format, which did not involve any significant code changes, but mainly Solaris Makefiles, lint library stubs, etc. These can be found under usr/src/lib/libtecla for the curious.

The more interesting work was tweaking zonecfg to use libtecla, which included the following steps, which are listed here in a way intended to make it easy to follow for other people wishing to do the same with their application(s):

  • header file:
    #include <libtecla.h>
  • some constant definitions:
    #define	MAX_LINE_LEN	1024
    #define	MAX_CMD_HIST	1024
  • global variable:
    static GetLine \*gl; /\* The gl_get_line() resource object \*/
  • initialization: near the beginning of main():
    	if ((gl = new_GetLine(MAX_LINE_LEN, MAX_CMD_HIST)) == NULL)
    Note that Z_ERR is a constant with the value of 1, but the important thing for the general case is that it is some non-zero value.
  • (optional) customization of tab-completion:
    	if (gl_customize_completion(gl, NULL, cmd_cpl_fn) != 0)
    Note that in the case of zonecfg, cmd_cpl_fn() is not particularly interesting, mainly consisting of lots of calls to cpl_add_completion() to cover various cases in the grammar; see the cpl_complete_word(3TECLA) man page for details.
  • clean-up: near the end of main():
    	(void) del_GetLine(gl);
  • the heart of it:
    If your program has some sort of construct like:
    	for (;;) {
    		prompt = prompt_value();
    		line = read_input();
    		if (line == NULL)
    then it would be rewritten thus:
    	for (;;) {
    		prompt = prompt_value();
    		line = gl_get_line(gl, prompt, NULL, -1);
    		if (gl_return_status(gl) == GLR_SIGNAL) {
    		if (line == NULL)
    In the case of zonecfg, however, it was slightly more complicated because of its use of lex(1)/yacc(1), so I needed to convert the char \*line into the FILE \*yyin, then call yyparse(). So the code became:
    	for (;;) {
    		prompt = prompt_value();
    		line = gl_get_line(gl, prompt, NULL, -1);
    		if (gl_return_status(gl) == GLR_SIGNAL) {
    		if (line == NULL)
    		(void) string_to_yyin(line);
    where the string_to_yyin() function is thus:
    static int
    string_to_yyin(char \*string)
    	if ((yyin = tmpfile()) == NULL) {
    		zone_perror(execname, Z_TEMP_FILE, TRUE);
    		return (Z_ERR);
    	if (fwrite(string, strlen(string), 1, yyin) != 1) {
    		zone_perror(execname, Z_TEMP_FILE, TRUE);
    		return (Z_ERR);
    	if (fseek(yyin, 0, SEEK_SET) != 0) {
    		zone_perror(execname, Z_TEMP_FILE, TRUE);
    		return (Z_ERR);
    	return (Z_OK);
    where execname is a global variable, zone_perror() is a perror(3C)-like function, and Z_TEMP_FILE is a constant which maps to an error message indicating "Problem creating temporary file", localized appropriately.
For those who would like to see all of zonecfg.c in context; it can be found at usr/src/cmd/zonecfg/zonecfg.c.

Technorati Tag:
Technorati Tag:

Wednesday Jan 26, 2005

building a better spam trap

I've spent a lot of time over the past couple of months trying out some new (and some not so new) anti-spam techniques. Note that this article assumes some familiarity with sendmail m4 macros; see $CFDIR/README for background and all sorts of details on these, where $CFDIR is one of:
  • /etc/mail/cf on Solaris 10
  • /usr/lib/mail on Solaris 7, 8 or 9
  • the cf sub-directory of the sendmail distribution for people "rolling their own"

These techniques are in the form of FEATURE and HACK m4 macros (the difference being that the former are provided and blessed by / Solaris whereas the latter are not, though a HACK may evolve into a FEATURE in a future release). For a HACK, one would use

in one's .mc file, likewise
When installing hacks, one must create $CFDIR/hack (if it does not already exist) and place hack-name.m4 in that directory. Note that the sendmail distribution comes with such a sub-directory but Solaris does not.

Also, to explain some terms used below: the access list is enabled by the FEATURE(`access_db') macro; details on this are in $CFDIR/README, both in its sub-section in the FEATURES section, and in the ANTI-SPAM CONFIGURATION CONTROL section. And FEATURE(`delay_checks') is strongly recommended, as it is needed to enable the overrule by an OK entry in the access list that I mention in a few places; this feature is also described in its subsection in the FEATURES section, as well as in the "Delay all checks" sub-section of the ANTI-SPAM CONFIGURATION CONTROL section.

Anyway, onto the details. In the order I started deploying them:

  • The first is HACK(`block _bad_helo'), written by Neil Rickert, a professor in the Computer Science Department at Northern Illinois University and a volunteer contributor. SMTP clients are supposed to send the client FQHN (fully qualified host name) as the HELO/EHLO parameter, but many broken clients send the server FQHN (or IP address) instead, or something without a ".". This rejects any such transmissions. The upside is that I have found it to block a good amount of spam, with no false positives for me. A couple of users of my personal domain have had some small number of false positives with it, though. And the down side is that it cannot be overruled by an OK entry in the access list. Bart has had a lot of troubles with this rule; apparently old versions of Netscape and early version of Mac's Mail.App got this wrong.
  • The second is a regular feature: DNS-based black-lists. I use (note: line wrapped for readability)
      FEATURE(`enhdnsbl', `',
            `"Spam blocked see:"$&{client_addr}',
    while Bart uses
      FEATURE(`dnsbl', `')dnl
    Both have proven extremely effective with very few false positives, and this feature, using whichever list, has the added virtue of allowing an OK override in the access list.
  • The last is HACK(`require_rdns'), also written by Neil Rickert. I enhanced Neil's original version so that it would allow an OK override in the access list. This enhanced version has been unbelievably effective, in sheer numbers, while also so far amazingly accurate (I estimated a false positive rate of 0.5% after a few weeks, but I think that may have gone even lower since I white-listed a few sites). As the name suggests, it requires that the SMTP client's IP address reverse map to some name, and also that the name forward map to the same address. (An IP address can have multiple A records; this merely requires that the original IP address is one of them.) This is the single most effective anti-spam rule I have ever deployed.

Overall, spam getting thru my personal domain's mail server to my users (including myself, my wife, my siblings, our mom, etc.) has dropped about 90% since I started using these techniques, despite the ever-increasing spam trends on the rest of the Internet.

Sunday Oct 10, 2004

what is relaying?

I just answered this in comp.unix.solaris for what seems like the millionth time, so I thought I might as well record the answer here for others who might be interested...

People who run mail servers need to be careful not to be an "open relay", or they will get added to various black-lists, and all sorts of mail delivery problems will ensue, as spammers take advantage of open relays when and wherever they find them.

Basically, a message has to either "start here" or "finish here" or it is considered a relay. There is a test and a parameter for each condition. The parameter for "finish here" is the domain part (i.e., right-hand side) of each recipient's e-mail address, and the test is "is that my host name?", though it is often extended to "is that host in my domain?". The parameter for "start here" is the IP address of the client side of the SMTP connection, and the test is "is this the loopback address" (i.e., "is this the local host?"), though that is also often extended to "does this IP address reverse-map to a name in my domain?"

Note that sendmail has disabled relaying by default since version 8.9, which first shipped with Solaris 7, and that the .mc file used to generate the default on Solaris contains this macro:

and that domain file contains:
which enables the extensions to which I alluded above. Details can be found in the README file, which is in /usr/lib/mail/ on Solaris 9 and earlier, but in /etc/mail/cf/ on Solaris 10; see the ANTI-SPAM CONFIGURATION CONTROL section of that file in particular.

Monday Oct 04, 2004

no more main vs subsidiary

A common gotcha when setting up a Solaris box on the Internet and trying to send mail has been:

Host unknown (Name server: host not found)
This is because Solaris has for many years shipped two sendmail config files: one for the "real" Internet ( and one for inside a firewall or elsewhere where DNS may not be configured properly, and average "dumb" hosts need to punt to some "smart" host ( As you might guess from the above message, that "smart" host is assumed to have the name mailhost.$m (where $m is the macro for the local domain, "" in the above example) . For backwards compatibility reasons, has always been a copy of rather than .

Well, starting in Solaris 10 build 63, this problem is fixed. More accurately, the fix is in sendmail version 8.13, which was introduced in s10_63; hopefully, 8.13 will be back-ported to Solaris 9 before too long. I should have thought of this years ago, but better late than never: by inventing a new FallbackSmartHost option, sendmail can handle either scenario in a single configuration:

If specified, the FallBackSmartHost will be used in a last-ditch effort for each host. This is intended to be used by sites with "fake internal DNS", e.g., a company whose DNS accurately reflects the world inside that company's domain but not outside.
This option is specified, with the value mailhost.$m as used to use, in the default, and there is no longer a or a . (Well, technically, they both can still be found, but only as sym-links to for backwards compatibility.)

Finally, after all these years, a grand unified sendmail configuration! :-)

Friday Oct 01, 2004

"rm -rf /" protection

Most people who have spent any time on any version of Unix know that "rm -rf /" is about the worst mistake you can make on any given machine. (For novices, "/" is the root directory, and -r means recursive, so rm keeps deleting files until the entire file system is gone, or at least until something like libc is gone after which the system becomes, as we often joke, a warm brick.) Well a couple of years ago one Friday afternoon a bunch of us were exchanging horror stories on this subject, when Bryan asked "why don't we fix rm?" So I did.

The code changes were, no surprise, trivial. The hardest part of the whole thing was that one reviewer wanted /usr/xpg4/bin/rm to be changed as well, and that required a visit to our standards guru. He thought the change made sense, but might technically violate the spec, which only allowed rm to treat "." and ".." as special cases for which it could immediately exit with an error. So I submitted a defect report to the appropriate standards committee, thinking it would be a slam dunk.

Well, some of these standards committee members either like making convoluted arguments or just don't see the world the same way I do, as more than one person suggested that the spec was just fine and that "/" was not worthy of special consideration. We tried all sorts of common sense arguments, to no avail. In the end, we had to beat them at their own game, by pointing out that if one attempts to remove "/" recursively, one will ultimately attempt to remove ".." and ".", and that all we are doing is allowing rm to pre-determine this heuristically. Amazingly, they bought that!

Anyway, in the end, we got the spec modified, and Solaris 10 has (since build 36) a version of /usr/bin/rm (/bin is a sym-link to /usr/bin on Solaris) and /usr/xpg4/bin/rm which behaves thus:

[28] /bin/rm -rf /
rm of / is not allowed

Thursday Sep 30, 2004

sendmail's new paradigm

Well, it's not all that new, as sendmail 8.12 was released over three years ago, but since few people pay attention to sendmail more than they have to, the major changes in 8.12 and their side effects may have gone unnoticed.

First, some background: sendmail has some historical notoriety from a security point of view, since it for many years was a setuid-root application, and thus a buffer overflow for example could make sendmail a vector for an unprivileged user to take over the system. Starting with version 8.12, however, sendmail lost its setuid bit in favor of becoming setgid to a new group.

To make queueing work, a split-queue model was created: /var/spool/mqueue as the traditional "server" queue with the same 750 root, bin permissions, and the new /var/spool/clientmqueue as the new (you guessed it) "client" queue with 770 smmsp, smmsp permissions. For the curious, "smmsp" is the new user and group invented for these purposes, having UID and GID values of 25 (chosen to match the SMTP port value in /etc/services), and it stands for "SendMail Message Submission Program".

The default configuration involving all "direct submission" messages [meaning messages submitted via sendmail invoked from the command line or comparably by programs such as mailx(1)] will go to the client queue. Messages in the client queue are then transferred to the SMTP server on the local host, which then proceeds in the traditional way. Thus running a sendmail daemon has become critical even for out-bound mail, at least from certain sources [such as cron(1M)]. See entry 3.44 of the sendmail FAQ for more details on this, as well as how to configure around this for hosts where accepting in-bound from other hosts is undesirable.

Friday Sep 17, 2004

Why HTML is bad for mail

Many people with whom I regularly correspond know that I have a strong preference against mail in "pure" HTML format. Here are my reasons, along with recommended alternatives.
  1. Spammers can hide tiny images: just a single pixel, small enough that your eye would likely miss it anyway, and the more insidious ones are the same color as the background. The URLs provided for your mail reader to fetch these images can contain encoded data which spammers can use to confirm your identity. I.e., each message they send can contain a slightly different URL, and when a mail reader fetches an image, it serves as confirmation for the spammers that the address corresponding to that particular URL is "live". This will in turn increase the likeliness of getting even more spam.
  2. Not all mail readers handle HTML well. Most modern mail readers do, especially those with a GUI, but many older mail readers, especially those which are screen-based, handle HTML badly or not at all. Some people may use a GUI mail reader at work but a screen-based one at home or when traveling.
  3. HTML takes up more bandwidth than plain text. Although this is not an issue in many environments (such as a high-speed LAN), in other environments (such a when traveling and stuck with a low-speed dial-up line) the extra bandwidth can be quite inconvenient.
The ideal alternative, when possible, is simply to send plain text, as it is sufficient for the vast majority of e-mail conversations.

When richer mark-up is needed, however, most mail programs which can generate HTML can also generate mixed text and HTML: the message's primary MIME type is multipart/alternative, with the first part being text/plain and the second part being text/html. Conforming mail readers will display HTML if they understand it, or plain text if they don't grok HTML (unless the user has configured it to display plain text by default).

For the curious, I use exmh at work and home but nmh (the CLI-based mail reading set of programs on which exmh is based) when on the road or in any other low-bandwidth environment. Exmh can display HTML, but it is much slower than displaying plain text, and not all constructs are well supported. I have mine configured to display plain text by default for multipart/alternative messages. I also have SpamAssassin configured to score "pure" HTML mail very highly, as the vast majority of such mail which I receive is indeed spam.

Thursday Sep 16, 2004

getting started

I post to Usenet occasionally about Zones, more often about sendmail. More to follow...



« December 2016