Wednesday Aug 01, 2007

Solaris Non-Executable Stack Concluded

Since publishing my two part series on non-executable stacks in the Solaris operating system, I received some very useful feedback and clarifications that I wanted to share with everyone. First, Vladimir Kotal commented on my first article that:

Having to grep(1) for the CPU features is really clumsy. Maybe psrinfo(1M) could be extended to print them out? (for every (virtual) CPU present in the system)

Frankly, I agree. After asking around however, today there does not appear to be a cleaner interface (although there is a bunch of discussion around adding one). Sherry Moore and Joe Bonasera were kind enough to point out that there is a programmatic way to access this information in the form of cpuid(7d). Joe also shared the following with me that you may find interesting:

The NX information doesn't belong in isainfo. isainfo, I'm told, is only meant to reflect processor capability information that is directly usable from user mode.

The NX bit feature has to do with page table construction which is not something you do from userland. What's a more interesting thing to know is "Does not specifying PROT_EXEC have any effect on this system, or is PROT_EXEC implicit for all PROT_READ segments?" Even cpuid doesn't help with that information as various bits of the OS memory subsystems might do different things along the way. For example if for some reason you're running a non-PAE 32 bit kernel, even though cpuid says that NX is supported, NX bits wont be used.

A similar issue has come up in the Open Solaris Xen project, in that many people want to know if their processor supports AMD-V or Intel VT-x. That information comes from CPUID, but is only usable from supervisor (either kernel or hypervisor) code, hence we haven't added it to isainfo. But it is a valid question to ask if the cpu/bios you have would support running such software w/o actually having it.

That said, Sherry did clue me in on a program called cpuid which can allow us to get this information and a lot more (subject to the issues noted by Joe above). Unfortunately, the cpuid program was developed for Linux and will not compile by default on Solaris:

blackhole$ gmake
cc -g -Wall -Wshadow -Wcast-align -Wredundant-decls -Wbad-function-cast -Wcast-qual -Wwrite-strings -Waggregate-return 
-Wstrict-prototypes -Wmissing-prototypes -D_FILE_OFFSET_BITS=64 -DVERSION=20070801 -o cpuid cpuid.c
cpuid.c:26:25: linux/major.h: No such file or directory
cpuid.c: In function `explain_errno':
cpuid.c:3191: error: `CPUID_MAJOR' undeclared (first use in this function)
cpuid.c:3191: error: (Each undeclared identifier is reported only once
cpuid.c:3191: error: for each function it appears in.)
cpuid.c: In function `real_setup':
cpuid.c:3472: warning: implicit declaration of function `makedev'
cpuid.c:3472: error: `CPUID_MAJOR' undeclared (first use in this function)
cpuid.c: In function `main':
cpuid.c:3751: warning: initialization discards qualifiers from pointer target type
cpuid.c:3752: warning: initialization discards qualifiers from pointer target type
cpuid.c:3753: warning: initialization discards qualifiers from pointer target type
cpuid.c:3754: warning: initialization discards qualifiers from pointer target type
cpuid.c:3755: warning: initialization discards qualifiers from pointer target type
cpuid.c:3756: warning: initialization discards qualifiers from pointer target type
cpuid.c:3757: warning: initialization discards qualifiers from pointer target type
gmake: \*\*\* [cpuid] Error 1

Luckily, the changes to get this program to work on Solaris were simple (Thanks Sherry!). All that we needed to do was remove the references to /dev/cpu/\* as that is a Linux-ism that does not exist on Solaris. Here is the complete diff for those wanting to try this at home:

blackhole$ diff linux-cpuid.c cpuid.c
> #if 0
> #endif
> #if 0
> #endif
> #if 0
> #endif

Clearly, if you wanted the program to work on either OS, you could just substitute the #if 0 strings for something like #if !defined(SOLARIS) and then just define SOLARIS in the CFLAGS parameter when compiling on Solaris. But I digress... With this simple change implemented, you can now compile the cpuid program on Solaris:

blackhole$ gmake
cc -g -Wall -Wshadow -Wcast-align -Wredundant-decls -Wbad-function-cast -Wcast-qual -Wwrite-strings -Waggregate-return 
-Wstrict-prototypes -Wmissing-prototypes -D_FILE_OFFSET_BITS=64 -DVERSION=20070801 -o cpuid cpuid.c
cpuid.c: In function `main':
cpuid.c:3757: warning: initialization discards qualifiers from pointer target type
cpuid.c:3758: warning: initialization discards qualifiers from pointer target type
cpuid.c:3759: warning: initialization discards qualifiers from pointer target type
cpuid.c:3760: warning: initialization discards qualifiers from pointer target type
cpuid.c:3761: warning: initialization discards qualifiers from pointer target type
cpuid.c:3762: warning: initialization discards qualifiers from pointer target type
cpuid.c:3763: warning: initialization discards qualifiers from pointer target type
gzip < >

These warnings can be safely ignored. With the program now compiled, let's give it a try and see what it can tell us about the NX bit:

blackhole$ ./cpuid | grep exec
      execution disable                      = false

Interesting. This system does not have the NX capability likely because I am running (Nevada in this case) in a Parallels VM which is 32-bit (reference Joe's note above). Let's give this a better test subject by trying it on a Sun X2100. This command is run from the global zone of a system running Solaris 10 11/06:

$ ./cpuid | grep exec
      no-execute page protection            = true

Careful observation will also show the AMD and Intel naming differences that I had talked about previously with respect to XD and NX.

Well, I think that I have talked about this subject to death. I hope that you found it interesting and perhaps a little educational. As always, I love to get your feedback! Before signing off, once again I would like to thank Sherry Moore and Joe Bonasera for sharing their knowledge and experience with me (and thereby with you)!

Take care,


Technorati Tag:

Wednesday Jul 25, 2007

Solaris Non-Executable Stack Continued

Previously, we covered some of the history and basics of Solaris non-executable stacks and how they can be enabled globally on both SPARC and x86/x64 systems. In this article, we extend that foundation by talking about how developers can configure their own programs to have non-executable stacks, regardless of the value of the global system setting, noexec_user_stack.

This little bit of magic is accomplished through the use of a linker map file. In the case of non-executable stacks, the linker map file in question is /usr/lib/ld/map.noexstk. Simply specifying this map file during a compilation or link will cause the resulting program to have a non-executable stack. Looking at the comments in this file, we see how this is accomplished:

#ident  "@(#)mapfile_noexstk    1.3     01/07/13 SMI"
# Copyright (c) 2001 by Sun Microsystems, Inc.
# All rights reserved.
# Linker mapfile to create a non-executable stack definition within an
# executable.
# The linker does not use this file automatically, so one must use the -M 
# option to cc or ld:
#       cc -M /usr/lib/ld/map.noexstk myprogram.c
stack = STACK ?RW;

If this sounds pretty straightforward and easy to use, that is because it is! Let's go ahead and give it a try! Before we begin, I would like to thank Scott Rotondo for sharing with me the following sample program. This program will attempt to execute code on the stack. Our test system is configured with noexec_user_stack=0 and we will compile our test program both with and without using the map file so that they can be compared with one another.

First, here is our test program:


int x = 0;


typedef void (\*funcptr)(void);

main(int argc, char \*\*argv)
        funcptr f;
        char code[100];

        /\* Copy the incr() function to the stack. \*/
        memcpy(code, (void \*)incr, sizeof(code));
        f = (funcptr)code;

         \* Increment x twice, once by calling incr() and
         \* once by running the copy on the stack.
        printf("x = %d\\n", x);
        printf("x = %d\\n", x);
        printf("x = %d\\n", x);
        return (0);

Now, let's compile the program (with and without the map.noexstk map file):

$ gcc -O -o incr incr.c
$ gcc -O -o incr-nx -Wl,-M,/usr/lib/ld/map.noexstk incr.c

(Thank you to Luke for pointing out a cleaner way to pass the linker map file using gcc!)

Note that if you were using the Sun C compiler, you could have used the following commands:

$ cc -O -o incr incr.c
$ cc -O -o incr-nx -M /usr/lib/ld/map.noexstk incr.c

So, how do we know that the program, incr-nx, has a non-executable stack? One of the easiest ways is to use the elfdump(1) command telling it to look for the program header type, PT_SUNWSTACK. The absence of this program header means that the program is effectively in a default configuration where (depending on the platform) the stack segment could be readable, writable as well as executable. If a PT_SUNWSTACK program header is found then the default is not being used, and we need only to look at the p_flags parameter to see what permissions are being assigned to the stack segment.

$ elfdump -p -N PT_SUNWSTACK incr
$ elfdump -p -N PT_SUNWSTACK incr-nx

Program Header[5]:
    p_vaddr:      0           p_flags:    [ PF_W PF_R ]
    p_paddr:      0           p_type:     [ PT_SUNWSTACK ]
    p_filesz:     0           p_memsz:    0
    p_offset:     0           p_align:    0

As you can see from the output of the two commands above, the incr program's stack segment is configured in the default manner and will therefore have an executable stack (unless of course the global system parameter noexec_user_stack is set to 1). On the other hand, the incr-nx program does have a PT_SUNWSTACK program header. Looking at the p_flags parameter, we see that this program's stack segment will have only the read (PF_R) and write (PF_W) flags enabled.

The next obvious question is whether these programs will behave differently. Certainly, we would expect them to given that they are configured to execute code on the stack yet such an operation is only permitted in one of the two programs. Let's take a closer look:

$ ./incr
x = 0
x = 1
x = 2
$ ./incr-nx
x = 0
x = 1
Segmentation Fault (core dumped)

If we had enabled logging of attempts to execute code on the stack using the noexec_user_stack_log parameter, we would have also seen a syslog message similar to:

$ tail -1 /var/adm/debug
Jul 25 22:11:36 quasar genunix: [ID 533030 kern.notice] NOTICE: incr-nx[12553] attempt to execute code on stack by uid 101

Pretty cool, eh? So with the simple addition of the linker map file, we can now deploy programs and services that will have non-executable stack segments (out of the box)! In fact, a large portion of the ON (operating system and networking) consolidation in the Solaris OS is already configured this way! In fact, even the Sun-contributed Firefox (that is also included in Solaris 10 and OpenSolaris) uses this mechanism to enable non-executable stacks. Yes, even OpenOffice/StarOffice and Xorg are in on the action! So, what are you waiting for? Give it a try today!

I hope you enjoyed this brief overview into Solaris non-executable stacks. As always, I would love to get your feedback and ideas. You can read more on this topic here and here.

Take care,


Technorati Tag:

Solaris Non-Executable Stack Overview

The ability to configure a Solaris system to run with non-executable stacks is not overly new. That functionality was originally introduced into the Solaris 2.6 operating system with the noexec_user_stack kernel parameter. Looking at the source code, this is how this parameter was documented (in usr/src/uts/common/vm/seg_vn.c):

 207 /\* 
 208  \* Patching this variable to non-zero allows the system to run with 
 209  \* stacks marked as "not executable".  It's a bit of a kludge, but is 
 210  \* provided as a tweakable for platforms that export those ABIs 
 211  \* (e.g. sparc V8) that have executable stacks enabled by default. 
 212  \* There are also some restrictions for platforms that don't actually 
 213  \* implement 'noexec' protections. 
 214  \* 
 215  \* Once enabled, the system is (therefore) unable to provide a fully 
 216  \* ABI-compliant execution environment, though practically speaking, 
 217  \* most everything works.  The exceptions are generally some interpreters 
 218  \* and debuggers that create executable code on the stack and jump 
 219  \* into it (without explicitly mprotecting the address range to include 
 220  \* PROT_EXEC). 
 221  \* 
 222  \* One important class of applications that are disabled are those 
 223  \* that have been transformed into malicious agents using one of the 
 224  \* numerous "buffer overflow" attacks.  See 4007890. 
 225  \*/ 

While non-executable stacks provide are a very useful technique for thwarting certain kinds of buffer overflow attacks, it should be noted that there exist other attack methods that do not rely on executable stacks. One such method was discussed back in 1999 on Bugtraq, but even in this case the author noted that there was inherent value in non-executable stacks (if only as an additional defense in depth layer):

Hopefully, these exploits demonstrate that it is important to make sure that programs that run at an elevated privilege are free of buffer overflow bugs. The stack protection will certainly help protect you from the majority of intruders, but moderately competent intruders will probably be able to bypass it.

Just as with minimization, hardening, and the deployment of services with reduced privilege, non-executable stacks are just another layer or tool to be used as part of a more comprehensive security architecture. But anyway, back to our story...

As with other kernel parameters, the non-executable stack state can be be adjusted (enabled or disabled) using the /etc/system file. For example, the following statement added to /etc/system would enable this feature:

set noexec_user_stack=1

As noted in the inline documentation above, experience has shown that "most everything works". In fact, the recommendation to enable this feature has been in Sun BluePrints since 1999 and similarly in the Solaris Security Toolkit since its inception. Looking even further, you find this common recommendation across the industry.

As a companion to this parameter, the noexec_user_stack_log parameter could be used to enable logging when this feature (if enabled) detected an attempt to run code from the stack. By default, this parameter is enabled if the noexec_user_stack parameter is enabled so no further action is required unless of course you want to prevent such logging. That has not stopped authors of tools and articles from recommending to enable it anyway using the command:

set noexec_user_stack_log=1

When this parameter is enabled and there is an attempt to execute code on the stack, a message such as the following will be generated and delivered via syslog to kern.notice:

Jul 25 14:48:02 quasar genunix: [ID 533030 kern.notice] NOTICE: myprog[12289] attempt to execute code on stack by uid 101

In this way, a system administrator can detect such attempts and take appropriate action.

Back in the days of Solaris 2.6, this parameter really only applied to the SPARC platform. Years passed and this feature continued to be available in Solaris 7, Solaris 8 and so on. As good fortune would have it, Intel and AMD got on board with the idea and the NX Bit was born. Technically speaking, Intel refers to its implementation as the XD Bit (for Execute Disable) while AMD has used the term NX (for No Execute), but for the purposes of Sun's implementation and this article, we will consistently use the term "NX" to refer to this functionality.

To find out if your system supports the NX bit, you can check in with the dmesg(1M) command:

$ dmesg | grep features
Jun 28 11:00:05 sec1 unix: [ID 126719] features: 1176fdf<cpuid,cmp,sse3,nx,asysc,sse2,sse,pat,cx8,pae,mca,mmx,cmov,pge,mtrr,msr,tsc,lgpg>

Similarly, if you have syslog configured to log messages, you can also get the information from your system log files:

$ grep "features:" /var/adm/debug
Jul 19 16:43:06 quasar unix: [ID 126719] features: 1076fff<cpuid,sse3,nx,asysc,sse2,sse,pat,cx8,pae,mca,mmx,cmov,de,pge,mtrr,msr,tsc,lgpg>

The first example was taken from a SunFire X2100 system whereas the second example was taken from an Ultra 20. The same commands should be able to be used on other x86/x64 systems in order to determine if this CPU feature is available.

On the SPARC platform, the non-executable stack functionality is available but disabled by default (for SPARC V8) in order to support a fully ABI-compliant execution environment. For 64-bit SPARC platforms, however, the SPARC V9 ABI specifies a non-executable stack by default. Note that 32-bit applications running on a 64-bit kernel do not automatically get this protection by default and would rely on the noexec_user_stack parameter being set to 1 for example.

On NX-capable x86/x64 platforms, Solaris OS uses the NX bit by default whenever PROT_EXEC is not specified. Stack segments, however, use PROT_EXEC by default, so the NX functionality must be explicitly enabled on these platforms to provide stack protection. As noted above, this can be globally configured using the noexec_user_stack parameter just as with SPARC-based platforms.

From the product documentation, it should be noted that a system administrator can disable all use of the NX bit (non-SPARC platforms) by using the eeprom(1M) command to set enforce-prot-exec to off. This variable is provided as a transition workaround for any system with legacy applications that are missing PROT_EXEC.

In this article, we have taken a brief look at the history of non-executable stacks in Solaris dating back to the original integration of this functionality in Solaris 2.6 all the way to the present. In the next article, we will talk a little bit about how this functionality can be enabled on a per-file basis in the Solaris 10 OS.

Take care,


Technorati Tag:

Wednesday Feb 28, 2007

Tracking Infected Telnet Worm Machines

Today, there has been a lot of discussion about the new telnet worm which exploits the recently announced telnet vulnerability in Solaris 10 and Nevada.

Aside from the usual recommendation of you should not be using telnet. You should be using SSH, I would like to cast a vote for the use of IP Filter. IP Filter is quick and easy to configure and can help give you visibility into attacks such as this. Beyond its initial use as an enforcement point (blocking access to services such as telnet), IP Filter is also a great tool to allow you to see what other systems are attempting to do to yours.

An IP Filter entry for the telnet worm may look something like:

Feb 27 15:26:38 blackhole ipmon[100]: [ID 702911 local0.warning] 15:26:38.269526 ip.tun0 @0:11 b,55039 ->,23 PR tcp len 20 52 -S I

With this format, you could quickly whip up a script to tell you who is knocking on your system's telnet door (even if telnet happens to be disabled - which is the case on my system). See:

blackhole$ getent hosts `grep  ipmon  /var/adm/debug | grep " b " |\\
   grep ",23 PR" | awk '{ print $13 }' | awk -F, '{ print $1 }' | sort -u`     europa     io     castor      pollux  orion  mercury
With just a little scripting, you can easily find out systems (particularly in an enterprise) that need some special love and attention.

Technorati Tag:

Tuesday Sep 26, 2006

Treo 700p on Nevada

Will wonders never cease? Today, I decided to plug my Treo 700p smart phone into my newly upgraded Solaris laptop. Honestly, I was not sure what would happen as this was the first time that I had tried to connect up a Palm device.

My goal for doing this was simple. I wanted to synchronize my calendar to my phone so that I would have a list of my appointments while I was on the road. I had wanted to use something more direct like SyncML, but that option was not available to me. Oh, well... I have been using Evolution lately to manage my appointments. What is interesting about my configuration is that my calendar is hosted on Sun's EdgeCal service which allows me to easily access and share my calendar from the Internet or within Sun. EdgeCal is basically a Sun Java System Calendar Server environment and I use the JESCS Evolution Connector to access EdgeCal. By the way, this all worked out of the box too!

So, back to today's experiment... Since Evolution already has an ability to synchronize with devices such as Palm Pilots, I decided to give that a try. The process was completely painless. I simply connected up the 700p via a USB port (actually on a USB hub since I am also using a USB keyboard and mouse), provided some basic settings information to Evolution (Pilot Synchronization Dialog) and hit the HotSync button. Evolution was able to not only find my device but also push the calendar information from EdgeCal to my phone in a matter of seconds. Way cool.

What is really nice is that I can also use the pilot-xfer command to also back up your device (to a ZFS partition in my case). You really have to love it when things just work.

Take care,


Technorati Tag:


This area of cyberspace is dedicated the goal of raising cybersecurity awareness. This blog will discuss cybersecurity risks, trends, news and best practices with a focus on improving mission assurance.


« February 2015