Sun Fire T2000 and Secure Applications

Sun Fire T2000 and Secure Applications

UltraSPARC T1 processor is the world's first Eco-responsible processor that uses patented CoolThreads (TM) chip multi-threading technology to leverage the threaded nature of Solaris (TM) 10. Combining the power of eight 4-way multithreaded cores, the UltraSPARC T1 processor offers 32 simultaneous processing threads while consuming power equivalent to a household light bulb.

PKCS#11 Compliant Cryptographic Accelerator for FREE!!

It goes further, in offering 8 on-chip Modular Arithmetic Units (MAU), one per core. These 8 MAUs extends the processor's capabilities to act as Cryptographic Accelerators at no additional cost. Solaris (TM) 10 combines the power of these 8 MAUs, and offers a device (NCP) that can do 14000 raw RSA ops/sec, more than sufficient for regular secure commercial applications like Online Banking.

Along with innovative features such as dTrace and Fault Management Architecture, Solaris (TM) 10, introduces a Cryptographic Framework that makes it easy for PKCS#11 based Cryptographic Consumers to plug into the NCP device.

ServerFarm Friendly System

With 4 onboard Gigabit ports, the 2U, UltraSPARC T1 based Sun Fire T2000 system consumes at least 50% less power than traditional Intel-based 2U systems and delivers at least 2.5x the performance. This, along with its free security offerings makes it ideal for a ServerFarm reducing the cost of power and cooling by at least 50%, allowing more room for expansion.

The Sun Fire T2000 can be used in almost any deployment including secure distributed applications such as:
  • Online Banking
  • Online Ecommerce
  • Online Auctioning
  • Secure Email
  • Secure WebServices Application

Two Cryptographic Solution

I will describe briefly how you can take advantage of this free Cryptographic Accelerator without compromising security or modifying any part of your application. For now I will only go over the important aspects of this unique and innovative offering. If you have specific question please post them here and I will try to answer them.

  • Userland Cryptographic Solution

    On Sun Fire T2000 systems, the Solaris (TM) Operating Environment is preconfigured to use the NCP module. Using the hooks provided by the Solaris Cryptographic Framework, PKCS#11 compliant consumers can offload the entire RSA processing into the NCP module significantly reducing CPU consumption thus allowing better scaling of applications.

    Servers like Apache should use the bundled openssl libraries and it's pkcs11 engine to hook into the NCP module. To use the pkcs11 engine with Apache set the following property in the Apache configuration file
    SSLCryptoDevice pkcs11
    The following is an example of openssl taking advantage of NCP.
    • First check the current NCP statistics using kstat
      # kstat -n ncp0 -s rsaprivate
      rsaprivate 0
    • then run the simple test
      # /usr/sfw/bin/openssl speed -engine pkcs11 rsa512 rsa1024
    • at the end of the test, check the kstat statistics again
      # kstat -n ncp0 -s rsaprivate
      rsaprivate 38166

    The above example shows that 38166 RSA-private jobs were handled by NCP.

    Other PKCS#11 based Cryptographic Consumers such as NSS, can hook into the NCP using the modutil command - an administrative utility provided with NSS:
    $ modutil -dbdir . -nocertdb -add "Solaris Cryptographic Framework" -libfile /usr/lib/ -mechanisms RSA

    In J2SE 1.5, the SunJSSE provider uses JCE exclusively for all of its cryptographic operations and hence any Java application compiled with J2SE1.5 can automatically take advantage of JCE features and enhancements. JCE's newly added support for PKCS#11 via the Sun PKCS#11 provider allows the SunJSSE provider in J2SE 1.5 to use the NCP for significant performance improvements. Thankfully the Sun Fire T2000 system comes with JDK1.5.0_B4, which has a PKCS#11 based provider "Sun PKCS#11".

  • Kernel Cryptographic Solution

    We did not stop here. We understand that there are SSL applications that cannot hook into the Solaris Cryptographic Framework because they do not use the PKCS#11 API. By using a simple command ksslcfg(1M), we can now off-load the entire SSL processing to a kernel-level SSL proxy server. This proxy server is able to use the Solaris Cryptographic Framework therefore NCP on behalf of the application. For example a secure J2EE application that runs only with JDK 1.4.2 and does not need to deal with Client-side Authentication, can take advantage of this solution. The next section gives a brief overview of the the technology and a following section describing the steps needed to configure the Sun Fire T2000 system for KSSL.
Introduction to KSSL in Solaris (TM) 10

The Kernel SSL proxy implements the SSL protocol such that a non-SSL application server will be able to handle SSL based client requests. The entire SSL processing is done in the kernel, thus the server program need to send/receive data in cleartext only.

The implementation adds a kernel module called kssl, which is responsible for the server side SSL protocol. This module acts as the SSL proxy server and is responsible for providing a cleartext proxy port to the application server and listening on the SSL port. It also manages keys and certificates and it is responsible for the SSL handshake with the clients and managing the the SSL session state information. The SSL handshake or SSL alert is handled asynchronously without the application server's knowledge or involvement.

Having said this, the encryption, decryption and message digest is still performed in the context of the application server and the cost of that SSL records processing is accrued by the application that benefits from it. When the application does a read() operation, the kssl module works in the ontext of the application to verify the MAC, decrypt the payload, strip out the SSL header and tail of the incoming record and copy the plaintext payload to the user-buffer supplied as an argument to the read() system call. Similarly, on a write() from application, the kssl module uses the application context to encrypt and compute the MAC of the outgoing message before actually sending out the encrypted message.

On UltraSPARC T1 based systems, along with NCP and the robust, secure Solaris (TM) Operating Environment, the kssl module is able to take advantage of all the features provided by the OS and the Solaris Cryptographic Framework. This solution is much better than traditional SSL proxy solution because:
  • it preserves end-to-end security: if the traditional SSL proxy device is outside the application server, the Plaintext data moving between the proxy device and the application server box can be intercepted and pose a security threat. Because kssl and the application reside on the same box, this problem does not exist on the Sun Fire T2000 system.
  • optionally it can seamlessly fall back to on-host software implementation of SSL in case the proxy is overwhelmed
  • if the traditional SSL proxy device is an IO card on the application server itself, due to limited resource availability on the card, like memory it cannot manage large number of SSL sessions, again this is not a problem on the Sun Fire T2000 system as the kssl module shares 32 gigabytes of host memory.
For the SSL Proxy module to be active for a given SSL port, the application server must listen on the proxy port. In fact even when the application is proxied for SSL data, it is still possible to have three listeners in the server program
  • one listening on regular cleartext port like HTTP port 80,
  • one listening on the SSL Proxy port through which the kssl module sends and receives clear text payload for secure clients,
  • a third port for managing encrypted data at user level, acting as a user-level SSL server. In such configuration client connections requesting a ciphersuite not supported by the kernel proxy are forwarded to the user-level SSL engine, as a fallback mechanism.
The kernel SSL proxy can be enabled for all ip addresses or for only a set of ip addresses. This gives great flexibility for applications that desire to run with and without kssl simultaneously. No reboot is necessary to enable or disable kssl. Also, the kernel SSL proxy is under SMF (Service Management Framework) Control which allows other SMF ready services to express dependencies on the the kssl service svc://network/ssl/proxy

How to use KSSL on the Sun Fire T2000 system

A secure application site is addressed by

It runs on Tomcat Server 5.x with SSL managed by the Tomcat server itself. This application server listens on port 80 for regular HTTP traffic and on Port 443 for Secure HTTP Traffic

We take this application scenario and deploy it on the Sun Fire T2000. No changes are required till this point. We stop the server.

We export the keys and certificate in a PK12 format file (cert.pk12), store the password in a file (cert.pass) and enable kssl on port 443 with proxy port at 8080. Given below is the command (for a full list of options to ksslcfg, see the end of this document)
ksslcfg create -f pkcs12 -i cert.pk12 -x 8080 -p cert.pass 443
We then configure a simple non-SSL listener on port 8080, by copying the port-80 directives in the server.xml and changing the port to 8080. We then restart the server.
From this point onwards, all secure requests will be proxied by the kssl module.

ksslcfg usage

  ksslcfg create -f pkcs11 [-d softtoken_directory] -T < token_label >
          -C < certificate_subject > -x < proxy_port > [options]
             [< server_address >] < server_port >
  ksslcfg create -f pkcs12 -i < certificate_file > -x < proxy_port > [options]
             [< server_address >] < server_port >
  ksslcfg create -f pem -i < certificate_file > -x  < proxy_port > [options]
             [< server_address >] < server_port >
  options are:
        [-c < ciphersuites >]
        [-p < password_file >]
        [-t < ssl_session_cache_timeout >]
        [-u < username >]
        [-z < ssl_session_cache_size >]
   ksslcfg delete [-v] [< server_address >] < server_port >

The multiple create options are for multiple certificate/key file format
  • from previously configured softtoken directory (see for more information about softtoken)
    -f pkcs11 [-d softtoken_directory] -T < token_label > -C < certificate_subject >
  • from a certificate exported in pkcs12 format file
    -f pkcs12 -i < certificate_file >
  • from a pem format file
    -f pem -i < certificate_file >
KSSL will be available as part of Solaris (TM) 10 Update-2 and as installable patches for Solaris (TM) 10 HW2 for Sun Fire T2000.

[ Technorati: NiagaraCMT]


Thanks for this great blog entry. I have tried to get Apache 2.2.0 running with the pkcs11 SSLEngine. As I understood, the engine is not experimental anymore. Anyway, when checking the kstats, it seems that the NCP device is not used. Have you made any tests with apache 2.2? Additionally, what I don't undestand is, according to the instruction on, why we have to use the prefork model, and why the server should not be started as root. Unfortunatly, the documentation is very thin, on the issue (yet!)

Posted by Mika Borner on February 26, 2006 at 03:18 PM PST #

Is there any API an application can use to be KSSL-aware? At a minimum, it would be nice to know that connections were using KSSL so they could be allowed access to functions that require security. Being able to request client identification and get the certificate presented would be nice too.

Posted by David Schwartz on March 20, 2006 at 12:00 AM PST #

[Trackback] Web サーバ絡みでもうひとつ. UltraSPARC T1 プロセッサに搭載されている RSA / DSA アクセラレーション機能の利用方法の解説が, 新着 BluePrint として公開された . (This BluePrint) provides a brief overview of SSL technology, as well as an introduction to the Sola...

Posted by tkudo's weblog on March 30, 2006 at 11:45 AM PST #

(for comment Posted by Mika Borner on February 26, 2006 at 11:18 PM PST ) Apologies for a long delay in response, I have been travelling lately. Has the setup worked for you with Apache 2.0.x ? I have observed some performance drop with Apache 2.2.0 compared to Apache 2.0.55 - have not really worked on it to root-cause it yet. It is true that the latest version of mod_ssl does not require using the EXPERIMENTAL flag. I just retried with Apache 2.2.0 and it works on my system here is how I compiled.. ./configure --prefix=/usr/local/apache22 --enable-mods-shared=all --enable-ssl --with-mpm=prefork --enable-so --enable-suexec --with-ssl=/usr/sfw make make install \*Please note the --with-ssl option. You should compile it on system which has atleast Solaris-10 and the bundled openssl installed. The mpm-worker seem to cause SSLSessionCache related issues that I am still working on to nail it down - If the SSLSessionCache is set to "none" it works okey with the mpm-worker model - but that defeats the presence of the cache itself.

Posted by Ning Sun on April 08, 2006 at 11:14 AM PDT #

Here is the URL for a blueprint we recently published on Niagara Crypto Acceleration, Solaris Crypto Framework and kernel SSL proxy. Let me know if you have any comments or questions.

Posted by Ning Sun on April 08, 2006 at 11:17 AM PDT #

(For comment posted by David Schwartz on March 20, 2006 at 08:00 AM PST) - I am sorry, but currently no such API exist at this point -

Posted by Ning Sun on April 08, 2006 at 11:20 AM PDT #

this is great

Posted by ben diyanni on May 23, 2006 at 02:45 AM PDT #

I am running into the same problems as Mike... To start with, I ran the SSL speed test to confirm the ncp unit was functioning properly :
# kstat -n ncp0 -s rsaprivate
name:   ncp0                            class:    misc
        rsaprivate                      0
#  openssl speed -engine pkcs11 rsa512 rsa1024
(output snipped)
#  kstat -n ncp0 -s rsaprivate
module: ncp                             instance: 0
name:   ncp0                            class:    misc
        rsaprivate                      31103
So you can see it is running. Next, I tried to make use of this in Apache 2.2.3, built from source against Sun's patched OpenSSL, using the relevant flags documented by the blueprint and above, and "SSLCryptoDevice pkcs11". SSL is working, but the numbers presented by "kstat -n ncp0" do not increment, so I do not believe the crypto unit is handling things. Anyone got this working on Apache 2.2 ?

Posted by Mark Round on December 16, 2006 at 09:22 PM PST #

I too am having problems getting this going with Apache (2.0.55). It works well by itself, but when trying to get it to work with SJSAS's mod_loadbalancer, it fails. Not sure if its a bug with pkcs11 or mod_loadbalancer, but its not good. As a workaround is it possible to get Apache to work with KSSL, so the pkcs11 directive isn't required?

Posted by Corey Johnston on May 21, 2007 at 10:29 AM PDT #

Hi, Can you Please tel us How to disable the Cores in T2000. It will be great if you answer the question Thank you Prakash

Posted by Prakash Channagouda on June 07, 2007 at 07:00 PM PDT #

To disable the cores, you could always put apps in zones which are restricted to only use certain processors. ie. Limit the number of processors available to a zone. Not exactly the same as disabling cores, as each thread is treated as a processor by the OS, but the nett effect should be the same.

Posted by Corey Johnston on June 07, 2007 at 09:10 PM PDT #

Also, the problem with SJSAS' loadbalancer and pkcs11 in Apache has been acknowledged by the Glassfish team as a bug in mod_loadbalancer. Apparently its scheduled for a fix in 9.1EE, but I was told that full support for Apache2 is not supported by SJSAS 8.2EE. Apparently they only really support SJS Webserver 7 with pkcs11 hardware support with the loadbalancer plugin.

Posted by guest on June 07, 2007 at 09:11 PM PDT #

Ways to disable cores: 1) do it at OS level using command "psradm -f <list of cpu ids>". You will need to make sure to disable all the cpus of a core to completely disable that core (and the MAU on that core). 2) do it at mc console via "diablecomponent" command. first do a "showcomponent" at mc> prompt and identify the CPUs to be disabled, eg "MB/CMP0/P0 MB/CMP0/P1 MB/CMP0/P2 MB/CMP0/P3" for core-0 and then run "disablecomponent" - you will need a separate command for each CPU to be disabled. After that do a "showcomponent" to confirm. Then reboot to make it effective.

Posted by guest on June 08, 2007 at 04:03 AM PDT #

Fantastic article. I have a question about the KSSL proxy option. When you say:

"This module acts as the SSL proxy server and is responsible for providing a cleartext proxy port to the application server and listening on the SSL port. It also manages keys and certificates ..."

It does mean, that if users are authenticating by certificates, this certificates stop at the KSSL Proxy and never go to the HTTP Server? I will need to use an SSL accelerator, but I need that certificates from the users pass to the final WEB server. It this possible with the KSSL Proxy?. Thanks a lot.

Posted by Manuel on July 09, 2007 at 05:15 PM PDT #

Post a Comment:
  • HTML Syntax: NOT allowed



« July 2016