Tuesday Nov 14, 2006

How to integrate PKCS11 engine patch into OpenSSL you built

Normally, you don't have to do this, as Jan Pechanec's two excellent blogs explain why this is so and talk about OpenSSL in Solaris and the PKCS#11 engine patch that Sun contributed to OpenSSL community. Please read the blogs and you will probably stop here :-).

If, for some reason, you have to build your own OpenSSL from source, and would still like to take advantage of MAU in UltraSPARC T1 on Sun Fire T2000/T1000 to speed up the RSA/DSA operations, I'll show you what I did.

First, download the PKCS#11 engine patch from OpenSSL's Contribution area. There are three PKCS#11 engine patch entries at the bottom of the list -- two for openssl-0.9.7d and one for openssl-0.9.7l. You have to apply the appropriate patch to either OpenSSL 0.9.7d or 0.9.7l, no other versions. For OpenSSL 0.9.7d, use pkcs11_engine-0.9.7d.patch.2006-09-12.gz as the other one suffers from Bug 6411001.

Before starting, there is one subtle thing to note -- the patch command. This PKCS#11 engine patch requires GNU patch and does not work with Solaris patch, so we have to use /usr/bin/gpatch.

Assume you want to patch OpenSSL 0.9.7l. At the directory level same as openssl-0.9.7l directory, unzip the patch:

#gunzip pkcs11_engine-0.9.7l.patch.2006-10-13.gz
Then cd into top-level OpenSSL directory and apply the patch
#cd openssl-0.9.7l
#gpatch -p 1 < ../pkcs11_engine-0.9.7l.patch.2006-10-13
patching file Configure
patching file Makefile.org
patching file README.pkcs11
patching file crypto/engine/Makefile
patching file crypto/engine/cryptoki.h
patching file crypto/engine/eng_all.c
patching file crypto/engine/engine.h
patching file crypto/engine/hw.ec
patching file crypto/engine/hw_pk11.c
patching file crypto/engine/hw_pk11_err.c
patching file crypto/engine/hw_pk11_err.h
patching file crypto/engine/hw_pk11_pub.c
patching file crypto/engine/pkcs11.h
patching file crypto/engine/pkcs11f.h
patching file crypto/engine/pkcs11t.h

(At this point, you can take a look at README.pkcs11. Most of the details are there.)

To build OpenSSL, you need to specify a PKCS#11 library on the system. On Solaris 10 or Solaris Express, it's libpkcs11.so under /usr/lib (32-bit) or /usr/lib/sparcv9 (64-bit). This is how I built 64-bit OpenSSL on a T2000 using Sun Studio cc:

#./Configure solaris64-sparcv9-cc shared --pk11-libname=/usr/lib/sparcv9/libpkcs11.so
Note that libcrypto.so and libssl.so will reside in the current directory after the make. To verify whether this self-built OpenSSL really takes advantage of MAU on T2000, do this:
#apps/openssl speed rsa1024 -engine pkcs11 -elapsed -multi 32

[ .. snip .. ]

available timing options: TIMES TIMEB HZ=100 [sysconf value]
timing function used:
                  sign    verify    sign/s verify/s
rsa 1024 bits 0.000078s 0.000022s  12846.6  45807.1
If it's close to 13000 signs/s, MAU is engaged. You can also use
#kstat -n ncp0 | grep rsa
to compare the RSA counters before and after the speed test.

Tuesday Dec 06, 2005

RSA Performance of Sun Fire T2000

You might have heard that UltraSPARC T1 has special hardware circuitry to accelerate certain crypto operations. In this blog I will show you what operations it is good at, and how good.

UltraSPARC T1 comes with Modular Arithmetic Unit (MAU) per core which can accelerate expensive modular arithmetic operations found in public key crypto algorithms such as RSA, DSA and DH. In Solaris, the utilization of MAU has to go through Niagara Cryptographic Provider (NCP) within Solaris Cryptographic Framework (SCF). Currently only RSA (up to 2048 bit) and DSA (up to 1024 bit) are supported by NCP.

On the Sun Fire T2000/T1000 with the UltraSPARC T1 processor, you can readily get a glimpse of the fast RSA operations performed by MAU. Here is an example for the popular 1024-bit and 2048-bit RSA on a Sun Fire T2000 with 1.2 GHz UltraSPARC T1 with 8 cores:

[watercloset]~> /usr/sfw/bin/openssl speed rsa1024 rsa2048 -engine pkcs11
engine "pkcs11" set.
Doing 1024 bit private rsa's for 10s: 10332 1024 bit private RSA's in 0.45s
Doing 1024 bit public rsa's for 10s: 25550 1024 bit public RSA's in 0.89s
Doing 2048 bit private rsa's for 10s: 2371 2048 bit private RSA's in 0.11s
Doing 2048 bit public rsa's for 10s: 10308 2048 bit public RSA's in 0.37s
OpenSSL 0.9.7d 17 Mar 2004
built on: date not available
options:bn(64,32) md2(int) rc4(ptr,char) des(ptr,risc1,16,long)
aes(partial) blowfish(ptr) 
compiler: information not available
available timing options: TIMES TIMEB HZ=100 [sysconf value]
timing function used: times
                  sign    verify    sign/s verify/s 
rsa 1024 bits   0.0000s   0.0000s  22960.0  28707.9
rsa 2048 bits   0.0000s   0.0000s  21554.5  27859.5

This invokes the OpenSSL speed test bundled with Solaris. The OpenSSL bundled with Solaris has PKCS#11 engine built-in which is necessary to access SCF (and thus MAU); if you download OpenSSL package and build it yourself, you will not be able to take advantage of MAU because it does not have PKCS#11 engine. Let's examine the performance numbers above. What we just did was to test the single-threaded RSA performance. Each RSA operation is run for 10 seconds. However, due to the timing errors in OpenSSL speed test in the single-threaded case, the throughput numbers at the bottom cannot be trusted when the operations are done in hardware. After some re-calculations we get:

                  sign    verify    sign/s verify/s 
rsa 1024 bits   0.0000s   0.0000s   1033.2   2550.0
rsa 2048 bits   0.0000s   0.0000s    237.1   1030.8

Are these numbers good? They are actually very good. Take 1024-bit RSA sign operation number, 1033.2, and compare it with the number on 3.6 GHz Xeon Dell box - 843.0. UltraSPARC T1 offers 20% more RSA performance at 1/3 clock rate and uses less power. Note that this is single-threaded test. As shown below, UltraSPARC T1 really dwarfs others in the multi-process test.

Now, let's look at multi-process RSA performance. This is where UltraSPARC T1 really shines. Do an OpenSSL speed test again, this time with the "-multi" option to invoke multiple processes to conduct RSA operations concurrently:

[watercloset]~> /usr/sfw/bin/openssl speed rsa1024 rsa2048 -engine pkcs11 -multi 32

[ intermediate output snipped..... ]

                  sign    verify    sign/s verify/s
rsa 1024 bits   0.0001s   0.0000s  12871.3  45148.1
rsa 2048 bits   0.0004s   0.0000s   2299.6  20425.3

We have used 32 processes to fully saturate the 32 hardware threads on UltraSPARC T1 to get the maximum throughput. Compare this with the results on the 2-way 3.6 GHz Xeon Dell PowerEdge 2850 (with hyperthreading on):

wgs93-187:~ openssl speed rsa1024 rsa2048 -multi 4

[ intermediate output snipped..... ]

                  sign    verify    sign/s verify/s
rsa 1024 bits   0.0005s   0.0000s   1943.2  34632.7
rsa 2048 bits   0.0031s   0.0001s    327.4  10891.9

For 1024-bit RSA sign operation (as commonly used in web server SSL handshaking), Sun Fire T2000 outperforms Dell PowerEdge 2850 by a whopping 6x! UltraSPARC T1 also excels when compared with the Sun Crypto Accelerator 4000, which can do 8000 1024-RSA signs/s. And remember, all this comes with just the Sun Fire T2000/T1000 box, no extra crypto accelerator card is needed.

In summary, if RSA/DSA operations consume a certain amount of CPU cycles in your application (e.g. HTTPS), Sun Fire T2000/T1000 with UltraSPARC T1 will offer you the biggest bang for the bucks with its per-core MAU and unique 8-core CMT architecture.




« April 2014