Cryptography Acceleration on UltraSPARC T2 Systems

Delivering on the same promise of Chip Multi-threading, this time, Sun is offering an even more powerful chip with more complete cryptographic features coupled with on-chip IO and integrated 2x10-GBPS network. A true System On Chip. The Ultra SPARC T2, being a successor of the UltraSPARC T1, combines the power of eight 8-way multithreaded cores offering 64 simultaneous processing threads while capping the power consumption when compared with any other comparable 8-core systems. More detailed information about the UltraSPARC T2 specification can be viewed here.

Zero Cost Cryptography

The UltraSPARC T1 processor offered 8 on-chip Modular Arithmetic Units (MAU), one per core. Today, these 8 MAUs are used to offload and accelerate RSA/DSA operations without compromising performance of the regular core functionality. RSA operation is an important component of the SSL full handshake.

The other part of cryptographic operation is encryption, decryption and hashing of sensitive data. Each core of the The UltraSPARC T2 processor contains a Streams Processing Unit (SPU) offering Encryption/Decryption and Hash-Operations offload engine, which can be used to offload DES, 3DES, AES-128, AES-192, AES-256, RC4, MD5, SHA1, SHA256. It also offers ECCp-160 and ECCb-163 used in Public Key exchange.

In addition to Encrypion/Decryption/Hash functionality, the UltraSPARC T2 processor has a on-chip Random Number Generator that is normally used by cryptographic applications for entropy data.
  • The eight MAUs, one for each core, are driven by the Niagara Crypto Provider (NCP) device driver in the Solaris 10 OS for both UltraSPARC T1 and UltraSPARC T2 processor. NCP supports hardware assisted acceleration of RSA and DSA cryptographic operations. It's NCP's responsibility to load balance between the MAUs to reduce programming complexity. NCP is pre-configured and enabled in the Solaris Cryptographic Framework on both T1 and T2 systems.
  • The eight SPUs, 1 per core on the UltraSPARC T2 processor are driven by the Niagara2 Crypto Provider (N2CP) device driver. N2CP supports hardware assisted acceleration DES, 3DES, AES, RC4, SHA1, SHA256, MD5, ECC and CRC32. It's N2CP's responsibility to load balance between the SPUs to reduce programming complexity. N2CP is pre-configured and enabled in the Solaris Cryptographic Framework on the T2 systems.
  • Random Number Generator unit is driven by the Niagara2 Random Number Generator (N2RNG) in the Solaris 10 OS.
These cryptographic offload and acceleration are available at no additional cost to software - thus zero cost cryptography becomes reality with the US T2 based systems. Performance numbers can be found here.

How to Configure

Just as on UltraSPARC T1, a PKCS#11 compliant cryptographic application on UltraSPARC T2 can take advantage of new cryptographic features via the Solaris Cryptographic Framework. N2CP, NCP and N2RNG are pre-configured and enabled by default on US T2 based systems. Secure applications such as Apache, SJS Web Server and Java Applications will need to be configured to go the PKCS#11 route if the default route is non-PKCS#11. Instructions given in my earlier blog for UltraSPARC T1 can still be used on UltraSPARC T2 except that new mechanisms can be added to offload encryption, decryption and hashing operations. For example, On US T1, to configure SJS Web Server NSS, we have:
    $ modutil -dbdir . -nocertdb -add "Solaris Cryptographic Framework" -libfile \\ /usr/lib/ -mechanisms RSA
On a UltraSPARC T2 system, this command can be changed to include other mechanisms such as AES or MD5
    $ modutil -dbdir . -nocertdb -add "Solaris Cryptographic Framework" -libfile \\ /usr/lib/ -mechanisms RSA:AES:MD5
To use the PKCS#11 engine with Apache set the following property in the Apache configuration file
    SSLCryptoDevice pkcs11
The OpenSSL PKCS#11 engine hooks up with Solaris Cryptographic Framework (SCF) and transparently offloads cryptographic operations to available hardware providers, in this case NCP and N2CP. For Java applications running JDK 1.5 or beyond, the ${java-home}/jre/lib/security/sunpkcs11-solaris.cfg can be modified to remove CKM_MD5, CKM_SHA_1, CKM_SHA256 from the disabled mechanisms list (the disableMechanisms line) so that these algorithms will be offloaded to N2CP.

Other complex mechanisms CKM_DSA_SHA1, CKM_MD5_RSA_PKCS , CKM_SHA1_RSA_PKCS, CKM_SHA256_RSA_PKCS, CKM_SHA384_RSA_PKCS and CKM_SHA512_RSA_PKCS listed above still can't be offloaded yet although the UltraSPARC T2 processor supports each SHA, MD5, DSA and RSA algorithm individually. To offload these complex mechanisms, we need to have JSSE or Metaslot break these complex mechansims into simple ones for provider to handle individually because we don't have a single provider that aggregates the NCP and N2CP capabilities. This issue is being tracked and worked on in CR #6337157.

A blueprint article with step by step instructions on how to take full advantage of US T2 hardware cryptographic features is currently in the works. I will update this blog with a link as soon as it's ready. If you have specific questions please post your questions here and I will try to answer them.

Kernel SSL Proxy

Traditionally, applications running on systems equipped with Cryptographic Accelerators need to pay the cost of moving data from User land to Kernel and back to User land for the purpose of encryption/decryption and hashing. If the application is a secure network application like secure webserver, then it should be possible to complete the cryptographic operation in the kernel itself and let the application handle clear text data both for in-bound and out-bound data. This technique would save cpu cycles spent in moving data from User land to kernel and back. This behaviour is analogous to SSL proxy devices, where the cryptographic opertations terminate outside the system, except in this case the cryptographic operation completes in the kernel of the same system where the application is running therefore preserving end-to-end security. Taking the approach of letting the kernel complete the cyrptographic operations, makes the kernel act as a SSL proxy. This technique of making the kernel act as a SSL proxy device is at the heart of Kernel Cryptographic Solution (KSSL) offered with the Solaris 10 OS.

An introduction to Kernel Cryptographic Solution (KSSL) is provided at my previous blog.

On UltraSPARC T1 and UltraSPARC T2 based systems, KSSL can offload the RSA/DSA operation to the MAU and thus avoid data movement overhead from User Land to Kernel and back.

On UltraSPARC T2 based systems, in addition of offloading RSA operation, KSSL also offloads some commonly used encryption algorithms like RC4,DES and 3DES along with HMAC (SHA1_HMAC, MD5_HMAC) computation of outbound packets.

A perfect match for datacenters

US T2 based systems provide 2x10-GBPS NICs onboard in addidtion to the 4 onboard gigabit ports. Just like the benefit from having on-chip Cryptographic Acceleration, these on-chip 10-GBPS NICs do not suffer from the limitations of bus bandwidth such as found with PCI-(x/ex) devices. US T1 and T2 based systems both allow additional PCI-ex based devices to be added for additional IO requirements such as disks or additional 10-GBPS cards. They also both offer virtualization techniques such as Zones and LDOMs. The on-chip cryptographic engine and the on-chip 10-GBPS interfaces can be virtualized or partitioned and are accessible even from a virtualized environment.

Packaging all these features in just 1U or 2U systems, makes both US T2 ideal for datacenters looking for expansion and or consolidation, with considerable savings in Power and Cooling. These systems can be used in any deployment scenario in the datacenter from edge tier to the backend database tier.


Related Links

Sun Fire T2000 and Secure Applications
Ultra-Fast Cryptography on the Sun UltraSPARC T2
Why on-chip accelerators?
UltraSPARC T2 - World's True System on a Chip
UltraSPARC T2 Supplement to the UltraSPARC architecture 2007


Using the n2cp0 provider with hardware-based hashing algorithms causes SSL handshake errors. When I disabled the hashing algorithms the SSL handshake worked and I could see incremental counts for both RSAPRIVATE (NCP) and AES (N2CP) drivers. Any help on how to make the hardware hashing algorithms work with Apache using the PKCS#11 library and openssl version 0.9.7d bundled with S10U4 ?

Posted by Jeroen on December 16, 2007 at 04:43 PM PST #

Hi Jeroen,

Which version of Solaris do you have? There is a bug for this exact problem:
It's been fixed in OpenSolaris. But there is no patch yet for Solaris 10 Updates. So if you want to get the fix now, you will need to switch to OpenSolaris community version - here is the DVD image at If you are already on some older version of OpenSolaris, to get the fix, you need to do a "bfu" to the following archive. Instructions on how to do bfu can be found at: Please read section 5.3. Remember you will need to have the OpenSolaris installed first before you can do this bfu.


Posted by Ning Sun on December 21, 2007 at 03:43 AM PST #

Hi Ning;
We're using httpclient in java to connect to an SSL web service. We're also using NCP and have CKM_MD5_RSA_PKCS (PKCS #1 MD5 With RSA Encryption) disabled in sunpkcs11-solaris.cfg.

When we attempt to open the connection we're getting the following error:
at javax.crypto.Cipher.getInstance(DashoA12275)

I think it's interesting that 1) the server certificate is CKM_MD5_RSA_PKCS and 2) the code works under windows.

In your opinion is this failure due to using the pkcs11 provider in Java and having CKM_MD5_RSA_PKCS in the disabled list? I have found a few others having this same issue but no open bug reports to explain it.

Posted by Nathan Wray on February 13, 2008 at 12:47 AM PST #

Hi Ning,
In your document: "USING THE CRYPTOGRAPHIC ACCELERATORS IN THE ULTRASPARC T1 AND T2 PROCESSORS" why is Prefork mode recommended for Apache 2.2.x web server versus Worker mode (threaded)?
thank you,

Posted by Emiro Uribe on August 21, 2009 at 01:19 AM PDT #

@Jeroen and @Ning: to get a patch for a bug in Solaris 10 update release, one has to raise a support call and make the Sun Services file an escalation. The patches for this particular bug (CR 6606361) were released in S10u5 time frame.

Posted by Vladimir Kotal on August 24, 2009 at 09:05 PM PDT #

@Emiro: The 819-5782 blueprint does not seem to mention the MPM-worker mode at all. I think this is because the Apache shipped with Solaris 10 by default only supports the prefork mode. In general the MPM-worker mode could be beneficial for SSL processing with crypto offloaded to the MUAs on Niagara CPUs (depending on workload). AFAIK officially supported Apache with worker mode module is available via Webstack:

Posted by Vladimir Kotal on August 24, 2009 at 09:33 PM PDT #

Great post and draw. Thank you for sharing.

Posted by links of london jewellery on November 23, 2009 at 10:37 AM PST #

prev. and mine comments are spam, remove it, thanks.

Posted by 1 on December 14, 2009 at 12:44 AM PST #

Dear Ning,

I am doing a project for my parallel programming course which deals with cryptography and I am supposed to write a c program which puts the burden of cryptography on hardware, I have read the blueprint you provided and I have faced with some questions. This is my system specifications:

SunOS 5.11 snv_34 sun4v sparc sun4v
scsi_vhci, instance #0
packages (driver not attached)
SUNW,builtin-drivers (driver not attached)
deblocker (driver not attached)
disk-label (driver not attached)
terminal-emulator (driver not attached)
dropins (driver not attached)
SUNW,asr (driver not attached)
kbd-translator (driver not attached)
obp-tftp (driver not attached)
ufs-file-system (driver not attached)

1. When I use the cryptoadm command it lists nothing in the kernel hardware providers, what does it mean? Does it mean the driver is not installed? How can I install or attach the driver?

2. In case I could attache the driver, how can I say explicitly that my program uses the n2cp for AES dec/enc rather than software providers. I am using the in my program.

Thanks a ton!

Posted by guest on January 10, 2011 at 03:59 AM PST #

Good luck, Ning hasn't posted here in two years.

Posted by Nathan Wray on January 10, 2011 at 04:09 AM PST #

For your first question, it could mean 1 of 2 things (or both)
1. the hw crypto was disabled when you got the system and or you do not have the necessary packages. To fix this issue - you will need to contact the source that provided you with the hardware.
2. the OS in use is very very old - You may want to switch to latest Solaris-11-Express or Solaris-10-U9
For your second hquestion, the driver attach should happen automatically - if there is a hardware provider, it will be used.

Posted by Ning on January 10, 2011 at 08:56 AM PST #

Hi Nathan, I thought I had responded to your earlier comment but apparently it didn't get posted for some reason (and I thought it did sorry about it) - I just dug out something from my old archive. My reply was:

\* Is this Apache HttpClient? Is it trying to pick a provider by itself?
\* I wrote a simple test case and that works fine on my system
Can you pl. try this out and see if it works? (you may want to
replace localhost with the server-name.
\* Just as a side note, if you are originating a connection from a box with
NCP configured, then NCP's feature is of no use?


public class TestSSL {
public static void main(String[] args) throws Exception {
Socket socket = SSLSocketFactory.getDefault().createSocket("localhost", 443);
try {
PrintWriter out = new PrintWriter(
new BufferedWriter(
new OutputStreamWriter(

out.println("GET / HTTP/1.0");

if (out.checkError())
"Test: error");

BufferedReader in = new BufferedReader(
new InputStreamReader(

String inputLine;
while ((inputLine = in.readLine()) != null)

} finally {

Posted by Ning on January 10, 2011 at 09:14 AM PST #

Hi Ning, thanks for posting back. It's been a few years but if I remember correctly we fixed the problem by enabling the provider in
the JDK settings. I think the big question at the time was, why did the Solaris JDK ship with that provider disabled?

Anyway good to hear from you

Posted by Nathan Wray on January 10, 2011 at 08:21 PM PST #

Post a Comment:
  • HTML Syntax: NOT allowed



« April 2014