PKCS#11 Engine Patch (including the token access) for OpenSSL 0.9.8l (el)

I have generated a PKCS#11 patch for OpenSSL 0.9.8l. It includes one new feature I have recently integrated into Nevada - RSA Keys by Reference. You can read the README here, and download the PKCS#11 patch.

As a test I've built the code on Solaris 10 and one Linux distro (which identified itself as GNU/Linux, I have no idea which distro was that). I had to make a couple of modifications since other systems do not have getpassphrase() function, and Linux distros have no strlcpy() function (because GNU C library has none). Hopefully I have not broken anything.

If you plan to use the patch, do not forget to check out examples and the PKCS#11 URI format in the presentation for the project. As usual, if you find problems with the patch, let me know please. And of course, if you find it really useful, you can say that as well :-)

The man page will be updated soon but before that gets to a public build it will take some time. So, below is the current draft change for the openssl(5) man page. Note that the man page in section 5 is for OpenSSL in Solaris, the one in section 1 is the original man page from the OpenSSL project. Obviously, we ship both. Use "man -s 5 openssl" to see the Sun's one, or "man -a" if unsure about sections.

openssl(5) planned addition:

  Accessing RSA Keys in PKCS#11 Keystores

     OpenSSL can access RSA keys in PKCS#11 keystores using the
     following functions of the ENGINE API:

       EVP_PKEY \*ENGINE_load_private_key(ENGINE \*e,
                const char \*key_id, UI_METHOD \*ui_method,
                void \*callback_data)

       EVP_PKEY \*ENGINE_load_public_key(ENGINE \*e,
                const char \*key_id, UI_METHOD \*ui_method,
                void \*callback_data)

     key_id, formerly for filenames only, can be now also set to
     a PKCS#11 URI. The EVP_PKEY structure is newly allocated and
     caller is responsible to free the structure later. To avoid
     clashes with existing filenames, "file://" prefix for
     filenames is now also accepted but only when the PKCS#11
     engine is in use. The PKCS#11 URI specification follows:

        pkcs11:[token=<label>][;manuf=<label>][;serial=<label>]
               [;model=<label>][;object=<label>]
               [;objecttype=(public|private|cert)]
               [;passphrasedialog=(builtin|exec:<file>)]

     The ordering of keywords is not significant. The PKCS#11
     engine uses the keystore for the slot chosen for public key
     operations whic is metaslot on a standardly configured
     machine. Currently, the PKCS#11 engine ignores "objecttype"
     keyword. The only mandatory keyword is "object" which is
     the key object label. For information on how to use a
     different, possibly hardware, keystore with metaslot see
     libpkcs11(3LIB).

     The token PIN is provided via "passphrasedialog" keyword and
     is either read from the terminal ("builtin") or from the
     output of an external command ("exec:<file>"). The PIN is
     used to log into the token and by default is deleted from
     the memory then. The keyword "pin" is intentionally not
     provided due to inherent security problems of possible use
     of a password in the process arguments.

     Due to fork safety issues the application must re-login if
     the child continues to use the PKCS#11 engine. It is done
     inside of the engine automatically if fork is detected and
     in that case, "exec:<file>" option of the "passphrasedialog"
     keyword can be used. Alternatively, an enviroment variable
     OPENSSL_PKCS11_PIN_CACHING_POLICY can be used to allow the
     PIN to be cached in memory and reused in the child. It can
     be set to "none" which is the default, "memory" to store
     the PIN in memory, and "mlocked-memory" keep the PIN in a
     locked page via mlock(3C). Note that PRIV_PROC_LOCK_MEMORY
     privilege is required in that case.

     Sensitive parts of private keys are never read from the
     token to the process memory no matter whether the key is
     tagged with sensitive flag or not. The PKCS#11 engine uses
     the public compoments as a search key to get a PKCS#11
     object handle to the private key.

     Note that in order to use the RSA keys by reference, high
     level API functions must be used, like RSA_public_decrypt(),
     EVP_PKEY_set1_RSA(), or EVP_SignInit(). Low level functions
     might go around the engine and thus fail to make use of the
     feature.

  Additional Documentation

      Extensive additional documentation for  OpenSSL  modules  is
      available       in      the      /usr/share/man/man1openssl,
      /usr/share/man/man3openssl, /usr/share/man/man5openssl,  and
      /usr/share/man/man7openssl directories.

      To view the license terms, attribution,  and  copyright  for
      OpenSSL, see /var/sadm/pkg/SUNWopensslr/install/copyright.

EXAMPLES

     Example 1: generate and print a public key stored in an
                already initilized PKCS#11 keystore. Note the
                use of "-engine pkcs11" and "-inform e".

        $ pktool gencert keystore=pkcs11 label=mykey \\
            subject="CN=test" keytype=rsa keylen=1024 serial=01
        $ openssl rsa -in \\
            "pkcs11:object=mykey;passphrasedialog=builtin" \\
            -pubout -text -engine pkcs11 -inform e
Comments:

Jan, the patch works well on Solaris. After one minor edit it compiles on OSX as well (with SoftHSM). However, I get the following error at runtime:
$ ./openssl rsa -in "pkcs11:object=mycert;passphrasedialog=builtin" \\
-pubout -text -engine pkcs11 -inform e
engine "pkcs11" set.
unable to load Private Key
31426:error:2609607D:engine routines:ENGINE_load_private_key:no load function:eng_pkey.c:121:

It works fine with your previous patch (i.e. with rsa -in pkcs11:mycert), so I think there is an include missing somewhere after you've split functions out of hw_pk11_err into hw_pk11. Let me know if you need more info.

Kind regards

Roy

Posted by Roy Arends on December 03, 2009 at 07:59 AM CET #

Roy, what was the minor edit? The error message means that the engine load function is not registered upon initialization of the engine, which is very strange. That would happen if OpenSSL was not build with RSA at all (macro OPENSSL_NO_RSA set), or if your slot could not do RSA (pk11_have_rsa set to false during mechanism probing):

#ifndef OPENSSL_NO_RSA
if (pk11_have_rsa == CK_TRUE)
{
if (!ENGINE_set_RSA(e, PK11_RSA()) ||
!ENGINE_set_load_privkey_function(e, pk11_load_privkey) ||
!ENGINE_set_load_pubkey_function(e, pk11_load_pubkey))
return (0);
#ifdef DEBUG_SLOT_SELECTION
fprintf(stderr, "%s: registered RSA\\n", PK11_DBG);
#endif /\* DEBUG_SLOT_SELECTION \*/
}
#endif /\* OPENSSL_NO_RSA \*/
#ifndef OPENSSL_NO_DSA

Posted by Jan on December 03, 2009 at 08:27 AM CET #

Roy, one more thing. You can change the following line in hw_pk11.c:

#undef DEBUG_SLOT_SELECTION

to:

#define DEBUG_SLOT_SELECTION 1

and rebuild OpenSSL. You will then get some debug output during the initialization of the engine. You will see what is the situation with RSA mechanisms.

Posted by Jan on December 03, 2009 at 08:31 AM CET #

Jan, the minor edit was in hw_pk11.c, to avoid the following error:

hw_pk11.c:806: error: expected identifier or ‘(’ before numeric constant
hw_pk11.c:807: error: expected identifier or ‘(’ before numeric constant
hw_pk11.c:2711: error: lvalue required as unary ‘&’ operand
hw_pk11.c:2712: error: lvalue required as unary ‘&’ operand
hw_pk11.c:2713: error: lvalue required as unary ‘&’ operand

The change was to avoid collision with true/false.

-static CK_BBOOL true = TRUE;
-static CK_BBOOL false = FALSE;
+static CK_BBOOL ctrue = TRUE;
+static CK_BBOOL cfalse = FALSE;

-{CKA_TOKEN, &false, sizeof (false)},
-{CKA_ENCRYPT, &true, sizeof (true)},
-{CKA_DECRYPT, &true, sizeof (true)},
+{CKA_TOKEN, &cfalse, sizeof (cfalse)},
+{CKA_ENCRYPT, &ctrue, sizeof (ctrue)},
+{CKA_DECRYPT, &ctrue, sizeof (ctrue)},

Posted by Roy Arends on December 03, 2009 at 09:08 AM CET #

Jan,

I've recompiled older and newer version with: #define DEBUG_SLOT_SELECTION 1

The results:
[older version]
./openssl rsa -in "pkcs11:mycert" -pubout -text -engine pkcs11 -inform e
PKCS#11 ENGINE DEBUG: provider: /usr/local/lib/libsofthsm.dylib
PKCS#11 ENGINE DEBUG: number of slots: 1
PKCS#11 ENGINE DEBUG: == checking rand slots ==
PKCS#11 ENGINE DEBUG: checking slot: 0
PKCS#11 ENGINE DEBUG: token label: test
PKCS#11 ENGINE DEBUG: this token has CKF_RNG flag
PKCS#11 ENGINE DEBUG: == checking pubkey slots ==
PKCS#11 ENGINE DEBUG: checking slot: 0
PKCS#11 ENGINE DEBUG: token label: test
PKCS#11 ENGINE DEBUG: potential slot: 0
PKCS#11 ENGINE DEBUG: setting found_candidate_slot to CK_TRUE
PKCS#11 ENGINE DEBUG: best so far slot: 0
PKCS#11 ENGINE DEBUG: chosen pubkey slot: 0
PKCS#11 ENGINE DEBUG: chosen rand slot: 0
PKCS#11 ENGINE DEBUG: pk11_have_rsa 1
PKCS#11 ENGINE DEBUG: pk11_have_random 1
PKCS#11 ENGINE DEBUG: registered RSA
PKCS#11 ENGINE DEBUG: registered random
engine "pkcs11" set.
PKCS#11 ENGINE DEBUG: myslot=0 optype=1
Enter PIN:
Modulus (1024 bit):
00:dc:32:3f:28:e3:6f:45:d1:15:c5:1c:de:e8:c8:
[snip]

[latest version]
./openssl rsa -in "pkcs11:object=mycert;passphrasedialog=builtin" -pubout -text -engine pkcs11 -inform e
PKCS#11 ENGINE DEBUG: provider: /usr/local/lib/libsofthsm.dylib
PKCS#11 ENGINE DEBUG: number of slots: 1
PKCS#11 ENGINE DEBUG: == checking rand slots ==
PKCS#11 ENGINE DEBUG: checking slot: 0
PKCS#11 ENGINE DEBUG: token label: test
PKCS#11 ENGINE DEBUG: this token has CKF_RNG flag
PKCS#11 ENGINE DEBUG: == checking pubkey slots ==
PKCS#11 ENGINE DEBUG: checking slot: 0
PKCS#11 ENGINE DEBUG: token label: test
PKCS#11 ENGINE DEBUG: no rsa/dsa/dh
PKCS#11 ENGINE DEBUG: == checking cipher/digest ==
PKCS#11 ENGINE DEBUG: checking slot: 0
PKCS#11 ENGINE DEBUG: checking mech: 122 not found
PKCS#11 ENGINE DEBUG: checking mech: 133 not found
PKCS#11 ENGINE DEBUG: checking mech: 121 not found
PKCS#11 ENGINE DEBUG: checking mech: 132 not found
PKCS#11 ENGINE DEBUG: checking mech: 111 not found
PKCS#11 ENGINE DEBUG: checking mech: 1082 not found
PKCS#11 ENGINE DEBUG: checking mech: 1082 not found
PKCS#11 ENGINE DEBUG: checking mech: 1082 not found
PKCS#11 ENGINE DEBUG: checking mech: 1081 not found
PKCS#11 ENGINE DEBUG: checking mech: 1081 not found
PKCS#11 ENGINE DEBUG: checking mech: 1081 not found
PKCS#11 ENGINE DEBUG: checking mech: 1091 not found
PKCS#11 ENGINE DEBUG: checking mech: 1086 not found
PKCS#11 ENGINE DEBUG: checking mech: 1086 not found
PKCS#11 ENGINE DEBUG: checking mech: 1086 not found
PKCS#11 ENGINE DEBUG: checking mech: 210 usable
PKCS#11 ENGINE DEBUG: checking mech: 220 usable
PKCS#11 ENGINE DEBUG: checking mech: 255 not found
PKCS#11 ENGINE DEBUG: checking mech: 250 usable
PKCS#11 ENGINE DEBUG: checking mech: 260 usable
PKCS#11 ENGINE DEBUG: checking mech: 270 usable
PKCS#11 ENGINE DEBUG: current_slot_n_cipher 0
PKCS#11 ENGINE DEBUG: current_slot_n_digest 5
PKCS#11 ENGINE DEBUG: best so far cipher/digest slot: 0
PKCS#11 ENGINE DEBUG: changing best so far slot to 0
PKCS#11 ENGINE DEBUG: chosen pubkey slot: 0
PKCS#11 ENGINE DEBUG: chosen rand slot: 0
PKCS#11 ENGINE DEBUG: chosen cipher/digest slot: 0
PKCS#11 ENGINE DEBUG: pk11_have_rsa 0
PKCS#11 ENGINE DEBUG: pk11_have_dsa 0
PKCS#11 ENGINE DEBUG: pk11_have_dh 0
PKCS#11 ENGINE DEBUG: pk11_have_random 1
PKCS#11 ENGINE DEBUG: cipher_count 0
PKCS#11 ENGINE DEBUG: digest_count 5
PKCS#11 ENGINE DEBUG: C_GetOperationState() not supported, setting digest_count to 0
PKCS#11 ENGINE DEBUG: registered random
engine "pkcs11" set.
unable to load Private Key
32986:error:2609607D:engine routines:ENGINE_load_private_key:no load function:eng_pkey.c:121:

Both use the exact same provider.

Posted by Roy Arends on December 03, 2009 at 09:29 AM CET #

Jan, I figured it out, and owe you an apology.

The "older" version was indeed the previous version of your patch, but with the check for CKM_RSA_X_509 removed from hw_pk11.c, since some HSMs (like softHSM in the example above) solely provide the CKM_RSA_PKCS mechanism.

Thanks for you getting me on track using DEBUG_SLOT_SELECTION

Posted by Roy Arends on December 04, 2009 at 03:32 AM CET #

Cool, I though it could be a bug and already put it on my todo list to check it out. I'm glad I can cross out that item now :-)

Posted by Jan on December 04, 2009 at 03:39 AM CET #

Hi Jan,

I have used this patch On AIX machine . While compilation on AIX with this patch ,I am getting this compilation error.

making all in crypto/engine...
xlc_r -I.. -I../.. -I../../include -DOPENSSL_THREADS -qthreaded -DDSO_DLFCN -DHAVE_DLFCN_H -DPK11_LIB_LOCATION="/usr/lib/pkcs11xcrypto/libpkcs11xcrypto.so" -DSSL_ALLOW_ADH -q32 -O -DB_ENDIAN -qmaxmem=16384 -qro -qroconst -c hw_pk11.c
"cryptoki.h", line 30.1: 1506-732 (W) pragma ident is not supported on the target platform.
"hw_pk11.c", line 786.40: 1506-275 (S) Unexpected text '/' encountered.
"hw_pk11.c", line 786.40: 1506-045 (S) Undeclared identifier usr.
"hw_pk11.c", line 786.40: 1506-045 (S) Undeclared identifier lib.
"hw_pk11.c", line 786.40: 1506-045 (S) Undeclared identifier pkcs11xcrypto.
"hw_pk11.c", line 786.40: 1506-045 (S) Undeclared identifier libpkcs11xcrypto.
"hw_pk11.c": 1506-168 (S) Initializer must be enclosed in braces.
make: 1254-004 The error code from the last command is 1.

Please let me know why this error is getting ?

Can this patch be used in AIX machine?

Thanks in Advance
Samuel

Posted by samuel on December 13, 2009 at 08:30 AM CET #

Samuel, Wolfgang's answer from security-discuss@opensolaris.org seems relevant:

in line 786 the path to the library is set. In your case the quotes are missing in the source and you may want to retry using

-DHAVE_DLFCN_H -DPK11_LIB_LOCATION=\\"/usr/lib/pkcs11xcrypto/libpkcs11xcrypto.so\\"

instead of

-DHAVE_DLFCN_H -DPK11_LIB_LOCATION="/usr/lib/pkcs11xcrypto/libpkcs11xcrypto.so"

Change the makefile or rerun configure with the additional quoting.

Posted by Jan on December 14, 2009 at 08:59 AM CET #

Hi Jan,

I have builded openssl with engine patch. But when ran the command

./openssl engine pkcs11
487610:error:25066067:DSO support routines:DLFCN_LOAD:could not load the shared library:dso_dlfcn.c:162:filename(/usr/lib/engines/libpkcs11.so): 0509-022 Cannot load module /usr/lib/engines/libpkcs11.so.
0509-101 The module has too many section headers
or the file is damaged.
487610:error:25070067:DSO support routines:DSO_load:could not load the shared library:dso_lib.c:244:
487610:error:260B6084:engine routines:DYNAMIC_LOAD:dso not found:eng_dyn.c:450:
487610:error:2606A074:engine routines:ENGINE_by_id:no such engine:eng_list.c:415:id=pkcs11

Why i am getting this error ? This engine patch is statically linked to openssl and it should not look for the library pkcs11.so. I am creating a shared library.

Even the command does not shows the pkcs11 engine ..

# ./openssl engine
(dynamic) Dynamic engine loading support

I do not have hardware acceralator in my system ? Is this because of that ..

Thanks
Rajan

Posted by rajan chittil on December 21, 2009 at 01:03 AM CET #

Rajan, you run it from a local directory:

# ./openssl engine
(dynamic) Dynamic engine loading support

I assume you have not installed the libs so the command probably uses OpenSSL libraries from the system. Try to set the library path to see if it helps:

# cd openssl-0.9.8l
# LD_LIBRARY_PATH=. ./apps/openssl engine

If unsure what libraries are used, run "ldd ./apps/openssl" while the env variable is set to the location of the built libraries.

Posted by Jan Pechanec on December 21, 2009 at 03:27 AM CET #

Hi Jan,

# LD_LIBRARY_PATH=. ./apps/openssl engine
(dynamic) Dynamic engine loading support

# ldd ./apps/openssl
./apps/openssl needs:
/usr/lib/libc.a(shr.o)
/usr/lib/libpthreads.a(shr_xpg5.o)
/home/rajan/pkcs/openssl-0.9.8k/libcrypto.a(libcrypto.so.0.9.8)
/home/rajan/pkcs/openssl-0.9.8k/libssl.a(libssl.so.0.9.8)
/unix
/usr/lib/libcrypt.a(shr.o)
/usr/lib/libpthreads.a(shr_comm.o)

what should ./openssl engine shows?

If i build static openssl ,I can see the pkcs11 engine from ./openssl engine command but shared library does not show that ?

Can you please paste me the output expected from ./openssl engine?

Thanks
Rajan

Posted by rajan chittil on December 21, 2009 at 06:53 AM CET #

Hi Jan,
I think when i am building shared library , pkcs11 engine need to dynamic loaded . so it is searching for the pkcs11.so lib from /usr/lib/engine path for plugging in the engine to openssl at run time .
Thanks
Rajan

Posted by rajan chittil on December 22, 2009 at 06:51 AM CET #

Rajan, the engine does not need to be dynamic in order to build shared libraries. In Solaris, we use the engine and the libraries are dynamic, of course.

You might be giving it a wrong PKCS#11 library name when configuring OpenSSL. If the PKCS#11 engine is not shown in the output of "openssl engine", and you are using the right openssl with the PKCS#11 support, it often suggests that the PKCS#11 library could not be loaded.

Posted by Jan Pechanec on December 22, 2009 at 07:01 AM CET #

Hi Jan,

I have used pkcs11 patch and was able to build it successfully.But i can see this error while running it.

openssl engine pkcs11
invalid engine "pkcs11"
engine routines:DYNAMIC_LOAD:init failed:eng_dyn.c:521:
no Engine_id:eng_list.c:433

Please help to solve this problem?

Thanks in advance
Joshi

Posted by joshi on December 30, 2009 at 12:06 AM CET #

Joshi, it might be the same problem as above. The PKCS#11 library you used when building OpenSSL doesn't seem to be loadable. Truss "openssl engine" to see why dlopen(3C) failed. The engine is static so when it tries to load it dynamically, something must have gone wrong already.

Posted by Jan Pechanec on January 04, 2010 at 03:17 AM CET #

Thanks Jan

I was able to successfully run the pkcs11 engine.

I have another query regarding the openssl.cnf.

what configuration should i used in the openssl.cnf so that i can tell openssl to use pkcs11 engine to perform crypto operation.

Thanks
Rajan

Posted by rajan chittil on January 11, 2010 at 04:29 AM CET #

Hi Jan,

My hardware do not have support for the C_VerifyRecover function .but it support C_Verify.I can see pk11_RSA_public_decrypt,it uses C_VerifyRecover since it is not supported , it fails here . Is it possible to use C_Verify instead of C_VerifyRecover ? if not , is there any solution to overcome this.
Please Help.
Thanks
Samuel

Posted by samuel on January 17, 2010 at 10:58 PM CET #

Hello Jan,

i tried to use your engine under linux with the opencrytoki library.
But I have problems to load the key. I have a softtoken with the label Soft and a key with label key. When i try to use it with the following uri : "pkcs11:object=key;token=Soft;passphrasedialog=builtin" the software does not ask for the pin and says "token attrs provided do not match". When I use it without the token=Soft parameter it asks for the pin but says it cannot find the key.
Do you have any suggestion for this problem?

Regards,

Achim Kanert

Posted by Achim Kanert on January 28, 2010 at 07:34 AM CET #

Achim, either the token is not called "Soft" or the pkcs11 library uses a different token which does not contain any key labeled "key". As explained in the man page section pasted to my blog entry, the PKCS#11 engine cannot itself change the token used, the attributes are there so that the URI can be used in other environments. I suggest to verify what token is used by default by the opencrytoki library, and also to verify the key label. J.

Posted by Jan on January 28, 2010 at 07:45 AM CET #

Hi Jan,

I ran through the openssl test suite and i can see many memory leak introduced here .

[18:08:26] 1783 file=dso_lib.c, line=380, thread=127038, number=30, address=20041728
[18:08:26] 1826 file=obj_lib.c, line=103, thread=127038, number=12, address=20043F28
[18:08:26] 1833 file=lhash.c, line=193, thread=127038, number=12, address=200441A8
[18:08:26] 1810 file=obj_lib.c, line=93, thread=127038, number=12, address=20043A98
[18:08:26] 1809 file=obj_lib.c, line=82, thread=127038, number=1, address=20043A38
[18:08:26] 1793 file=obj_lib.c, line=82, thread=127038, number=1, address=2001D608
[18:08:26] 1796 file=obj_dat.c, line=241, thread=127038, number=8, address=20027B68
[18:08:26] 1830 file=obj_dat.c, line=247, thread=127038, number=8, address=200440B8
[18:08:26] 1829 file=obj_dat.c, line=245, thread=127038, number=8, address=20044058
[18:08:26] 1817 file=lhash.c, line=193, thread=127038, number=12, address=20043D18
[18:08:26] 1802 file=lhash.c, line=193, thread=127038, number=12, address=200438E8
[18:08:26] 1791 file=lhash.c, line=121, thread=127038, number=64, address=20043458
[18:08:26] 1795 file=obj_lib.c, line=103, thread=127038, number=12, address=20043628
[18:08:26] 1828 file=obj_dat.c, line=243, thread=127038, number=8, address=20043FF8
[18:08:26] 1823 file=a_object.c, line=334, thread=127038, number=24, address=20043DF8
[18:08:26] 1840 file=hw_pk11.c, line=828, thread=127038, number=52, address=200444A8
[18:08:26] 1780 file=dso_lib.c, line=103, thread=127038, number=40, address=200415C8
[18:08:26] 1808 file=a_object.c, line=334, thread=127038, number=24, address=200439C8
[18:08:26] 1827 file=obj_dat.c, line=241, thread=127038, number=8, address=20043F98
[18:08:26] 1790 file=lhash.c, line=119, thread=127038, number=96, address=20043398
[18:08:26] 1839 file=hw_pk11.c, line=828, thread=127038, number=52, address=20044418
[18:08:26] 1824 file=obj_lib.c, line=82, thread=127038, number=1, address=20043738
[18:08:26] 1811 file=obj_lib.c, line=103, thread=127038, number=12, address=20043B08
[18:08:26] 1838 file=hw_pk11.c, line=828, thread=127038, number=52, address=20044388
[18:08:26] 1801 file=lhash.c, line=193, thread=127038, number=12, address=20043878
[18:08:26] 1781 file=stack.c, line=125, thread=127038, number=20, address=20041648
[18:08:26] 1837 file=hw_pk11.c, line=819, thread=127038, number=52, address=200442F8
[18:08:26] 1794 file=obj_lib.c, line=93, thread=127038, number=12, address=200435B8
[18:08:26] 1836 file=hw_pk11.c, line=812, thread=127038, number=52, address=20044268
[18:08:26] 1818 file=lhash.c, line=193, thread=127038, number=12, address=20043D88
[18:08:26] 1803 file=lhash.c, line=193, thread=127038, number=12, address=20043958
[18:08:26] 1835 file=hw_pk11.c, line=805, thread=127038, number=52, address=20043358
[18:08:26] 1792 file=a_object.c, line=334, thread=127038, number=24, address=200434F8
[18:08:26] 1815 file=obj_dat.c, line=247, thread=127038, number=8, address=20043C98
[18:08:26] 1814 file=obj_dat.c, line=245, thread=127038, number=8, address=20043C38
[18:08:26] 1800 file=lhash.c, line=193, thread=127038, number=12, address=20043808
[18:08:26] 1831 file=lhash.c, line=193, thread=127038, number=12, address=20043BD8
[18:08:26] 1799 file=obj_dat.c, line=247, thread=127038, number=8, address=200437A8
[18:08:26] 1784 file=dso_dlfcn.c, line=350, thread=127038, number=30, address=200417A8
[18:08:26] 1843 file=hw_pk11.c, line=828, thread=127038, number=52, address=20044658
[18:08:26] 1812 file=obj_dat.c, line=241, thread=127038, number=8, address=20043B78
[18:08:26] 1798 file=obj_dat.c, line=245, thread=127038, number=8, address=20043748
[18:08:26] 1842 file=hw_pk11.c, line=828, thread=127038, number=52, address=200445C8
[18:08:26] 1816 file=lhash.c, line=193, thread=127038, number=12, address=200436E8
[18:08:26] 1825 file=obj_lib.c, line=93, thread=127038, number=12, address=20043EB8
[18:08:26] 1782 file=stack.c, line=127, thread=127038, number=16, address=200416B8
[18:08:26] 1832 file=lhash.c, line=193, thread=127038, number=12, address=20044138
[18:08:26] 1841 file=hw_pk11.c, line=828, thread=127038, number=52, address=20044538
1111 bytes leaked in 48 chunks

I can even see memory leaks in dso_lib.c iand stack.c.This memory leak was not there before .

what need to do to solve this memory leak.

Thanks in advance
Samuel

Posted by samuel on February 01, 2010 at 06:36 AM CET #

Samuel, I have no idea why (especially) all those non-PKCS#11 engine files are included there since we haven't touched them. Unfortunately, I have no free cycles to take a look at it any time soon.

Posted by Jan on February 01, 2010 at 07:05 AM CET #

Hi Jan ,

I can see this memory leak was reported earlier too.
Please visit this link
http://blogs.sun.com/janp/entry/fixed_pkcs_11_engine_patch

Please let me how it can be fixed...?

Thanks
Samuel

Posted by samuel on February 01, 2010 at 07:12 AM CET #

Hi Samuel,

I can even this leaking but it can be fixed easily by walking through the code . Check whether you called ENGINE_cleanup() API at the exit of the program.

Hi Jan,

I modified the openssl test suite of RSA and when I run it ,I can see some segmentation fault plus Encryption and decrytion failure .

I will paste the test suite code and error message .Hope so you will be able to help me in fixing this issue .

I am sorry ,I am attaching such a long mail . I cannot find to attach file so i am copying the test case here

#include <stdio.h>
#include <string.h>

#include "e_os.h"

#include <openssl/crypto.h>
#include <openssl/err.h>
#include <openssl/rand.h>
#include <openssl/bn.h>
#ifdef OPENSSL_NO_RSA
int main(int argc, char \*argv[])
{
printf("No RSA support\\n");
return(0);
}
#else
#include <openssl/rsa.h>

#define SetKey \\
key->n = BN_bin2bn(n, sizeof(n)-1, key->n); \\
key->e = BN_bin2bn(e, sizeof(e)-1, key->e); \\
key->d = BN_bin2bn(d, sizeof(d)-1, key->d); \\
key->p = BN_bin2bn(p, sizeof(p)-1, key->p); \\
key->q = BN_bin2bn(q, sizeof(q)-1, key->q); \\
key->dmp1 = BN_bin2bn(dmp1, sizeof(dmp1)-1, key->dmp1); \\
key->dmq1 = BN_bin2bn(dmq1, sizeof(dmq1)-1, key->dmq1); \\
key->iqmp = BN_bin2bn(iqmp, sizeof(iqmp)-1, key->iqmp); \\
memcpy(c, ctext_ex, sizeof(ctext_ex) - 1); \\
return (sizeof(ctext_ex) - 1);

static int key1(RSA \*key, unsigned char \*c)
{
static unsigned char n[] =
"\\x00\\xAA\\x36\\xAB\\xCE\\x88\\xAC\\xFD\\xFF\\x55\\x52\\x3C\\x7F\\xC4\\x52\\x3F"
"\\x90\\xEF\\xA0\\x0D\\xF3\\x77\\x4A\\x25\\x9F\\x2E\\x62\\xB4\\xC5\\xD9\\x9C\\xB5"
"\\xAD\\xB3\\x00\\xA0\\x28\\x5E\\x53\\x01\\x93\\x0E\\x0C\\x70\\xFB\\x68\\x76\\x93"
"\\x9C\\xE6\\x16\\xCE\\x62\\x4A\\x11\\xE0\\x08\\x6D\\x34\\x1E\\xBC\\xAC\\xA0\\xA1"
"\\xF5";

static unsigned char e[] = "\\x11";

static unsigned char d[] =
"\\x0A\\x03\\x37\\x48\\x62\\x64\\x87\\x69\\x5F\\x5F\\x30\\xBC\\x38\\xB9\\x8B\\x44"
"\\xC2\\xCD\\x2D\\xFF\\x43\\x40\\x98\\xCD\\x20\\xD8\\xA1\\x38\\xD0\\x90\\xBF\\x64"
"\\x79\\x7C\\x3F\\xA7\\xA2\\xCD\\xCB\\x3C\\xD1\\xE0\\xBD\\xBA\\x26\\x54\\xB4\\xF9"
"\\xDF\\x8E\\x8A\\xE5\\x9D\\x73\\x3D\\x9F\\x33\\xB3\\x01\\x62\\x4A\\xFD\\x1D\\x51";

static unsigned char p[] =
"\\x00\\xD8\\x40\\xB4\\x16\\x66\\xB4\\x2E\\x92\\xEA\\x0D\\xA3\\xB4\\x32\\x04\\xB5"
"\\xCF\\xCE\\x33\\x52\\x52\\x4D\\x04\\x16\\xA5\\xA4\\x41\\xE7\\x00\\xAF\\x46\\x12"
"\\x0D";

static unsigned char q[] =
"\\x00\\xC9\\x7F\\xB1\\xF0\\x27\\xF4\\x53\\xF6\\x34\\x12\\x33\\xEA\\xAA\\xD1\\xD9"
"\\x35\\x3F\\x6C\\x42\\xD0\\x88\\x66\\xB1\\xD0\\x5A\\x0F\\x20\\x35\\x02\\x8B\\x9D"
"\\x89";

static unsigned char dmp1[] =
"\\x59\\x0B\\x95\\x72\\xA2\\xC2\\xA9\\xC4\\x06\\x05\\x9D\\xC2\\xAB\\x2F\\x1D\\xAF"
"\\xEB\\x7E\\x8B\\x4F\\x10\\xA7\\x54\\x9E\\x8E\\xED\\xF5\\xB4\\xFC\\xE0\\x9E\\x05";

static unsigned char dmq1[] =
"\\x00\\x8E\\x3C\\x05\\x21\\xFE\\x15\\xE0\\xEA\\x06\\xA3\\x6F\\xF0\\xF1\\x0C\\x99"
"\\x52\\xC3\\x5B\\x7A\\x75\\x14\\xFD\\x32\\x38\\xB8\\x0A\\xAD\\x52\\x98\\x62\\x8D"
"\\x51";

static unsigned char iqmp[] =
"\\x36\\x3F\\xF7\\x18\\x9D\\xA8\\xE9\\x0B\\x1D\\x34\\x1F\\x71\\xD0\\x9B\\x76\\xA8"
"\\xA9\\x43\\xE1\\x1D\\x10\\xB2\\x4D\\x24\\x9F\\x2D\\xEA\\xFE\\xF8\\x0C\\x18\\x26";

static unsigned char ctext_ex[] =
"\\x1b\\x8f\\x05\\xf9\\xca\\x1a\\x79\\x52\\x6e\\x53\\xf3\\xcc\\x51\\x4f\\xdb\\x89"
"\\x2b\\xfb\\x91\\x93\\x23\\x1e\\x78\\xb9\\x92\\xe6\\x8d\\x50\\xa4\\x80\\xcb\\x52"
"\\x33\\x89\\x5c\\x74\\x95\\x8d\\x5d\\x02\\xab\\x8c\\x0f\\xd0\\x40\\xeb\\x58\\x44"
"\\xb0\\x05\\xc3\\x9e\\xd8\\x27\\x4a\\x9d\\xbf\\xa8\\x06\\x71\\x40\\x94\\x39\\xd2";

SetKey;
}

static int key2(RSA \*key, unsigned char \*c)
{
static unsigned char n[] =
"\\x00\\xA3\\x07\\x9A\\x90\\xDF\\x0D\\xFD\\x72\\xAC\\x09\\x0C\\xCC\\x2A\\x78\\xB8"
"\\x74\\x13\\x13\\x3E\\x40\\x75\\x9C\\x98\\xFA\\xF8\\x20\\x4F\\x35\\x8A\\x0B\\x26"
"\\x3C\\x67\\x70\\xE7\\x83\\xA9\\x3B\\x69\\x71\\xB7\\x37\\x79\\xD2\\x71\\x7B\\xE8"
"\\x34\\x77\\xCF";

static unsigned char e[] = "\\x3";

static unsigned char d[] =
"\\x6C\\xAF\\xBC\\x60\\x94\\xB3\\xFE\\x4C\\x72\\xB0\\xB3\\x32\\xC6\\xFB\\x25\\xA2"
"\\xB7\\x62\\x29\\x80\\x4E\\x68\\x65\\xFC\\xA4\\x5A\\x74\\xDF\\x0F\\x8F\\xB8\\x41"
"\\x3B\\x52\\xC0\\xD0\\xE5\\x3D\\x9B\\x59\\x0F\\xF1\\x9B\\xE7\\x9F\\x49\\xDD\\x21"
"\\xE5\\xEB";

static unsigned char p[] =
"\\x00\\xCF\\x20\\x35\\x02\\x8B\\x9D\\x86\\x98\\x40\\xB4\\x16\\x66\\xB4\\x2E\\x92"
"\\xEA\\x0D\\xA3\\xB4\\x32\\x04\\xB5\\xCF\\xCE\\x91";

static unsigned char q[] =
"\\x00\\xC9\\x7F\\xB1\\xF0\\x27\\xF4\\x53\\xF6\\x34\\x12\\x33\\xEA\\xAA\\xD1\\xD9"
"\\x35\\x3F\\x6C\\x42\\xD0\\x88\\x66\\xB1\\xD0\\x5F";
static unsigned char dmp1[] =
"\\x00\\x8A\\x15\\x78\\xAC\\x5D\\x13\\xAF\\x10\\x2B\\x22\\xB9\\x99\\xCD\\x74\\x61"
"\\xF1\\x5E\\x6D\\x22\\xCC\\x03\\x23\\xDF\\xDF\\x0B";

static unsigned char dmq1[] =
"\\x00\\x86\\x55\\x21\\x4A\\xC5\\x4D\\x8D\\x4E\\xCD\\x61\\x77\\xF1\\xC7\\x36\\x90"
"\\xCE\\x2A\\x48\\x2C\\x8B\\x05\\x99\\xCB\\xE0\\x3F";

static unsigned char iqmp[] =
"\\x00\\x83\\xEF\\xEF\\xB8\\xA9\\xA4\\x0D\\x1D\\xB6\\xED\\x98\\xAD\\x84\\xED\\x13"
"\\x35\\xDC\\xC1\\x08\\xF3\\x22\\xD0\\x57\\xCF\\x8D";

static unsigned char ctext_ex[] =
"\\x14\\xbd\\xdd\\x28\\xc9\\x83\\x35\\x19\\x23\\x80\\xe8\\xe5\\x49\\xb1\\x58\\x2a"
"\\x8b\\x40\\xb4\\x48\\x6d\\x03\\xa6\\xa5\\x31\\x1f\\x1f\\xd5\\xf0\\xa1\\x80\\xe4"
"\\x17\\x53\\x03\\x29\\xa9\\x34\\x90\\x74\\xb1\\x52\\x13\\x54\\x29\\x08\\x24\\x52"
"\\x62\\x51";

SetKey;
}

static int key3(RSA \*key, unsigned char \*c)
{
static unsigned char n[] =
"\\x00\\xBB\\xF8\\x2F\\x09\\x06\\x82\\xCE\\x9C\\x23\\x38\\xAC\\x2B\\x9D\\xA8\\x71"
"\\xF7\\x36\\x8D\\x07\\xEE\\xD4\\x10\\x43\\xA4\\x40\\xD6\\xB6\\xF0\\x74\\x54\\xF5"
"\\x1F\\xB8\\xDF\\xBA\\xAF\\x03\\x5C\\x02\\xAB\\x61\\xEA\\x48\\xCE\\xEB\\x6F\\xCD"
"\\x48\\x76\\xED\\x52\\x0D\\x60\\xE1\\xEC\\x46\\x19\\x71\\x9D\\x8A\\x5B\\x8B\\x80"
"\\x7F\\xAF\\xB8\\xE0\\xA3\\xDF\\xC7\\x37\\x72\\x3E\\xE6\\xB4\\xB7\\xD9\\x3A\\x25"
"\\x84\\xEE\\x6A\\x64\\x9D\\x06\\x09\\x53\\x74\\x88\\x34\\xB2\\x45\\x45\\x98\\x39"
"\\x4E\\xE0\\xAA\\xB1\\x2D\\x7B\\x61\\xA5\\x1F\\x52\\x7A\\x9A\\x41\\xF6\\xC1\\x68"
"\\x7F\\xE2\\x53\\x72\\x98\\xCA\\x2A\\x8F\\x59\\x46\\xF8\\xE5\\xFD\\x09\\x1D\\xBD"
"\\xCB";

static unsigned char e[] = "\\x11";

static unsigned char d[] =
"\\x00\\xA5\\xDA\\xFC\\x53\\x41\\xFA\\xF2\\x89\\xC4\\xB9\\x88\\xDB\\x30\\xC1\\xCD"
"\\xF8\\x3F\\x31\\x25\\x1E\\x06\\x68\\xB4\\x27\\x84\\x81\\x38\\x01\\x57\\x96\\x41"
"\\xB2\\x94\\x10\\xB3\\xC7\\x99\\x8D\\x6B\\xC4\\x65\\x74\\x5E\\x5C\\x39\\x26\\x69"
"\\xD6\\x87\\x0D\\xA2\\xC0\\x82\\xA9\\x39\\xE3\\x7F\\xDC\\xB8\\x2E\\xC9\\x3E\\xDA"
"\\xC9\\x7F\\xF3\\xAD\\x59\\x50\\xAC\\xCF\\xBC\\x11\\x1C\\x76\\xF1\\xA9\\x52\\x94"
"\\x44\\xE5\\x6A\\xAF\\x68\\xC5\\x6C\\x09\\x2C\\xD3\\x8D\\xC3\\xBE\\xF5\\xD2\\x0A"
"\\x93\\x99\\x26\\xED\\x4F\\x74\\xA1\\x3E\\xDD\\xFB\\xE1\\xA1\\xCE\\xCC\\x48\\x94"
"\\xAF\\x94\\x28\\xC2\\xB7\\xB8\\x88\\x3F\\xE4\\x46\\x3A\\x4B\\xC8\\x5B\\x1C\\xB3"
"\\xC1";

static unsigned char p[] =
"\\x00\\xEE\\xCF\\xAE\\x81\\xB1\\xB9\\xB3\\xC9\\x08\\x81\\x0B\\x10\\xA1\\xB5\\x60"
"\\x01\\x99\\xEB\\x9F\\x44\\xAE\\xF4\\xFD\\xA4\\x93\\xB8\\x1A\\x9E\\x3D\\x84\\xF6"
"\\x32\\x12\\x4E\\xF0\\x23\\x6E\\x5D\\x1E\\x3B\\x7E\\x28\\xFA\\xE7\\xAA\\x04\\x0A"
"\\x2D\\x5B\\x25\\x21\\x76\\x45\\x9D\\x1F\\x39\\x75\\x41\\xBA\\x2A\\x58\\xFB\\x65"
"\\x99";

static unsigned char q[] =
"\\x00\\xC9\\x7F\\xB1\\xF0\\x27\\xF4\\x53\\xF6\\x34\\x12\\x33\\xEA\\xAA\\xD1\\xD9"
"\\x35\\x3F\\x6C\\x42\\xD0\\x88\\x66\\xB1\\xD0\\x5A\\x0F\\x20\\x35\\x02\\x8B\\x9D"
"\\x86\\x98\\x40\\xB4\\x16\\x66\\xB4\\x2E\\x92\\xEA\\x0D\\xA3\\xB4\\x32\\x04\\xB5"
"\\xCF\\xCE\\x33\\x52\\x52\\x4D\\x04\\x16\\xA5\\xA4\\x41\\xE7\\x00\\xAF\\x46\\x15"
"\\x03";

static unsigned char dmp1[] =
"\\x54\\x49\\x4C\\xA6\\x3E\\xBA\\x03\\x37\\xE4\\xE2\\x40\\x23\\xFC\\xD6\\x9A\\x5A"
"\\xEB\\x07\\xDD\\xDC\\x01\\x83\\xA4\\xD0\\xAC\\x9B\\x54\\xB0\\x51\\xF2\\xB1\\x3E"
"\\xD9\\x49\\x09\\x75\\xEA\\xB7\\x74\\x14\\xFF\\x59\\xC1\\xF7\\x69\\x2E\\x9A\\x2E"
"\\x20\\x2B\\x38\\xFC\\x91\\x0A\\x47\\x41\\x74\\xAD\\xC9\\x3C\\x1F\\x67\\xC9\\x81";

static unsigned char dmq1[] =
"\\x47\\x1E\\x02\\x90\\xFF\\x0A\\xF0\\x75\\x03\\x51\\xB7\\xF8\\x78\\x86\\x4C\\xA9"
"\\x61\\xAD\\xBD\\x3A\\x8A\\x7E\\x99\\x1C\\x5C\\x05\\x56\\xA9\\x4C\\x31\\x46\\xA7"
"\\xF9\\x80\\x3F\\x8F\\x6F\\x8A\\xE3\\x42\\xE9\\x31\\xFD\\x8A\\xE4\\x7A\\x22\\x0D"
"\\x1B\\x99\\xA4\\x95\\x84\\x98\\x07\\xFE\\x39\\xF9\\x24\\x5A\\x98\\x36\\xDA\\x3D";

static unsigned char iqmp[] =
"\\x00\\xB0\\x6C\\x4F\\xDA\\xBB\\x63\\x01\\x19\\x8D\\x26\\x5B\\xDB\\xAE\\x94\\x23"
"\\xB3\\x80\\xF2\\x71\\xF7\\x34\\x53\\x88\\x50\\x93\\x07\\x7F\\xCD\\x39\\xE2\\x11"
"\\x9F\\xC9\\x86\\x32\\x15\\x4F\\x58\\x83\\xB1\\x67\\xA9\\x67\\xBF\\x40\\x2B\\x4E"
"\\x9E\\x2E\\x0F\\x96\\x56\\xE6\\x98\\xEA\\x36\\x66\\xED\\xFB\\x25\\x79\\x80\\x39"
"\\xF7";

static unsigned char ctext_ex[] =
"\\xb8\\x24\\x6b\\x56\\xa6\\xed\\x58\\x81\\xae\\xb5\\x85\\xd9\\xa2\\x5b\\x2a\\xd7"
"\\x90\\xc4\\x17\\xe0\\x80\\x68\\x1b\\xf1\\xac\\x2b\\xc3\\xde\\xb6\\x9d\\x8b\\xce"
"\\xf0\\xc4\\x36\\x6f\\xec\\x40\\x0a\\xf0\\x52\\xa7\\x2e\\x9b\\x0e\\xff\\xb5\\xb3"
"\\xf2\\xf1\\x92\\xdb\\xea\\xca\\x03\\xc1\\x27\\x40\\x05\\x71\\x13\\xbf\\x1f\\x06"
"\\x69\\xac\\x22\\xe9\\xf3\\xa7\\x85\\x2e\\x3c\\x15\\xd9\\x13\\xca\\xb0\\xb8\\x86"
"\\x3a\\x95\\xc9\\x92\\x94\\xce\\x86\\x74\\x21\\x49\\x54\\x61\\x03\\x46\\xf4\\xd4"
"\\x74\\xb2\\x6f\\x7c\\x48\\xb4\\x2e\\xe6\\x8e\\x1f\\x57\\x2a\\x1f\\xc4\\x02\\x6a"
"\\xc4\\x56\\xb4\\xf5\\x9f\\x7b\\x62\\x1e\\xa1\\xb9\\xd8\\x8f\\x64\\x20\\x2f\\xb1";

SetKey;
}

static int pad_unknown(void)
{
unsigned long l;
while ((l = ERR_get_error()) != 0)
if (ERR_GET_REASON(l) == RSA_R_UNKNOWN_PADDING_TYPE)
return(1);
return(0);
}

static const char rnd_seed[] = "string to make the random number generator think it has entropy";

int main(int argc, char \*argv[])
{
ENGINE \*e;
const char \*engine_id = "pkcs11";
ENGINE_load_builtin_engines();
e = ENGINE_by_id(engine_id);
if(!e)
/\* the engine isn't available \*/
return 0;
if(!ENGINE_init(e)) {
/\* the engine couldn't initialise, release 'e' \*/
ENGINE_free(e);
return 0;
}
if(!ENGINE_set_default_RSA(e))
/\* This should only happen when 'e' can't initialise, but the previous
\* statement suggests it did. \*/
abort();
int err=0;
int v;
RSA \*key;
unsigned char ptext[256];
unsigned char ctext[256];
static unsigned char ptext_ex[] = "\\x54\\x85\\x9b\\x34\\x2c\\x49\\xea\\x2a";
unsigned char ctext_ex[256];
int plen;
int clen = 0;
int num;
int n;

CRYPTO_malloc_debug_init();
CRYPTO_dbg_set_options(V_CRYPTO_MDEBUG_ALL);
CRYPTO_mem_ctrl(CRYPTO_MEM_CHECK_ON);

RAND_seed(rnd_seed, sizeof rnd_seed); /\* or OAEP may fail \*/

plen = sizeof(ptext_ex) - 1;

for (v = 0; v < 6; v++)
{
key = RSA_new();
switch (v%3) {
case 0:
clen = key1(key, ctext_ex);
break;
case 1:
clen = key2(key, ctext_ex);
break;
case 2:
clen = key3(key, ctext_ex);
break;
}
if (v/3 >= 1) key->flags |= RSA_FLAG_NO_CONSTTIME;
// printf("ptext=%s \\n plen=%d",ptext_ex,plen);
//printf("ctext=%s\\n",ctext);
printf("\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* RSA_PKCS1_PADDING \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\\n");
num = RSA_public_encrypt(plen, ptext_ex, ctext, key,
RSA_PKCS1_PADDING);

printf("clen=%d num=%d",clen,num);
if (num != clen)
{
printf("PKCS#1 v1.5 encryption failed!\\n");
err=1;
goto oaep;
}

num = RSA_private_decrypt(num, ctext, ptext, key,
RSA_PKCS1_PADDING);

if (num != plen || memcmp(ptext, ptext_ex, num) != 0)
{
printf("PKCS#1 v1.5 decryption failed!\\n");
err=1;
}
else
printf("PKCS #1 v1.5 encryption/decryption ok\\n");

oaep:
printf("\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* RSA_PKCS1_OAEP_PADDING \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\\n");
num = RSA_public_encrypt(plen, ptext_ex, ctext, key,
RSA_PKCS1_OAEP_PADDING);
if (num == -1 && pad_unknown())
{
printf("No OAEP support\\n");
goto next;
}
if (num != clen)
{
printf("OAEP encryption failed!\\n");
err=1;
goto next;
}

num = RSA_private_decrypt(num, ctext, ptext, key,
RSA_PKCS1_OAEP_PADDING);

if (num != plen || memcmp(ptext, ptext_ex, num) != 0)
{
printf("OAEP decryption (encrypted data) failed!\\n");
err=1;
}
else if (memcmp(ctext, ctext_ex, num) == 0)
printf("OAEP test vector %d passed!\\n", v);

/\* Different ciphertexts (rsa_oaep.c without -DPKCS_TESTVECT).
Try decrypting ctext_ex \*/

num = RSA_private_decrypt(clen, ctext_ex, ptext, key,
RSA_PKCS1_OAEP_PADDING);

if (num != plen || memcmp(ptext, ptext_ex, num) != 0)
{
printf("OAEP decryption (test vector data) failed!\\n");
err=1;
}
else
printf("OAEP encryption/decryption ok\\n");
printf("\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* decrypting corrupted ciphertext RSA_PKCS1_OAEP_PADDING\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\\n");

/\* Try decrypting corrupted ciphertexts \*/
for(n = 0 ; n < clen ; ++n)
{
int b;
unsigned char saved = ctext[n];
for(b = 0 ; b < 256 ; ++b)
{
if(b == saved)
continue;
ctext[n] = b;
num = RSA_private_decrypt(num, ctext, ptext, key,
RSA_PKCS1_OAEP_PADDING);
if(num > 0)
{
printf("Corrupt data decrypted!\\n");
err = 1;
}
}
}
next:
RSA_free(key);
}

CRYPTO_cleanup_all_ex_data();
ERR_remove_state(0);

CRYPTO_mem_leaks_fp(stderr);
ENGINE_cleanup();
#ifdef OPENSSL_SYS_NETWARE
if (err) printf("ERROR: %d\\n", err);
#endif
return err;
}
#endif

This is the error message I am getting

PKCS #1 v1.5 encryption/decryption ok
OAEP encryption failed!
clen=50 num=-1PKCS#1 v1.5 encryption failed!
OAEP encryption failed!
clen=128 num=-1PKCS#1 v1.5 encryption failed!
OAEP encryption failed!
clen=64 num=64num2=8
PKCS #1 v1.5 encryption/decryption ok
Segmentation fault(coredump)

Please help me solve this problem.

Thanks
Rajan

Posted by rajan chittil on February 02, 2010 at 07:33 AM CET #

I'm sorry Rajan, those are your changes, I think you are on your own here :-) I really don't have any cycles I could spend on this issue.

Posted by Jan Pechanec on February 02, 2010 at 08:08 AM CET #

Hi Jan,

I have not changed any code in openssl pkcs11 engine code . I just wrote one test case to use openssl pkcs11 engine API but can see this failure message . Is my testcases wrong ? Please direct me so that i can fix this issue .

Thanks
Rajan

Posted by rajan chittil on February 02, 2010 at 08:56 AM CET #

Hi Jan,

I have tested this code and seems like some problem is there with the code . Can you please test above test case and verify it occurs in ur sytem too .

Note : I have not modified any piece of code .

Looking forward for your help.

Thanks in advance
Rajan

Posted by rajan chittil on February 02, 2010 at 10:14 AM CET #

Hi Jan,

I hope you might have run this test case .This testcases will give segmentation fault.I think it will not take more than 5 minutes for you to test it .

Hope so you will test this scenario too.

Thanks
Rajan

Posted by rajan chittil on February 03, 2010 at 08:41 AM CET #

Dear Jan,
What a great job !
I'm currently using a CentOS 5.4 with :
- OpenSC to generate RSA key pairs
- OpenSSL to make use of them.

When I try a few sample openssl commands, it works fine with openssl rsa or req commands but fails when using the openssl ca command. Any idea what the reason can be?

Othe point, I also tried to set the OPENSSL_PKCS11_PIN_CACHING_POLICY environement variable but the values 'memory' or 'mlocked-memory' forces openssl to hang ?

Thanks for your help.

Regards,

Olivier

Posted by Olivier LIBON on February 14, 2010 at 04:42 PM CET #

Olivier, I have no idea why 'ca' subcommand fails. You also don't provide any details on that. Wrt PIN caching, it works in OpenSolaris but I haven't tested it on any Linux distro, and unfortunately I don't have cycles to do that now. If you find out what is the problem, please let me know.

Posted by Jan on February 15, 2010 at 04:06 AM CET #

Hi Jan,

I have used opensolaris pkcs11 engine and was able to successfully
build build pkcs11 engine with openssl and I have tested this with openssl test suit and it all works fine .

I am trying to use pkcs11 engine with OpenSSH .But I am get the following error when I am trying to coonect client to server .

Disconnecting: Bad packet length in ssh.

Is this problem with openssl pkcs11 engine or with ssh code .

Please help

Thanks in advance
Jais

Posted by Jais michael on February 22, 2010 at 01:05 PM CET #

Hi Jan,

This issue happens at the end of communication ie after client gives the password for the login id .I have gone through the PKCS11#API traces and finds that all the API returns 0 (successfully executed). I am not able to make out what is happening .

One more thing I have noticed is that it happens only if CIPHER and Some other Crypto operation (like DIGEST or Random operation)is used .If CIPHER operation is alone used (Activated ) others are disabled ,It works fine .

Please help me solve this issue .

Thanks
Jais

Posted by Jais michael on February 22, 2010 at 01:35 PM CET #

Hi Jan,

Just want to know from you .To use OpenSSL pkcs11 engine ,Does it require to change the OpenSSH code .I want through the SunSSH code and can see huge changes in SunSSH code to use OpenSSL pkcs11 engine .

Thanks in advance
Jais

Posted by Jais michael on February 23, 2010 at 02:48 AM CET #

hi Jais, I believe the problem is with PKCS#11 fork safety issues. I had to do significant changes to SunSSH to avoid those problems. OpenSSH privilege separation code is completely different, I don't know what or if there are any changes needed.

See my blog entry on HW support in SunSSH, I think I discuss the problem there.

Posted by Jan on February 23, 2010 at 03:52 AM CET #

Hi Jan,

I need to port openssl with PKCS#11 patch on windows. Are there any instructions how to port the PKCS#11 patch on windows?

Thanks,

Victor

Posted by Victor on February 25, 2010 at 10:22 AM CET #

Hi Jan,

I see OpenSSL 1.0 is out. Do you plan to provide a port of the PKCS#11 engine patch for it?

Thanks.

-John

Posted by John Center on March 29, 2010 at 11:51 AM CEST #

Hi John, we are quite limited as to what we can say to external people now. However, I think it's quite safe to say that I have no idea if I can and/or will provide the patch, and/or when.

I'd suggest to apply the current patch (it will not apply cleanly) and then to try to resolve the problems manually. I don't think any \*.c or \*h file will need any change, it might be just Makefile. It shouldn't be hard to fix when you take patched files from 0.9.8k and compare with those from 1.0.0.

Hope this helps. Good luck, Jan.

Posted by Jan on March 29, 2010 at 12:19 PM CEST #

Hi Jan,

I tried to align the pkcs#11 engine patch to openssl 1.0, but I still have issues. The resolving of the patches were pretty easy, but I had to enable md2 support, as it was turned of as default in openssl 1.0.

So I was able to compile it, but it still doesn't find the engine.

#/home/daniel/openssl/bin/openssl speed rsa -engine pkcs11
invalid engine "pkcs11"
4278680460:error:25066067:DSO support routines:DLFCN_LOAD:could not load the shared library:dso_dlfcn.c:185:filename(/home/daniel/openssl/lib/engines/libpkcs11.so): ld.so.1: openssl: fatal: /home/daniel/openssl/lib/engines/libpkcs11.so: open failed: No such file or directory
4278680460:error:25070067:DSO support routines:DSO_load:could not load the shared library:dso_lib.c:244:
4278680460:error:260B6084:engine routines:DYNAMIC_LOAD:dso not found:eng_dyn.c:450:
4278680460:error:2606A074:engine routines:ENGINE_by_id:no such engine:eng_list.c:417:id=pkcs11
4278680460:error:2506406A:DSO support routines:DLFCN_BIND_FUNC:could not bind to the requested symbol name:dso_dlfcn.c:287:symname(bind_engine): ld.so.1: openssl: fatal: bind_engine: can't find symbol
4278680460:error:2506C06A:DSO support routines:DSO_bind_func:could not bind to the requested symbol name:dso_lib.c:294:
4278680460:error:260B6068:engine routines:DYNAMIC_LOAD:DSO failure:eng_dyn.c:463:

If it is okay for you I would offer my current patch between openssl 1.0 and a pkcs#11 patched openssl variant for other readers here in the blog. Eventually someone knows how to fix it.

Please let me know if thats okay and I will host the "test-patch" at my server to offer it to the others.

Cheers,

Daniel

Posted by Daniel on April 06, 2010 at 07:16 AM CEST #

Jan,
some more details (truss output)

[...]
stat("/home/daniel/openssl/lib/engines/libpkcs11.so", 0xFFBFBF18) Err#2 ENOENT
stat("/home/daniel/lib/libpkcs11.so", 0xFFBFBF10) Err#2 ENOENT
stat("/lib/libpkcs11.so", 0xFFBFBF10) Err#2 ENOENT
stat("/usr/lib/libpkcs11.so", 0xFFBFBF10) = 0
resolvepath("/usr/lib/libpkcs11.so", "/usr/lib/libpkcs11.so.1", 1023) = 23
open("/usr/lib/libpkcs11.so", O_RDONLY)
mmap(0x00010000, 32768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_ALIGN, 3, 0) = 0xFF060000
mmap(0x00010000, 188416, PROT_NONE, MAP_PRIVATE|MAP_NORESERVE|MAP_ANON|MAP_ALIGN, -1, 0) = 0xFEF50000
mmap(0xFEF50000, 89545, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_TEXT, 3, 0) = 0xFEF50000
mmap(0xFEF76000, 24528, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_INITDATA, 3, 90112) = 0xFEF76000
mmap(0xFEF7C000, 488, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_ANON, -1, 0) = 0xFEF7C000
munmap(0xFEF66000, 65536) = 0
mmap(0x00000000, 8192, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, -1, 0) = 0xFEF70000
memcntl(0xFEF50000, 10136, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
close(3) = 0
[...]
fstat64(2, 0xFFBFBA98) = 0
invalid engine "pkcs11"
write(2, " i n v a l i d e n g i".., 24) = 24
4278680460:error:25066067:DSO support routines:DLFCN_LOAD:could not load the shared library:dso_dlfcn.c:185:filename(/home/daniel/openssl/lib/engines/libpkcs11.so): ld.so.1: openssl: fatal: /home/daniel/openssl/lib/engi
nes/libpkcs11.so: open failed: No such file or directory
write(2, " 4 2 7 8 6 8 0 4 6 0 : e".., 282) = 282
4278680460:error:25070067:DSO support routines:DSO_load:could not load the shared library:dso_lib.c:244:
write(2, " 4 2 7 8 6 8 0 4 6 0 : e".., 105) = 105
4278680460:error:260B6084:engine routines:DYNAMIC_LOAD:dso not found:eng_dyn.c:450:
write(2, " 4 2 7 8 6 8 0 4 6 0 : e".., 84) = 84
4278680460:error:2606A074:engine routines:ENGINE_by_id:no such engine:eng_list.c:417:id=pkcs11
write(2, " 4 2 7 8 6 8 0 4 6 0 : e".., 95) = 95
4278680460:error:2506406A:DSO support routines:DLFCN_BIND_FUNC:could not bind to the requested symbol name:dso_dlfcn.c:287:symname(bind_engine): ld.so.1: openssl: fatal: bind_engine: can't find symbol
write(2, " 4 2 7 8 6 8 0 4 6 0 : e".., 201) = 201
4278680460:error:2506C06A:DSO support routines:DSO_bind_func:could not bind to the requested symbol name:dso_lib.c:294:
write(2, " 4 2 7 8 6 8 0 4 6 0 : e".., 120) = 120
4278680460:error:260B6068:engine routines:DYNAMIC_LOAD:DSO failure:eng_dyn.c:463:
write(2, " 4 2 7 8 6 8 0 4 6 0 : e".., 82) = 82
sigaction(SIGALRM, 0xFFBFC6C0, 0xFFBFC760) = 0
[...]

Posted by Daniel on April 06, 2010 at 07:51 AM CEST #

hi Daniel, it looks that openssl cannot find the PKCS#11 library you used to configure it with:

4278680460:error:25066067:DSO support routines:DLFCN_LOAD:could not load the shared library:dso_dlfcn.c:185:filename(/home/daniel/openssl/lib/engines/libpkcs11.so): ld.so.1: openssl: fatal: /home/daniel/openssl/lib/engines/libpkcs11.so: open failed: No such file or directory

I don't mind to put the patch for 1.0.0 here as a contribution from the community but I'd really like it was verified as working before that. It seems you are almost there.

Posted by Jan on April 06, 2010 at 08:33 AM CEST #

Hi Jan,

I always follow your blog and build all older openssl patches from you to the current openssl core version.

Up to 0.9.8 it worked fine when using engine pkcs11, but even in 0.9.8 I never had a /home/daniel/openssl/lib/engines/libpkcs11.so.

I have a truss of 0.9.8n with your latest pkcs11 patch (-l):

getpid() = 27999 [27998]
stat("/usr/lib/libpkcs11.so", 0xFFBFF048) = 0
resolvepath("/usr/lib/libpkcs11.so", "/usr/lib/libpkcs11.so.1", 1023) = 23
open("/usr/lib/libpkcs11.so", O_RDONLY) = 3
mmap(0x00010000, 32768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_ALIGN, 3, 0) = 0xFF150000
mmap(0x00010000, 188416, PROT_NONE, MAP_PRIVATE|MAP_NORESERVE|MAP_ANON|MAP_ALIGN, -1, 0) = 0xFEF50000
mmap(0xFEF50000, 89545, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_TEXT, 3, 0) = 0xFEF50000
mmap(0xFEF76000, 24528, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_INITDATA, 3, 90112) = 0xFEF76000
mmap(0xFEF7C000, 488, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_ANON, -1, 0) = 0xFEF7C000
munmap(0xFEF66000, 65536) = 0
mmap(0x00000000, 8192, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, -1, 0) = 0xFF140000
memcntl(0xFEF50000, 10136, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0

As you can see, it even not tries to load the engine from /home/daniel/openssl/lib/engines ... However if I enter a typo in the engine name, it looks similar to the openssl 1.0 scheme:
("engine pkcs-typo-11" instead of "engine pkcs11")
[...]
stat("/home/daniel/openssl/lib/engines/libpkcs-typo-11.so", 0xFFBFC188) Err#2 ENOENT
stat("/home/daniel/openssl/lib/libpkcs-typo-11.so", 0xFFBFC180) Err#2 ENOENT
stat("/lib/libpkcs-typo-11.so", 0xFFBFC180) Err#2 ENOENT
stat("/usr/lib/libpkcs-typo-11.so", 0xFFBFC180) Err#2 ENOENT
stat("/usr/krb5/lib/libpkcs-typo-11.so", 0xFFBFC180) Err#2 ENOENT
stat("/usr/vendor/sybase/lib/libpkcs-typo-11.so", 0xFFBFC180) Err#2 ENOENT
stat("/usr/local/lib/libpkcs-typo-11.so", 0xFFBFC180) Err#2 ENOENT
fstat64(2, 0xFFBFBCE8) = 0
invalid engine "pkcs-typo-11"
write(2, " i n v a l i d e n g i".., 30) = 30
28047:error:25066067:DSO support routines:DLFCN_LOAD:could not load the shared library:dso_dlfcn.c:162:filename(/home/daniel/openssl/lib/engines/libpkcs-typo-11.so): ld.so.1: openssl: fatal: /home/daniel/openssl/lib/engines/libpkcs-typo-11.so: open failed: No such file or directory
write(2, " 2 8 0 4 7 : e r r o r :".., 287) = 287
28047:error:25070067:DSO support routines:DSO_load:could not load the shared library:dso_lib.c:244:
write(2, " 2 8 0 4 7 : e r r o r :".., 100) = 100
28047:error:260B6084:engine routines:DYNAMIC_LOAD:dso not found:eng_dyn.c:450:
write(2, " 2 8 0 4 7 : e r r o r :".., 79) = 79
28047:error:2606A074:engine routines:ENGINE_by_id:no such engine:eng_list.c:415:id=pkcs-typo-11
write(2, " 2 8 0 4 7 : e r r o r :".., 96) = 96
28047:error:25066067:DSO support routines:DLFCN_LOAD:could not load the shared library:dso_dlfcn.c:162:filename(libpkcs-typo-11.so): ld.so.1: openssl: fatal: libpkcs-typo-11.so: open failed: No such file or directory
write(2, " 2 8 0 4 7 : e r r o r :".., 217) = 217
28047:error:25070067:DSO support routines:DSO_load:could not load the shared library:dso_lib.c:244:
write(2, " 2 8 0 4 7 : e r r o r :".., 100) = 100
28047:error:260B6084:engine routines:DYNAMIC_LOAD:dso not found:eng_dyn.c:450:
write(2, " 2 8 0 4 7 : e r r o r :".., 79) = 79
sigaction(SIGALRM, 0xFFBFC910, 0xFFBFC9B0) = 0
[...]

If I select pkcs11 in the old openssl 0.9.8 it even not tries to load an external engine as the support must be included. But I see the loading scheme like in OpenSSL 1.0.0 when I enter a wrong engine name inside 0.9.8.

So I come to the conclusion that I missed something to include pkcs11 support directly into openssl 1.0.0.

How could I force to made a libpkcs11.so in lib/engines in the old openssl 0.9.8 ?

Thanks,

Daniel

Posted by Daniel on April 06, 2010 at 12:37 PM CEST #

Aha, I understand, it just tries to load the PKCS#11 library until it finds one. Hmm, it looks like 1.0.0 might have changed something but I'm not sure what it could be. The patch didn't apply cleanly - are you sure you did those manual changes right? If you did, there is a change in 1.0.0 and we will have to fix the engine.

We do not ship 1.0.0 yet and unfortunately, I do not have free loops to take a look at it, I'm really sorry :-( What I can suggest is that you truss both 0.9.8 and 1.0.0 and carefully compare the whole results. You should find where they differ. Sorry I can't help more.

Posted by Jan Pechanec on April 06, 2010 at 12:48 PM CEST #

If someone would like to have a look, here is my (faulty) patch try:
http://www.flinkmann.de/~daniel/temp/test-patch-openssl10-with-pkcs11.tgz

Cheers,

Daniel

Posted by Daniel on April 06, 2010 at 12:50 PM CEST #

Hi Jan,

Yes I had to manually change the rejected patch parts. I hope I have done it right, but as it is still not working, I could have some something wrong or the engine loading is changed.

Just to ask it again: How could I force the Configure/Makefile to create a libpkcs11.so in lib/engines in the old openssl 0.9.8 ?

Posted by Daniel on April 06, 2010 at 05:30 PM CEST #

It does not create a dynamic engine, it's a static one. We could not deliver dynamic engine support in Solaris due to US export rules. The shared object needed externally is the PKCS#11 library providing the PKCS#11 API.

Posted by Jan on April 07, 2010 at 04:31 AM CEST #

Hello,

Your name keeps coming up as I'm searching for an answer to an issue I am seeing on a Sparc Solaris 10 server. Thanks for any help you can give.

How can I disable PKCS #11 support for openssl AES-128-CBC?

A library was static compiled on another server with openssl 0.9.8g. The library calls EVP_aes_128_cbc(). On the server below with PKCS #11 support for AES-128-CBC, the funtion returns an encrypted string that is not standard - can't be decrypted on other servers with same passphrase and key. The same operation on the server with PKCS #11 engine active but not supporting AES-128-CBC produces standard encrypted strings.

I suspect an issue with redirection to the PKCS# 11 engine, perhaps the passphrase or the key is being mangled. At least its symmetric so that particular server can decrypt its own strings.

openssl enc AES-128-CBC with or without the engine specified is producing standard encrypted strings.

What I'd like to do is disable PKCS #11 support for AES-128-CBC from openssl - but I can't figure out how to do it.

$ /usr/sfw/bin/openssl version
OpenSSL 0.9.7d 17 Mar 2004 (+ security patches to 2006-09-29)
$ /usr/sfw/bin/openssl engine -vvv -t -c
(pkcs11) PKCS #11 engine support
[RSA, DSA, DH, RAND, DES-CBC, DES-EDE3-CBC, AES-128-CBC, RC4, MD5, SHA1]
[ available ]
SO_PATH: Specifies the path to the 'pkcs#11' shared library
(input flags): STRING

-> /usr/sfw/bin/openssl version
OpenSSL 0.9.7d 17 Mar 2004 (+ security fixes for: CVE-2005-2969 CVE-2006-2937 CVE-2006-2940 CVE-2006-3738 CVE-2006-4339 CVE-2006-4343 CVE-2007-5135 CVE-2008-5077 CVE-2009-0590)
/usr/sfw/bin/openssl engine -vvv -t -c
(pkcs11) PKCS #11 engine support
[RSA, DSA, DH, RAND]
[ available ]
SO_PATH: Specifies the path to the 'pkcs#11' shared library
(input flags): STRING

Thanks!
-Lonny

Posted by Lonny Niederstadt on April 23, 2010 at 04:08 PM CEST #

Hello,

I'm trying to use following - all 64 bit versions

openCryptoki-2.2.2-rc7
openssl-0.9.8l
pkcs11_engine-0.9.8l.2009-11-19

Basically I applied the patch to the openssl as mentioned in the README.

Everything compiles without error and runs too, except that I don't see the output mentioned about pkcs11.

Output I see:

./openssl engine
(dynamic) Dynamic engine loading support
(4758cca) IBM 4758 CCA hardware engine support
(aep) Aep hardware engine support
(atalla) Atalla hardware engine support
(cswift) CryptoSwift hardware engine support
(chil) CHIL hardware engine support
(nuron) Nuron hardware engine support
(sureware) SureWare hardware engine support
(ubsec) UBSEC hardware engine support

When I do

strace ./openssl engine

======
execve("./openssl", ["./openssl", "engine"], [/\* 37 vars \*/]) = 0
brk(0) = 0xa805000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaaaaaab000
uname({sys="Linux", node="localhost.localdomain", ...}) = 0
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=65686, ...}) = 0
mmap(NULL, 65686, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2aaaaaaac000
close(3) = 0
open("/lib64/libdl.so.2", O_RDONLY) = 3
read(3, "\\177ELF\\2\\1\\1\\0\\0\\0\\0\\0\\0\\0\\0\\0\\3\\0>\\0\\1\\0\\0\\0\\20\\16`\\326:\\0\\0\\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=23360, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaaaaabd000
mmap(0x3ad6600000, 2109696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3ad6600000
mprotect(0x3ad6602000, 2097152, PROT_NONE) = 0
mmap(0x3ad6802000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x3ad6802000
close(3) = 1
open("/lib64/libpthread.so.0", O_RDONLY) = 3
read(3, "\\177ELF\\2\\1\\1\\0\\0\\0\\0\\0\\0\\0\\0\\0\\3\\0>\\0\\1\\0\\0\\0\\200W\\240\\326:\\0\\0\\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=145592, ...}) = 0
mmap(0x3ad6a00000, 2204528, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3ad6a00000
mprotect(0x3ad6a16000, 2093056, PROT_NONE) = 0
mmap(0x3ad6c15000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15000) = 0x3ad6c15000
mmap(0x3ad6c17000, 13168, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x3ad6c17000
close(3) = 0
open("/lib64/libc.so.6", O_RDONLY) = 3
read(3, "\\177ELF\\2\\1\\1\\0\\0\\0\\0\\0\\0\\0\\0\\0\\3\\0>\\0\\1\\0\\0\\0p\\332\\341\\325:\\0\\0\\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1713088, ...}) = 0
mmap(0x3ad5e00000, 3494168, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3ad5e00000
mprotect(0x3ad5f4c000, 2097152, PROT_NONE) = 0
mmap(0x3ad614c000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x14c000) = 0x3ad614c000
mmap(0x3ad6151000, 16664, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x3ad6151000
close(3) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaaaaabe000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaaaaabf000
arch_prctl(ARCH_SET_FS, 0x2aaaaaabeec0) = 0
mprotect(0x3ad614c000, 16384, PROT_READ) = 0
mprotect(0x3ad6c15000, 4096, PROT_READ) = 0
mprotect(0x3ad6802000, 4096, PROT_READ) = 0
mprotect(0x3ad5c1b000, 4096, PROT_READ) = 0
munmap(0x2aaaaaaac000, 65686) = 0
set_tid_address(0x2aaaaaabef50) = 30921
set_robust_list(0x2aaaaaabef60, 0x18) = 0
futex(0x7fff173468cc, FUTEX_WAKE_PRIVATE, 1) = -1 ENOSYS (Function not implemented)
rt_sigaction(SIGRTMIN, {0x3ad6a05370, [], SA_RESTORER|SA_SIGINFO, 0x3ad6a0e4c0}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x3ad6a052b0, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x3ad6a0e4c0}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=10240\*1024, rlim_max=RLIM_INFINITY}) = 0
brk(0) = 0xa805000
brk(0xa826000) = 0xa826000
rt_sigaction(SIGPIPE, {SIG_IGN}, {SIG_DFL}, 8) = 0
futex(0x3ad68030ec, FUTEX_WAKE, 2147483647) = 0
open("/home/smishra/fips/install/lib/pkcs11/PKCS11_API.so64", O_RDONLY) = 3
read(3, "\\177ELF\\2\\1\\1\\0\\0\\0\\0\\0\\0\\0\\0\\0\\3\\0>\\0\\1\\0\\0\\0`<\\0\\0\\0\\0\\0\\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=72989, ...}) = 0
mmap(NULL, 2158344, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x2aaaaaac0000
mprotect(0x2aaaaaace000, 2097152, PROT_NONE) = 0
mmap(0x2aaaaacce000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xe000) = 0x2aaaaacce000
close(3) = 0
socket(PF_FILE, SOCK_DGRAM, 0) = 3
fcntl(3, F_SETFD, FD_CLOEXEC) = 0
connect(3, {sa_family=AF_FILE, path="/dev/log"}, 110) = -1 ENOENT (No such file or directory)
close(3) = 0
stat("/home/smishra/fips/install/sbin/pkcsslotd", {st_mode=S_IFREG|0755, st_size=88247, ...}) = 0
socket(PF_FILE, SOCK_STREAM, 0) = 3
fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK) = 0
connect(3, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
close(3) = 0
socket(PF_FILE, SOCK_STREAM, 0) = 3
fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK) = 0
connect(3, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
close(3) = 0
open("/etc/nsswitch.conf", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=1696, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaaaaccf000
read(3, "#\\n# /etc/nsswitch.conf\\n#\\n# An ex"..., 4096) = 1696
read(3, "", 4096) = 0
close(3) = 0
munmap(0x2aaaaaccf000, 4096) = 0
open("/etc/ld.so.cache", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=65686, ...}) = 0
mmap(NULL, 65686, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2aaaaaccf000
close(3) = 0
open("/lib64/libnss_files.so.2", O_RDONLY) = 3
read(3, "\\177ELF\\2\\1\\1\\0\\0\\0\\0\\0\\0\\0\\0\\0\\3\\0>\\0\\1\\0\\0\\0\\340\\37\\0\\0\\0\\0\\0\\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=53880, ...}) = 0
mmap(NULL, 2139432, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x2aaaaace0000
mprotect(0x2aaaaacea000, 2093056, PROT_NONE) = 0
mmap(0x2aaaaaee9000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x9000) = 0x2aaaaaee9000
close(3) = 0
mprotect(0x2aaaaaee9000, 4096, PROT_READ) = 0
munmap(0x2aaaaaccf000, 65686) = 0
open("/etc/group", O_RDONLY) = 3
fcntl(3, F_GETFD) = 0
fcntl(3, F_SETFD, FD_CLOEXEC) = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=773, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaaaaccf000
read(3, "root:x:0:root\\nbin:x:1:root,bin,d"..., 4096) = 773
close(3) = 0
munmap(0x2aaaaaccf000, 4096) = 0
getuid() = 0
socket(PF_FILE, SOCK_STREAM, 0) = 3
fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK) = 0
connect(3, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
close(3) = 0
socket(PF_FILE, SOCK_STREAM, 0) = 3
fcntl(3, F_GETFL) = 0x2 (flags O_RDWR)
fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK) = 0
connect(3, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
close(3) = 0
open("/etc/passwd", O_RDONLY) = 3
fcntl(3, F_GETFD) = 0
fcntl(3, F_SETFD, FD_CLOEXEC) = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=2004, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaaaaccf000
read(3, "root:x:0:0:root:/root:/bin/bash\\n"..., 4096) = 2004
close(3) = 0
munmap(0x2aaaaaccf000, 4096) = 0
geteuid() = 0
open("/etc/passwd", O_RDONLY) = 3
fcntl(3, F_GETFD) = 0
fcntl(3, F_SETFD, FD_CLOEXEC) = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=2004, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaaaaccf000
read(3, "root:x:0:0:root:/root:/bin/bash\\n"..., 4096) = 2004
close(3) = 0
munmap(0x2aaaaaccf000, 4096) = 0
open("/etc/localtime", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=3519, ...}) = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=3519, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaaaaccf000
read(3, "TZif2\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\4\\0\\0\\0\\4\\0\\0\\0\\0"..., 4096) = 3519
lseek(3, -2252, SEEK_CUR) = 1267
read(3, "TZif2\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\5\\0\\0\\0\\5\\0\\0\\0\\0"..., 4096) = 2252
close(3) = 0
munmap(0x2aaaaaccf000, 4096) = 0
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=3519, ...}) = 0
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=3519, ...}) = 0
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=3519, ...}) = 0
socket(PF_FILE, SOCK_DGRAM, 0) = 3
fcntl(3, F_SETFD, FD_CLOEXEC) = 0
connect(3, {sa_family=AF_FILE, path="/dev/log"}, 110) = -1 ENOENT (No such file or directory)
close(3) = 0
munmap(0x2aaaaaac0000, 2158344) = 0
open("/home/smishra/fips/fips_openssl/install/ssl/openssl.cnf", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=9374, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaaaaac0000
read(3, "#\\n# OpenSSL example configuratio"..., 4096) = 4096
read(3, "_name ]\\ncountryName\\t\\t\\t= Country "..., 4096) = 4096
read(3, " an SSL server.\\n# nsCertType\\t\\t\\t="..., 4096) = 1182
read(3, "", 4096) = 0
close(3) = 0
munmap(0x2aaaaaac0000, 4096) = 0
open("/proc/meminfo", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaaaaac0000
read(3, "MemTotal: 4042388 kB\\nMemFre"..., 4096) = 777
close(3) = 0
munmap(0x2aaaaaac0000, 4096) = 0
rt_sigaction(SIGPIPE, {SIG_IGN}, {SIG_IGN}, 8) = 0
brk(0xa847000) = 0xa847000
fstat(1, {st_mode=S_IFREG|0644, st_size=10102, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaaaaac0000
brk(0xa83f000) = 0xa83f000
write(1, "(dynamic) Dynamic engine loading"..., 363(dynamic) Dynamic engine loading support
(4758cca) IBM 4758 CCA hardware engine support
(aep) Aep hardware engine support
(atalla) Atalla hardware engine support
(cswift) CryptoSwift hardware engine support
(chil) CHIL hardware engine support
(nuron) Nuron hardware engine support
(sureware) SureWare hardware engine support
(ubsec) UBSEC hardware engine support
) = 363
exit_group(0) = ?

======

This is my pkcs11 library

open("/home/smishra/fips/install/lib/pkcs11/PKCS11_API.so64", O_RDONLY) = 3

What am I missing here?

I'll be very thankful for any help.

Thanks
Shravan

Posted by shravan mishra on May 14, 2010 at 05:31 PM CEST #

Hello,

I' really not making any progress here. I have done everything as mentioned in the opencryptoki do and openssl doce for making installing but nothing seems to work. I'm sure I'm making mistake(s) along the way but some help will really be appreciated.

Now I have installed everything under /usr/local but still to no avail.

Following is the information, I can provide more if needed, please let me know.

[root@localhost bin]# pwd
/usr/local/bin

[root@localhost bin]# ./openssl engine
(dynamic) Dynamic engine loading support
(4758cca) IBM 4758 CCA hardware engine support
(aep) Aep hardware engine support
(atalla) Atalla hardware engine support
(cswift) CryptoSwift hardware engine support
(chil) CHIL hardware engine support
(nuron) Nuron hardware engine support
(sureware) SureWare hardware engine support
(ubsec) UBSEC hardware engine support

[root@localhost bin]# ldd openssl
libdl.so.2 => /lib64/libdl.so.2 (0x0000003ad6600000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003ad6a00000)
libc.so.6 => /lib64/libc.so.6 (0x0000003ad5e00000)
/lib64/ld-linux-x86-64.so.2 (0x0000003ad5a00000)

dir -l /usr/lib64/pkcs11/
total 140
-rwxr-xr-x 1 root root 137056 Sep 17 2007 libcoolkeypk11.so
lrwxrwxrwx 1 root root 37 May 17 16:39 PKCS11_API.so -> /usr/local/lib/pkcs11/PKCS11_API.so64

ldd /usr/lib64/pkcs11/PKCS11_API.so
libc.so.6 => /lib64/libc.so.6 (0x00002aaaaaccd000)
libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaab023000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaaab227000)
/lib64/ld-linux-x86-64.so.2 (0x0000555555554000)

Please let me know what am I missing.

Thanks much.
Shravan

Posted by shravan mishra on May 17, 2010 at 05:38 PM CEST #

hello,

i'm tryning to compile patched openssl (0.9.8l) with opencryptoki (2.3.1) and it seems to be fine until I want to try it:

$ ./apps/openssl engine pkcs11
6426:error:2506406A:DSO support routines:DLFCN_BIND_FUNC:could not bind to the requested symbol name:dso_dlfcn.c:261:symname(bind_engine): /home/szopen/testing/lib/engines/libpkcs11.so: undefined symbol: bind_engine
6426:error:2506C06A:DSO support routines:DSO_bind_func:could not bind to the requested symbol name:dso_lib.c:294:
6426:error:260B6068:engine routines:DYNAMIC_LOAD:DSO failure:eng_dyn.c:463:
6426:error:2606A074:engine routines:ENGINE_by_id:no such engine:eng_list.c:419:id=pkcs11

the library is found, but where's that engine??

Posted by zbigniew jasinski on June 24, 2010 at 06:37 AM CEST #

Hi,

I had the same problem when trying to adapt the 0.9.8 patch to 1.0.0. PKCS11 engine did not show when running "openssl engine".

I my case I had placed the loading of the PKCS11 engine inside an #ifndef that caused it not to be compiled in. This was in the file crypto/engine/eng_all.c

After I moved the loading of the PKCS11 engine outside this #ifndef, it now seems to work. Here is the patch for eng_all.c that worked for me:

--- openssl-1.0.0a.orig/crypto/engine/eng_all.c 2009-07-01 16:55:58.000000000 +0200
+++ openssl-1.0.0a/crypto/engine/eng_all.c 2010-08-04 14:15:53.731090000 +0200
@@ -112,6 +112,9 @@
ENGINE_load_capi();
#endif
#endif
+#ifndef OPENSSL_NO_HW_PKCS11
+ ENGINE_load_pk11();
+#endif
}

#if defined(__OpenBSD__) || defined(__FreeBSD__) || defined(HAVE_CRYPTODEV)

Regards,
Svein Olav Bjerkeset

Posted by Svein Olav Bjerkeset on August 04, 2010 at 07:52 AM CEST #

can openssl'ssl lib use engine to choose symmetric algorithm? and how?

Posted by wangwei on October 28, 2010 at 03:20 AM CEST #

Did you plan to update the pkcs11 engine package to work with the last openssl release (1.0.0)?

Thank you

Posted by Geoffrey G. on November 26, 2010 at 07:29 AM CET #

Post a Comment:
  • HTML Syntax: NOT allowed
About

Jan Pechanec

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today