Bugs the bite: C_WrapKey returnning CKR_MECHANISM_INVALID
By izick on Mar 02, 2009
When looking through a truss of the output, metaslot calls the ncp crypto accelerator to do the wrapping operation; however, as displayed by 'cryptoadm list -mv', ncp does not support wrapping. This situation was fixed in OpenSolaris under bug 6543159. OpenSolaris now checks if the provider supports the operation before calling it. If it does not support the operation, the provider is skipped. But this is Solaris 10, which does not have the 6543159 fix. So when the C_WrapKey operations fails on ncp, it return CKR_MECHANISM_INVALID. So we know why the error exists, but why is metaslot returning that error code?
Metaslot was built so that one provider fails, it will try another provider if available. So continuing down the truss output, metaslot does failover to pkcs11_softtoken. But the C_WrapKey call fails with an error of CKR_WRAPPING_KEY_INCONSISTENT. Now we are on to something.. Both providers failed
So why did we not see pkcs11_softtoken's correct error code, but saw ncp's incorrect error code? Well it was a design decision for metaslot. We can only report one error code.. Metaslot makes the assumption by using the first error code, CKR_MECHANISM_INVALID. In 99% of the cases, where both providers support an operation it's safe to bet that they will both have the same error code. This is just that 1%.
So after fixing the CKR_WRAPPING_KEY_INCONSISTENT issue in the application, the operation worked. Under the hood, it still fails on the ncp call, but succeeds on the pkcs11_softtoken call and that success is returned.
If such an error happens to you, the best way to find the true error code is to run the below:
$ truss -f -u pkcs11_softtoken\*::
2>&1 | grep C_WrapKey