Ensuring FIPS compliance is crucial for securing sensitive data and meeting compliance objectives. For those of you unfamiliar with the term, FIPS (Federal Information Processing Standards) are a set of standards and guidelines issued by the U.S. National Institute of Standards and Technology (NIST) for use in federal computer systems. This blog provides a step-by-step guide to enabling FIPS mode on Oracle Linux 9 with MySQL, covering essential configurations for OpenSSL, SSH, and MySQL itself.
Now let’s go through the 10 steps you need to take to enable FIPS.
Step 1 – Select a FIPS certified Linux and OpenSSL 3 Operating System
Review certifications
We will demonstrate using Oracle Linux 9
Step 2 – Assess your server
- What version of OpenSSL is installed?
- Is the OS in FIPS mode?
- Is OpenSSL config in FIPS mode?
What version or OpenSSL is installed?
# openssl version -v
OpenSSL 3.0.7 1 Nov 2022 (Library: OpenSSL 3.0.7 1 Nov 2022)
Well, that’s a bit old. Let’s update
# sudo dnf update openssl
# openssl version -v
OpenSSL 3.2.2 4 Jun 2024 (Library: OpenSSL 3.2.2 4 Jun 2024)
That’s better!
Is the OS in FIPS mode?
# sudo fips-mode-setup --check
FIPS mode is disabled.
Is OpenSSL config in FIPS mode
# openssl list --providers
Providers:
default
name: OpenSSL Default Provider
version: 3.2.2
status: active
No a FIPS provider is not listed – OpenSSL is NOT in FIPS mode.
Step 3 – Install MySQL
- Enterprise installation details
- Community installation details
Check that MySQL is linked to OpenSSL 3
# ldd /usr/sbin/mysqld | grep ssl
libssl.so.3 => /lib64/libssl.so.3 (0x00007fa868470000)
MySQL is linked to the local OpenSSL. “libssl.so.3” indicates we are using OpenSSL 3.
If not, then – you may have
- Installed the wrong image – for example an OL8 image on an OL9 OS.
- Installed an old “Generic Linux” tar
- Installed a newer “Generic Linux” tar that is linked to the tar included MySQL OpenSSL 3 library.
- An odd LD_LIBRARY_PATH setting.
- An older OS that does not include OpenSSL 3
Start MySQL Server
# sudo systemctl start mysqld
Step 4 – Make sure my SSH access to this server will work once FIPS is enabled
On the Client
- In my test my original PEM failed – it was NOT FIPS compliant
# ssh -i rsa512 localhost
sign_and_send_pubkey: signing failed: error:04066078:rsa
routines:RSA_EAY_PRIVATE_ENCRYPT:key size too small
- You will need to create a passworded private PEM EC private key
- Use ecdsa-sha2-nistp256 or other FIPS compliant crypto when creating the key
On server
- Remove non-FIPS crypto from opensshd
- # sudo vi /etc/crypto-policies/back-ends/opensshserver.config
- # sudo systemctl restart sshd
On my test server here’s what I had after removing non-FIPS from – Ciphers, MACs, and KexAlgorithms
Ciphers aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes128-gcm@openssh.com,aes128-ctr
MACs hmac-sha2-256-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha1,umac-128@openssh.com,hmac-sha2-512
GSSAPIKexAlgorithms gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-
KexAlgorithms curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512
HostKeyAlgorithms ecdsa-sha2-nistp256,ecdsa-sha2-nistp256-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519,ssh-ed25519-cert-v01@openssh.com,sk-ssh-ed25519@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,rsa-sha2-256,rsa-sha2-256-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-512-cert-v01@openssh.com
PubkeyAcceptedAlgorithms ecdsa-sha2-nistp256,ecdsa-sha2-nistp256-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519,ssh-ed25519-cert-v01@openssh.com,sk-ssh-ed25519@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,rsa-sha2-256,rsa-sha2-256-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-512-cert-v01@openssh.com
CASignatureAlgorithms ecdsa-sha2-nistp256,sk-ecdsa-sha2-nistp256@openssh.com,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,sk-ssh-ed25519@openssh.com,rsa-sha2-256,rsa-sha2-512
Various Linux DISA STIGs define requirements for OpenSSH configuration in FIPS mode. Please refer to those documents if DISA STIG compliance is required.
Step 5 – Enable OS FIPS
# sudo fips-mode-setup –enable
# sudo fips-mode-setup –check
Installation of FIPS modules is not completed.
FIPS mode is disabled.
Before you reboot
If you have not –
- On a FIPS enabled compute Create a Private Key with a password
- Create the public key
- Update your public key for SSH.
- # cd .ssh/
- # vi authorized_keys
Step 6 – Reboot Linux
# sudo reboot
SSH back to the server and check the status of FIPS
# sudo fips-mode-setup --check
FIPS mode is enabled.
OR –
# openssl list –providers
…
fips
name: Oracle Linux 9 OpenSSL FIPS Provider
version: 3.0.7-b27cdeb3ba51be46
status: active
Now your kernel is in FIPS mode.
Step 7 – Run some tests on MySQL
# mysql -u root -p
mysql> show variables like '%fips%';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| ssl_fips_mode | OFF |
+---------------+-------+
1 row in set (0.00 sec)
So – given that MySQL OpenSSL FIPS was enabled via the OS. This is as expected.
mysql> SHOW SESSION STATUS LIKE ‘Ssl_cipher_list’;
TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_CCM_SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES256-CCM:ECDHE-ECDSA-AES128-CCM:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-CCM:DHE-RSA-AES128-CCM:DHE-RSA-CHACHA20-POLY1305
I see some non-fips for example – TLS_CHACHA20_POLY1305_SHA256
FIPS is enforced – using non FIPS crypto fails –
# mysql -u root -p --host=localhost --protocol=TCP --ssl-mode=REQUIRED --tls-version=TLSv1.3 --tls-ciphersuites=TLS_CHACHA20_POLY1305_SHA256
Enter password:
ERROR 2026 (HY000): SSL connection error: error:0A0000B5:SSL routines::no ciphers available
------------
Let’s try a FIPS compliant algorithm
# mysql -u root -p --host=localhost --protocol=TCP --ssl-mode=REQUIRED --tls-version=TLSv1.3 --tls-ciphersuites=TLS_AES_128_CCM_SHA256
For a MySQL TCP connection - non-FIPS is not working – FIPS is enforced – good!
The kernel setting limited the TLS connections
Now for another non-FIPS crypto test.
mysql> select md5(8);show warnings;
+----------------------------------+
| md5(8) |
+----------------------------------+
| c9f0f895fb98ab9159f51fd0297e236d |
+----------------------------------+
So WHY DOES THIS TEST WORK? When MD5 is NOT FIPS 140-2 / 3 compliant
- FIPS mode at OS kernel level does NOT change default provider settings in OpenSSL’s configuration file.
- In the case where MySQL uses OS resources to create the TLS connection – its enforced by the kernel calls.
- MySQL relies on that property to set FIPS flag ON/OFF. So next lets enable OpenSSL FIPS.
Step 8 – Enabling FIPS for OpenSSL
For this step I followed the directions here https://docs.openssl.org/3.0/man7/fips_module/#making-all-applications-use-the-fips-module-by-default
# openssl version -d
OPENSSLDIR: “/etc/pki/tls”
# sudo ls /etc/pki/tls/
cert.pem certs ct_log_list.cnf fips_local.cnf misc openssl.cnf openssl.d private
We vi the openssl.cnf. add the reference to the fips_local.cnf, and then edit per below.
# sudo vi /etc/pki/tls/openssl.cnf
… from the above documentation add …
config_diagnostics = 1
openssl_conf = openssl_init
.include /usr/local/ssl/fipsmodule.cnf
[openssl_init]
providers = provider_sect
alg_section = algorithm_sect
[provider_sect]
fips = fips_sect
default = default_sect
[default_sect]
activate = 1
[algorithm_sect]
default_properties = fips=yes
—
Once you have made these changes – check to make sure the FIPS provider is listed.
# openssl list –providers
You should next reboot the server to assure all linked applications are using FIPS mode for OpenSSL.
Step 9 – Run FIPS tests
# mysql -u root -p –host=localhost –protocol=TCP –ssl-mode=REQUIRED –tls-version=TLSv1.3 –tls-ciphersuites=TLS_CHACHA20_POLY1305_SHA256
Enter password:
ERROR 2026 (HY000): SSL connection error: error:0A0000B5:SSL routines::no ciphers available
————
The error was expected – demonstrates FIPS compliant
mysql -u root -p –host=localhost –protocol=TCP –ssl-mode=REQUIRED –tls-version=TLSv1.3 –tls-ciphersuites=TLS_AES_128_CCM_SHA256
mysql> select md5(8);show warnings;
+----------------------------------+
| md5(8) |
+----------------------------------+
| 00000000000000000000000000000000 |
+----------------------------------+
1 row in set, 1 warning (0.00 sec)
+---------+------+------------------------------------------------------------------------+
| Level | Code | Message |
+---------+------+------------------------------------------------------------------------+
| Warning | 4073 | SSL fips mode error: FIPS mode ON/STRICT: MD5 digest is not supported. |
+---------+------+------------------------------------------------------------------------+
1 row in set (0.00 sec)
The OS and OpenSSL – thus MySQL Server – are now in FIPS mode.
Step 10 – Test your stack
Test
- All TCP/IP TLS MySQL connections that can not support FIPS algorithms will now fail.
- Not just on this server but for all users and applications
- Typically this would only occur when using old out of date MySQL database drivers.
- Or if the connection string specifically references a non-FIPS crypto algorithm.
- All applications and services running on the Linux server
- And agents, connections to agents or servers, etc
- Use the # sudo systemctl list-units –type services command to list the currently active service units on your system.
- For example – these services might not support FIPS
- CIFS mounts
- SSSD
- Jenkins
- Samba
Conclusion
In conclusion, this guide has walked you through the critical steps of enabling FIPS mode on an Oracle Linux 9 system with MySQL, focusing on the interplay between the operating system, OpenSSL, and MySQL configurations.
We’ve demonstrated how to update OpenSSL, verify FIPS compliance at the OS level, install and configure MySQL, and crucially, address SSH access to prevent lockouts post-FIPS enablement. By meticulously adjusting OpenSSH configurations and ensuring FIPS-compliant cryptographic algorithms are used, we’ve secured remote access.
Furthermore, we’ve explored the nuances of FIPS enforcement within MySQL, highlighting the distinction between OS-level FIPS enforcement for TLS connections and the need for explicit OpenSSL configuration to restrict non-compliant algorithms like MD5 within MySQL queries. The successful implementation of FIPS mode, as validated by our tests, demonstrates that while the OS kernel enforces FIPS for system-level cryptographic operations like TLS, OpenSSL’s configuration controls application-specific cryptographic usage.
Ultimately, this process ensures that your system adheres to stringent security standards, providing a robust and compliant environment for sensitive data. Remember, meticulous planning and testing, especially concerning SSH access, are essential for a smooth transition to FIPS mode. By following these steps and understanding the underlying principles, you can confidently deploy and manage a FIPS-compliant MySQL server on Oracle Linux 9.
As always, thank you for using MySQL!
References