Put IPsec to work in YOUR application

Hello coders!

Most people know that you can use ipsecconf(1m) to apply IPsec policy enforcement to an existing application. For example, if you wish to only allow inbound telnet traffic that's under IPsec protection, you'd put something like this into /etc/inet/ipsecinit.conf or other ipsecconf(1m) input:

# Inbound telnet traffic should be IPsec protected
{ lport 23 } ipsec { encr_algs any(128..) encr_auth_algs md5 sa shared}
             or ipsec { encr_algs any(128..) encr_auth_algs sha1 sa shared}

Combine that with appropriate IKE configuration or manual IPsec keys, and you can secure your telnet traffic against eavesdropping, connection hijacking, etc.

For existing services, using ipsecconf(1m) is the most expedient way to bring IPsec protection to bear on packets.

For new services, or services that are being modified anyway, consider using per-socket policy as an alternative. Some advantages to per-socket policy are:

  • Per-socket policy is stored internally in network session state (the conn_t structure in OpenSolaris). Entries from ipsecconf(1m) are stored in the global Security Policy Database (SPD). No global SPD entries means lower latency for fresh flow creation, and less lock acquisition.
  • Per-socket bypass means fewer bypass entries in global SPD. If I bypass remote-port 80 using ipsecconf(1m), I can, in theory, enter the system with a remote TCP packet with port=80. There's an RFE (6219908) to work around this, but per-socket is still quicker. I'd love a web proxy with the ability to set per-socket bypass.

The newly SMF-ized inetd(1m) would be a prime candidate for per-socket policy. See RFE 6226853, and this might be something someone in the OpenSolaris community would like to tackle!

Let's look at the ipsec_req_t structure that's been around since Solaris 8 in /usr/include/netinet/in.h:

 \* Different preferences that can be requested from IPSEC protocols.
#define IP_SEC_OPT              0x22    /\* Used to set IPSEC options \*/
#define IPSEC_PREF_NEVER        0x01
#define IPSEC_PREF_REQUIRED     0x02
#define IPSEC_PREF_UNIQUE       0x04
 \* This can be used with the setsockopt() call to set per socket security
 \* options. When the application uses per-socket API, we will reflect
 \* the request on both outbound and inbound packets.

typedef struct ipsec_req {
        uint_t          ipsr_ah_req;            /\* AH request \*/
        uint_t          ipsr_esp_req;           /\* ESP request \*/
        uint_t          ipsr_self_encap_req;    /\* Self-Encap request \*/
        uint8_t         ipsr_auth_alg;          /\* Auth algs for AH \*/
        uint8_t         ipsr_esp_alg;           /\* Encr algs for ESP \*/
        uint8_t         ipsr_esp_auth_alg;      /\* Auth algs for ESP \*/
} ipsec_req_t;
The ipsec_req_t is a subset of what one can specify with ipsecconf(1m) in Solaris 9 or later, but it matched what one could do with Solaris 8's version. Algorithm values are derived from PF_KEY (see /usr/include/net/pfkeyv2.h for values), as below. One could also use getipsecalgbyname(3nsl). If I wish to set a socket to use ESP with AES and and MD5, I'd set it up as follows:
        int s;  /\* Socket file descriptor... \*/
        ipsec_req_t ipsr;


        /\* NOTE: Do this BEFORE calling connect() or accept() for TCP sockets. \*/
        ipsr.ipsr_ah_req = 0;
        ipsr.ipsr_esp_req = IPSEC_PREF_REQUIRED;
        ipsr.ipsr_self_encap_req = 0;
        ipsr.ipsr_auth_alg = 0;
        ipsr.ipsr_esp_alg = SADB_EALG_AES;
        ipsr.ipsr_esp_auth_alg = SADB_AALG_MD5HMAC;
        if (setsockopt(s, IPPROTO_IP, IP_SEC_OPT, &ipsr, sizeof (ipsr)) == -1) {
                bail();  /\* Ugggh, we failed. \*/
        /\* You now have per-socket policy set. \*/
Notice I mentioned setting the socket option BEFORE calling connect() or accept? This is because of a phenomenon we implement called connection latching. Basically, connection latching means that once an endpoint is connect()-ed, the IPsec policy (whether set per-socket or inherited from the state of the global SPD at the time) latches in place. We made this decision to avoid keeping policy-per-datagram state for things like TCP retransmits.

One thing per-socket policy does not address is the case of unconnected datagram services. In a perfect world, we could have IPsec policy information percolate all the way to the socket layer, where an application can make fully-informed per-datagram decisions on whether or not a particular packet was secured or not. It's a hard problem, requiring XNET sockets (to use sendmsg() and recvmsg() with ancillary data).

BTW, if you want to bypass whatever global entries are in the SPD, you can zero out the structure, and set all three (ah, esp, self_encap) action indicators to IPSEC_PREF_NEVER. You need to be privileged (root or "sys_net_config") to use per-socket bypass, however.

So modulo the keying problem (setting up IKE or having both ends agree on IPsec manual keys), you can put IPsec to work right in your application. In fact, if you use IKE, you can let IKE sort out permissions and access control (by using PKI-issued certificates, self-signed certificates, or preshared keys) and have policy merely determine the details of the protection required.

EDITED: This entry brought to you by the Technorati tags , , and .

Dan, I'm wondering if per-socket policies might be more secure in some situations vs. system-wide policies. For example, say I have a client process that creates a UDP socket using an ephemeral port number and sends messages to a server on a well known UDP port. I could write a system-wide policy that encrypts that UDP traffic, but then if an evil process comes up, it too can create a UDP client socket and send messages to the server, which the server will accept since they are properly encrypted. But if I used a per-socket policy, couldn't I ensure that only my client can get messages through to the server? (Or could the evil process also create it's own per-socket policy to defeat this?) thanks, Dave

Posted by Dave Spears on May 22, 2006 at 10:03 AM EDT #

If you have a malicious user on the same system, he/she could also use per-socket policy. We could, however, improve that situation.

Today, the IP_SEC_OPT socket option is not privilege-checked except for the BYPASS case. One could imagine a finer-grained privilege set that allows only certain users to set IP_SEC_OPT on a socket.

That's a nifty idea for a multi-user system with potentially adversarial users. Hold on a second while I file an RFE...

... there we go. RFE 6429874 now states this problem. (The link may not be active yet, but it will be soon.)

Thanks for the suggestion!

Posted by Dan McDonald on May 24, 2006 at 03:50 AM EDT #

hii i have one question. i hope you will help me i am using IPsec Tool for two normal computer and assign same policy and rules to both. than i use ehternal Packet capturing tool. i use ping connection between to computer and capture packects between two computers. everything works fine. but in ESP with AH in IPsec, as per format of frame at the end of packect we have ESP Payload and Authenticated Data.(Hash code)but i dont found Authenticated data so i just want to know where is that field is located?? i hope u get my point. Thank you AAnal

Posted by aanal on October 28, 2006 at 06:49 AM EDT #

AAnal --> Please either engage OpenSolaris discussions for this sort of question, or if you're a paying customer, exercise your Sun support options.

Posted by Dan McDonald on October 30, 2006 at 07:35 AM EST #

Post a Comment:
  • HTML Syntax: NOT allowed



« July 2016