IP Security (IPSec)

Dr. Natarajan Meghanathan Assistant Professor of Computer Science Jackson State University E-mail: [email protected]

The IP Security Protocol suite (IPSec) is implemented at the IP layer, so it does not require any change to existing TCP, UDP and application layer protocols. IPSec is designed to address the fundamental shortcomings of the IP layer such as being subjected to spoofing, eavesdropping and session hijacking. The basis of IPSec is called a security association, which is basically the set of security parameters for a secured communication channel. Each host can have several security associations in effect for current communications with different remote hosts. A security association is identified using a security parameter index (SPI) a 32-bit identifier and the IP address of the partner host on the other side of the association. The SPI and the partner IP address are used to index to the security association database (SADB) that has information about the other characteristics of the different security associations Two protocols have been developed to provide packet-level security for both IPv4 and IPv6:
IP Authentication Header, AH (Next Header protocol ID: 51) provides integrity, authentication and non-repudiation. IP Encapsulating Security Payload, ESP (Next Header protocol ID: 50) provides confidentiality, along with authentication and integrity protection.

IPSec

IPSec
By implementing security at the IP level, an organization can ensure secure networking not only for applications that have security mechanisms (like PGP, Kerberos, TLS); but, also for the many security-ignorant applications. When implemented in a firewall or router, it provides strong security that can be applied to all traffic crossing the perimeter. Traffic within a company or workgroup does not incur the overhead of security-related processing. can be transparent to end users. There is no need to train users on security mechanisms, issue keying material on a per-user basis, or revoke keying material when users leave the organization. Note: IPSec is optional for IPv4; but, mandatory for IPv6.

Typical IPSec Scenario

IPSec implemented at end user (Transport Mode)

IPSec implemented at firewall/ router (Tunnel Mode)

Source: Figure 19.1 from William Stallings Cryptography and Network Security, 5th Edition

IPSec
A security association is characterized by the following parameters:
Encryption algorithm (DES, AES and etc.) Encryption key Encryption parameters such as the Initialization Vector Integrity/ Authentication algorithms (keyed-HMAC algorithms) and the key Lifespan of the association

We cannot simply use a cryptographic hashing algorithm like SHA or MD5 that gives a message authentication code dependent only on the message to be sent. Note that to achieve both integrity and authentication simultaneously with the Authentication header, we want to compute a message authentication code that is dependent on both the message and the authenticity of the sender of the message. If the two communicating parties can agree on a shared secret session key, then the secret key could be used in combination with a cryptographic hash function to obtain what is called a keyed-Hash Message Authentication Code (HMAC). Any iterative cryptographic hash function like SHA-1 or MD5 may be used in the calculation of an HMAC; the resulting MAC algorithm would be accordingly termed as HMAC-SHA-1 or HMAC-MD5. The cryptographic strength of the HMAC depends upon the cryptographic strength of the underlying hash function, the size and quality of the key and the size of the hash output.

IPSec
A security association (SA) is uni-directional. For two hosts to communicate in either directions, SAs have to be established separately in either directions. For host A to securely send data packets to host B, and make host B to believe that the data packet did come from host A, it should establish a SA with host B. Such a SA is said to be outbound at A and inbound at B. An IPSec header of a datagram sent from host A to host B, should have the secure features of the SA that is inbound at B and similarly the IPSec header of a datagram sent from host B to host A should have the secure features of the SA that is inbound at A. Prior to establishing a IPSec SA, the two end hosts need to exchange their public-key certificates digitally certified by a trusted third-party certificate authority (CA). This is done through the Internet Key Exchange (IKE) protocol. Once two hosts have exchanged each others public-key certificates, then they are said to have established an IKE Security Association (IKE SA).
Establishing an IKE SA is a pre-requisite to establish an IPSec SA.

IPSec
Establishing a Security Association
Host A wishing to send data packets to host B needs to establish an inbound security association with host B Host A picks a SPI that has not been yet chosen for communications with host B and sends a SA Establishment Request to B. This message is encrypted with the public key of Host B and contains the following:
SPI for the inbound SA channel at host A (i.e., the outbound SA channel at host B) Lifespan of the association.
This could be negotiated by host B.

The packet-level security protocol chosen (protocol ID: 51 for AH or 50 for ESP)
This could be also negotiated by host B.

If AH is chosen, then the list of keyed-HMAC algorithms that could be used is specified. Host B will choose one from this list if it wishes to receive packets from host A. If ESP is chosen, then the list of keyed-HMAC algorithms along with the list of encryption algorithms and key-derivation functions that could be used will be sent.

Hosts A and B will basically exchange a series of messages negotiating the lifespan of the association, the packet-level security protocol to be used and the appropriate keyed-HMAC and encryption algorithms, key-derivation function to be used.

IPSec
Establishing Security Associations
All negotiation messages will be encrypted at the sender side using the public key of the receiver and will be decrypted with the private key of the receiver at the receiver side. Hosts A and B agree on a shared session key using the Diffie-Hellman exchange algorithm. The shared session key would be used for the keyed-HMAC algorithm. Each host would use the shared session key and the key-derivation function agreed upon to derive the secret key that would be used for encryption of the data at host A and decryption of the data at host B.

Summary of Steps to establish a security association from host A to host B

IKESA: Exchange of public-key certificates between hosts A and B Sending the SA Establishment Request from host A to B. Negotiation of the parameters, algorithms and key-derivation functions for SAA->B Diffie-Hellman exchange algorithm to agree on a shared session key that will also be used for keyed-HMAC algorithm Generation of the shared secret key for encryption/decryption using the session key and the agreed upon key-derivation function

IPSec Authentication Header

Next Header identifies the transport layer protocol Payload length (AH length) The length of the whole Authentication Header (AH) in 32-bit words Reserved This field is reserved for future use and must be zero. SPI 32-bit field to identify the SA Sequence Number This is a monotonically increasing identifier (incremented, starting from 0, for every datagram sent on the SA) that is used to assist in antireplay protection. If the sequence number value reaches 232-1, the SA has to be terminated and a new SA has to be formed. This prevents replay attacks. Authentication Data: This is the integrity/ authentication check value (keyedHMAC) calculated over the entire packet including the header fields that do not change at the intermediate hosts. The size of the keyed-HMAC may vary with each SA and may not be exactly multiple of 32-bits. If this is the case, the HMAC will be padded.

IP4 Datagram with Authentication Header

(variable length, padded if required)

Source: http://unixwiz.net/techtips/iguide-ipsec.html

IPSec ESP Header

SPI 32-bit field to identify the SA Sequence Number This is a monotonically increasing identifier (incremented for every datagram sent on the SA) that is used to assist in anti-replay protection. Payload data the data to be transferred. Padding Used with some block ciphers to pad the data to the full length of a block. Pad length size of padding in bytes. Next header identifies the transport layer protocol Authentication Data: This is the integrity/ authentication check value (keyed-HMAC) calculated over only the SPI, Sequence Number in the ESP header, the actual data, padding data, pad length and the next header field.

IP4 Datagram with ESP Header

Pkt len + ESP Hdr len

+ ESP Hdr len

TCP header +

Source: http://unixwiz.net/techtips/iguide-ipsec.html

IPSec Transport Mode

Both the AH and ESP options support two modes of use: the transport and tunnel modes. Figure below shows the protocol architecture for the transport mode when ESP is used. Transport mode operation provides confidentiality for any application that uses it, thus avoiding the need to implement confidentiality in every individual application.

ESP
encrypted

Source: Figure 19.9 (a) from William Stallings Cryptography and Network Security, 5th Edition

IPSec Tunnel Mode

Tunnel mode is used to encrypt an entire IP packet (also referred to as context protection). Note that the Transport mode encrypts only the payload portion of the IP datagram (referred to as content protection). In the Tunnel mode, after the AH or ESP fields are added to the IP packet, the entire packet plus security fields is treated as the payload of new "outer" IP packet with a new outer IP header. The entire original, or inner, packet travels through a "tunnel" from one point of an IP network to another; no routers along the way are able to examine the inner IP header. Tunnel mode is useful in a configuration that includes a firewall or other sort of security gateway that protects a trusted network from external networks. In this latter case, encryption occurs only between an external host and the security gateway or between two security gateways. With tunnel mode, a number of hosts on networks behind firewalls may engage in secure communications without implementing IPsec.

IPSec Tunnel Mode

Figure below shows the protocol architecture for the tunnel mode when ESP is used.

ESP

encrypted

Source: Figure 19.9 (b) from William Stallings Cryptography and Network Security, 5th Edition

IPSec AH Transport Mode IPSec AH Tunnel Mode

Source: Figure 11.12, 11.13 from Conklin and White Principles of Computer Security, 2nd Edition

IPSec Transport Mode

A VPN using IPSec Tunnel Mode

Source: Figure 19.7 from William Stallings Cryptography and Network Security, 5th Edition

References
William Stallings Cryptography and Network Security, Prentice Hall, 5th Edition, January 2010 Wikipedia http://unixwiz.net/techtips/iguide-ipsec.html