How tunnels work

An IPSec tunnel is typically thought of as a discrete two-way logical connection between two systems. However, each tunnel is actually made up of complementary unidirectional connections called Security Associations, or SAs.

Each SA is identified by a 32-bit number called the Security Parameter Index or SA ID.

SAs define the attributes of an IPSec tunnel, the most important of which is the security protocol being used.

The security protocol determines how data traffic which passes through the tunnel is to be encoded and decoded. There are two different protocols; either or both may be used in any given IPSec tunnel, depending on what form of security is required.

• The Authentication Header (AH) protocol provides origin authentication for the entire IP packet, but not data confidentiality.

• The Encapsulating Security Payload (ESP) protocol provides both origin authentication and data confidentiality (encryption). However, ESP's origin authentication only protects the payload of the IP packet, not the IP header.

These security protocols operate by encoding every IP packet sent over the tunnel, using a predetermined algorithm.

• Both protocols use an asymmetric hash algorithm to provide origin authentication. Some IPSec platforms have several different algorithms available, but the present implementation is limited to the Keyed MD5 algorithm.

• The ESP protocol uses a symmetric (secret key) algorithm to encrypt packets for data confidentiality. The available algorithms are CDMF (40-bit) and DES-CBC (56-bit). DES-CBC is a more secure algorithm, and should be used instead of CDMF whenever possible.

Note: The DES-CBC algorithm is only available in MPTS V5.4 (WR_8610) and above. MPTS V5.3 (which came with TCP/IP V4.1) did not include the DES module due to U.S. Government export restrictions which were in effect at the time of release.

Transport vs. tunnel mode

With either protocol, there are two different ways in which IPSec may encode an IP packet.

• In transport mode, the original IP packet has its payload encoded, but not its IP header. The AH or ESP header is placed around the payload portion of the packet. In the case of ESP, the original IP header information (including the packet's source and destination information) is not encrypted. AH, however, provies origin authentication for the entire packet, except for certain mutable fields.

  Transport mode is normally used in IPSec tunnels between individual hosts. Gateways, on the other hand, are not even required to support transport mode (although most do).

• In tunnel mode, the entire original IP packet is encoded, then encapsulated as the payload of a new IP packet. This provides complete protection for the original IP packet (but not for the new, outermost, IP header).

  Tunnel mode is required when either end of the IPSec tunnel is a gateway, unless the actual data traffic is destined for the gateway itself. Tunnel mode has a higher overhead than transport mode.

Note that the term 'tunnel mode' has no relation to the term 'IPSec tunnel'. An IPSec tunnel may use either transport or tunnel mode.

When using tunnels in transport mode, you must disable the 'MTU Path Discovery' feature of the IP layer. This can be done with the command

    inetcfg -set mtudiscover 0