6. Setting Up The VPN Software: IPSec

Linux 2.6 comes with IPSec code; when building your kernel, be sure to enable the following, either as modules or compiled into the kernel:

If your distribution doesn't come wtih a package for them, the latest versions of Racoon and setkey can be found at http://ipsec-tools.sourceforge.net/

6.1. Network Configuration for IPSec

Since it was designed by the IETF, IPSec works seamlessly (with a couple exceptions) with TCP/IP; we can therefore assign 10.42.1.1 to the eth1 interface of the wired endpoint, and 10.42.1.2 to the wireless AP.

6.2. Racoon

Racoon needs to be running on both the wired and wireless endpoints of the tunnel; the configuration is slightly different in each case. One common thing needs to be done on both ends, and that is making the CA root certificate available to Racoon. Racoon looks for a file whose name is the CA certificate's hash, with .0 appended. Execute the following in the directory where you will keep the certificates Racoon will use, on both the wired and wireless tunnel endpoints.

ln -s <ca-cert-file> `openssl x509 -noout -hash < <ca-cert-file>`.0

6.2.1. Wired Endpoint

The wired endpoint is always going to be the IKE responder, so we will put it in passive mode, and tell it to automatically generate SPD entries from the data given to it by the initiator (wireless endpoint).

path certificate "/path/to/certificates";

remote anonymous {
	exchange_mode main;
	doi ipsec_doi;
	proposal_check strict;
	generate_policy on;
	passive on;

	certificate_type x509 "<cert-file>" "<key-file>";

	my_identifier asn1dn;
	peers_identifier asn1dn;

	proposal {
		dh_group 2;
		encryption_algorithm 3des;
		hash_algorithm sha1;
		authentication_method rsasig;
	}
}

sainfo anonymous {
	pfs_group 2;
	encryption_algorithm aes, 3des;
	authentication_algorithm hmac_sha1, hmac_md5;
	compression_algorithm deflate;
}

6.2.2. Wireless Endpoint

The wireless side is the IKE initiator, so passive mode is off, and policy will be set via a config file rather than generated automatically.

path certificate "/path/to/certificates";

remote anonymous {
	exchange_mode main;
	doi ipsec_doi;
	proposal_check strict;

	certificate_type x509 "<cert-file>" "<key-file>";

	my_identifier asn1dn;
	peers_identifier asn1dn;

	proposal {
		dh_group 2;
		encryption_algorithm 3des;
		hash_algorithm sha1;
		authentication_method rsasig;
	}
}

sainfo anonymous {
	pfs_group 2;
	encryption_algorithm aes, 3des;
	authentication_algorithm hmac_sha1, hmac_md5;
	compression_algorithm deflate;
}

6.3. Setting Security Policies

Two computers communicating via IPSec create a security association, or SA, describing the parameters of the connection. In order to determine when an SA needs to be negotiated, the IPSec software consults a security policy database (SPD), which is managed by the setkey utility.

6.3.1. Wired Endpoint

The generate_policy option in the configuration file for Racoon obviates the need for a manual SPD configuration file on the wired endpoint.

6.3.2. Wireless Endpoint

#!/usr/sbin/setkey -f

flush;
spdflush;

# Communication between wireless nodes will not be encrypted.
spdadd 10.42.1.0/24 10.42.1.0/24 any -P out none;
spdadd 10.42.1.0/24 10.42.1.0/24 any -P in none;

# Otherwise, it gets tunneled.
spdadd 10.42.1.100/32 0.0.0.0/0 any -P out ipsec 
	esp/tunnel/10.42.1.100-10.42.1.1/require;
spdadd 0.0.0.0/0 10.42.1.100/32 any -P in ipsec 
	esp/tunnel/10.42.1.1-10.42.1.100/require;

6.4. Transmission Issues

The overhead incurred in an ESP tunnel can occasionally create stalls in transmission when fragmentation or path discovery fails. One way to fix the problem is to lower the wireless endpoint's advertised maximum segment size. The ip(8) command can handle this, via its route subcommand:

ip route change 10.42.1.0/24 dev ath0 proto kernel scope link \
   src 10.42.1.100 advmss 1350
ip route change default via 10.42.1.1 dev ath0 advmss 1350

1350 is a safe value guaranteeing that everything will fit into a standard Ethernet frame.

6.5. Making It Work With DHCP

But rather than hardcode the client IP and network interface in all the configuration hitherto, a little scripting can make this configurable on the fly after our DHCP client has obtained an IP for us.

6.5.1. SPD Rules

We can create a template something like the following to use for our wireless SPD configuration, and use sed later on to put the proper values in place of %NET%, %MY_IP%, and %GW%:

#/usr/sbin/setkey -f

flush;
spdflush;

# Communication between wireless nodes will not be encrypted.
spdadd %NET% %NET% any -P out none;
spdadd %NET% %NET% any -P in none;

# Otherwise, it gets tunneled.
spdadd %MY_IP%/32 0.0.0.0/0 any -P out ipsec
        esp/tunnel/%MY_IP%-%GW%/require;
spdadd 0.0.0.0/0 %MY_IP%/32 any -P in ipsec
        esp/tunnel/%GW%-%MY_IP%/require;

6.5.2. client-ipsec-config.sh

The following script takes one argument: the interface over which we'll be running IPSec.

#!/bin/sh
#
# client-ipsec-config.sh
# Autogenerate spd configs and ip route configuration from templates.

if [ -z $1 ]
then
        echo "Usage: client-ipsec-config.sh <iface>"
        exit 1
fi

# Get the IP address of the desired interface.
IFACE=$1
netroute=`/sbin/route -n | grep $IFACE | head -n 1`
gwroute=`/sbin/route -n | grep $IFACE | grep 'UG'`
NET=`echo $netroute | cut -d\  -f1`
NETMASK=`ip addr show $IFACE | egrep "inet " | cut -d/  -f2 | \
         cut -d\  -f1`
GW=`echo $gwroute | cut -d\  -f2`
MY_IP=`/sbin/ifconfig $IFACE | grep inet | cut -d: -f2 | \
       cut -d\  -f1`

# Edit our template file
TEMPLATE=/path/to/template
SETKEY_CONF=/path/to/setkey/conf
SCRIPT="s/%MY_IP%/$MY_IP/g;s/%GW%/$GW/g;s/%NET%/$NET\/$NETMASK/g"
sed -e "$SCRIPT" $TEMPLATE > $SETKEY_CONF

# Decrease our advertised MSS.
MSS="1350"
ip route change $NET/$NETMASK dev $IFACE proto kernel scope link \
   src $MY_IP advmss $MSS
ip route change default via $GW dev $IFACE advmss $MSS