We have two different problems to overcome in securing an 802.11 network:
Controlling access to the 802.11 media (RF) is difficult if not impossible, and as has been stated, it is easy to break or spoof basic control methods. How can we isolate our wireless system from the rest of the network, and allow only people we trust to send packets to other places?
As we have also stated, the default encryption used by 802.11 is easy to break. How can we use better encryption, and prevent or hinder an attacker from snooping on our traffic?
It would be even better if we could somehow combine the solutions to these problems into one system.
Traditionally, when two parties desired to communicate secretly, they had to each possess some key, a word or number or algorithm used both to encrypt and decrypt the message; if the key was known by a third party, the messages were no longer secret.
In the 1970s, several cryptologists developed the idea of public-key cryptography, which changes the conditions needed for breaking a cryptogram. Two different keys are used: an encryption key, which is made public, and a decryption key, which is known only to the person receiving an encrypted communication. It is either impossible (in the ideal case) or prohibitively difficult (in the real case) to determine the private (decryption) key from the public (encryption) key; thus, secure communication can be undertaken without the need for exchanging a shared secret. The well-known RSA (Rivest-Shamir-Adelman) algorithm is a public-key system.
Most public-key algorithms are far slower than shared-secret algorithms of equivalent security, too slow in most cases to be used for encrypting communications in real-time. However, public-key cryptography has become popular for the following applications relevant to our problem:
Authentication. If one party encrypts some random data with another party's public key, and it can be sent back decrypted, the party of the first part can have reasonable assurance that the party of the second part is not someone spoofing party #2's identity.
Distribution of shared secrets. A public-key algorithm is used between two peers to distribute keys for a shared-secret algorithm, and that algorithm is used to encrypt the rest of the communication.
If we want to use public-key cryptography for either authentication or distributing shared secrets, we need to make sure that we have received the right key, and are not being fooled by a man-in-the-middle attack. In such an attack, a third party silently inserts himself into a communications channel, spoofs the identity of the other two parties, each in turn, and intercepts all communications between them. How can we be sure we are not being so fooled?
One way is to physically meet the person we want to communicate with and get a CD with the certificate on it. This starts to get cumbersome, especially when that person is on the other side of the world. It would be nice if he could publish his certificate on his website, and we could download it, and still verify that it is his.
Another way is to agree on a trusted third party, to whom we send our public keys, augmented with some naming information and called certificates. After careful scrutiny, the trusted third party signs our certificates with his private key. We call this trusted third party a certificate authority, or CA.
The CA also has a certificate, which we must obtain by some secure means; for example, we go out for coffee with him and he hands us a CD containing the certificate. The CA could also publish his certificate online, along with a digest of its contents; we then verify the digest, first by computing it ourselves and checking the value we got against the published value, and then perhaps by calling our CA on the phone and having him read the digest to us. This whole system of obtaining and signing certificates is known as public-key infrastructure.
For securing our 802.11 network, we will act as both CA and client wishing to be certified; we will create our own root certificate and private key, and sign certificates we create for our VPN machines. The OpenSSL (http://www.openssl.org/) software suite can generate the necessary certificates and private keys, and can be configured to act as a CA.
A private network is any network to which access is physically restricted; a virtual private network is any system that simulates a private network, usually using encryption, over some untrusted public network. In addition to encryption, a virtual private network uses tunneling, a networking trick wherein a packet formatted for some protocol becomes the data carried by another packet of the same protocol; i.e., an IP packet is the data payload of another IP packet. Our 802.11 system qualifies as an untrusted public network, and since VPNs usually use very strong encryption, we have found a way to solve problem #2.
Most VPNs also do some kind of authentication between virtual private nodes, and thus VPN software can also help solve problem #1, in concert with a firewall that only allows packets that have come from the VPN to go beyond the wireless network. Anyone who can't create a VPN connection is stopped from transmitting further.
We will consider two VPN systems: IPSec and OpenVPN.
IPSec is the older of the two, and was created by the Internet Engineering Task Force (IETF), the research body of the Internet Society. There are several protocols in the system; we will focus on two: Internet Key Exchange (IKE), for peer authentication and key distribution, and Encapsulated Security Payload (ESP), for encrypted transmission. With the KAME IPSec tools under Linux 2.6, IKE is implemented by the Racoon key-exchange daemon; it can be set up to use PKI for authenticating clients, which we will do. A supporting program called setkey manages security databases (called SAD and SPD) for Racoon and for the kernel code which implements ESP.
IPSec was designed before NAT was widely used to stave off the IPv4 address shortage, and since it operates at the network layer, encapsulating IP packets in tunnel mode, bad things can happen if NAT is running anywhere between the two endpoints of the tunnel. An extension by the name of NAT-T has been proposed and partially implemented for ESP, and the Linux 2.6 IPSec tools can use it; still, NAT can cause major headaches for an IPSec VPN. In the case of our network, however, this is not an issue.
OpenVPN is newer than IPSec; the project was started in 2002 by Jim Yonan. Instead of creating new protocols to handle VPN security, OpenVPN is a modular design using existing software components:
OpenSSL for PKI authentication and encryption
Kernel IP tunneling drivers (*BSD, Solaris, Linux, and MacOS) or a simulation thereof (Windows)
Compared to IPSec, its configuration is simpler, and no major security flaws have been discovered yet. It is a promising system, and may in the end relegate IPSec to the dustbin of history. (As with all predictions of the future, may is the operative word here.) Since OpenVPN uses SSL/TLS, it has no problems interacting with NAT.