Routing means deciding at each interface in a network where the packet is intended to go. Usually when we talk about routing we mean IP routing. That is the "kind" of routing I will be talking about. IP routing happens at layer three in the OSI model, it has nothing at all to do with the MAC layer. Routing divides broadcast domains. This means that ARP traffic stops at the router. ARP traffic is a layer two function used to discover which MAC host has a specific IP address. In large bridged domains (broadcast domains) ARP traffic can generate substantial network traffic.
Bridging means repeating at each interface in a network all packets which appear on one side to the other side. A bridge is a repeater. A hub is a "multipoint" repeater. You plug in eight wires on a hub and a packet coming into any wire goes out the other seven. Bridging happens at layer two in the OSI model. That is the MAC layer. It has nothing to do with TCP/IP. As an aside, all this means that a layer two switch is in actual fact a layer two router because, a packet coming in one port on the switch is only repeated on the port where the destination computer is connected. The layer two switch "routes" packets between ports using MAC addresses and the spanning tree protocol. Layer two switching is easily a whole order of magnitude faster than IP routing because IP routing requires a lot more CPU and a lot more software.
Now that we understand what bridging and routing are for our purposes, lets have a look at how they play together to move packtes to the right hosts.
When a computer wants to communicate with another computer using TCP/IP. It goes thru the following process:
1) Check the destination IP address and compare it with it's own IP address and netmask to see if it needs to send it to it's gateway. If it knows it's gateway it will go to step number 2, if it doesn't know it's gateway it may send an ARP request out and the gateway can proxyarp for the remote host, which is essentially skipping to step number 3. What happens in this unusual case is that the gateway "spoofs" the local host into believing the remote host is on the local segment. Always set your default gateway.
2) If the remote host is on a different IP network, contact the gateway computer and forward it to the gateway computer for ROUTING to the remote host. This is done by sending a packet with the gateway's MAC address and the remote systems IP address to the gateway via the MAC address of the gateway. The gateway (router) understands what to do with such a packet.
3) If the remote host is on the local IP network, check the ARP cache and see if it has an MAC/IP pairing in the cache for that host.
4) If it has the MAC/IP pairing for that host prepare the packet and send it DIRECTLY to that host. Note that it uses the MAC address to accomplish this.
5) If it does not have the MAC/IP pairing in the ARP cache, send an ARP request. When the other computer hears the ARP request it responds with an ARP reply, the local machine sticks the MAC/IP pairing in its ARP cache and then it talks directly to the remote host. Note again that all computers on the local network communicate using MAC addresses.
A couple of things stand out here. How can a computer know that an ARP request is for it? Well, every computer on the LAN listens to FF:FF:FF:FF:FF:FF (the MAC broadcast address) and the ARP request goes to that address. The ARP request looks sorta like this, "Yo' alla yall! Who has IP ADDRESS 184.108.40.206?" The ARP reply looks sorta like this, "Ayup! IP ADDRESS 220.127.116.11 is at MAC ADDRESS 01:20:03:40:05:60."
Now since that ARP request goes to the MAC broadcast address, it is repeated across every bridged or switched port in the entire LAN. This is one place where problems can show up.
For example, suppose you have a Cisco router you use to bridge 120 DSL clients onto your network. All of those customers are on the same T-1 circuit on that router. That T-1 is carrying frame-relay back to each subscriber. Essentially you have 120 subscribers sharing 120 different LOGICAL circuits on that T-1. There is no way for the router to know which MAC address is on which IP address at the far end because it is bridging. Therefore when ONE ARP request goes to that Cisco it has to repeat that single request 120 times, once for each logical connection across that T-1. That is 120 packets. Now imagine that a worm scans your Class-C and causes your gateway router to generate fifty ARP requests in one second. That causes the bridging Cisco to generate 6000 ARP packets across that T-1. Clearly bridging can be a problem in a busy network.
Now suppose that we had the exact same Cisco router but rather than use it to bridge those DSL subs onto the network we use it to route those subs onto the network. We subnet our Class-C network and use the top 128 addresses behind the Cisco and the bottom 128 addresses in front of the Cisco. This means we need to run some kind of routing protocol on the main network or we need to set static routes in our primary router(s). Our centralized DHCP server will no longer work because DHCP is a LAYER TWO protocol and is not broadcast across routers. We solve this by running a DHCP server(s) on our client facing Cisco(s). We do need to ensure that their pools of addresses DO NOT conflict with one another. DHCP makes provisions for this. We now must require customers to login using RADIUS or some other routable authentication protocol, or via a captive portal. To reduce our problems with ARP storms we have substantially increased our maintenance overhead.
Wouldn't it be sweet if we could have our cake and eat it too? I want to bridge these DSL subs onto my backbone and force them to login before they can go anywhere or do anything. I want to be able to hand them an IP address by looking at their MAC address or by reading their user name during login. I want to eliminate ARP storms caused in situations where multiple logical connections cross one physical link. I want a centralized server in my main offices which lets me completely control every aspect of each individuals traffic because every packet passes thru it. I want to be able to firewall them, shape their traffic, turn off their account, and account for their data. I want to know who is logged on which logical port and to be able to sniff that specific port with little or no hassel so that I can identify the one with a worm quickly and disable the account as quickly.
Enter the PPPoE server.
PPPoE (Point to Point Protocol over Ethernet) is now built into Windows XP and RP-PPPoE is a free aftermarket you can add to W98, WMe, W2K. You can build your own PPPoE server using Linux or FreeBSD. A PPPoE server allows you to logically bridge your clients to a host in your NOC where it logs them in, gives them an address, and does whatever else you can do with a Linux/FreeBSD box.
You don't get ARP storms because everyone is using the MAC address of the PPPoE server on the public side of the PPPoE server (essentially layer two NAT) and the PPPoE server knows which MAC addresses have what IP addresses all the time. You can assign NATed non-routable addresses to some users and public routed addresses to other users and you can firewall as desired by individual user login.
Why is this cool? It is a hybird. It lets you bridge your clients to a single server where you make all the decisions. Your clients can't talk to each other unless the PPPoE server allows it, even if they are actually associated with the same access point, because all IP traffic must go thru the PPPoE server. It lets you allow anyone to associate with one of your AP's but if they don't have a username and password they can't get an IP address and if they forge an IP address they can't get past the PPPoE server. This means any subscriber of yours can connect to the Internet from anywhere in your network.
That's the holy grail, transparent roaming at layer two.
I built a PPPoE server and we have been running it on our network for a few months now. Since we are a RADIUS shop, I configured it to use our RADIUS server as it's source of authentication data. I am quite pleased with the control it gives me. We are busily moving all of our DSL subscribers over to it and we are now experimenting with micropops backhauled via DSL thru the PPPoE server. Folks, this is the ticket. Do you want to stick a dumb AP in the parking lot of an apartment building and let multiple clients login to your network in such a way that if one of them gets a worm you will know which one it is? You can do that with a PPPoE server. Do you want to place a half dozen AP's in a neighborhood and let anyone who has an account with you access your network from any of those AP, while roaming freely? This technology will do that for you.
But the question was do I bridge or route? I think you bridge where you should bridge and you route where you should route, choose carefully because the answer is not obvious most of the time.
Credit to DaDogs (Michael) for this one!
Feedback received on this FAQ entry: