Wireless Service Providers → Mikrotik PPPoE

Those of you using Mikrotik as a PPPoE concentrator. How are you blocking customer to customer traffic? Are you using a separate box for the PPPoE concentrator from your Internet router?

If your customers are using PPPoE, how can there be any customer to customer traffic? It should all go through the PPPoE server, correct?

Yes there isn't any customer to customer traffic if you are using a pppoe concentrator. Each customer is in an isolated tunnel from their cpe/router to your concentrator.

Yes but lets say I give customer1 172.16.49.10 and customer2 172.16.49.11. Customer2 can still ping Customer1.

Does this eliminate Multicast and Broadcast traffic or do I need to implement firewall rules as well?

Yes, they can ping each other, but that traffic is all going to the pppoe concentrator and then to the other customer. Basically think of it like every customer's pppoe session is a VPN back to the server. When the customers ping each other, it's not much different than just pinging any other IP on the internet, with the exception that if they are your customers, the pings don't go out to the internet.Assuming you don't have the CPE bridged, then yes, all multicast and broadcast traffic (other than what your AP's, CPE's, or other backbone equipment might be generating) is isolated to the LAN side of the customer's CPE. I see a little broadcast traffic for things like the mikrotik "neighbors" discovery, but that's about all I see on my whole layer 2 backbone network.

So currently the configuration I have is the Mikrotik PPPoE concentrator still uses my Cisco router as the gateway address it hands out to each customer.

In this configuration does all traffic still pass through the PPPoE concentrator or does the PPPoE concentrator only authenticate and hand out the IP so the CPE can talk to the Cisco?

Why don't you use your Cisco for pppoe ?

You can get about 500 subscribers comfortably on a 3825 with hardly any CPU load. (I can't ever recall seeing more then 15% on the graph).

Also to segment multicast and broadcast traffic split your broadcast domains up with vlans.

It's also a good idea to nat at the customer cpe to prevent them from exploiting their layer 2 connection to your network.

Well you can block these attacks on the switch port it just automatically shuts down the port to the whole ap if you have a naughty customer. Then you have to re-enable it and it's a pain in the ass.

If you don't nat the cpe they can overwhelm your gear in many ways. Broadcast floods, MAC address flooding arp cache poisoning you ap's management connections for anything in the same vlan as them.

Also If you didn't obey best practices and vlan1 is on production switch ports they can re-tag and vlan hop to elsewhere in your network.

The only inner client traffic we block is our management. It Keeps the transit costs for p2p down .

Edit: to block your subscriber mgmt traffic if your on Cisco just assign a ACL to your pppoe virtual template.

Don't think about pppoe like DHCP. It's not really "handing out" the addresses or gateway to the customers.

An easier way to think about it is that the IP for the customer is added to a the PPPoE concentrator, and a dedicated private tunnel is created to the customers device.

When the customer puts traffic on the connection, the CPE funnels all traffic through that PPPoE tunnel to the concentrator, and then the concentrator is responsible for passing the traffic on to where it needs to go, whether that be to the gateway and on to the internet, or to another tunnel if it is destined for another customer.The PPPoE tunnel on the CPE will always talk directly to (and only to) the pppoe concentrator it authenticated with. Someone here put together a nice PPPoE guide awhile back, but I don't remember who it was or where it would have been posted.

If I am running multiple PPPoE servers on the same flat network can they all use the same subnet for customers?

Yup that's all done via radius, but they will become unevenly loaded.

Especially if one boots faster then the other and you have a power outage.

I ended up spreading the load by just running each bras on a different vlan ranges.

In a pinch I can just ssh in and move another bras to cover another network segment. If one of them fails.

Running them for different customers so if customers of PPPoE1 lose connection they do not authenticate to PPPoE2.

What I don't want:

Edge ->Fiber hop->Wireless hop->PPPoE-> Customers

Edge ->Fiber hop->PPPoE->Customers

I don't want the customers in the second segment to connect to the PPPoE concentrator in the first hop putting additional load on a wireless link.

You can seggregate parts of your network with a VLAN, placing the desired customers in those VLANs, and only put one or two PPPoE servers in each VLAN at strategic locations.

The PPPoE servers "WAN" interfaces and your border routers can then live in another VLAN which exists network wide, allowing you to limit where L2 traffic can go on the network, but still provide complete access for L3 traffic.

e.g.

»docs.google.com/a/snnap. ··· fas/edit

If you consider that red is a VLAN that exists network wide, allowing L3 traffic to go anywhere it needs to, but blue and purple are also 2 individual VLANs that keep customers from an individual tower local to that tower, such that they can only reach their local PPPoE server.

In practice you can use the same VLAN at each tower for your PPPoE customers, just as long as you prevent that VLAN from being trunked outside the local tower switch. This can make provisioning a bit easier if you dont have to remember which VLAN is used at which tower.

Also, slightly contrary to what OHSrob says, allocation of IPs is not always strictly via RADIUS. It is possible to give each PPPoE server its own pool of addresses which it can assign directly by itself. In a situation like this you will need to ensure that each PPPoE server has a non-overlapping pool of IPs to ensure that two people on the same network segment do not end up with the same IP address, as they do not coordinate between themselves.

Running multiple PPPoE servers on the same network segment can also be a good idea, in the event one fails, customers will automatically re-authenticate via any other available server on the same network segment, minimising downtime. This is a natural operational mechanism of the PPPoE protocol itself.

I thought that this
»mum.mikrotik.com/present ··· rbel.pdf

was a cool take on load balancing multiple mikrotik PPPOE servers

thought i knew mikrotik, i don't know crap after that. my head is still spinning

+1

worst presentation formatting ever.
nice premise -- but waaaaay to "flashy" and not easy enough to read.

q.

I don't know -anything- about MT so maybe that is an advantage since I am not confounded by what I know (or don't know).

This is how I see it...

In the existing configuration each of the PPPPoE servers was set up to do only one type of transaction specific to a single service class, and, as a consequence of the significant disparity in the number of clients in each service class, the server load became unbalanced.

The new solution has each of the servers configured to accept transactions of any service class, and the load balancer polls the PPPoE servers to see what the current client load is, then it directs the next client wishing to connect to the server that has the lowest load.

It also looks like the new configuration has better fall-back since no class is left without service on single or even multiple PPPoE server fails.

lol... yes the presentation is a tad offensive to the eyes but the idea and method was solid... it's not whitepaper after all... There was a video at that MUM with the actual test rig and performance/balancing when under load. They did a good job of proving the model befor dumping it into production.

having had to present many times -- a good (and simple) presentation template will do wonders to get your point across. it also makes it easier for people to emulate the work that you've done. i'm not saying that you need to make it plain with a white background -- but the 'kiss' principle works here very well.yes. not knowing 'tik -- i make approximations based on what John Galt6 stated above. knowing the api and how to script along with the use of routing protocols is the key here. while it was a very creative use to solve the problem -- each vendor has their own hooks to make what was shown possible. i just wish the presentation was a bit better so i could read it all meh. i don't always test my configuration. but when i do -- i do it in prod.
;-P

q.

Wow thats an incredibly complicated way to achieve load balancing across multiple PPPoE servers when you could:

1. Put them all in the same L2 domain and let the client sort out which one it wants to use - it wont be perfect but it will be reasonably well spread
2. Stick a LAC in front of your LNS and let the LAC spread the load amongst the LNS - again not perfect, but probably more predictably well spread given the LAC gets to choose randomly from a "static" list of destinations

But, I will give them credit for coming up with a pretty cool solution really. I will file this away in the "if I ever need to do something similar ....." pile in my mind.

Though I probably wouldnt have used BGP to do the load balancing, but instead toyed with the IGP and link costs. Causing unneccessary BGP churn is just not nice for the network, especially if its big.

Looks more like a sales presentation with their logo being about 1/5 of the vertical screen real estate.