Wireless Service Providers → [Tech Ops] Network topology suggestion for small wisp ???

i want to change my network topology. Now most my APs and Stations re Bridge mode, Some of the stations re in routed mode .( use 9 main AP in 5 ghz band - 70 locom5 station attached to main 9 APs and 50 AP in 2.4 ghz band attached to 70 locom5 stations) You Can see that in attached file. So in peak times my clients network is slowdown. I guess this is bcz broadcast and arp packets. If i change my network topology to Vlan in tower side is it helpful for me .

I wait yours suggestion

Thank you in advance

I came from a PPPoE background where each tower (PoP) did PPPoE to a central server. That works for sure but demands routing overhead which may not be necessary.

I have seen a network of layer two bridged APs (600 of them) deliver service to maybe 15000 simultaneous users. There was no PPPoE in that network, but that network was 100% mobile users.

The advantage of layer two service to a central IP address server (DHCP) is that once you are in the system you can roam from AP to AP.

In my mind, today (as opposed to what I have said historically), the ability to roam is very important in a network which supports roaming users, not so much in a network that is delivering fixed wireless.

Still, I like the layer two model. Bridge it to your central service.

Before you try solving what you guess it is, you should verify that is the problem. I'd start by sniffing some segments at off peak and peak time and making sure that it is Broadcast and/or ARP that is causing the slow down. My guess is that you will find that it is not.

He should be graphing each radio and pinging to each location. That will show you exactly where the problem is. Products like Aircontrol for your radio and The Dude from Mikrotik would go a long way to verifying your problem

For full time latency and loss monitoring of everything from ICMP to Radius I'd recommend Smokeping. Excellent tool.

While Dude or AirControl will graph resource/bandwidth utilization, neither of those tools will show that it is Broadcast traffic in particular, Which is his guess. The way to do that is with something like a Netflow analyzer, or a packet sniffer spanned into key locations.

True, but chances are it's not the problem any way. But, a giant bridge has it's faults to begin with, so... It could be a lot of things

Agreed. My own guess is collocation interference on the backhauls and AP's, but to fix the problem he needs to find it first.

We have done alot of bridged to routed conversions. If you come up with a new ip scheme you can have the bridged and routed segments running alongside each other. This way you can convert sites and customers over with little to no downtime.

Do you have any good resources on how to change. I've always ran bridged, but am looking to go to routed.... but only have enough info to be dangerous. The concepts make sense, but it's setting up the routing that i'm having a little trouble with.

The best thing i've found so far are these 3.

»oreilly.com/catalog/cisc ··· h05.html
»www.mywisptraining.com/w ··· uted.pdf
»Wireless ISP FAQ »Bridging or Routing Your Network

I would first ask myself can your network go routed? You have devices in your network acting as bridges or switches that can also route?

Is your bridged network using RSTP or STP for failover on backup links? Then you are going to want to use something like OSPF.
If you need to install routers at sites that don't have any then you could slowly convert each site to a routed site. You could install your router as a "bridge" to get the physical work done, then a few hours overnight convert a section of your network to routed.

My network: 100% routed with mikrotik, Each site has a single router. OSPF for failover and MPLS/VPLS Tunnels back to a central PPPoE Server. It works very well.

If you don't have much experience with routing I would set up a lab to simulate it all on, before moving to your network making you money. And count on stuff going wrong.

Thanks!

I really only have 2 towers at this point, with about 90 subs total between the 2. They are skewed though, like 20 on one tower and 70 on the other.

I have an ERL3 that I started playing with yesterday in my home network on a 172 subnet Right now there is some STP for redundancy. but I'm curious about the affects of going routed. The plan would be to start small.

I get the overall concepts of routing, but when it comes to creating the rules, they didn't seem to be forwarding through to the next hop.

i would like to chime in here as a proponent for routed core with mpls running. i'm not sure of the featureset available with the mt hardware -- but if you set up a scalable, redundant layer-3 network with an mpls overlay -- you can move things wherever you need to. need to support a customer with private wan -- set up an l3vpn. customer needs l2 reachability between points for some clustering apps, etc -- set up a vpws/vpls link between sites. in this way -- you can provision services over the top for specific cases (and charge more money for them) while still keeping a flexible and highly-available core.

my thoughts.

q.

+1 tubbynet.

Im a big fan of routed networks, and with PPPoE for the access layer. Centralised or distributed BRAS, either way you have much more control over who gets on your network, and can much more easily count the amount of data they have used up/down and charge excess or drop their speed via shaping/policing if required.

It also means you can simply disable their RADIUS entry to prevent them getting on the network if they dont pay their bill, and this can be automated to a high degree.

i draw a new one what s yours offers is it Good for using vlans

A fair amount of the older MT hardware, and even some of the current stuff has a problem with MAX MTU still being too small when cramming MPLS, VPLS, VLAN and PPPOE together, which, thanks to MPLS can be a tough problem to figure out. MT can handle all of the rest if you don't have that restriction/problem. Check MTU on everything before going to MPLS... and then check it again. Where you might need to a less elegant option is IP in IP rather than MPLS/VPLS.

But PPPOE to NAS at the tower site then into VLANs over MPLS and OSPF to the head end is the cat's ass if your gear and your brain can handle it for certain.

In my last job I spent about 30% of my time teaching TCP/IP.... Please forgive me if I'm talking below your knowledge level but this might help....Keep in mind this pertains to IP Version 4.

The Final step in datagram delivery is dependent on MAC address of the destination because the signalling 'on the wire' uses MAC addressing to specify the address of the intended recipient - that's how a device knows that it needs to process the signal coming in 'on the wire' - because THAT datagram is being sent to IT'S MAC address. Only MAC addresses, *NOT IP addresses*, are used for the *final phase*, or the 'on the wire' phase of Datagram delivery.

I usually equate IP addresses to postal codes and MAC address to individual house numbers on a street (AKA the wire).... that seems to help.

Routers change both source and destination MAC addresses (along with other things) inside data packets when they route them because each Router is effectively 'delivering' that packet to the next hop's MAC address, but Layer 2 bridges do not change MAC addresses because they do not break up broadcast domains.

So in bridging it 'just works' because both ends know the other end's MAC and can/will signal directly to that MAC, where-as in routing the recipient only knows that the datagram came from the the MAC address of it's 'next hop', which is the adjacent router's interface on the same segment.

Long story (or a 30 hour training course) made short; in my experience, what messes most people up in routing, is that just because YOU know how to get THERE doesn't mean that the other end knows how to get BACK to YOU if there is a router in between. That's why you need static routes or a routing protocol like RIP, OSPF Etc

I've found it's very helpful to start with the OSI model and have the student understand the concepts from Layer 1 up.... you don't need to get above Layer 4 in most cases before it all comes together for them. As part of that, understanding ARP is critical to understanding and troubleshooting IP bridging and routing, once they get the concept of MACs mapping to IP addresses and ARP Tables it's easy to explain static routing and then dynamic routing concepts.

Hope that helps.

S

yes -- because its not always known by those who run networks how many bytes of overhead each operation tacks on. vlan tags, pppoe, one mpls label, a vpls id, etc -- all have overhead that decreases the amount of available payload. however -- if you've got a head for these things (or can find handy-dandy references) -- i'm certain that mt hardware has a ping option to set the 'df' bit. simply change payload sizes until you drop pings.because at that point -- you're running your network very similar to a dsl provider, which has been proven over the years. plus -- depending on the options that your gear supports -- its very possible to push all kinds of policy (shaping/traffic profiles, qos tagging, ip addressing, etc) all via the pppoe login.

q.

I was running bridged up until about 2 months ago.
Started placing mikrotik routers at each site where we had one more sub-site running off it.

Beach village 93kms from core, 4x backhaul hops + 300 metres AP-CPE
Previously bridged the whole way - They would run a speedtest.net mini test to a web server in our core and get 1mbit down / 3mbit up. UDP was about 8/8

After putting routers at the first two sites, the speedtest went up to 4.5 down / 4 up - they were still bridged a couple of hops or 50kms of the chain but it made a big improvement.

IP in IP will have more overheads than MPLS/VPLS.

IP in IP encaps an IP packet inside another one. MPLS just changes the ethertype of the outer Ethernet frame and chucks 4 or 8 bytes of headers on (typically).

You'll probably save 12+ bytes of overhead with MPLS compared to IP in IP.

MPLS is much more efficient as a transport technology than anything besides native.

e.g.

1500 byte IP packet + 14 bytes of Ethernet + 4 bytes of VLAN tag = 1518 payload you want to carry.

For IP in IP add 20 bytes for wrapper IP headers + 14 bytes of wrapper Ethernet + 4 bytes of wrapper VLAN = 1556 (maybe more depending on the layer 4 protocol used).

Or for MPLS just add 8 bytes for 2 labels (which is usually sufficient for most MPLS carriage) + 14 bytes of wrapper Ethernet + 4 bytes of wrapper VLAN = 1544.

This is the issue I still suffer from. In my network my pppoe MTU is 1488 after MPLS/VPLS . One day when old hardware is removed I will run a 1500 byte pppoe MTU.
(I have a lot of old hardware).

is there a way to see a list of hardware that suffers this max mtu issue?

Mikrotik publishes a list on their wiki. Ubnt is either 1524 (old) or 2024 (new). Some Mikrotik hardware (rb750 for example) had a chip change without a model number change so older 750 may have small MTU support. (1524/1522).

Yup I'm not arguing - just clearing up the Q you had mentioned about MT gear in particular.

Agreed. I meant where the gear cannot handle MPLS for whatever reason... IP in I Pis what we did before MPLS.

»wiki.mikrotik.com/wiki/M ··· erBoards