Industry Forums > Wireless Service Providers > Routing question

Routing question

If you have two upstream providers and both speak BGP, you can tell your router to prefer to move traffic through one of them. It's not too dissimilar from assigning an interface metric on a Linux box that has two interfaces and two gateways.

see the part here about local preferences:

»www.cisco.com/en/US/tech/tk365/t···31.shtml

This can be fine tuned enough so that if one of your upstreams has a much better path to, for example, Youtube, you can move all traffic to/from the Google-Youtube ASNs through that provider.

reply to raytaylor
Im not sure Id get into BGP necessarily especially if you don't have any portable IP space.

If you have 2 DSL providers and you are getting public IPs from both of them, you don't have much choice but to make sure that IPs from DSL provider 1 go out DSL provider 1 only, and same with DSL provider 2. With NAT its easy to load balance, but once you start having your customers have public IPs the load balancing thing changes.
--
OptionsDSL Wireless Internet
»www.optionsdsl.ca

reply to raytaylor
From what I am reading in your post BGP isn't going to do much for you. If you can't speak BGP to two providers or you are not trying to control the advertisement of your own address space you are not going to gain anything.

Yea what he said.

reply to raytaylor
BGP doesnt route. It only builds a routing table.

And typically by default, BGP will only install a single route in the routing table, the shortest one by AS hops (thats the number of networks the route has traversed to reach you), not the shortest overall hops...! This can usually be tuned, refer to BGP multipath.

But its up to the router to move the packets, so you need to determine what mechanism it offers to move traffic from one link to another if the first link becomes full.

But having said that, you cant control the path that data will take to get back to you - the Internet is asymmetrical in that the path data takes in one direction is not guaranteed to be the same path it takes in reverse. Other providers routing policies may also override any traffic engineering that you try.

reply to thewalrus
Yes I have my own ip space assigned to be by apnic that is in theory portable with BGP - I can take my ip addresses to any isp that will allow me to bgp peer? with them

Based on that, I think you will need to continue NATing, because its the only way to guarantee that data takes the the same path back to you. Otherwise you could very likely end up with all of your return traffic taking one path (the one everyone else sees as the best path), and congesting one of your downstream links, resulting in slow performance for *everything* that happens to be using that link.

Or maybe you need to selectively NAT some traffic, while allowing other traffic to route natively. But you would still likely end up with one return path being more heavily utilised than the other.

reply to raytaylor
Can you get faster DSL lines?

I used to have multiple DSL lines, 3 actually each with their own /28, so I just evened out the customers across all the lines whilst giving them all public IPs. No its not dynamic load balancing but it did work well and wasn't too much to maintain up to about 60 customers. I set the mikrotik to simply do a "if it came in on this interface, make sure it goes back out this one too" sorta thing.. Worked well. (interface routing introduced in v3)

MLPPP came next removing that need, then 100 meg fiber..
Wow all seems so long ago now. lol

Back to topic..
--
OptionsDSL Wireless Internet
»www.optionsdsl.ca

reply to raytaylor
Nah no DSL provider in NZ will do multiple ip addresses over a DSL line.

I just had a chat with my upstream and i am going to double my payments and get triple the bandwidth. Will keep a dsl line which i can switch on natting for as a backup, while still keeping my customers without a public ip address load balanced.

reply to raytaylor
You can at least prepend which will help to balance out inbound traffic but that's kind of hard if you only have 1mb on each feed unless you are selling really low bandwidth to each customer.

reply to raytaylor

Yeah i have read that.
Snap provide and i think now partially own my competitor so im not really keen to use them. They data also has to travel an extra 1200km to get to the auckland peering exchange which is only 400km from me via almost every other isp in the country.

Would prefer to stay away - and i doubt i would get that sort of service now they have grown into a bigger company with (corporate-ised)than the 3 network techs starting up in their garage.

But even so, i still cant fully route and traffic load balance the way i want. The snap connection would only have a 768k upload speed too.

So i have decided to just get a cheaper and better performing dsl connection from a national ISP based in auckland, and enable natting as a backup when my main upstream goes down.

reply to raytaylor
There is one other possibility. Its not ideal for a few reasons (one being MTU really gets messed around with), but it could help.

You'd need a few DSL lines at your NOC or POP or where ever you currently have your gear, enough to get the bandwidth you want, and another router colo'd somewhere you have access to better bandwidth.

You could then setup series of GRE/IPIP tunnels, one per DSL connection to link you into your remote router. You then load balance traffic across those tunnels, and at the other end it routes in/out via higher bandwidth services.

Alternatively, do any of your current ISPs support MLPPP? You could login with two ADSL services under the same username, and PPP takes care of bonding them together into a single fatter pipe.

Me and a friend who runs another wisp thought about that. Decided it was going to be too much trouble and a drive to auckland to reboot or troubleshoot a microtik box isnt the best idea
Decided it would be too unreliable, and I would have to drive to auckland to fix the problem.

I have basically decided to double my bandwidth with my main upstream provider who routes to me, and just keep the dsl line for the customers that I continue to NAT and will load balance them across one of the routed IPs and the DSL.

I have almost got my per megabit pricing already at $233 which is awesome considering i was paying around $1000 when i started, and then got it down to $533

Me and my upstream have talked and I am going to run a p2p link direct to his office (runs another wisp) and because he focuses mainly on business clients, I am able to get a very high burst. During my evening peak period when all his business customers have gone home and arent using the bandwidth he would normally have set aside for them.

At the moment he has me on a trango SU which isnt very fast so he cant give me much burst.

So right now i barely need a megabit during the day which is what my current CIR is. Its going to go up to 2mbits under our new plan. But in the evenings, he has a whole lot of spare bandwidth that i will have access to as i enter my peak hours. And its going to allow me to get 6mbits between 6pm and 8am. Then it gradually drops down to my 2mbit CIR during his workday peak hours of 8am to 6pm

I am quite happy with that. Might really only need the dsl as a failover rather than a load balancer now that i think about it.

I also have an appliansys cachebox which gives me heaps more performance too.

There are ways to get around that.

Have a phone line with a modem connected to a console server with multiple console lines, one of which is connected to your router.

Also have, for example, an APC Masterswitch, also connected to the console server.

If you have a fault, botch some config, whatever, dial into the console server, connect to the Masterswitch and reboot the device, or connect to the router console instead and fix the configuration or roll it back.

But either way, sounds like youre getting things sorted out.