dslreports logo
 
    All Forums Hot Topics Gallery
spc
Search similar:


uniqs
3229
ArkiMage
join:2001-06-30
Kingsport, TN

ArkiMage

Member

BGP, Hardware, etc...

We got a Class C network from ARIN in 1995. For years it wasn't routed at all but for the past 4 or 5 years a local ISP advertises it and static routes it to us. I'm considering an upgrade, bringing in multiple fiber providers and advertising the network ourselves. Right now we only have an _old_ Cisco router, not appropriate for BGP. So, a couple of questions if you don't mind...

Current recommendations on a router that could have 2 (Layer 2 delivery) 10/10mbit fiber connections and advertise a Class C via BGP through them? Router hardware is faster now, is RAM the only real concern?

I've never setup or worked with BGP myself. The local Telco LEC and the local cable/internet company have both provided fiber connectivity quotes. 5 to 10mbit is their minimum to pull fiber to us but not wanting to limit ourselves, we might want to get 20 or more mbit from each of them in a year or two(?) so adequate hardware for that is a must.

So, assuming we get connections from 2 providers and route advertise our network to the world with BGP. Just how good and/or quick is "failover"? If one provider totally goes belly up, will my customers be able to hit our web sites without interruption at all? Outbound and inbound routing will instantly or within minutes or within hours know to route via the still valid provider? Packets "in transit" will still arrive, or be dropped and re-transmit via the other provider?

Two providers for redundancy. I'm trying to make sure they connect to us from separate neighboring cities running fiber on poles down different ends of the road so as to prevent a single pole being clipped taking them both out. One connects to the world primarily via Sprintlink's network and the other AT&T. Neither buys capacity from the other and the leave the area on separate fiber backbone routes. So, any words of wisdom or suggestions on other considerations I might not have thought of yet?

One wildcard, the current ISP delivers IP to us wirelessly. The connection is rather unreliable though. Based on a business arrangement we get some bandwidth from them for free (they host a tower on our property via which they sell connectivity to some neighboring businesses). So I'd not like to lose that, it's free so why not. But the bandwidth is fairly limited and the quality isn't the greatest in the world. Will I easily be able to have 3 providers via BGP but primarily keep this one in reserve, have basically no outbound/inbound packets traversing them unless BOTH primary fiber providers experience outages or cuts?

Thanks!

PS. Suggestions of RTFM first would even be welcomed, if you have suggestions or pointers to good reading material, please post it...

PSS. Sticking with the Class C we already have a perfectly fine idea? No reason to abandon that or do anything different? All I need is to pay $500(?) for an ASN and away we go?
aryoba
MVM
join:2002-08-22

1 edit

aryoba

MVM

1st of all I'm not sure why moderator moved this thread from Cisco forum to Networking forum since there is a question relating to Cisco router.

Moving forward, there are several factors in considering network device choice to run BGP. They are

* Budget or financial constraints
* Application throughput/performance requirement
* Future growth expectations
* High Availability expectation level

For 20 MBps pipe on each ISP link (with total ISP of two), you would probably prefer something like Cisco 3825, 3845, Catalyst 4503, or higher model.

As to failover scenario for a single Class C, your only feasible choice is probably Active/Standby between the two ISP although Active/Active is possible assuming you and your two ISP have proper BGP Community string.

FYI, Active/Active scenario convergence time is in general shorter than Active/Standby scenario convergence time. In other word, your customers should feel minimal or no interruption when accessing your website during one ISP outage if you have Active/Active scenario.

As of choosing ISP for BGP, I usually prefer Tier 1 ISP. Whenever possible, you should have direct BGP peering to Sprint and to AT&T. If this is not feasible choice, you need to make sure that both your ISP has direct BGP peering between each other.

Frankly, I never heard BGP peering connection via wireless. Since you are considering fiber link, it is most likely you will use the fiber link to deliver BGP peering and not using the wireless for the BGP peering.

With a 3rd ISP (Wireless ISP), you may be able to announce the subnet from the wireless ISP to the two previous ISP over the fiber link. Make sure that all three ISP are aware of it so they can do necessary adjustment on their end.

boognish
Premium Member
join:2001-09-26
Baton Rouge, LA

boognish to ArkiMage

Premium Member

to ArkiMage
BGP can be announce over wireless we have a 2.5 through »www.broadbandip.net/html ··· ess.html and then some T-1s through other providers. When one goes down it is not really noticeable to customers at all. The routes seem to favor the T-1 lines.

rolande
Certifiable
MVM,
join:2002-05-24
Dallas, TX
ARRIS BGW210-700
Cisco Meraki MR42

rolande to ArkiMage

MVM,

to ArkiMage
If you are looking at this level of redundancy, then you need to consider having 2 routers (1 for each circuit) and run iBGP between them. Routers can and do fail. For disaster recovery you should also have a second location with similar connectivity or use a colo. Depending on the features you intend to deploy and the future bandwidth scalability, I would recommend no less than a pair of 3825's with 512MB DRAM each.

If you expect to be at or above 20Mbps on each circuit with continued growth, I would lean towards a pair of 7206VXR's. You might want to look at Juniper M series gear too. You can typically scale the Juniper gear alot farther for your dollar in the same chassis.

carp
Rejected
join:2002-10-30

carp to ArkiMage

Member

to ArkiMage
Is there a specific reason you want fiber pulled? You could order DS3 and possibly consider SONET or SONET protected. Don't forget to apply for an AS number.
JTY
join:2004-05-29
Ellensburg, WA

JTY to ArkiMage

Member

to ArkiMage
For a reasonably priced router to handle this, I'd recommend a Juniper J series, something like a 4350.

packetpusher
Premium Member
join:2005-03-22
Oakville, ON

packetpusher to ArkiMage

Premium Member

to ArkiMage
Heya,
I'd definitely recommend reading Internet Routing Architectures (»www.amazon.ca/Internet-R ··· 9&sr=8-1). You can influence routing choices to a certain point that would make the wireless link the least preferable, however nothings really perfect. That being said I'd still advertise over it. In terms of hardware, I'll second the multiple routers using VRRP and iBGP between them. I'd also check out www.vyatta.com to use as your router, and grab a couple Dell servers to use as hardware.
ArkiMage
join:2001-06-30
Kingsport, TN

ArkiMage

Member

Thanks for all the great answers and assistance everyone!

If Active/Standby means one connection is used and the other just waits to take over in the event of a failure, that doesn't sound near as appealing. I'd like to be able to *use* the bandwidth and routing of both. Sounds like from what you're saying a/a can "failover" quickly so no real reason not to(?)

We're very big on redundancy so the two router idea I like the sound of. I'm not sure how equipment behind both of them can have a single default route, but that brings me to the next item.

Thanks for the book suggestion, I obviously have a lot of reading to do and intend just that.

The Cable provider is Charter and they have fiber on the pole 100' from us. From the LEC I've gotten quotes and fiber is suggested by them (Embarq). They'd have to extend their fiber network about 1/4 mile to get to us. Also fiber providers are going to demarc as layer 2 ethernet. A DS3 or other similar connections wouldn't be would they?

Cisco, Juniper, etc... Experience with Cisco here, none with Juniper. Lots of experience with Linux and Open Source in general. I was wondering if anyone would recommend something like Vyatta. I hadn't read any of the info on their site until now and didn't know if that type of solution could compete from a speed/latency standpoint. Sounds like as long as a reasonable PC is used, it can work very well.

Level3 is also within 1/4 mile of us but won't build out their network to us, too costly. Sigh...

So one new question. On the point of going with tier-1 providers. Embarq quotes a "metro-ethernet" price as transit and then an internet port charge separately. Would it be advisable to investigate using them only as transport back to the POP and then connectivity to AT&T or Sprint from there? Versus buying IP from both of these providers, Embarq who locally connect solely to Sprintlink and Charter who locally connect solely to AT&T. How important is the distinction as long as neither one oversell too much?

Thanks!

rolande
Certifiable
MVM,
join:2002-05-24
Dallas, TX
ARRIS BGW210-700
Cisco Meraki MR42

rolande

MVM,

said by ArkiMage:

Sounds like from what you're saying a/a can "failover" quickly so no real reason not to(?)
BGP route failover on the Internet can take several minutes to "fully" propagate. Usually the failure is noticed within about 60 seconds or so.
said by ArkiMage:

We're very big on redundancy so the two router idea I like the sound of. I'm not sure how equipment behind both of them can have a single default route, but that brings me to the next item.
You can use HSRP or VRRP to setup a redundant virtual next hop address between the routers for transparent failover. It is possible to configure 2 HSRP or VRRP groups and then set either router primary for one of the virtual addresses with the opposite router as the backup. Then you configure dual, equal cost default routes to these virtual IP addresses.
said by ArkiMage:

Also fiber providers are going to demarc as layer 2 ethernet. A DS3 or other similar connections wouldn't be would they?
Neither is the fiber connection by default unless they are doing long haul ethernet. Any provider can drop off their own equipment to demarc any type of WAN connection and deliver it as ethernet.
said by ArkiMage:

Embarq quotes a "metro-ethernet" price as transit and then an internet port charge separately. Would it be advisable to investigate using them only as transport back to the POP and then connectivity to AT&T or Sprint from there? Versus buying IP from both of these providers, Embarq who locally connect solely to Sprintlink and Charter who locally connect solely to AT&T. How important is the distinction as long as neither one oversell too much?
Upstream provider diversity is the key to getting the most out of redundant connections. Yes Tier 1 providers are good to connect to. They get you as close as possible to the largest portion of the Internet. However, there is something to be said for Tier2 providers as an alternate connection. NTT Verio, Savvis, Level 3, or Qwest among others can provide a much better complement of diverse paths to customer endpoints than going with a second or third Tier 1 provider.

In your case, more than likely, Embarq is not providing the backhaul transport to the destination you want. So it could get even more expensive in that you pay for Embarq's transport, then you pay for local loop costs to get to Level 3 and then you pay an Internet port charge from Level 3. Pick your poison.
aryoba
MVM
join:2002-08-22

2 edits

aryoba

MVM

said by ArkiMage :
Sounds like from what you're saying a/a can "failover" quickly so no real reason not to(?)

Probably I should elaborate on the Active/Active and Active/Standby scenario. In implementation, Active/Active could be in a form of announcing the same single Class C to both providers with equal cost for both inbound and outbound.

On the other hand, Active/Standby could be in a form of announcing the single Class C to one provider and announcing the Class C supernet to another provider with equal cost for both inbound and outbound. Active/Standby could also be in a form of announcing the single Class C to both providers with unequal cost (either for inbound, outbound, or both).

With a good network design, even using Active/Standby scenario network problem due to primary ISP meltdown will not be noticeable to your customers. This is why it is quite essential to have both your Tier 2 ISP to have direct BGP peering with each other; as Rolande brought up regarding Tier 1 ISP preference over Tier 2.
said by ArkiMage :
Experience with Cisco here, none with Juniper

There are good features in Juniper that are not available in Cisco. There are also good features in Cisco that are not available in Juniper. One of the good feature in Juniper that is not available in Cisco is storing multiple configuration in the same Flash memory for easy configuration reverse (or Juniper calls it rollback).

On the other hand, one of the good feature in Cisco that is not available in Juniper is the ACL counter. When you have Cisco equipment as your Internet gateway, ACL counter feature is useful to detect (let's say) DOS attack that strike certain IP address on certain TCP or UDP port number.