Equipment Support > Hardware By Brand > Cisco > BGP Question

BGP Question

Im not Rolande but I have a suggestion for you

You could advertise all of your routes to the 3640 from your 7200's and prepend a couple of extra AS hops to the routes it advertises back out to the opposite 7200, that way BGP can always remain active, but your main links will take the traffic because they have the "shortest/preferred path". In the event that one of your main links go down the next shortest path which happens to be via your 3640 will kick into action.

Hopefully that helps, and I may be wrong, so hopefully Rolande can help out too

I think at some point that (or some version of it) has been tried. Currently the 3640 has 7 AS prepends on it, but apparently in the past it's advertisements would be taken as the preferred path by the upstream carrier there, and throw off the works. I wasn't around for those days, so I can't speak accurately to specifics.

One thing I've looked at is the Multi Exit Discriminator attribute. If I understand my reading correctly, I *should* be able to set metrics on the advertisements from the 7200s of 100, and a metric of 500 on the advertisements from the 3640 of something higher, say 500.

In theory, I think that should force BGP to select the two main links as preferred, and if the BGP sessions to there drop, the higher metric link of the backup site would switch.

Still reading on this tho, I could be wrong.
--
Find me on WoW Realm Eonar....just don't tell my boss that's why I ordered that extra T1......

reply to SkellBasher
Sounds like a plan. Changing the route metrics could also work.

Perhaps another method would be to use EIGRP between the 7200's and the 3640. EIGRP routes have a higher (less preferable) metric than BGP so it could also work. 90 vs 20 respectively IIRC.

reply to SkellBasher
One of the good ( or bad things depending how you look at it ) with BGP is the adaptability it has. With BGP there are several ways to "tweak" attributes to get desired and optimal routing.

Now it seems you already understand the aspect of the 3640 needing to be advertising the less preferred routes. There are a few ways to do this. Some of which you already brought up.

1) Shortest AS path

2) Orgin ID ( IGP>EGP>? )

3) MED

You mentioned that AS path was "already tried" with no luck. If for whatever reason the AS prepending didn't work. My next suggestion would be to go the MED route as you talked about. Something to remember. By default Cisco routers will mark routes received with a MED of 0. However even if you configure the sending router to give a MED of say 500. The receiving router still possibly wont chose this as the "worst link" you have to enable the command "bgp always-compare-med" on all BGP speaking routers. Hope this helps some

DP
--
"Knowledge is contagious, infect"

reply to SkellBasher
This scenario can work for you. It does take time for new routing announcements to propagate across the Internet. I think you are looking at a minimum 3-5 minute reconvergence delay.

With that said, there are tons of ways you can tune BGP to do what you want. Personally, if I were you, I would fire up BGP and leave it active all the time with your Disaster Recovery provider. Just use a route-map to prepend your AS a ton of times (many more than 7) at the beginning of your advertisements. This way, when both of your main links fail, your backup routes will start being selected much faster than if you had to wait for a new advertisement to converge. The backup BGP routes would already exist across every Internet router. They would only be selected in the event that the other 2 routes are removed from BGP if your primary BGP peers are disconnected.

There are other things you can do as people have listed here, but the most effective solution is AS prepending on the Internet. Many providers do not accept MEDs. Even if they did, that can be much more difficult to manage. MEDs typically only work if all paths have a positive MED value. The majority of providers will not enable 'always-compare-med'. Not to mention you can't control what they do with their BGP processes. They could enable the 'best-as-worst' MED option and then you'd be screwed.

I would adjust the AS prepends to a much higher number to solve the problem you saw before. You need to figure out how many AS hops your backup provider sees to your primary routes. It must be 7 or more. I would just bump your AS prepends up to like 15 at least. That should solve the problem.

If you want, post the 3640's config minus passwords and mask any identifiable config and subnets etc.
--
Ignorance is temporary...stupidity lasts forever!

Before I forget, thank you everyone for your comments and advice, much appriciated!!

The AS prepend option seems to be the best. At present it's this (And I was wrong, it was 8 prepends):

route-map Choice1-out permit 10
match ip address NOC-Prepend
set as-path prepend 7321 7321 7321 7321 7321 7321 7321 7321

So I'll bump it up to 15 and see what happens. MED seems to be out of the question, as I was told by two of the three carriers in question that they may accept the MED attribute, and the third didn't even know what I was talking about, so I left it alone. It would seem that MED, although great in theory, is one of those options that is not practical in production use for manageability reasons.

I'm working on getting access from my disaster recovery carrier to see how many AS Hops they see to my main site. I don't get the full BGP route table there, I only advertise, so I can't look there.

**Update. I'm only allowed to do 10 AS prepends, then I error out. Restriction of the hardware or IOS?

For reference:

Cisco Internetwork Operating System Software
IOS (tm) 3600 Software (C3640-I-M), Version 12.2(28), RELEASE SOFTWARE (fc5)
Technical Support: »www.cisco.com/techsupport
Copyright (c) 1986-2005 by cisco Systems, Inc.
Compiled Thu 24-Feb-05 15:52 by kellmill
Image text-base: 0x60008930, data-base: 0x60A3E000

ROM: System Bootstrap, Version 11.1(20)AA2, EARLY DEPLOYMENT RELEASE SOFTWARE (fc1)
ROM: 3600 Software (C3640-I-M), Version 12.2(28), RELEASE SOFTWARE (fc5)

XXXXXXX uptime is 8 weeks, 4 days, 22 hours, 20 minutes
System returned to ROM by reload at 17:38:43 EST Fri Mar 5 1993
System image file is "flash:c3640-i-mz.122-28.bin"

cisco 3640 (R4700) processor (revision 0x00) with 98304K/32768K bytes of memory.
Processor board ID 27581893
R4700 CPU at 100Mhz, Implementation 33, Rev 1.0
Bridging software.
X.25 software, Version 3.0.0.
2 Ethernet/IEEE 802.3 interface(s)
2 FastEthernet/IEEE 802.3 interface(s)
DRAM configuration is 64 bits wide with parity enabled.
125K bytes of non-volatile configuration memory.
8192K bytes of processor board System flash (Read/Write)

Configuration register is 0x2102
--
Find me on WoW Realm Eonar....just don't tell my boss that's why I ordered that extra T1......

reply to rolande

Network Diagram

Post a copy of the whole BGP config as well as route-maps and/or relevant ACLs or prefix lists etc. on your backup router and mask any identifiable information like networks and AS numbers etc. You must have something wrong in your prepend config for it to not properly advertise.

Also, BGP doesn't broadcast anything. It only sends and receives update messages that describe the routes in the router's BGP Routing Information Base or RIB. BGP does take time to update new or changed attributes on its routes. You may have to do a 'clear ip bgp * soft out' to kick it in the butt so it walks the RIB immediately and sends out any updates right away.
--
Ignorance is temporary...stupidity lasts forever!

Here's the BGP configs on the backup router:

router bgp AAAA
no synchronization
no bgp log-neighbor-changes
neighbor XXXX remote-as ZZZZ
neighbor XXXX ebgp-multihop 4
neighbor XXXX password YYYY
neighbor XXXX next-hop-self
neighbor XXXX route-map Choice1-out out

ip access-list standard NOC-Prepend
permit BBBB 0.0.31.255

route-map Choice1-out permit 10
match ip address NOC-Prepend
set as-path prepend [AS # 10 times]
--
Find me on WoW Realm Eonar....just don't tell my boss that's why I ordered that extra T1......

reply to SkellBasher
Just something to add,If your seeing both routes in the BGP table but not the routing table. You need to use "maximum-paths" command to have both routes in the routing table.

DP
--
"Knowledge is contagious, infect"

reply to SkellBasher
Where are you originating the route advertisement from? I see no network statement in this router for the route you are advertising. Are you running iBGP and advertising it on another router internally or are you redistributing the route from an IGP protocol somewhere?
--
Ignorance is temporary...stupidity lasts forever!

Sorry, I took that line out of the running config. Once I add it back, the problem of this router's advertisments becoming the only ones seen externally happens.

When I have it there, it's in the normal form:

network BBBB mask 255.255.224.0
--
Find me on WoW Realm Eonar....just don't tell my boss that's why I ordered that extra T1......

Have you gone to a route server and done a sh ip bgp for your prefix(es) when you advertise the backup route? I'd be curious to see if your backup provider is stripping your AS path somehow or else something really strange is going on with your advertisement to that provider.
--
Ignorance is temporary...stupidity lasts forever!

I was using a Telus route server to watch changes. Right now, with the backup router isde (as far as BGP is concerned):

route-views.on>sh ip bgp XXX
BGP routing table entry for XXX, version 6810612
Bestpath Modifiers: deterministic-med
Paths: (1 available, best #1)
Not advertised to any peer
852 3356 3549 [MY AS x1]
154.11.63.85 from 154.11.63.85 (154.11.0.148)
Origin IGP, localpref 100, valid, external, best
route-views.on>

BGP routing table entry for XXXX, version 6829081
Bestpath Modifiers: deterministic-med
Paths: (1 available, best #1)
Not advertised to any peer
852 7911 13609 [MY AS x11]
154.11.63.85 from 154.11.63.85 (154.11.0.148)
Origin IGP, localpref 100, valid, external, best
Dampinfo: penalty 482, flapped 1 times in 00:00:52
route-views.on>

So, I am presuming that the 10 prepends are being advertised out correctly, as I'm seeing 11 in the path from the route server external to everybody. But, what I can't figure out is why I only see that one route. Everything that I have read leads me to beleive that this longer route should be added as a backup route, as you said, leaving the primary, shorter AS Path untouched.
--
Find me on WoW Realm Eonar....just don't tell my boss that's why I ordered that extra T1......

So the prefix is showing up with the proper prepending in that particular route server's BGP table. When you say that you "only see that one route" explain that a little better. Where are you only seeing that one route? What providers are selecting the path to your backup provider as best for that route that clearly has a much longer AS path? Where have you tested from to validate this result? Can you show the route and its corresponding BGP table entry?

On your router at the backup site, you will never learn the valid prefix from your primary BGP peers, as long as you are advertising the same prefix. BGP will see the prefix announcement originating from the same AS across an eBGP peer so it believes it is a routing loop and it will not accept it inbound from your backup provider. As long as your backup router is in the same AS as your primary BGP peers it will never learn any prefixes advertised from those routers since they are in the same AS.

Technically, all routers within the same AS must have connectivity to each other via iBGP if they need to all see the same routes. In your case, this is a backup site, so you should not care about connectivity from there to your primary location using this duplicate address space.
--
Ignorance is temporary...stupidity lasts forever!

I have been assuming (most likely incorrectly) that when viewing my prefix from external route servers , I should see two paths listed, one showing the 'normal' path through the ASes connected to the NOC, and a 2nd showing the extended path to the backup site through the backup AS. Im guessing that this is not the case?

Im starting to get some differing information from the parties involved here, so I'm not going to go much further with this until I can verify everything. In additon, I think I'm a little over my head with this, and want to understand this more before I continue playing with it.
--
Find me on WoW Realm Eonar....just don't tell my boss that's why I ordered that extra T1......

You are correct if no attributes are being modified on your advertisements by any of your providers. However, this looks strangely like a local preference issue here. The only problem is I am assuming you are checking remote route servers that are not in the same AS as your backup provider. That would eliminate that as an option.

Default local preference for all routes in BGP is 100. If you manually tune the local pref higher, then all routers within that iBGP process will prefer that particular route. The actual symptom you will see on the router is that only the BGP path with the higher local pref will actually be injected into the BGP RIB on all iBGP routers, let alone into the routing table.

Anyway, this gets more confusing as we go further. I may need to see more detail offline to actually isolate the problem.
--
Ignorance is temporary...stupidity lasts forever!

One thing that may or may not apply, and this is something I was recently schooled on on a mailing list is this:

Many providers (a surprising amount, judging by my netflow stats) are buying transit from Cogent, WIlliams and other "bargain basement" providers. If they see that they can reach you via say a Level3 or UUNet, but they also see you via their (cheaper) Williams or Cogent pipe, guess which one they pick?

Even if you prepend the bejesus out of your announcements, if one of your upstreams is buying transit from one of these bottom feeders, no amount of prepending will help. Dig around a bit and you'll find a big, old-school provider that is "transit free" does some very interesting stuff to shove as much traffic as possible off to Cogent.

It would be helpful to know who transit provider #3 is. From my limited view, I only see the original poster's AS via 1239 (but via three different paths). I haven't gone route-server hunting on it though...

This would only explain the symptom if by luck the route server he is checking happens to be using local preference to prefer the routes via the cheap transit provider. Otherwise, how would you explain the primary paths not even showing up in the BGP RIB when they advertise the backup path?

That is a very evil thing to do to customers or even transit non-customer routes. They should set MEDs instead of local pref so at least all paths continue to be advertised. Squashing the "backup" advertisement is not an Internet friendly thing to do.
--
Ignorance is temporary...stupidity lasts forever!