dslreports logo
 
    All Forums Hot Topics Gallery
spc
Search similar:


uniqs
6837

yaplej
Premium Member
join:2001-02-10
White City, OR

2 edits

yaplej

Premium Member

Route distribution for dual WAN

Click for full size
Topology
I am in the process of getting a second WAN setup on our network. Currently we have an MPLS WAN that's using BGP to distribute routes. In our primary offices we are distributing those into OSPF. The second network will be Metro-E and Ethernet over Coax MAN so we can run OSPF on it.

With this configuration I would expect that the OSPF routes will take preference by default because the redistributed routes from BGP will be external routes or E2 routes. I am not sure if the E2 routes from BGP will keep the E2 tag through OSPF area 3 to avoid routing loops. So this is our primary concern and am not sure if we should just make OSPF area 3 part of area 0 instead or if there is a better solution.

I was also wondering if we could modify the preferred route for specific traffic and force it to take a route to the MPLS network or to the Metro-E network. Our preference would be to force VoIP traffic to the MPLS network and force the rest to the Metro-E network such as CIFS, IMAP ect.

My first thoughts were to PBR to pass the traffic to either the MPLS or Metro-E router through the 100Mbps link between them but I'm not sure if we can use it conditionally like that so it would only modify the route if both routes are available.

Your thoughts would be much appreciated.

sk1939
Premium Member
join:2010-10-23
Frederick, MD
ARRIS SB8200
Ubiquiti UDM-Pro
Juniper SRX320

sk1939

Premium Member

The issue I see is that since your using two different routing protocols, the tweaking for both is quite different. BGP allows quite a bit of customization as to what it does, but you REALLY better KNOW WHAT your doing before you start to mess with it...sort of like IPv6.

OSPFv2 I know allows for a certain amount of traffic shaping but I don't know whether or not it would meet your needs. Also, BGP by default trusts it's own local programmed routes over anything else, so if you want OSPF to be dominant you'd have to adjust priority. In addition, I'm assuming that both of these routers are located on the same site, as if they are two different sites and your doing site to site VPN you may have additional issues.

If your going to go the PBR route your going to have to setup route maps and configure EVERY protocol that you want where, which if you just want VOIP over DS3 you would allow VOIP protocols followed by a VOIP anything else. Personally though, I'm not sure why you would keep the DS3 line though, as Metro-E should have more than enough bandwidth for both applications, other than for redundancies sake, which you have just relatively eliminated by allowing only voice traffic over the DS3. You also as a security measure should stop the BGP and OSPF routes from propagating out of the others port.

Finally, the setup your using seems to me like it would over-complicate things, why not just use both togather and set priority on the links? Load balancing seems like it would work much better and be less of a hassle to setup, as you could adjust traffic flow as needed, rather than having fixed bandwidth for both. Also, why use OSPF over EIGRP? EIGRP is much easier to setup if your running all Cisco gear, not to mention little extras like built in encryption.
meta
join:2004-12-27
00000

meta

Member

My advice is to get off OSPF as a WAN protocol. Transition entirely to BGP and use its capabilities to acomplish your goals.

yaplej
Premium Member
join:2001-02-10
White City, OR

yaplej

Premium Member

We want to keep both for redundancy in case one goes down we can keep our offices up and running through the other. Also the EoC connections don't have QoS in the network so we want any prioritized traffic to go over the MPLS network where we have more control over it.

BGP is just used to distribute routes in the WAN because it's the only routing protocol the MPLS provider supports without a bunch of extra stuff trying to justify why we need to run something else and honestly we don't need anything BGP works quite well there.

Our Metro-E MAN is just a flat broadcast network so it's my understanding that BGP will not work without configuring each node on that as a neighbor to the others. That seems rather clunky unless BGP can run in some type of broadcast mode.

sk1939
Premium Member
join:2010-10-23
Frederick, MD
ARRIS SB8200
Ubiquiti UDM-Pro
Juniper SRX320

sk1939

Premium Member

BGP is run primarily on IP transit networks, and is really configurable but it is complex to do. If you aren't completely sure what your doing I wouldn't mess with it at all, this is not a CCNA level thing that your doing.

You still haven't answered as to why you are using OSPF over EIGRP, especially given that your area 3 in your diagram should be your area 0. Like I said though, why don't you just utilize both links at the same time rather than limit traffic type on one to the other.

yaplej
Premium Member
join:2001-02-10
White City, OR

yaplej

Premium Member

We are already using BGP on the MPLS network and its working great. I dont plan on changing how we are using that. If I were not so busy right now I would be studying more for the CCNP ROUTE exam honestly it would probably help me a little in this project.

As to why we are using OSPF at DC1 and DC2 is because our CORE equipment does not support EIGRP. So as of right now we have two separate OSPF Area 0 instances one for each "data center". The Metro-E network does not exist yet so we could create a new OSPF area, expand Area 0 or use EIGRP on the Metro-E then redistribute that into OSPF like we are doing with BGP. Creating full meshed iBGP peering in the Metro-E would take a lot of manual configuration and I would rather avoid that.

The reason we want to force VoIP traffic over the MPLS is because of the QoS available in the MPLS network. Its not available on most of the Metro-E network so VoIP could be effected in a negative way. The reason we would want to sent the rest of the traffic to the Metro-E is because of the bandwidth. Its faster so the rest of the traffic can benefit from the speed improving user experience.

So we do plan on using both networks just for specific traffic types depending on the traffic requirements. That's where I think PBR would come into play. I know that I could force all VoIP traffic out one interface or another but I dont know if I could do that only if a route to the destination network exists out that interface.

For example DS1 at OFFICE3 goes down so we dont want to PBR VoIP traffic to the MPLS network for OFFICE3 so the PBRing would need to make sure there was still a route to OFFICE3 out the interface before sending it. Making PBR dependant on the existence of a route to the destination network somehow.
aryoba
MVM
join:2002-08-22

1 edit

aryoba to yaplej

MVM

to yaplej
said by yaplej:

With this configuration I would expect that the OSPF routes will take preference by default because the redistributed routes from BGP will be external routes or E2 routes. I am not sure if the E2 routes from BGP will keep the E2 tag through OSPF area 3 to avoid routing loops. So this is our primary concern and am not sure if we should just make OSPF area 3 part of area 0 instead or if there is a better solution.

You can either stay with OSPF or migrate to BGP. Should you consider to stay with OSPF, the choices are the following

1. Make both DC-1, DC-2, and the point-to-point Metro-E interconnecting DC-1 and DC-2 as the same Area 0

2. Assign the Metro-E as the Area 0 where DC-1 is non-Area 0 (i.e. something like Area 1 for DC-1 and Area 2 for DC-2).

With either case, make sure that each DC-1 and DC-2 must have different BGP AS domain ID. Otherwise you can't use the MPLS as backup in case the Metro-E breaks.

As meta See Profile mentioned, migrate to BGP would be simpler solution.
aryoba

2 edits

aryoba to yaplej

MVM

to yaplej
said by yaplej:

Creating full meshed iBGP peering in the Metro-E would take a lot of manual configuration and I would rather avoid that.

You don't have to create full-meshed iBGP peering since you can use Route Reflectors or Confederations when necessary.
said by yaplej:

So we do plan on using both networks just for specific traffic types depending on the traffic requirements. That's where I think PBR would come into play. I know that I could force all VoIP traffic out one interface or another but I dont know if I could do that only if a route to the destination network exists out that interface.

Personally I would stay away from PBR since in the long run it could be very messy business and un-scalable solution. Should you stay with OSPF, you could implement multiple areas where one for data and another for voice. Should you decide to migrate to BGP, you could implement more BGP AS where one for data and another for voice or implement multiple OSPF domains as IGP when one OSPF domain for data and another OSPF domain for voice.
aryoba

1 edit

aryoba to yaplej

MVM

to yaplej
said by yaplej:

Our Metro-E MAN is just a flat broadcast network so it's my understanding that BGP will not work without configuring each node on that as a neighbor to the others. That seems rather clunky unless BGP can run in some type of broadcast mode.

With any types of network (broadcast, point-to-point, or non-broadcast), there has to be BGP session between BGP neighbors otherwise it won't work. Which router will be the BGP neighbors and if the neighbor relationship would be iBGP or eBGP, that depend on how you design the network.
meta
join:2004-12-27
00000

meta

Member

There are alot of options to design a network with the topology described above.

Unless this is a pure lab exercise, i strongly suggest running BGP and NOT ospf or eigrp for WAN connectivity. That was a lesson learned by most companies back in the early 2000's with their L2 frame-relay networks.

Should you choose otherwise, my rate starts at 55$ an hour and i expect untangling a mess like that to be at least 2 months. Please have your management budget accordingly.

;P

yaplej
Premium Member
join:2001-02-10
White City, OR

yaplej

Premium Member

Thanks for the tips on BGP. Iv only used BGP very little for our MPLS network. Never read about a Route Reflector or Confederation.

I assume I should make DC1-ROUTER2 and DC2-ROUTER2 Route Reflectors and configure all the nodes on the Metro-E to peer with those two routers. That would be a lot easier than fully meshing 40+ nodes. Bleh.

Basically the Metro-E network and the link between ROUTER1 and ROUTER2 would all be iBGP ie same (private) AS number 65001.

The MPLS network uses our providers AS so I should be able to route voice out using that assuming a route exists and data using the private AS assuming at route exists there.

How would do you prioritise what AS to use for particular class of traffic though?
meta
join:2004-12-27
00000

meta

Member

First of all, using the same ASN in this case isnt particularly viable for path redundancy. You will need a different ASN for every site, and every datacenter. In doing so, each remote site will have to peer with the MPLS network, and both datacenters.

Due to the topology this will create, the primary path will always be via the metro ethernet link, and use the MPLS network as a backup path. This will create a hub and spoke topology between the remote sites and head-end sites.

sk1939
Premium Member
join:2010-10-23
Frederick, MD

sk1939

Premium Member

If there is anyone who knows about BGP here, nosx is the one to ask. I deal more with switching nowadays than routing, and since we don't do transport, precious little do I do with BGP.
aryoba
MVM
join:2002-08-22

aryoba to yaplej

MVM

to yaplej
said by yaplej:

The MPLS network uses our providers AS so I should be able to route voice out using that assuming a route exists and data using the private AS assuming at route exists there.

How do you mean by saying that the MPLS network uses your provider AS? Posting your MPLS network topology with associated BGP AS and some description would help clarify things.
said by yaplej:

I assume I should make DC1-ROUTER2 and DC2-ROUTER2 Route Reflectors and configure all the nodes on the Metro-E to peer with those two routers. That would be a lot easier than fully meshing 40+ nodes. Bleh.

Basically the Metro-E network and the link between ROUTER1 and ROUTER2 would all be iBGP ie same (private) AS number 65001.

As meta See Profile mentioned, it would be simpler when each site has its own AS. You could have the same AS for both DC-1 and DC-2, however it would complicate things.
said by yaplej:

How would do you prioritise what AS to use for particular class of traffic though?

We can start discussing that when agreeable BGP network design is in place
aryoba

aryoba to sk1939

MVM

to sk1939
said by sk1939:

I deal more with switching nowadays than routing, and since we don't do transport, precious little do I do with BGP.

(LAN) Switching is fine, although dealing and troubleshooting Spanning Tree issue could cause headaches. With routing, the worst you can have is a routing loop or lost (blackholed) routing. With switching, the network shuts down when there is a Layer-2 loop
aryoba

aryoba to meta

MVM

to meta
said by meta:

Should you choose otherwise, my rate starts at 55$ an hour and i expect untangling a mess like that to be at least 2 months. Please have your management budget accordingly.

;P

Rate of $55/hr (assuming that is the number you get while your company charges more) is still low. I know some companies charge $300/hr easy for this kind of work

yaplej
Premium Member
join:2001-02-10
White City, OR

yaplej

Premium Member

Click for full size
BGP Overview
We peer to our MPLS network using ASN 65001 our neighbor AS is the providers public ASN ### in the diagram. We use the same private ASN for all our sites.

At DC2 we also add a community string that lowers the priority of the routes sourced from that site in the MPLS network. This is because we advertise a default route from both DC1 and DC2 with DC1 being the primary. Only the providers network uses the community our other equipment does nothing with it.

Here is our BGP config from DC2. I know its bad to redistribute an entire routing protocol but I just have not gotten around to cleaning it up. I will remove that and add network statements for each network I need to advertise eventually.
router bgp 65001
 no synchronization
 bgp log-neighbor-changes
 network 172.31.9.1 mask 255.255.255.255
 redistribute ospf 1 match internal external 1 external 2
 neighbor ab.abc.bcd.161 remote-as ###
 neighbor ab.abc.bcd.161 send-community
 neighbor ab.abc.bcd.161 route-map bgpcomm out
 default-information originate
 distance bgp 115 115 115
 no auto-summary
route-map bgpcomm out
route-map bgpcomm permit 10
 match ip address bgpcomm10
 set community ###:90
ip access-list extended bgpcomm10
 permit ip any any
 

I think there are 1024 private ASN so we could go around and assign each site one of those private ASNs. It would take a little time but our MPLS provider wont have any issues with that. Ill just need to get them on the phone to reconfigure their end of the BGP association between our CPE and their edge router for each site.

From what I have read about route reflectors it would turn our full meshed Metro-E network into a Hub-and-Spoke network because all the routes would be sourced from either DC1 or DC2. I would like to keep the full mesh capability if possible on the Metro-E network but it does not look like that's possible with BGP unless you explicitly configure fully meshed BGP peering between all 40+ of the nodes again Bleh. Maybe I misunderstood how the route reflector server and clients work.

sk1939
Premium Member
join:2010-10-23
Frederick, MD

sk1939 to aryoba

Premium Member

to aryoba
Tell me about it, Span Tree can be a pain to troubleshoot...and combine that with my predecessor who believed that a 3750 is a 48 port router...its taken a loooong time to finally sort things out.

Sorry for the thread-jack.
meta
join:2004-12-27
00000

meta to yaplej

Member

to yaplej
We use the same private ASN for all our sites.
^--so that is the first major issue. That is going to create years of routing headaches as a network grows. From tracking down where a prefix is originating from, to multipath loop prevention, that decision can and should be changed.

Every site, every datacenter should have a unique BGP ASN assigned. Furthermore, aAll the routing devices within the site (wan, core, distribution, etc) should participate within iBGP within the site.

With that topology, redistribution is unnecessary. All endpoint prefixes will be carried in BGP, and OSPF is just used for loopbacks and transit interfaces within a given BGP AS.

Route-reflectors can be configured within sites on edge routers (in this case wan routers) to reduce the number of configured neighbors. This permits very large scalability both within a site and across the network.

A routing topology like this with a single protocol carrying all of the prefixes will allow for granular traffic management, uniform routing policy, and ease of support & troubleshooting.

There are some bad habbits in the bgp configuration posted as well. Manually setting the distance should be avoided, default-info orig should be off as the default route should be injected by your upstream internet gateway, and redistribution should be avoided. The community sent looks like it is changing the local preference of everything advertised to 90, which is odd. Workarounds like setting admin distance, local preference, and redistribution make this look like a network in need of redesign.

There are alot of topology and configuration options available but to make the right choices you need to understand where the traffic is coming from and going to. For example, what percentage of the traffic is between remote sites, as opposed to the datacenters?

yaplej
Premium Member
join:2001-02-10
White City, OR

yaplej

Premium Member

said by meta:

We use the same private ASN for all our sites.
^--so that is the first major issue.

Ok so that will be my first step to assign a unique private ASN to each site.
said by meta:

All the routing devices within the site (wan, core, distribution, etc) should participate within iBGP within the site.

With that topology, redistribution is unnecessary. All endpoint prefixes will be carried in BGP, and OSPF is just used for loopbacks and transit interfaces within a given BGP AS.

Our "CORE" switches cannot run BGP only OSPF. I can run BGP between the WAN routers but thats about it. I would prefer not to redistribute also but just have not gotten around to fixing that.
said by meta:

Route-reflectors can be configured within sites on edge routers (in this case wan routers) to reduce the number of configured neighbors. This permits very large scalability both within a site and across the network.

Correct me if I am wrong but this does turn a full meshed network into a hub-and-spoke network yes? If SITE 4 neeeds to route to SITE 5 will it route back to DC1 or DC2 first then to its destination? When I read about a route reflector thats how it sounded to me.
said by meta:

A routing topology like this with a single protocol carrying all of the prefixes will allow for granular traffic management, uniform routing policy, and ease of support & troubleshooting.

Sounds good.
said by meta:

There are some bad habbits in the bgp configuration posted as well. Manually setting the distance should be avoided, default-info orig should be off as the default route should be injected by your upstream internet gateway, and redistribution should be avoided.

Our Internet edge is just a static route right now. The firewall does not run BGP and is running OSPF with the CORE switches. Default-info-orig gives a default route to the MPLS and the rest of our sites. That might need some work I can peer with our Internet provider with BGP and they will advertise a default route to us. Still have the problem with lack of BGP support at our core.
said by meta:

The community sent looks like it is changing the local preference of everything advertised to 90, which is odd.

This is to lower the preference of the default route coming from DC2 into the MPLS network. So outbound internet traffic will default to DC1. Its how our MPLS provider said to configure it. They have a policy that community 90 is a lower preference. The reason for this might be gone now.
said by meta:

Workarounds like setting admin distance, local preference, and redistribution make this look like a network in need of redesign.

The admin distance setting needs to be removed. Thats something residual from before we had a separate CORE and WAN layer.
said by meta:

There are alot of topology and configuration options available but to make the right choices you need to understand where the traffic is coming from and going to. For example, what percentage of the traffic is between remote sites, as opposed to the datacenters?

Most traffic is from remote sites to DC1 or DC2 there is a considerable volume of traffic between DC1 and DC2. Most traffic between sites is VoIP traffic.

Phew giving my fingers a workout.
meta
join:2004-12-27
00000

meta

Member

I think we need a whiteboard session ;P

Regarding the any-to-any vs hub-and-spoke topology, both can be constructed with caveats in an L2 wan scenario. While the BGP peering relationships form a hub/spoke topology with route reflectors, traffic can still flow from spoke to spoke directly with recursive next-hop lookup. Data doesnt necessarily have to flow along the same lines as the neighbor relationships.

Regarding local preference, usually if you want to make prefixes secondary, its easier to use a transitive BGP attribute like as-path. In your case, it would be simpler by performing a simple prepend rather than a throw-catch scenario with community values trusting the PE to manipulate local preference accordingly.

Is the bgp-free-core an issue with licensing or platform?

I think it would be beneficial to draw a detailed visio including the internet, l3 wan, l2 wan, both "datacenters", and a sample remote site. Let me know when you have time on skype and ill start scribbling.

yaplej
Premium Member
join:2001-02-10
White City, OR

yaplej

Premium Member

If using a route reflectors still allows any-to-any traffic flows then that sounds like a great solution. Its so obvious once you know about it.

If prepending the as-path would be easier Im all for it. I think maybe that's something to address after reassigning new private ASNs to each of our sites.

The bgp-free-core is a platform issue the core switches are Dell PC6248's and Dell has not found it in their interest to implement BGP on them yet. Its a nice cheap, fast platform but lacking some of the more advanced features.

Today has been pretty busy trying to implement GETVPN on our MPLS network. Very odd issue... for another post. I will work on the diagrams and trying to indicate all the stupid stuff I done.
meta
join:2004-12-27
00000

meta

Member

What hardware are you running? Are all the WAN routers fairly recent? If this isnt a mostly legacy network there are alot of options for secure multipoint communication over both WAN clouds.

yaplej
Premium Member
join:2001-02-10
White City, OR

yaplej

Premium Member

Click for full size
Overview

Office
 
The routers are all 2811's and 7206VXRs at each DC. I am about done getting GETVPN setup though and it should work over both WAN clouds with very little changes. Just add the crypto map on the second WAN facing interface and be done with it.

I created this network overview for you. Keep in mind that Metro-E does not exist yet. The EDGE routers have a static default route to the Internet and are the source for default-info-orig to the rest of the OSPF area. I have talked with our provider and we can receive a default route from them using BGP.

ROUTER1 is also a source for default-info-orig in OSPF and BGP. The BGP admin distance was increased to prevent a default route loop so the default route from OSPF will be preferred.

The CORE switches do not support BGP maybe in a future update or if I can get new switches.

The remote sites are super simple just a 2811 and a few switches.

Yes, its sad but thats one big OSPF Area 0.
aryoba
MVM
join:2002-08-22

1 edit

aryoba

MVM

said by yaplej:

ROUTER1 is also a source for default-info-orig in OSPF and BGP. The BGP admin distance was increased to prevent a default route loop so the default route from OSPF will be preferred.

Are you referring to the DC1-ROUTER1 and DC2-ROUTER1 when you say ROUTER1 that has OSPF default-information origin?

I wonder why you need such OSPF default-information origin command on the ROUTER1. Is the BGP MPLS providing default gateway from somewhere? Maybe a default gateway in form of static route configuration on the ROUTER1 to provide reachability to other sites?
said by yaplej:

The CORE switches do not support BGP maybe in a future update or if I can get new switches.

Since your network is small, you could get away without CORE switches running BGP. Running BGP on the DC ROUTERs and EDGEs should be fine at this point.

What are the CORE switches anyway? Non-Cisco switches?
said by yaplej:

The remote sites are super simple just a 2811 and a few switches.

What are the switches? Catalyst 2950?
said by yaplej:

Yes, its sad but thats one big OSPF Area 0.

The OSPF Area 0 of DC-EDGE, DC-ASA; and the OSPF Area 0 of DC-CORE, DC-ROUTER; are they the same Area 0? No redistribution between the two Area 0?

yaplej
Premium Member
join:2001-02-10
White City, OR

yaplej

Premium Member

said by aryoba:

Are you referring to the DC1-ROUTER1 and DC2-ROUTER1 when you say ROUTER1 that has OSPF default-information origin?

Yes. DC1-ROUTER1 and DC2-ROUTER1 are both running default-info-orig in both OSPF and BGP.

They provide a default route to/from the local OSPF network and to/from the BGP network. So if the default route from OSPF is lost ROUTER1 will advertise the default route coming from the other site from BGP to the local OSPF network so devices can still get out to the Internet through the remote DC.
said by aryoba:

What are the CORE switches anyway? Non-Cisco switches?

They are Dell PC6248s.
said by aryoba:

What are the switches? Catalyst 2950?

Yes, the office switches are all 2950s.
said by aryoba:

The OSPF Area 0 of DC-EDGE, DC-ASA; and the OSPF Area 0 of DC-CORE, DC-ROUTER; are they the same Area 0? No redistribution between the two Area 0?

Yes, they are the same area no need for redistribution between them.
aryoba
MVM
join:2002-08-22

aryoba

MVM

said by yaplej:

said by aryoba:

Are you referring to the DC1-ROUTER1 and DC2-ROUTER1 when you say ROUTER1 that has OSPF default-information origin?

Yes. DC1-ROUTER1 and DC2-ROUTER1 are both running default-info-orig in both OSPF and BGP.

They provide a default route to/from the local OSPF network and to/from the BGP network. So if the default route from OSPF is lost ROUTER1 will advertise the default route coming from the other site from BGP to the local OSPF network so devices can still get out to the Internet through the remote DC.

I don't think this is a good design since the "default route" pointing through the MPLS BGP network is not really default gateway. A default gateway should be pointing to outside network (i.e. the Internet) or be pointing to non-local network. Remote sites' IP addresses are not outside network, they are still "known" networks.

Are you doing BGP peering with the MPLS provider routers? If yes, then you should receive BGP route advertisement from them regarding networks of remote sites. When this is the case, you should never put static routes on the ROUTERs to reach remote sites since the ROUTERs receive the remote site IP addresses dynamically via BGP. In other words, you would only receive default gateway (the IP address of 0.0.0.0) from the ISP router (iNet).
aryoba

aryoba to meta

MVM

to meta
said by meta:

I think we need a whiteboard session ;P

I would agree

But no matter, I think we are moving forward to the right direction. There are a lot of network design issues to clean up, however I'm confident we can help you dealing with them and keeping up the good work. So far I think you've done a good job

yaplej
Premium Member
join:2001-02-10
White City, OR

yaplej to aryoba

Premium Member

to aryoba
said by aryoba:

I don't think this is a good design since the "default route" pointing through the MPLS BGP network is not really default gateway. A default gateway should be pointing to outside network (i.e. the Internet) or be pointing to non-local network. Remote sites' IP addresses are not outside network, they are still "known" networks.

Its probably not the best way to do it but if our Internet connection goes down it was how I could get the Internet traffic routed to the other connection. There is a default route out both sites.
said by aryoba:

Are you doing BGP peering with the MPLS provider routers? If yes, then you should receive BGP route advertisement from them regarding networks of remote sites. When this is the case, you should never put static routes on the ROUTERs to reach remote sites since the ROUTERs receive the remote site IP addresses dynamically via BGP. In other words, you would only receive default gateway (the IP address of 0.0.0.0) from the ISP router (iNet).

DC1-ROUTER1, DC2-ROUTER2 and OFFICE routers all peer with the MPLS provider routers. The only place we have static routes is the EDGE routers pointing to the Internet (iNet).

When distributing OSPF into BGP and BGP into OSPF at DC1-ROUTER1 and DC2-ROUTER1 the default route was not being included. That's why I had to turn on default-info-orig on each of routing protocol so when one of the Internet connections went down Internet traffic would begin flowing through the MPLS network to other site and out the other Internet connection. Tested this several times actually.
aryoba
MVM
join:2002-08-22

aryoba

MVM

said by yaplej:

said by aryoba:

I don't think this is a good design since the "default route" pointing through the MPLS BGP network is not really default gateway. A default gateway should be pointing to outside network (i.e. the Internet) or be pointing to non-local network. Remote sites' IP addresses are not outside network, they are still "known" networks.

Its probably not the best way to do it but if our Internet connection goes down it was how I could get the Internet traffic routed to the other connection. There is a default route out both sites.
said by aryoba:

Are you doing BGP peering with the MPLS provider routers? If yes, then you should receive BGP route advertisement from them regarding networks of remote sites. When this is the case, you should never put static routes on the ROUTERs to reach remote sites since the ROUTERs receive the remote site IP addresses dynamically via BGP. In other words, you would only receive default gateway (the IP address of 0.0.0.0) from the ISP router (iNet).

DC1-ROUTER1, DC2-ROUTER2 and OFFICE routers all peer with the MPLS provider routers. The only place we have static routes is the EDGE routers pointing to the Internet (iNet).

When distributing OSPF into BGP and BGP into OSPF at DC1-ROUTER1 and DC2-ROUTER1 the default route was not being included. That's why I had to turn on default-info-orig on each of routing protocol so when one of the Internet connections went down Internet traffic would begin flowing through the MPLS network to other site and out the other Internet connection. Tested this several times actually.

If I'm not mistaken, it sounds like you need to do manual configuration on DC-ROUTERs when there is an outage, which include static route (default gateway) implementation and have OSPF on DC-ROUTERs announce it using the default-information originate command. Am I correct?

Some clarification; what is the MPLS BGP AS number? Is it the same 65001 as your current DC1 AS number? Or else?

The Metro-E fiber between DC1 and DC2; and the EoC to remote offices are not in place yet, correct?