 Need BB join:2001-12-21 Rochester, NY | MPLS VPN I am working on setting up MPLS to get a better understanding of how it works. I am currently designing the network in emulation before moving it over to the real hardware of 3725, 2600xm, and 1760 routers.
I currently have an L3 VPN setup via MPLS between the two 1700 routers but I am confused about the VPN setup. Will this setup allow the two 1700 routers to share the same subnet so computer at one side can connect to the computers on the other or do the networks have to be separate?
Additionally I would like the two 1700 routers to also access the internet (via a connection from one of the 3700 routers) besides just a having VPN between each over. How would I go about setting this up?
Thanks |
|
 nosx join:2004-12-27 00000 kudos:5 | As to the first question, no, not with an L3 MPLS VPN. however if you setup EPLS (pseudowire crossconnects), VPLS (vfi crossconnects), you can deliver devices on the same subnet across an MPLS wan. However this requires real high end equipment (think 7600s or GSR12400s)
As to the second question, there are ALOT of ways to deliver internet service into an L3 MPLS VPN. I would go with one of the following: 1) per-vpn firewall contexts on a separate security device (asa/fwsm). this is how major providers would go about doing it. 2) per-vpn nat pool with internet in the global routing table, i can provide a config example if u want to go this direction. Remember that both of these approaches need to be done on a PE router rather than a P router. (PE vs PCORE) as the p-cores have no knowledge of VPNs or customer routing/bridging other than the outer most labels (transport labels) in the stack.
Anyways, I build MPLS aware services relatively often so if you have any specific questions i can certainly help. The easiest thing might be for you to zip up txt files of all your running configs and we can have a conversation with more specific configuration examples.
Good luck, have fun, mpls is wonderful stuff! |
|
 Need BB join:2001-12-21 Rochester, NY | Great that explains a great deal. I didn't know that L2VPN/VPLS require higher end hardware so it looks like I have to use L3 VPN. I don't have access to a firewall so I would greatly appreciate if you could post the vpn nat pool configs.
The goal of the project is to experiment with MPLS VPN, QOS, and MPLS TE. I attached my network diagram and configs. The mpls core is not inter-meshed since all the traffic will eventual go through a Cisco ONS OC3 optical transport link between each side (rochester to boston).
Any insight into my current setup would be greatly appreciated since I am just following some Cisco Press books.
Thanks |
|
 nosx join:2004-12-27 00000 kudos:5 2 edits | Im going to spend some time reading into your config more when i have time. Below is a config example, but as i said before, its for a PE and there are MANY different ways to accomplish this kind of thing. It really depends on how many customers you have, and how complex their networks are, and if there is a good deal of address space overlap etc.
ip vrf cust1
rd 1:1
route-target export 1:1
route-target import 1:1
ip vrf cust2
rd 1:2
route-target export 1:2
route-target import 1:2
ip vrf cust3
rd 1:3
route-target export 1:3
route-target import 1:3
ip vrf services
rd 65000:1
route-target export 65000:1
route-target import 65000:1
interface FastEthernet0/0.3
ip vrf forwarding services
ip nat enable
no ip route-cache
interface FastEthernet0/0.210
encapsulation dot1Q 210
ip vrf forwarding cust1
ip address 10.3.210.1 255.255.255.0
ip nat enable
no ip route-cache
interface FastEthernet0/0.220
encapsulation dot1Q 220
ip vrf forwarding cust2
ip address 10.3.220.1 255.255.255.0
ip nat enable
no ip route-cache
ip route vrf cust1 0.0.0.0 0.0.0.0 FastEthernet0/0.3 10.3.0.1
ip route vrf cust2 0.0.0.0 0.0.0.0 FastEthernet0/0.3 10.3.0.1
ip route vrf services 0.0.0.0 0.0.0.0 10.3.0.1
ip nat pool NAT 10.3.0.61 10.3.0.61 netmask 255.255.255.0
ip nat source list 1 pool NAT vrf cust1 overload
ip nat source list 1 pool NAT vrf cust2 overload
access-list 1 permit 10.3.210.0 0.0.0.255
access-list 1 permit 10.3.220.0 0.0.0.255
If you are trying to do a L2 MPLS VPN between exactly two locations, you could try to get an ethernet pseudowire working. I can give you some config for that, but since L2VPN is basically hardware specific, you need to login to the router and run a show command:
show mpls l2transport hw-capability interface $INTERFACE
where the interface specified is the one facing the L2 network on either side, such as FA0/0 that would connect to a LAN that was to be spanned across the WAN. I have successfully crossconnected old ATM and frame equipment, so ethernet isnt the only protocol supported. |
|
 Need BB join:2001-12-21 Rochester, NY | Great! No rush... when ever you get the chance to look over the configs it would be greatly appreciated!
Thanks |
|
 nosx join:2004-12-27 00000 kudos:5 | I do have two real quick questions that arent so related to MPLS VPN so much as P/PE functionality.
Why do you have network 0.0.0.0 255.255.255.255 in the OSPF config? Usually I just throw "ip ospf 1 area 0" on loopbacks and transits. This is useful as point to point interfaces are frequently unnumbered (as to not waste transit /30s and keep track of them all)
The loopbacks in BGP network 1.1.1.1 mask 255.255.255.255 network 1.1.1.2 mask 255.255.255.255 network 1.1.1.3 mask 255.255.255.255 network 1.1.1.4 mask 255.255.255.255 are also questionable. If they are in OSPF, you could accidentally cause a loop in the event of an equipment failure or outage. They shouldnt be in the BGP table, ospf will be responsible for the routing table to share TE information. BGP doesnt participate in TE.
Always double check that your neighbors send-community both. some say extended.
Also, quick reminder, the default route should only be installed in a VRFs routing table, not the global. Global should only contain transit/loopbacks of LSRs. Most IOS will not install a label for the default route by default so any packets without a specific prefix to another known LSR (if the IGP P/PE topology doesnt match on all the LSRs) you may get IP routing rather than label switching and blackholed traffic due to broken label switch path.
For every VRF you may want to assign two (or more) route-targets. This is done by most providers incase you ever need to build a hub/spoke topology for a VPN or extranet/leak selectively between VPNs.
You will want to change "mpls ldp router-id Loopback0" to "mpls ldp router-id Loopback0 force". Sometimes IOS is just flakey about that ID and its not easy to clear the LDP process.
Again, to run the nat virtual interface (NVI) "ip nat enable" stuff above, you will need to create per VRF address-families on the 3725 and turn it into a full PE even if there are no actual interfaces in a VRF. You will just be redistributing static and adding default routes for those VPNs. Remember that on lower end platforms performing vrf-aware nat may cause soft-switching of packets so I wouldnt count on great performance. I would really suggest installing a pair of ASA 5520's or better in multicontext mode, using a fast ethernet or gigabit ethernet interface and tagging subinterfaces in different VPNs forced through different contexts to reach the internet.
Ill provide more config when i get a chance at work today. |
|
 aryobaPremium,MVM join:2002-08-22 kudos:3 | said by Need BB:I currently have an L3 VPN setup via MPLS between the two 1700 routers but I am confused about the VPN setup. Will this setup allow the two 1700 routers to share the same subnet so computer at one side can connect to the computers on the other or do the networks have to be separate? said by nosx:As to the first question, no, not with an L3 MPLS VPN. however if you setup EPLS (pseudowire crossconnects), VPLS (vfi crossconnects), you can deliver devices on the same subnet across an MPLS wan. However this requires real high end equipment (think 7600s or GSR12400s) Actually you can use L3 MPLS VPN to provide same-broadcast-domain connection, depending on how the MPLS cloud is setup. And no, you don't have to use high-end equipment to have this kind of setup. You can actually use old lower-end equipments such as 2600 series (non-XM) routers and 2950 switches.  |
|
 aryobaPremium,MVM join:2002-08-22 kudos:3 | reply to Need BB said by Need BB:The goal of the project is to experiment with MPLS VPN, QOS, and MPLS TE. I attached my network diagram and configs. The mpls core is not inter-meshed since all the traffic will eventual go through a Cisco ONS OC3 optical transport link between each side (rochester to boston). If you can post the configuration instead of using zip file, it would be more helpful and simpler to read for everybody here to analyze.  |
|
 aryobaPremium,MVM join:2002-08-22 kudos:3 | reply to nosx said by nosx:I do have two real quick questions that arent so related to MPLS VPN so much as P/PE functionality. Why do you have network 0.0.0.0 255.255.255.255 in the OSPF config? I too have a question. Why OSPF? Or even, why do you use dynamic routing at all as IGP for the MPLS cloud/core? If the MPLS core setup is simple as the diagram at your first post, then static routes should be good enough.
If you have to use some dynamic routing as IGP for the MPLS core, then why don't you consider IS-IS instead of OSPF? IS-IS is simple enough to deploy, no LSA to deal with that may take too much resources and slow things up, and is pretty much standard IGP for L3 MPLS VPN. |
|
 nosx join:2004-12-27 00000 kudos:5 2 edits | I havent seen a same-broadcast-domain connection across an mpls cloud outside of L2 VPN's. Can you give an example?
He did post the configs: Project_configs.zip 4,675 bytes Router Configs
OSPF is the gold standard for MPLS deployments worldwide. I have yet to see a live deployment using IS-IS, and when you talk about operational support, OSPF knowledge is much more prevalent. Dont use statics because OSPF carries metrics that are copied into BGP MED as well as adding the possibility of future TE and easy network expansion.
This is how I have seen network-facing interfaces configured:
interface Loopback0
ip address X.X.X.6 255.255.255.255
ip ospf 1 area 0
!
interface POS0/0/1
ip unnumbered Loopback0
ip ospf 1 area 0
mpls ip
!
router ospf 1
log-adjacency-changes
!
mpls ldp router-id Loopback0 force
!
router bgp 123
bgp always-compare-med
no bgp default ipv4-unicast
bgp log-neighbor-changes
bgp deterministic-med
bgp dynamic-med-interval 60
timers bgp 7 21
neighbor PEER_RR peer-group
neighbor PEER_RR remote-as 123
neighbor PEER_RR update-source Loopback0
neighbor Y.Y.Y.4 peer-group PEER_RR
!
address-family vpnv4
neighbor PEER_RR send-community both
neighbor Y.Y.Y.4 activate
bgp scan-time 15
exit-address-family
!
Some equipment with sham-links has a bgp command "bgp redistribute-internal" but im not enough of an expert to claim if thats necessary or not. |
|
 Need BB join:2001-12-21 Rochester, NY | This info is great. I will give it a try and see what I come up with!
Thanks |
|
 Need BB join:2001-12-21 Rochester, NY | So after thinking about my configs I have to implement NAT on the 3725 to access the Internet. See attached diagram.
I am confused how to set up NAT on the 3700 for multiple subnets. I have only done NAT on a single subnet but for this network I want all the routers to be able to reach the Internet. Any insight on how to set this up would be appreciated.
Thanks |
|
 nosx join:2004-12-27 00000 kudos:5 1 edit | Can you send me the config for the internet gateway 3725?
and actually, if you have a terminal server in your lab or some other way for me to telnet or ssh into the equipment id love to take a look at it a little deeper. |
|
 Need BB join:2001-12-21 Rochester, NY | That is the issue I don't have any configs for the 3725. I have a single DHCP IP to the Inerent from outside the lab and want to redistribute this into the lab network.
The 3725 config is the same as before except I changed some of the stuff that was suggested. Here is the current config:
version 12.4 service timestamps debug datetime msec service timestamps log datetime msec no service password-encryption ! hostname Router3725.1 ! boot-start-marker boot-end-marker ! no aaa new-model memory-size iomem 5 ip cef ! mpls label protocol ldp multilink bundle-name authenticated ! ! archive log config hidekeys ! ! interface Loopback0 ip address 1.1.1.1 255.255.255.255 ip ospf 1 area 0 ! interface FastEthernet0/0 ip address 10.1.1.1 255.255.0.0 ip ospf 1 area 0 speed 100 full-duplex mpls ip ! interface Serial0/0 ip address 10.2.1.1 255.255.0.0 ip ospf 1 area 0 mpls ip clock rate 2000000 ! interface FastEthernet0/1 ip address 10.100.1.1 255.255.0.0 ip ospf 1 area 0 speed 100 full-duplex mpls ip ! interface Serial0/1 ip address 10.101.1.1 255.255.0.0 ip ospf 1 area 0 mpls ip clock rate 2000000 ! interface FastEthernet1/0 ip address dhcp speed 100 full-duplex ! router ospf 1 log-adjacency-changes ! router bgp 100 no synchronization bgp cluster-id 1 bgp log-neighbor-changes neighbor PE peer-group neighbor PE remote-as 100 neighbor PE update-source Loopback0 neighbor PE route-reflector-client neighbor PE send-community both neighbor PE send-label neighbor 1.1.1.2 remote-as 100 neighbor 1.1.1.2 update-source Loopback0 neighbor 1.1.1.2 send-community both neighbor 1.1.1.2 send-label neighbor 1.1.1.3 peer-group PE neighbor 1.1.1.4 peer-group PE no auto-summary ! address-family vpnv4 neighbor PE send-community both neighbor PE route-reflector-client neighbor 1.1.1.2 activate neighbor 1.1.1.2 send-community both neighbor 1.1.1.3 activate neighbor 1.1.1.4 activate exit-address-family ! ip forward-protocol nd ! ip http server no ip http secure-server ! ! mpls ldp router-id Loopback0 force ! ! control-plane ! ! line con 0 exec-timeout 0 0 line aux 0 line vty 0 4 login ! ! end -- AWD Turbo Power |
|
 nosx join:2004-12-27 00000 kudos:5 4 edits | I have never attempted NVI config on a non-edge router without an interface that was a member of a customer VRF. Also I have never done interface nat with NVIs (only POOL) for outbound internet connections. This config is a best guess thats untested and might require tweeking and troubleshooting:
conf t
ip vrf VRF_SERVICES
rd 1.1.1.1:999
route-target export 100:999
route-target import 100:999
exit
ip vrf Business1
rd 1.1.1.1:1
route-target export 100:1
route-target import 100:1
exit
int fa1/0
ip vrf forwarding VRF_SERVICES
no ip dhcp client request dns-nameserver
no ip dhcp client request static-route
ip address dhcp
ip nat enable
no ip route-cache
exit
int loop101
!This is a failsafe loop int for testing to turn
!this router into a PE with Customer facing interfaces.
ip vrf forwarding Business1
ip address 192.168.255.1 255.255.255.255
ip nat enable
no shut
exit
ip route vrf Business1 0.0.0.0 0.0.0.0 FastEthernet1/0 DHCP
ip route vrf VRF_SERVICES 0.0.0.0 0.0.0.0 FastEthernet1/0 DHCP
ip nat source list 1 interface FastEthernet1/0 vrf Business1 overload
access-list 1 permit 192.168.100.0 0.0.0.255
access-list 1 permit 192.168.200.0 0.0.0.255
access-list 1 permit 192.168.255.0 0.0.0.255
router bgp 100
address-family ipv4 vrf Business1
redistribute static
redistribute connected
no auto-summary
no synchronization
exit-address-family
exit
end
|
|
 aryobaPremium,MVM join:2002-08-22 kudos:3 | reply to Need BB said by Need BB:So after thinking about my configs I have to implement NAT on the 3725 to access the Internet. See attached diagram. Are you saying that you want your PE router does NAT?
Why do you want a PE router does NAT?
Typically PE routers as part of MPLS core/cloud do not participate in configuration types other than the MPLS-related configuration themselves. |
|
 nosx join:2004-12-27 00000 kudos:5 1 edit | Its one of the valid ways to provide internet service to your VPN customers. When you do so, it basically turns whatever router INTO a PE because you have to run mp-bgp and define VRFs and routing for your customers VPNs. Like we discussed above, a better solution is to hairpin through a multicontext firewall (much more common setup) but the nat virtual interface (NVI) stuff is still prevalent to be on the CCIE service provider lab so its out there too.
How involved you are in your customers routing depends greatly on what kind of business you are in. Verizon or ATT woudnt do this kind of trickery, but a small managed services centric service provider with only a couple POPs might do this for some customers. Its a cheap (as in no capital expense) way to charge your customers for more service. |
|
 Need BB join:2001-12-21 Rochester, NY | I am understanding the MPLS architecture much better now. So it would be best to have the Internet connected to the 2650XM PE router. The issue is that the 2650XM only have one WAN port which will connect to the core 3725 routers. The 2650XM also has the 16ESW switching module but I don't think this card is routable. I did read that you can route using VLAN trunking, but this wouldn't allow an external connection to the Internet. It seems that the existing hardware is limiting how I setup and configure the devices. -- AWD Turbo Power |
|
 nosx join:2004-12-27 00000 kudos:5 2 edits | I would home the internet connection into an existing PE router.
I would also make sure that any route reflectors are NOT in the traffic path from one PE to another.
Segmenting the role of PCORE, PE, and RR are important. The PCORE routers should not participate in BGP at all, they have no need to. Their primary responsability is to forward MPLS frames by outer-most transit label. The intelligence should reside on the PE for BGP and customer routing, as well as LSP construction and selection. The route-reflectors might hang off a PCORE but shouldnt participate in MPLS, and when their links are configured steps should be taken (very high interface IGP/OSPF costs) to make sure that they dont become transit paths for traffic transversing the cloud.
Making sure that you have a solid architectural foundation ensures that when your network grows (and all do eventually) that it doesnt become unmanagable or difficult to support and maintain.
Also, a good practice for numbering things like RD's is to use the devices loopback : unique number per PE. This makes troubleshooting routing across the cloud simpler (you can instantly tell which PE the originating VRF is on) as well as ensure that RDs remain unique across the cloud.
Make sure that you dont accidentally install a default route in the global routing table. MPLS likes explicit paths to a PE. If there is an IGP screwup, and you lose a path to a PE loopback, you dont want to blackhole the traffic out a default ipv4 route in the global table. Make sure any default routes are in VRF routing instances only. |
|
 Need BB join:2001-12-21 Rochester, NY | So after considering my options given the limited hardware I came up with the following setup. I used the existing PE router to connect to the Internet via an ESW16 Etherswitch module. An issue was that I couldn't configure the Etherswitch module to perform NAT, so I used an existing linksys router. There isn't much Internet traffic, only some 64K voice SIP trunks, so the linksys should be up to the task.
An issue is that all the routers can reach the Internet but the CE devices can't unless I ping from the loopback IP. This is a problem since I want to connect some equipment (the Redcoms) directly to the 2650's and have the 2650 perform L3 VPN across the MPLS cloud. I would like this Redcoms to also be able to reach the Internet but this isn't working. I am using two 1760 routers to emulate the static IP of the Redcoms in the diagram.
Any insight into the setup and configuration would be greatly appreciated. The support so far has been awesome and really has allowed me to learn about MPLS/BGP. -- AWD Turbo Power |
|
|
|