Topic 'Staging a new network' in forum 'Photos of Broadband gear' - dslreports.com

Re: Staging a new network

Thu, 05 Nov 2009 09:54:28 EDT

unstable posted : very sexy. thanks for the share.

Re: Staging a new network

Tue, 27 Oct 2009 18:10:55 EDT

TomS_ posted : Possibly. :-)

I dont think it would be too bad, but I dont think its really worth it except to remove all of the static routes from the various boxes.

I toyed with the idea, including setting all of the other devices gateway to the IP of the console server, and adding "no ip redirects" to the console servers LAN interface. That way I would be able to access all of our devices via IP.

But all devices are connected via serial to the console server, and I think this will be fine.

Re: Staging a new network

Tue, 27 Oct 2009 18:08:08 EDT

tubbynet posted :

said by TomS_:

I could also run OSPF over the tunnel, but why complicate things even more.

isn't something like ospf a little top heavy for gprs?

q.
--
"...if I in my north room dance naked, grotesquely before my mirror waving my shirt round my head and singing softly to myself..."

Re: Staging a new network

Tue, 27 Oct 2009 18:06:37 EDT

TomS_ posted : Yeah, crafty routing describes it best I think. I could also run OSPF over the tunnel, but why complicate things even more. :-)

Re: Staging a new network

Tue, 27 Oct 2009 18:05:31 EDT

tubbynet posted : pretty nifty.
its just a lot of crafty routing - nothing difficult, but just not obvious.
nice setup.

q.

Re: Staging a new network

Tue, 27 Oct 2009 18:03:12 EDT

TomS_ posted : Yeah its not too shabby.

Heres a couple more bits of pr0n, because pr0n is good. :D

The tower with the mass of dishes isnt ours, and its not the local mobile cell either. Our tower is the one to the left.

Heres the output of a "sh cell x/x/x radio" command for the HWIC-3G:

>sh cell 0/3/0 radioCurrent Band = GSM 900, Channel Number = 75Current RSSI = -70 dBmBand Selected = GSM EGSM/DCS

Unfortunately, the HWIC-3G doesnt do 3G in 900mhz, but our mobile provider does (and only in rural areas) so were stuck with GPRS, but its still quite reasonable just for console access.

Shot of the gear
Antenna mounted outside the hut

Re: Staging a new network

Tue, 27 Oct 2009 14:30:46 EDT

cow116 posted : thats actually very cool

Re: Staging a new network

Tue, 27 Oct 2009 04:54:20 EDT

TomS_ posted : Ok so, referring to the diagram.

The primary goal of the design is to allow IP connectivity to console servers 1..n when the primary network connectivity is down. The primary network connectivity could go down due to the local router failing, an upstream router failing, a network link failing somewhere, etc etc.

The core router distributes a default route, and each of the local routers will pass this on via an OSPF session to their adjacent console server. The subnet which the central console server is also distributed to each console server via OSPF (this becomes important later on).

Each console server also has a loopback interface with a /32 on it. On the core router there are matching static routes with a metric of 255 pointing towards the centra console server so that they dont interfere with routes learned via OSPF.

The central console server then has static routes to each of the other console servers via the appropriate neighbor IP over a multipoint GRE tunnel which is maintained using DMVPN.

So under normal network conditions when the primary backhaul is functional, each console server has a default route and the subnet the central console server is in learned via OSPF, and using OSPF each console server also lets the rest of the network know about its own loopback address. So normally, all traffic will be routing using the primary backhaul.

When the primary network fails somewhere, console servers beyond that point will lose their default route (including the subnet the central console server is in), and the rest of the network will no longer know about those console servers, at which a static via the 3G connection will fire up.

Once the console servers lose their OSPF default route, a static default route with metric 255 then exists via the GRE tunnel which will establish to the central console server. There is also a steering route for the subnet which the central console server lives in which points out of the 3G connection so that the tunnel can establish (important!).

So now that the network has failed somewhere, a console server only knows how to reach the central console server by routing traffic over the 3G connection.

At this point the GRE tunnel establishes using DMVPN.

On the core router you will no longer find an OSPF route for the console servers that are affected. Instead you will find static routes for their loopback IPs pointing to the central console server.

So traffic that then wants to go to the affected console servers will route into the core router, to the central console server, and over the MPGRE tunnel to the appropriate console server.

The console server will then route traffic back via the GRE tunnel to the central console server, through the core router, etc etc.

Thats the basic operation of it. I hope that made sense (Im tired as hell from a 5:30am start this morning).

Let me know if I need to clarify anything. :-)

Re: Staging a new network

Mon, 26 Oct 2009 17:23:36 EDT

TomS_ posted : Oh yeah, I forgot about this. Sorry.

Have drawn the diagram, just needed to put together a post with details of how it works.

Hopefully get that done when I get home tonight. Too busy to do it right now. :)

Re: Staging a new network

Mon, 26 Oct 2009 12:40:09 EDT

cow116 posted : *bumping* hope the wireless didnt kill yah

Re: Staging a new network

Wed, 23 Sep 2009 13:42:22 EDT

tomdlgns posted : that looks somewhat fun.

Re: Staging a new network

Sun, 20 Sep 2009 04:28:26 EDT

tubbynet posted :

said by TomS_:

I'll write something up with a diagram to explain how it works. Stay tuned.

you got me on pins and needles! :D

q.
--
"...if I in my north room dance naked, grotesquely before my mirror waving my shirt round my head and singing softly to myself..."

Re: Staging a new network

Sat, 19 Sep 2009 10:42:37 EDT

TomS_ posted :

said by tubbynet:

very interesting. i'd love to see more about this.

I'll write something up with a diagram to explain how it works. Stay tuned.

Re: Staging a new network

Fri, 18 Sep 2009 01:15:14 EDT

joshb posted : Very Nicely done...Looks good :)

On a side note,

We are busy staging a system right now at the office and we have so much equipment we have it strung out all over the office... The bench area, a couple office's and the board room... Running big 50+ foot cat5 cables between room...
--
R.I.P Mom We miss you.

Re: Staging a new network

Wed, 16 Sep 2009 11:09:07 EDT

cooldude9919 posted :

said by tubbynet:

said by TomS_:

I could have static, however, Ive chosen to create a DMVPN between each of the 3G enabled console servers and a central console server, so the IPs are dynamic, and it doesnt really matter what the WAN IP of the 3G service is.

I can share more detail on exactly how I am doing this if you like.

interesting. i've never really played with dmvpn too much. most of our customers just have a site or two, and in that case they generally just opt for site to site tunnels using asa hardware (especially because the remote sites are easily served by a limited-license asa5505).
i assume that by running a dmvpn, the spoke sites negotiate with the hub to provide the tunnel. once the tunnel is constructed, you are essentially given access to all "internal" addressing behind each site. you then have all routing and data run over your main link, so only management traverses the 3g (hence the oob, part).

very interesting. i'd love to see more about this.

q.

We run a ~130 spoke 3 hub dmvpn net work. Spoke to spoke tunnels are made on the fly as needed by the given traffic. Given in our setup all Ip's are static, but it doesnt have to be that way and the spokes can be on a dynamic IP. Works quite well.

Re: Staging a new network

Tue, 15 Sep 2009 17:23:34 EDT

tubbynet posted :

Re: Staging a new network

Tue, 15 Sep 2009 09:36:08 EDT

cow116 posted : *giggles* thats awesome

Re: Staging a new network

Tue, 15 Sep 2009 03:52:42 EDT

TomS_ posted :

said by cow116:

fiber connectivity goes down but u still have 3g up so you remote in over the 3g and see whats up over an RDP session?

Yes thats basically what Im doing. :-)

said by tubbynet:

TomS_ , are you registering the 3g radio ips using a dynamic dns service?

I could have static, however, Ive chosen to create a DMVPN between each of the 3G enabled console servers and a central console server, so the IPs are dynamic, and it doesnt really matter what the WAN IP of the 3G service is.

I can share more detail on exactly how I am doing this if you like.

Re: Staging a new network

Mon, 14 Sep 2009 18:23:59 EDT

tubbynet posted : TomS_ , are you registering the 3g radio ips using a dynamic dns service? i have thought about the feasibility of using 3g for oob, but even when using my sprint card for extended periods of time i have noticed that my connection will bounce and with it, my ip. or do you have a static ip address with each radios service?

q.
--
"...if I in my north room dance naked, grotesquely before my mirror waving my shirt round my head and singing softly to myself..."

Re: Staging a new network

Mon, 14 Sep 2009 13:42:46 EDT

cow116 posted : *grins from a geeky thought* so tom with the HWIC 3g cards in there you're saying that in addition to OOB access you can also use "web access" for example you have a linux or windows box behind them at one of the towers.... fiber connectivity goes down but u still have 3g up so you remote in over the 3g and see whats up over an RDP session?
--
Grambonet Networks
»cowkilla.dyndns.org/cisco

Re: Staging a new network

Mon, 14 Sep 2009 09:07:14 EDT

TomS_ posted : I consider dialin OoB to be the more reliable method, and use it whereever I can.

But some of the sites I am rolling out are too far away from any PSTN infrastructure, and/or its simply too expensive to get a PSTN line put in, so 3G it is.

Weve also used some GSM based modems in the past. The cool thing about the 3G solution is that you can maintain IP access to your gear, rather than just dial into a console server. Will be interesting to see how well it works in the real world (seems to work pretty well in the lab). :)

Re: Staging a new network

Sat, 12 Sep 2009 22:14:12 EDT

cooldude9919 posted : lol we still use dial backup for OOB access. Old and slow, but works for the most part :)

Re: Staging a new network

Sat, 12 Sep 2009 21:24:31 EDT

LazMan posted : Loving the in-chassis OOB access!

We're using Raven 1x wireless modems for OOB at some sites, but they are an external "box" - the HWIC option is cool! :D

Re: Staging a new network

Fri, 11 Sep 2009 13:22:03 EDT

kewlkeed posted : Awesome! I love benching a huge network on my desk hehehe.

I just did one recently with 25 nodes for what will be spread across nearly 500KM, all running gigabit speeds. I never took pics sadly... I'll remember for next time.

Staging a new network

Fri, 11 Sep 2009 06:50:31 EDT

TomS_ posted : Im in the process of rolling out a new network, and for the last two weeks Ive been staging it in the lab, configuring everything and making sure that it works as required/expected etc.

21 Cisco devices, mostly 2811 routers, all 3560 series switches, and a 7401 hand-me-down from elsewhere in the network that was no longer necessary.

Some gear has already been rolled out so is not pictured.

4-5 new towers are also being built. Most of this gear will be located at those towers, with some being located at a PoP elsewhere, and some colocated at an existing tower.

I'll be trying to get photos of the rest of the gear to share as we roll it out.

The antennas sticking out of some of the 2811's are for an Out of Band remote access solution (using HWIC-3G's) which is used when the primary network goes down so that we can get to things like console servers, masterswitches, etc.

Enjoy. :)

All of the gear
Mess of power cables