dslreports logo
site
 
    All Forums Hot Topics Gallery
spc

spacer




how-to block ads


Search Topic:
uniqs
4485
share rss forum feed


TomS_
Git-r-done
Premium,MVM
join:2002-07-19
London, UK
kudos:5

1 recommendation

Juniper linecard(s)

First post in a while!

Im sure some people have seen big routers and ever wondered what, if anything, the linecards have on them. They can look kind of unassuming, with just a bunch of ports.

Some cards are more complex than others, and these are definitely in the complex pile... e.g. IIRC the Cisco Catalyst 4500 linecards are quite simple, with basically a backplane connection to the Sup where all switching is done, not really a lot onboard. 6500 linecards get a bit more complex, with more supporting hardware for higher performance and other features.

Attached is a photo of an FPC3 from a Juniper T640 router, and I can have a photo of an FPC4 for a T1600 within the next couple of days if anyone is interested (though it'll probably look much the same)




FPC3's can take up to 4 PICs, which are modules that contain the physical port(s), and control the physical media - basically just layer 1 functionality. In this FPC there are 4 x 10GE PICs on the right with XENPAK optics.

On the left are where things get a little more interesting. There are 3 mezzanine boards:

* Top and bottom: packet forwarding engines
* Middle: processor module

Yep, each line card has its own little computer onboard to control it. It runs a PPC processor, has its own RAM, and talks with the host processor modules via internal ethernet links. Im not 100% certain of its function, but Im sure its got something to do with controlling the PFEs and other things related to the particular linecard its on.

The PFEs take packets coming in from the PICs, do various things with them, and interface the linecard to the switching boards installed at the rear of the chassis to move the packet between incoming and outgoing interfaces. They'll also take packets coming in from the switching boards and direct them out to the appropriate PIC. On these linecards, each PFE only serves two PICs, so even on a single linecard there is some redundancy.

If you stick an FPC3 from a T640 and an FPC3 from a T320 side by side, they will look almost identical, except for the fact two ports and a PFE are shaved off (also the T320 linecard is physically smaller.) A T320 really almost is 1/2 a T640.

The FPC4's for the T1600 are the same size as the T640 linecards, except that they only take two PICs, but the PICs can have more ports. For example, an FPC4 can have up to 8 x 10GE ports in a single slot, instead of 4 for an FPC3 (or two on a T320), providing 160Gbps full duplex per slot, and 1.28Tbps per chassis.

So if youve ever wondered why these things have such a high price tag, theres really a lot of brains involved. But with the PICs being relatively "dumb" sometimes you have to wonder why they are so damned expensive...

HELLFIRE
Premium
join:2009-11-25
kudos:18

Okay, half of that gave me a headache... but for some reason my inner geek heart went pitter-patter

Thanks for the show and tell TomS_, and please do post some more of that gear.

Regards



TomS_
Git-r-done
Premium,MVM
join:2002-07-19
London, UK
kudos:5
reply to TomS_

Here comes the plural (finally).

First off, a T320 FPC2. As you can see, it is similar to the T640 FPC above, basically just "chopped in half" (missing one PFE which would drive two more PICs if they existed.)


T320 FPC2


The difference between an FPC2 and FPC3 in basic terms is that an FPC3 can do 10Gbps per PIC, whereas an FPC2 can only do 2.5Gbps. This FPC2 was pulled out to upgrade from two 4xSTM-4 (or 1xSTM-16) PICs to two 4xSTM-16 PICs.

Next up is an FPC4 from a T1600. Were into big boy territory now.


T1600 FPC4


Again you can see two PFEs and the processor module wedged in between. The presence of some rather beefed up heatsinks on the PFEs tells you this thing means business because ...


4x10GE PIC


Heres a 4 port 10GigE module.

Theres also a 4 port STM-64/OC-192 module which looks pretty much identical, and has the ability to do inverse multiplexing to form an STM-256/OC-768 - that gives you somewhere in the region of 38Gbps of bandwidth, which is a lot of Internets...

The FPC4s are capable of 100Gbps each which (full duplex) gives you 1.6Tbit/sec of forwarding capacity, hence T1600.

A T1600 system can also take older FPCs so you can support legacy interfaces. In the two T1600's I built, we installed an FPC3 with 3 x STM-64/OC-192 PICs and a 4xSTM-16/OC-48 PIC.

(Clarification: I know I said 1.28Tbit in my first post, but thats for a chassis fully populated with 10GigE interfaces only.)

voxframe

join:2010-08-02
reply to TomS_

Not that it's my business... But what's just a ballpark figure on those things...

*Cringe factor*



TomS_
Git-r-done
Premium,MVM
join:2002-07-19
London, UK
kudos:5

Ive *heard* (but TBH Im not actually certain) that the 4x10GE PICs run just into the 6 figure range. That could also be list price...

A fully kitted router is going to cost comfortably in the $millions. But when you have to buy a router that big, the cost is really secondary.



tubbynet
reminds me of the danse russe
Premium,MVM
join:2008-01-16
Chandler, AZ
kudos:1

said by TomS_:

Ive *heard* (but TBH Im not actually certain) that the 4x10GE PICs run just into the 6 figure range. That could also be list price...

what are you doing with kit that big, tom? are you handling core functions (just gathered from the 4x10gbe pic) or is this your edge handoff platform (in which case -- you've got big pipes coming in/out of your a/s)?
didn't realize you guys handled that much juniper -- unless these purchases were made before the crs3 was readily available at $reasonable_cost.

q.
--
"...if I in my north room dance naked, grotesquely before my mirror waving my shirt round my head and singing softly to myself..."


TomS_
Git-r-done
Premium,MVM
join:2002-07-19
London, UK
kudos:5

This is for core.

Ive since moved on from my previous employer in Aus, this kit belongs to my new employer (in case you thought it belonged to my Aus employer who was a 100% Cisco shop.)



tubbynet
reminds me of the danse russe
Premium,MVM
join:2008-01-16
Chandler, AZ
kudos:1

said by TomS_:

This is for core.

Ive since moved on from my previous employer in Aus, this kit belongs to my new employer (in case you thought it belonged to my Aus employer who was a 100% Cisco shop.)

saw the new locale under your name. i vaguely recall you saying your previous employer was 100% cisco. what stuck out more to me was the fact that you were doing sdh over microwave (stm-4, if memory serves) and that 10gbe would be (a) a huge conversion and (b) waaaaaay future-proof for what you were doing.

i'm a geek -- i remember the specifics

q.
--
"...if I in my north room dance naked, grotesquely before my mirror waving my shirt round my head and singing softly to myself..."


TomS_
Git-r-done
Premium,MVM
join:2002-07-19
London, UK
kudos:5

Close.

STM-1s actually, but the radios could do enough to handle 2xSTM-4.

Even for us back then GigE was massive, and we only really used that in the very core of our network where everything got aggregated in. 10GE would have been stupid.


HELLFIRE
Premium
join:2009-11-25
kudos:18
reply to TomS_

said by TomS_:

But when you have to buy a router that big, the cost is really secondary.

When you buy a router that big, you KNOW you're in the big leagues.

Thanks for the show and tell TomS_, hope with your new employer more is in the pipeline.

Regards

sk1939
Premium
join:2010-10-23
Mclean, VA
kudos:10
reply to TomS_

It's mid six figures. A PIC with 8 GbE ports still costs in the range of $120,000 USD, a 4 port STM-4 PIC is about $160,000, and a single port 100 GbE module is in the neighborhood of $300,000.