how-to block ads
|
whiteybulger
join:2003-03-11
Belmont, MA | Does packet forging reduce network traffic?
I don't know how this works, but if they're just injecting fake packets to slow a given application, doesn't that actually increase network traffic? | |
koitsu
Premium
join:2002-07-16
Mountain View, CA
| said by whiteybulger  :I don't know how this works, but if they're just injecting fake packets to slow a given application, doesn't that actually increase network traffic? The trade-off is substantial. Two TCP RST packets are about ~54 bytes in size each. Consider what sort of bandwith savings they're inducing by sending 108 bytes of traffic every time someone tries to seed. | |
gaforces
United We Stand, Divided We Fall
join:2002-04-07
Santa Cruz, CA | Its like a smart targeted DoS attack. Interfering with packets = Denial of Service. | |
Jack2131
@sbcglobal.net
| reply to whiteybulger
P2P REALLY slows down an entire network. If someone initiates a couple popular torrents on my dsl line, everything screaches to a halt/slow motion browsing. It's why I bought a Ubicom-based router to "degrade" the p2p applications while I'm doing other things. If they didn't do that I'm sure ALOT more people would be complaining. But hey, they could spend more on infrastructure, but p2p can consume an entire line no matter how fast it is. | |
knightmb
Everybody Lies
join:2003-12-01
Franklin, TN
 ·AT&T DSL Service
| said by Jack2131 :
P2P REALLY slows down an entire network. If someone initiates a couple popular torrents on my dsl line, everything screaches to a halt/slow motion browsing. It's why I bought a Ubicom-based router to "degrade" the p2p applications while I'm doing other things. If they didn't do that I'm sure ALOT more people would be complaining. But hey, they could spend more on infrastructure, but p2p can consume an entire line no matter how fast it is.
It's not just P2P, you realize that someone doing a single file upload on your DSL at maximum send rate will basically produce the same effect. It's just the limitations of a NAT based router. If you had two static IP address and each computer had it's own, the one doing P2P would have a lot less affect on the second computer since this would call into the TCP/IP even split of bandwidth between the two instead of the "race to get the packet out first" that is one of the limitations of a NAT with multiple computers behind it.
It's not easy to demonstrate on a modem with a single LAN to WAN setup, but try a DSL or Cable modem (usually has to be business class) that has multiple ports to support multiple static IP address for each port and the problem is only noticeable from a benchmarking upload speed standpoint rather than "nothing is working" view. | |
fiberguy
My views are my own.
Premium
join:2005-05-20
| reply to Jack2131
So that's the reason why when I downloaded Spiderman III from your computer it took forever!! What's your address so I can sue you! How dare you purposely slow my free movie download for your own selfish reasons..   | |
koitsu
Premium
join:2002-07-16
Mountain View, CA
| reply to knightmb
said by knightmb :It's not just P2P, you realize that someone doing a single file upload on your DSL at maximum send rate will basically produce the same effect. This is why packet prioritisation (read: QoS) is important. The term QoS is used incorrectly all over the place now, which is quite irritating -- so if you want a good read about how QoS can truly improve performance over slower (read: non-100mbit) links, try this. It specifically documents how to solve the above issue (re: downloading a single file at full speed causing all other applications to perform horribly) using pf ALTQ on OpenBSD or FreeBSD. I'm positive Linux has similar.
However, there is one thing I should point out about P2P vs. downloading a file at high speed via HTTP:
From a TCP state perspective (read: NAT, or anything that tracks TCP states in general), P2P is hell. The number of states which have to be tracked for torrents where there's thousands of peers is bordering on nuts. I wouldn't be surprised if a hundred users downloading the same torrent (a thousand peers each) on a single Juniper M20 could cause the routing engine to slow significantly. So from *that* perspective, P2P does in fact slow down a network.
But in that situation, you're overselling capacity, and should look into getting another M20. *shrug* That's just the way it goes. That's reality, and some ISPs want to try and pinch pennies in every way possible rather than realising users are paying for said growth.
--
Making life hard for others since 1977.
I speak for myself and not my employer/affiliates of my employer. | |
funchords
Hello
Premium,MVM
join:2001-03-11
Washington, DC
·Verizon Online DSL
 ·Skype
| Very good points -- just wanted to clear something up (since you cleared up a few things for others)...
said by koitsu :The number of states which have to be tracked for torrents where there's thousands of peers is bordering on nuts. I wouldn't be surprised if a hundred users downloading the same torrent (a thousand peers each) on a single Juniper M20 could cause the routing engine to slow significantly. Although users can often change some of these settings, users generally do not connect to more than (some number between) 30-70 other peers in a swarm. This is by design -- there is no added efficiency in doing so. The knee of the curve of diminishing returns falls between 50-80 (depending on a lot of factors, most of which change from moment to moment).
Also keep in mind that 80%-ish to 90%-ish are on Microsoft TCP/IP stacks. They won't connect at a rate faster than 10 clients per second, and it stops attempting ANY new connections any time there are 10 unanswered connection attempts. (This is a security limit added by Microsoft.) So the problem isn't as bad as some might imagine.
1. The number of connections actually established is far less than many imagine; and
2. The rate at which clients can seek connections is usually limited.
Keep in mind that a lot of these kids have the cheapo router with just a couple of MB of RAM available for maintaining their own NAT tables and other operating variables/caches and etc.. With a few exceptions, most of these work under normal P2P use.
--
Robb Topolski -= funchords.com =- Hillsboro, Oregon
"We don't throttle any traffic," -Charlie Douglas, Comcast spokesman, on this report. | |
gal
join:2006-12-21
Toledo, OH | reply to whiteybulger
They are actually sending packets to impersonate one of the machines asking the other to stop sending information therefore stopping the service. It can be restarted if the machines continue to try after a certain peroid of time. | |
koitsu
Premium
join:2002-07-16
Mountain View, CA
| reply to funchords
All very good points, as usual. 
And I had completely forgotten about the "maximum 10 half-open TCP connection" ordeal on Windows (I build my own Windows CDs using nLite, which lets you patch TCPIP.SYS to address that).
--
Making life hard for others since 1977.
I speak for myself and not my employer/affiliates of my employer. | |
Anonymous_
Anonymous
Premium
join:2004-06-21
127.0.0.1
clubs:
 ·RoadRunner Cable
·Time Warner Cable
·Time Warner VOIP
1 edit | reply to funchords
said by funchords :Very good points -- just wanted to clear something up (since you cleared up a few things for others)... said by koitsu :The number of states which have to be tracked for torrents where there's thousands of peers is bordering on nuts. I wouldn't be surprised if a hundred users downloading the same torrent (a thousand peers each) on a single Juniper M20 could cause the routing engine to slow significantly. Although users can often change some of these settings, users generally do not connect to more than (some number between) 30-70 other peers in a swarm. This is by design -- there is no added efficiency in doing so. The knee of the curve of diminishing returns falls between 50-80 (depending on a lot of factors, most of which change from moment to moment). Also keep in mind that 80%-ish to 90%-ish are on Microsoft TCP/IP stacks. They won't connect at a rate faster than 10 clients per second, and it stops attempting ANY new connections any time there are 10 unanswered connection attempts. (This is a security limit added by Microsoft.) So the problem isn't as bad as some might imagine. 1. The number of connections actually established is far less than many imagine; and 2. The rate at which clients can seek connections is usually limited. Keep in mind that a lot of these kids have the cheapo router with just a couple of MB of RAM available for maintaining their own NAT tables and other operating variables/caches and etc.. With a few exceptions, most of these work under normal P2P use. i have my set to 16 million i am running SP2
ALSO my ROUTER HAS 32MEGABYTES OF RAM ,8MB flash
WRT-54GS
This is a security limit IS USE LESS With in minutes millions of computers would have the worm
ok let say one computer has the limit of 10 and are sending out an worm
10 > 100 > 1000 > 10,000 > 100,000 > 1,000,000 X 60 seconds 60,000,000 computers with the worm
--
Ω? | |
espaeth
Digital Plumber
Premium,MVM
join:2001-04-21
Minneapolis, MN
·voip.ms
·Vitelity VOIP
·Callcentric
·VoiceStick
·ViaTalk
·Comcast
·Embarq
| reply to koitsu
said by koitsu :I wouldn't be surprised if a hundred users downloading the same torrent (a thousand peers each) on a single Juniper M20 could cause the routing engine to slow significantly. So from *that* perspective, P2P does in fact slow down a network. But in that situation, you're overselling capacity, and should look into getting another M20. It doesn't work that way... at all. Comcast doesn't use Juniper hardware, but the Cisco 7600s they use share a commonality with the M20 in that they're both hardware-based routing platforms. These platforms use TCAM to perform pattern matching that can do access control lists / address translation / NetFlow tracking at line rate.
With standard memory (like that in your PC) you supply an address location and then you can read or write the contents of the memory at that position. TCAM is pretty much the opposite of that -- you give it a bit pattern and it returns the index position where that particular entry exists. This process is extremely fast because it is baked into silicon as a specific operation on an ASIC and does not have the overhead of being a process running in software on a generic-use CPU; it can complete a full lookup in just a few nanoseconds. (significantly faster than the network processing demands of microsecond/sub-millisecond timing)
Whether your hardware based router is tracking a single flow, or the maximum number of flows their TCAM can store patterns for there is absolutely no measurable impact to forwarding performance. The most common use of state tracking on standard routers is gathering NetFlow traffic statistics data. The Netflow tracking flow looks something like this:
You gather statistics for as many flows as your memory will allow and simply forward anything else once you run out of memory to store additional patterns. The lookup time is roughly the same if the pattern matches or not, and the counter increment process uses a negligible amount of ASIC processing time.
The real issue is fairness in TCP flows when the link starts to become saturated. Each TCP session is like a car on the freeway; trying to go as fast as the road will allow but constantly having to adjust for other cars. TCP is greedy just like cars on the freeway as well, as soon as a gap opens up it speeds up to immediately take up that extra space. When congestion starts to occur on a link segment every TCP session slows down to roughly the same top speed because every session follows the same rules when faced with congestion. That's where the problem with BitTorrent really starts to come into play. Normally with TCP sessions slowing down in roughly the same manner that is a relatively fair situation on the network in terms of distribution of impact, but in the case of BT where you have several flows this fairness rapidly disappears. Say congestion gets to the point that flows can't grow to be faster than 40kbps. For you with BitTorrent you can still max out your upstream connection because you're establishing multiple TCP connections with 40kbps each. For me uploading content to my web server via FTP, the most network resources I get out of the deal is 40kbps because I'm only uploading my content in a single flow.
That's why closing down connections is effective: it gets the TCP-Flow per user ratio to be more consistent across the board and helps to give each user a fighting chance at a better share of the available bandwidth. | |
koitsu
Premium
join:2002-07-16
Mountain View, CA
| said by espaeth :It doesn't work that way... at all. You're always here to make my life hell, man. Just kidding.
Thank you for the crash course on hardware-based routing. I've always wondered how that was done and whether or not it was just a buzzword for "dedicated routing engine card" for offloading routing vs. on the main CPU (which I believe M20s and M40s use the equivalent of a Pentium II).
I've known Cisco uses hardware-based routing for quite some time, and was always told it was "incredibly fast" but never was given an actual description as to how it worked, so again, thanks!
--
Making life hard for others since 1977.
I speak for myself and not my employer/affiliates of my employer. | |
funchords
Hello
Premium,MVM
join:2001-03-11
Washington, DC
·Verizon Online DSL
·Skype
1 edit | reply to espaeth
said by espaeth :When congestion starts to occur on a link segment every TCP session slows down to roughly the same top speed because every session follows the same rules when faced with congestion. I'm with you so far.
said by espaeth : That's where the problem with BitTorrent really starts to come into play. Normally with TCP sessions slowing down in roughly the same manner that is a relatively fair situation on the network in terms of distribution of impact, but in the case of BT where you have several flows this fairness rapidly disappears. You (and many, many others) are forgetting the fact that the first bottleneck for a residential Comcast user is his modem. You can't send any faster than that modem will accept data -- and that modem is your residential gateway. When it's maxed out, the only open flows you are restricting are your own. Maxing out your cable modem does not affect your neighbors at all.
From the modem to the CMTS, its DOCSIS. This is the second bottleneck, but it doesn't matter for two reasons. First and foremost, the paragraph above. Secondly, since it's not TCP/IP, then it doesn't know about sockets or how many you have nor where they are going. Basically, if your DOCSIS node is saturated, your byte-for-byte throughput is impacted by the same X% regardless of how many TCP/IP sockets you have open. DOCSIS can't give someone with 40 sockets an advantage because it doesn't know about sockets. It only has one flag that says "I have data to send -- tell me when." There is no flag that says "I have 40 sockets with data to send"
The third bottleneck is the Comcast network gateway. Unlike the other two, this TCP/IP to TCP/IP bottleneck is not about technology limits, it's about money. If its irrelevance is not obvious, I'm happy to discuss -- but at that connection point, there's nothing different from Comcast's network to Verizon's network.
WARNING - SHOCKING NEWS AHEAD
Web Browsers open two uploading ports simultaneously to each server to a page. As soon as they're done, they close the port. Typically 2-5 ports are actively uploading at the same time.
Now, here's the shocking piece of news to everyone: Most BitTorrent peer connections are idle at any one time -- SUBSTANTIALLY USING NO BANDWIDTH. Regardless of how many connections you have open, you're only going to be uploading on 3 or 4 sockets at a time -- the rest are choked and are using 4-9 Bytes every 20-60 seconds waiting for their turn.
When congestion happens on a TCP/IP segment, then you are right that all of the sockets that are actively using bandwidth on the bottlenecked segment will be decreased to a lower, roughly equal amount. For us using Comcast, that's the segment between your computer and the Cable modem. But people SHOULD NOT get the idea that because a P2P application has 40 open sockets and a web browser has 5 that P2P has a 40/5 advantage across the Internet. The only sockets that matter are the ones using a high amount of bandwidth (ones faster than the new congestion-created ceiling will eventually be), and in CATV-Internet, the only ones that really matter are the ones that are uploading from your own side of the modem.
This whole Sandvine thing is a farce. It's about saving money paid to backbone vendors -- money to transit the bandwidth customers have already purchased. It has nothing to do with DOCSIS and it has nothing to do with network congestion.
--
Robb Topolski -= funchords.com =- Hillsboro, Oregon
"We don't throttle any traffic," -Charlie Douglas, Comcast spokesman, on this report. | |
funchords
Hello
Premium,MVM
join:2001-03-11
Washington, DC
·Verizon Online DSL
·Skype
2 edits | reply to Anonymous_
said by Anonymous_ :i have my set to 16 million i am running SP2 You only have about 4000 useful ephemeral ports, so any number above that is, at best, meaningless or, at worst, allocating resources you cannot use. (My bet is it's just meaningless, but I've never tested it.)
There is also the limitation created by how fast your system can pump out SYN requests. I've never calculated or tested that either, but my guess is that its far less than those settings.
said by Anonymous_ :ALSO my ROUTER HAS 32MEGABYTES OF RAM ,8MB flash WRT-54GS Now you're just bragging. Is that sold as a higher-duty device, or is that typical for the normal WRT-54Gs?
said by Anonymous_ :This is a security limit IS USE LESS With in minutes millions of computers would have the worm ok let say one computer has the limit of 10 and are sending out an worm 10 > 100 > 1000 > 10,000 > 100,000 > 1,000,000 X 60 seconds 60,000,000 computers with the worm I can't prove usefulness, but I can say that the way malware is spread has definitely changed since the limit was added. But that could be just because Social Engineering and Phishing was both easier and more profitable.
That said, you're thinking about this the wrong way. The worms were being being sent to random TCP/IP addresses as fast as the system would open sockets. In most cases, those connection attempts won't answer because they are random and there is no machine at that address, or because the machine is firewalled.
The only change made in SP2 says that if you have over 10 UNANSWERED OUTGOING connection ATTEMPTS IN PROGRESS, then the 11th or higher will be delayed until one of those first 10 is either cancelled, answered, rejected, or times out.
Now 10 is a little too low for my tastes, but 30-50 should be just fine. Even today's P2P applications (half of them are firewalled) should perform equally well (connecting wise) between a limit of 40 and those with no limit. That's because most of the addresses attempted will answer.
But even if you stick with 10, you're not sacrificing transfer speed. You're only taking SLIGHTLY longer to initially connect when you start up a torrent. Once you're connected up, the speed is the same -- patch or no patch.
Where Microsoft blew it, IMHO, is by making this impossible to user-configure -- and then they explained it so poorly that most people just decided to patch it rather than to understand and choose the setting they felt was most appropriate for how they use the Internet. Instead, they decided for us and that just pissed people off!
--
Robb Topolski -= funchords.com =- Hillsboro, Oregon
"We don't throttle any traffic," -Charlie Douglas, Comcast spokesman, on this report. | |
koitsu
Premium
join:2002-07-16
Mountain View, CA
2 edits | said by funchords :said by Anonymous_ :ALSO my ROUTER HAS 32MEGABYTES OF RAM ,8MB flash WRT-54GS Now you're just bragging. Is that sold as a higher-duty device, or is that typical for the normal WRT-54Gs? It's a consumer device made by Linksys. The first-generation WRT54GS units contains 32MB of RAM, and 8MB of flash. Future generations of that model contained 16MB RAM and 4MB flash, to save on manufacturing costs.
The first generation model ran Linux, while future generations ran a custom VxWorks firmware and made it very difficult to re-flash to Linux. There are some available "hack" methods now which allow you to turn a VxWorks-based unit into a Linux-based unit, but it's not worth the pain if you ask me. It's easier to buy a WRT54GL or WRTSL5GS (see below).
The same applies to the WRT54G (note the lack of S), where original revisions were Linux-based and had more RAM/flash, but Linksys dropped that down as well (in some cases down to 4MB RAM and 1MB flash).
Linksys eventually made a product -- still available and VERY popular -- called the WRT54GL, which specifically runs Linux, and is marketed as such. It's intended for consumers who want to use third-party Linux-based firmwares or develop their own.
There is also the WRTSL5GS, which uses Linux too (on both 1.0 and 1.1 revisions).
The point here is this: Linux's IP stack is significantly more reliable than some third-party stack. The VxWorks-based firmware have a track record of IP stack bugs (such as users being able to crash the router by using BitTorrent, or even worse, seemingly "random" websites not being accessible for no reason -- all turned out to be bugs in the IP stack on the VxWorks firmwares). Many third-party firmwares adjust the default TCP stack settings, which significantly helps with P2P applications (this is likely what you're remembering causing issues).
I happen to run the Tomato firmware on my WRTSL54GS, which includes all of those settings by default; it comes out of the box with a soft-limit of 4096 connection limit, but you can increase that through the UI. I use the defaults. I've had 4 or 5 torrents going simultaneously, with about 400-500 peers and 20-30 seeds connected to each one, without a single issue. (I also increase the half-open connection limit on XP SP2 from 10 to 16384 using a TCPIP.SYS patch).
--
Making life hard for others since 1977.
I speak for myself and not my employer/affiliates of my employer. | |
funchords
Hello
Premium,MVM
join:2001-03-11
Washington, DC
·Verizon Online DSL
·Skype
| I remember scavenging my city looking for a "Rev 4" of the WRT-54G. Rev 5 was all that could be found, and it was not 3rd-party FW friendly. The WRT-54GL was announced, but nobody had one yet.
SO, to this date I've never played with any 3rd-party firmware. I don't really need it, but I just love to tweak.!
--
Robb Topolski -= funchords.com =- Hillsboro, Oregon
"We don't throttle any traffic," -Charlie Douglas, Comcast spokesman, on this report. | |
koitsu
Premium
join:2002-07-16
Mountain View, CA
| said by funchords :I remember scavenging my city looking for a "Rev 4" of the WRT-54G. Rev 5 was all that could be found, and it was not 3rd-party FW friendly. The WRT-54GL was announced, but nobody had one yet. SO, to this date I've never played with any 3rd-party firmware. I don't really need it, but I just love to tweak.! If you ever feel up to playing again, give the WRT54GL a try. It's guaranteed to work with all third-party firmwares available right now, there's no anti-VxWorks hackery needed like on the WRT54G v4 and onwards, and it's affordable (around US$50 after rebate on Amazon right now).
I continue to recommend to everyone the WRT54GL, speaking purely based on experience and overall familiarity with the units, ditto with having actually done some development work on them (on HyperWRT, tofu, and OpenWRT). They're great little units. | |
espaeth
Digital Plumber
Premium,MVM
join:2001-04-21
Minneapolis, MN
·voip.ms
·Vitelity VOIP
·Callcentric
·VoiceStick
·ViaTalk
·Comcast
·Embarq
| reply to koitsu
said by koitsu :You're always here to make my life hell, man. Just kidding. Just trying to keep the facts in line.
said by koitsu :Thank you for the crash course on hardware-based routing. I've always wondered how that was done and whether or not it was just a buzzword for "dedicated routing engine card" for offloading routing vs. on the main CPU (which I believe M20s and M40s use the equivalent of a Pentium II). Ternary Content-Addressable Memory is really the magic that makes hardware routing possible. It really started with Ethernet switches taking advantage of this solution to rapidly determine which port a particular frame is destined towards. In hardware routers today you will have several TCAM instances for Netflow, the adjacency (ARP) table, the forwarding table (ie, Cisco Express Forwarding/CEF), and Access Control lists. The CPU on these boxes only handles the tasks of running the user interface to the router, running the routing protocols and building the associated forwarding table in TCAM after routes are updated, and handling maintenance traffic like SNMP and ICMP. | |
espaeth
Digital Plumber
Premium,MVM
join:2001-04-21
Minneapolis, MN
·voip.ms
·Vitelity VOIP
·Callcentric
·VoiceStick
·ViaTalk
·Comcast
·Embarq
| reply to funchords
said by funchords :You (and many, many others) are forgetting the fact that the first bottleneck for a residential Comcast user is his modem. You can't send any faster than that modem will accept data -- and that modem is your residential gateway. When it's maxed out, the only open flows you are restricting are your own. Maxing out your cable modem does not affect your neighbors at all. Saturation on the upstream is an "n-user" problem. My car is only so big, and I can get on the freeway at 3am and maybe not run into a single car. When I get that same freeway at 7:30am all of a sudden it's bumper to bumper and running slow. You can't look at singular TCP connections, you have to look at the situation once critical mass is achieved.
said by funchords :From the modem to the CMTS, its DOCSIS. This is the second bottleneck, but it doesn't matter for two reasons. First and foremost, the paragraph above. Secondly, since it's not TCP/IP, then it doesn't know about sockets or how many you have nor where they are going. Basically, if your DOCSIS node is saturated, your byte-for-byte throughput is impacted by the same X% regardless of how many TCP/IP sockets you have open. DOCSIS can't give someone with 40 sockets an advantage because it doesn't know about sockets. TCP is an end-to-end stateful protocol with sequencing and flow control. It adjusts its flow rate based on a number of criteria including max window size, round-trip time, and observed packet loss. If the DOCSIS node is busy there will inevitably be additional queuing delay, and TCP sensing an increase in round-trip time will back off on the transfer rate a bit. The maximum transfer rate (per second) for a TCP session is the (TCP Window) * (Number of Round-trip intervals per second). Assuming the TCP Window is artificially limited (true for the vast majority of Windows boxen) then queuing delay will result in a steady decrease in the performance of each TCP flow because of the reduced number of round-trip intervals per second.
said by funchords :The third bottleneck is the Comcast network gateway. Unlike the other two, this TCP/IP to TCP/IP bottleneck is not about technology limits, it's about money. If its irrelevance is not obvious, I'm happy to discuss -- but at that connection point, there's nothing different from Comcast's network to Verizon's network. This whole Sandvine thing is a farce. It's about saving money paid to backbone vendors -- money to transit the bandwidth customers have already purchased. It has nothing to do with DOCSIS and it has nothing to do with network congestion. The MSO to Internet gateway points are the cheapest part of the whole service equation. Unless Comcast is being intentionally deceptive with their PTR names (highly unlikely) you can pretty accurately piece together the bulk of their network environment.
The uBR CMTS hardware connects to the UR01/02 user routers via Gigabit ethernet (hence the gi-2-3 PTR records, signifying the gig interface in slot 2, port 3 of the 7600)
The UR01/02 gear connects to the AR01/02 area aggregation routers via 10GigE (hence the te-2-3 PTR records)
The AR01/02 routers connect to their transport peers via 10GigE in the vast majority of cases.
The cost isn't in this part of the network because you get a better per-mbps pricing with bigger circuits and large commits, and having 14 million pairs of eyeballs as a bargaining chip doesn't hurt either. If you look at how Comcast is routing their traffic they have fewer egress points for their national footprint than there are states in the nation. Just skim the Comcast forum and gather traceroute statistics and you'll find that there are only a few dozen external connection points to the Comcast network. It's not the modest handful of Internet connections that drives your costs, it's the hundreds of CMTS units and thousands of nodes deployed across the cable plant. When they turn up additional capacity for Internet peering it can be used by every single customer in the region; the cost/benefit ratio is quite favorable. It's also easily expandable because the limited number of peering locations make for nice places to consolidate fiber interconnects to their transport vendors. This means that it's easy to get more capacity in a big hurry. Need another 10GigE? You can rapidly provision new capacity until you run out of DWDM wavelengths and physical pairs of fiber.
The cable plant is a completely different story: there's a finite amount of frequency space and the frequency range that can be used for the upstream traffic is a minuscule chunk of that. The common example of a node layout is a central hub with "strings" running North, West, East, South. When service was originally built out all 4 strings likely shared the same upstream and downstream channels for HSI, and as the service grew these were eventually split out. That means if you have upstream channes 1-4 available, you can break them out so that North 1-4 are different than South 1-4, etc. The real problem comes when one segment like "West" becomes heavily loaded; at that point once you are out of frequency space your only option for providing more segment bandwidth is to split the copper plant and insert another HFC node. At that point you're talking about 5-6 figures when you work out right-of-way for the fiber path, laying additional conduit, pouring the cement slab to hold the node (if required), getting power to the node, the cost of the node itself, plus the cost of additional CTMS ports at the head-end to support the fiber connections for that node. The labor involved isn't simply a "cut and connect" operation either, as all of the amplifiers for that entire segment will need to adjusted or in some cases moved to more ideal positions on the coax plant.
The major cost of providing the service is in the last mile, not the head-end. | |
|
