how-to block ads
|
beerbum
Premium
join:2000-05-06
Reading, PA
clubs:
| The Article - in readable format
From: Dave Burstein [daveb@dslprime.com]
Sent: Tuesday, July 29, 2008 6:10 AM
To: David Farber
Subject: Practical Congestion Solutions (Real Soon Now)
Folks
David Reed asked about how is Comcast managing traffic today, and what is likely later in the year. I have some of the data. I would enormously appreciate better information on what we can expect from DOCSIS 3.0 upstream when it's widely deployed (?late 2009.) I'm guessing it will be very good, but traffic models or other info that translate from 160/120 shared to realistic expectations per user would really help. My guess is 50/25 will be achieved 95+% of the time and that most of the cable upstream problems will be solved, but I just don't have enough data.
Takeaways:
1) The traffic at AT&T is rapidly shifting from p2p to streaming video such as YouTube and Hulu, per data below on the record and factchecked at a senior technical level. This matters, because any p2p solution will not do very much if p2p is much less of the traffic. It might postpone the need for an upgrade for 6 or 12 months, but p2p throttling can't do much more for most networks.
2) The updated model from Sandvine, Comcast's main supplier, allows much less obtrusive shaping. It can be tuned to neighborhood CMTS units rather than a wide territory, can automatically shut off when congestion clears, and enables more careful policies. Things like "if the user has uploaded more than 3 gigabytes in the last five days, reduce that connection to 256K until the congestion clears." Whether Comcast will choose a system like that, and whether the software will deliver everything promised, is not yet known. It certainly could come closer to "reasonable" than what they are doing now.
3) There really is a problem on (at least some) cable upstreams today, based on what I hear from people I respect who have the data. My hope - which won't be tested until 2009 - is that the DOCSIS 3.0 upstream will resolve most or all of the problems for the next few years. Full DOCSIS 3.0 has a minimum of 120 megabits upstream (shared) among typically 300 homes, something like 400K per subscriber. Current cable modems typically have 8 to 30K per subscriber. This is a huge difference.
Verizon, AT&T, and Free.fr are strongly on the record they do not have significant congestion problems. They do not have a shared local loop, and have allocated much more bandwidth per customer. I've sat at the network console of a large ISP and seen they had essentially no congestion amongst the millions of subscribers they served. They allocate 250K per subscriber, much more than current cable. So my best guess is that when cable gets to levels like that, the problem will drastically diminish. DOCSIS 3.0 120 meg shared downstream is now available to nearly 20 million homes (8M at J:COM Japan, millions more at Numericable France, Virgin UK, Videotron Quebec, Korea, and some others.) Very few systems can send at 50 and 100 megabits, leaving enough capacity for customers to actually receive at 50-100 megabits despite the sharing. Unfortunately, the upstream is not ready. Vendors say 2008, many think late 2009. 50/25 to 20M Comcast customers (promised for 2010) and many others is totally destabilizing if the telco doesn't have fiber and the cablecos price like they already are in Japan (100 meg is only about $5 more) and France (100 meg, shared, is effectively $25-30 as part of a bundle.) The price will be the key.
Separately, a note came from Singapore to this list as I was writing, sensibly discussing whether problems are due to the ISP or the carrier. I wish we could think like that over here, but in the U.S. and Canada something over 95% of consumers get everything from the telco or cableco. We just don't have many surviving ISPs.
---------------
I unfortunately don't have the time to integrate these longer comments with the above, so forgive the length. If I had more time, I'd write shorter. The AT&T traffic data at the end is particularly interesting; the rest is not thoroughly factchecked yet but I think mostly on target.
1) The latest Sandvine quarterly call described what they say is now available for testing. Unlike the current models, which need to do something broad like slowing all the p2p across large systems, the new software allows much finer tuning, to the level of the usage of the individual customer. It essentially is a policy server tuned to let policies be set for each customer and enforced in real time. Choice of policy would be by the carrier.
The first improvement is that it can read the network and the individual CMTS, and only invoke policy if that customer is on a congested CMTS. This dramatically cuts how much is shaped.
The second improvement is that it can choose who to shape in a much more sophisticated way. Currently, they get total usage for each customer, so implementing a 250 gig cap for an individual or grossly shaping a whole region is what they are mostly limited to (I believe). Now, they can set a policy that chooses to shape only those who use more than 4 gig upstream between 6 p.m. and 10 p.m. , and only partially shape them (say to 256K up) if that's enough. It can then turn off the shaping automatically when the circuit drops to a more normal pattern (say, less than 60% utilization for 15 minutes.)
Contrast that to today, where apparently Comcast is leaving the stuff on for 18 hours a day, and filtering closer to the core, affecting many locations without problems. (The primary problem is at the neighborhood CMTS, and the current system I believe requires shaping dozens or hundreds at a time because the boxes are deeper in the network.)
How sensible will Comcast be? I can't answer that, but since Sandvine is designing the software working closely with their large customers like Comcast, I suspect the capabilites they are discussing are what the customer intends to use.
None of which means we can let up, but suggests much of the problem can be technically fixed.
2) Neither Comcast nor any of the other cablecos has shared enough data publicly, but privately some have provided information I believe I can trust about one crucial question. There is a real problem with (at least some) cable upstream, which typically designs for 6 to 30 kbps per subscriber. p2p frequently outruns that, even if they do ordinary upgrades doubling and tripling the total shared upstream but staying within that range.
The very interesting question is whether the DOCSIS 3.0 upstream will have similar problems. Verizon and AT&T, with far more effective bandwidth per user, aren't seeing a similar problem. So my first guess is that the the p2p congestion issue is important up to a certain level of bandwidth, but not above that. This is totally unproven, and the best cable engineers I've asked are unsure until they get thousands of live customers. DOCSIS 3.0 is at least a 12x (vs 10 meg upstream cable), for many a 30x (versus 2-3 meg upstream systems.) There's a lot of folks with this "voracious p2p grabs any upstream you can give it" idea, which seems to correspond to some of the experience on cablecos today,
However, Verizon, AT&T, and Free.fr tell me they simply do not have this problem. Bell Canada provided the CRTC with data that shows similar - the only problems of significance were at the back of the DSLAM, which I've determined was they haven't upgraded the link out of the back for many years, and have many OC-3's,etc. I've sat at the network console watching the realtime network map for millions of subscribers. The only congestion at that moment was at an interconnection with Telecom Italia, where Italia refuses to upgrade their side of the link. I believe this is typical of their experience; the network engineer told me he has authority and budget to simply upgrade any link that is a problem. They have fiber throughout, all IP, and are running it as the prototype "stupid network." If they see a problem, they add another Gig-E to the fiber link.
Comcast is going all 3.0 by 2010 - 20 million homes. So one of the most important questions is whether that will be enough to avoid upstream problems. All insight welcome. I have some interesting new data points. AT&T is seeing an overall slight decline in bandwidth demand growth per user, with p2p becoming less and less of a factor. YouTube, Hulu, and other commercial streamers are growing much more rapidly.
Here's a story I have, factchecked by a VP at AT&T Labs.
20% Drop in p2p on AT&T Backbone
Other video, like YouTube and Hulu, twice as high
Easily a third of AT&T's downstream traffic is now “web audio-video,” far more than p2p and the gap is widening rapidly. Hulu and YouTube are taking over, while p2p is fading away on DSL networks. One likely result is that managing traffic by shaping p2p is of limited and declining use, perhaps buying a network 6 months or a year before needing an upgrade. The p2p traffic shaping debate should be almost over, because it simply won't work very much longer.
Jason Hillery provided some current AT&T information. p2p is currently still growing but “at a slower pace than other traffic.” On the Tier 1 AT&T backbone p2p actually dropped 20% during a period last year. AT&T Labs VP Charles Kalmanek points out that a shift in customer mix, rather than an absolute drop in overall p2p, may explain that surprising statistic. Around the world, the trend is clear: web traffic continues to grow at something like 25-40% per user each year, right in line with the trend since 2001. Video is growing rapidly, but not enough to change the trend so far.
Direct from AT&T
“Overall traffic on the AT&T IP backbone network is growing at a pace of more than 50 percent per year. This growth is a combination of customer growth and growth in traffic per customer. Average growth in traffic per customer is about 25-30 percent per year.
To gauge the application breakdown of broadband traffic, we measure downstream traffic during the weekly busy hour. With this measure, as of June 2008, traffic was about 1/3 Web (non video/audio streams), 1/3 Web video/audio streams, and 1/5 P2P (with other applications making up the remainder). For the first time in June 2008, Web video/audio was the highest traffic-generating application over our IP backbone network.
As for the trends we've seen over the past few months:
-- Web video and audio is growing at a much higher pace than overall traffic (more than 70 percent/year);
-- P2P traffic continues to grow, though at a slower pace than other traffic;
-- and Web traffic is growing at a pace consistent with overall growth.”
Many of the policy people believe that p2p is a ravenous monster that is devouring the Internet. The data show that simply isn't true, with the possible but important exception of some cable upstream. Traffic management has become so political few are willing to talk on the record, so reconciling conflicting data is very hard. It appears that large carriers are not having congestion problems on DSL or fiber backhaul, but cable upstream is. With friends, I'm trying to figure out what's going on and in particular whether the problem disappears with DOCSIS 3.0 which will have 12-30+ times more upstream bandwidth. Advice and data very welcome.
Video streamed/downloaded from sites like Daily Motion, Hulu, YouTube and others are coming to dramatically dominate web traffic. YouTube's market share is tending down, Hulu, BBC, Facebook presumably climbing. Heise has a German report that general interest in music over the web is dramatically down, consistent with my belief that most people who want music collections already have more than they can listen to. Hulu has now added movies, free with commercials. Most are old blockbusters The Fifth Element, Ghostbusters, Men in Black, Jerry Maguire and other old favorites. They have also Kurosawa's great Kagemusha, Quills, Enter the Ninja, Some Like It Hot,
AT&T has sensible plans to handle the load without disruption. They are already moving from 10 gig to 40 gig in the core, and planning a transition to 100 gig in a few years. The current projections are they can do these upgrades without raising capex, bringing per bit costs down along a Moore's Law curve and keeping bandwidth costs per user essentially unchanged. Most of the optical vendors believe they can meet those goals, although some worry that the pace of innovation may slow down as the optical components industry is struggling.
Sorry I didn't have the time to write something shorter.
db
Editor, DSL Prime | |
karlmarx
join:2006-09-18
iraq
 ·Fairpoint Communic..
| So, even with Docsis 3, why are they promising 5mb/subscriber per 300 people, when they only have 120MB total? That does not make sense. Provide 1mb/subscriber, and with 300 people, that should be good enough. I mean, if you only have 1mb/sec, then you sure as hell can't use 4GB/uploads in 4 hours. Why don't they just sell what they can provide.
Do the math.
Assume 20% of their subscribers use P2P. Out of 300 houses, that would be 60 users. If every user does 5mb/sec, it would take a total of 300mb/sec to just service those users. And they already admitted they only have 120mb/sec. Solution: SELL 1mb/sec upstream! Then, those 60 users can only use 60mb/sec, and that still leaves 60mb/sec for the other 80%. Gee, very simple solution.
Oh, wait.. hmm.. Verizon provides me with 15mb/sec upstream. And they have 200 people per node, but an upstream capacticy of 1.2GB/sec. That means that 40 people at 15mb/sec use a total of 600mb/sec, and that leaves 600mb/sec for the remaining 160 people. Hmm.. which one is a better system?
Which would you choose, if the price was the same. 1mb/sec upload or 15mb/sec upload. Not a very hard decision there. 30mb/sec down or 6mb/sec down?
--
The happiest countries are the most secular. The struggle AGAINST corporations is the struggle FOR humanity! | |
JasonOD
@comcast.net
| reply to beerbum
So it's more of a shift from P2P to legit web video as content providers have made more available (hulu, for example), not less total traffic, just a shift. I bit of irony there I guess.
And if byte-billing doesn't work (which seems to be the thrust of the article, since it downplays capacity conerns), I smell new opportunities for ISP's. Unlike p2p, it's easy to find the bigger content houses like Hulu (again, for example). With a massive presence on ISP's networks, I'd think the ISP's would have room to negotiate revenue sharing or have the video sites deliver contextually related ads that benefit ISP's concerns. | |
espaeth
Digital Plumber
Premium,MVM
join:2001-04-21
Minneapolis, MN
 ·voip.ms
·Vitelity VOIP
·Callcentric
·VoiceStick
·ViaTalk
 ·Comcast
·Embarq
| reply to karlmarx
said by karlmarx  :So, even with Docsis 3, why are they promising 5mb/subscriber per 300 people, when they only have 120MB total? Outside of testing with Spirent, Smartbits, Ixia, or similar there is no real-world IP network scenario where all ports of attachment are maxed out.
The only time we've ever seen anything close to that on the Internet is when SQL Slammer first hit.
Your density estimate is also off a bit. For example, for Comcast they have published infrastructure stats here, they have an average of 468 homes passed per node. That's houses capable of getting a signal, but doesn't reflect actual subscriber density. They also say they have 102k HFC nodes. Last year Comcast had about 13.5 million HSI subscribers, so 13.5 mil / 102k nodes = ~132 HSI subs per node. | |
funchords
Hello
Premium,MVM
join:2001-03-11
Washington, DC
·Verizon Online DSL
·Skype
| reply to JasonOD
said by JasonOD :
So it's more of a shift from P2P to legit web video as content providers have made more available (hulu, for example), not less total traffic, just a shift.
I wouldn't be surprised to find two other factors:
First,
The "legit" content providers generally encode either less generously and perhaps also more efficiently than the "pirate" counterparts. So the overall size of the same content is smaller, even if the content providers add commercials.
Second,
But the biggest sea change is probably the declining sense that content will disappear. A lot of what I've learned about P2P has to do with psychology -- that P2P users tend to download more content today on the assumption that it will become unavailable for download tomorrow.
The "legit" sites have (finally) started to drive that fear out of the minds of viewers and I think we're only seeing the beginnings of the effect. That mindset is still very much out there, and content providers are still just getting started in getting their stuff out there.
The industry has to be careful not to abuse the viewers, now. P2P users aren't a tolerant nor trusting bunch. It's just one Sony DRM scandal away from reversing this very positive trend, and too many obtrusive distractions embedded in their content will also kill progress.
--
Robb Topolski -= funchords.com =- Hillsboro, Oregon
More fun, more features, Join BroadbandReports.com, it's free...
| |
-
|
