dslreports logo
site
 
    All Forums Hot Topics Gallery
spc

spacer




how-to block ads


Search Topic:
uniqs
119
share rss forum feed


espaeth
Digital Plumber
Premium,MVM
join:2001-04-21
Minneapolis, MN
kudos:2

2 edits

1 recommendation

reply to sortofageek

June 2008 - Yet another DOCSIS 3.0 install - Minneapolis MN

Click for full size
Side profile of cable modem, DSL modem, VoIP ATA, Firewall, and Media server
Click for full size
Front profile of modem
Click for full size
Modem w/ flash
I know sorto' was trying to keep core DOCSIS 3.0 discussions in this thread, so even though this topic got derailed a bit a while back I'm going to post my install info here. (hope that works!)

Ordering

I first signed up for the service the day it was available, and then the following day after I found out upload speeds on the 8/768k plan were bumped up to 8/2 I called to cancel. I wasn't that thrilled about the required professional install ($39.95?) and was still a little uncertain if I really wanted it bad enough to justify the $149.95 price tag.

Last week I finally broke down and decided to go forward with the order again. I called and spoke with a sales rep who said that all I needed to do was stop by the office and pick up the modem and they would provision everything for me there. The Comcast office is about a half mile from my house and is right on my drive home, so stopping there was no problem at all.

Unfortunately once I got to the office, they told me that they aren't handing out the 2505s and that a professional install would need to be scheduled. The rep said that as long as I was there she would schedule the appointment for me, but she had to call for support because she didn't know how to code the order.*

Install

My install window was 3-5pm yesterday, but the first tech showed up a little early at around 2:45pm. The tech asked where I wanted the modem installed so I showed him my setup in the basement; he gave me a puzzled look for a few seconds because apparently his work order just said "Install HSI service" (* might be related to how the order was coded). He said he wasn't sure why a tech was dispatched for a modem swap, and that he would call his supervisor to confirm a few things. After he got off the phone he said that he couldn't do the install and that a commercial tech needed to be sent out.

About 20 minutes later the commercial tech arrived with the modem and proceeded to compete the swap. This ended up taking longer than I expected, only because for some reason the system wouldn't let him swap my owned SB5100 with the 2505 from his phone/PDA app that he was using for provisioning. A quick call to the office so they could manually change the assigned modem and things were up and running.

The service

The Scientific Altanta 2505 modem provided has 3 lights to indicate lock on each of the downstream channels, and a single upstream light. For those who care, the bootup sequence is a lock on DS1, followed by a lock on US, then DS2 & DS3 lock and ONLINE lights up.

I've only been able to do a little bit of testing, but the results aren't quite what I expected. It appears that the 50/5 service (provisioned as 55mbps/5.5mbps) is actually 3x18.33mbps/5.5mbps. This is important because it appears that the configuration is setup for per-flow load balancing across the channels instead of per-packet. This is likely configured this way to prevent out-of-order TCP packets between the channels. Since each DOCSIS channel is still shared infrastructure, it's possible that heavy traffic on one channel could delay delivery of packets more than the other two. In a best case scenario this would slow the rate at which your computer will increase the TCP window size and keep your transfers artificially capped for speed; in a worst case scenario your box will interpret the out-of-sequence packets as dropped packets and your transfer rates will turn to utter garbage. The net effect of this configuration is that each TCP session will be limited to about 18.33mbps downstream.

With my SB5100 on the 8/2 tier with Powerboost I would hit 3MB/sec downloading a 100MB test file from a box I have colo'd at Gigenet in Chicago, and after 30-40MB it would drop down to 1.0MB/sec. With the 2505 my transfer starts out at 2.2-2.3MB/sec and ends at 2.2-2.3MB/sec. So Powerboost was actually faster for each transfer initially, but the DOCSIS3.0 transfer still wins out because it's dead consistent at 2.2-2.3MB/sec for the entire transfer.
--21:48:17--  http://cluster/100mb.bin
Resolving cluster... 
Connecting to cluster|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 104857600 (100M) [application/octet-stream]
Saving to: `100mb.bin'
 
100%[===========================================================>] 104,857,600 2.21M/s   in 45s    
 
21:49:03 (2.22 MB/s) - `100mb.bin' saved [104857600/104857600]
 

Most speedtest sites use multiple TCP connections (which would be balanced across the channels, since it's per-flow balancing) to test throughput, so Speedtest.net and others report numbers that are right on the money for the service:



Others have reported that the Minneapolis market was going to test a 25mbps connection, but based on what I'm seeing on the 50/5 service that was likely postponed because it would have resulted in 3 x 8.33mbps channels, or 2 x 12.5mbps channels.

Q: Do you get a unique IP address with the 50/5 service?

I didn't. In fact, I got the exact same IP as I had with my SB5100 once I got hooked up with the 2505.

Q: Are there usage limits on the 50/5 service?
I don't use my connection enough that I'll probably ever find out, but according to this guy, yes: »Review of Comcast by apok86

Conclusion

As much as I can have a conclusion about the service after just over 24 hours, overall I'd have to say I'm pleased with the results. The 5mbps upstream speeds are absolutely magical; uploading files to one of my web servers was remarkably quick last night. One thing I did notice, however, is that my overall latency went up about 2-3ms with the new modem; my server in Chicago always had a consistent 17-18ms round trip time, now it's consistently 19-21ms.

That's my $0.02 for now -- I'll post more information as I have it.


EG
The wings of love
Premium
join:2006-11-18
Union, NJ
kudos:10

Re: Yet another DOCSIS 3.0 install

That is very interesting info espaeth. Thanks for sharing !


espaeth
Digital Plumber
Premium,MVM
join:2001-04-21
Minneapolis, MN
kudos:2

1 edit
reply to espaeth

Adjusting TCP stack for 50mbps - CentOS Linux

Update
Now that I've had a few days to test, I've been able to determine that my upper limit of 18mbps is purely coincidental based on standard TCP window scaling parameters and the fact that every remote device I can use for testing is > 18ms away. I went back and read the DOCSIS3 spec regarding their bonding solution, and they address out of order delivery by having massive buffering capabilities in the modem, which re-orders packets in the correct sequence before dumping them out the Ethernet interface. I spent a bit of time playing around with TCP tuning parameters on one of my boxes I colo @ Gigenet in Chicago, and I've been able to get transfers to hit 50mbps in a single TCP flow.

Server kernel version:
Linux cluster 2.6.18-53.1.21.el5xen #1 SMP Tue May 20 10:31:46 EDT 2008 i686 i686 i386 GNU/Linux
 

Server apache version:
Server version: Apache/2.2.3
Server built:   Jan 15 2008 20:33:30
 

Server TCP modifications:
net.core.rmem_max = 8388608
net.core.wmem_max = 8388608
net.ipv4.tcp_rmem = 4096        87380   8388608
net.ipv4.tcp_wmem = 4096        87380   8388608
net.ipv4.tcp_no_metrics_save = 1
net.ipv4.tcp_moderate_rcvbuf = 1
net.core.netdev_max_backlog = 5000
net.ipv4.tcp_congestion_control = cubic
net.ipv4.tcp_abc = 2
 

wget result: (TCP windows didn't grow to hit 6MB/sec until 50MB into the transfer)
--07:17:26--  http://cluster/100mb.bin
Resolving cluster... 
Connecting to cluster|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 104857600 (100M) [application/octet-stream]
Saving to: `100mb.bin'
 
100%[===========================================================>] 104,857,600 6.04M/s   in 25s    
 
07:17:52 (3.93 MB/s) - `100mb.bin' saved [104857600/104857600]
 

To get a better net result, I created a larger test file. After 50MB into the transfer speeds stayed consistently between 5.5MB/s and 6.2MB/sec.
--08:01:17--  http://cluster/500mb.bin
Resolving cluster... 
Connecting to cluster|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 524288000 (500M) [application/octet-stream]
Saving to: `500mb.bin'
 
100%[===========================================================>] 524,288,000 6.10M/s   in 93s    
 
08:02:50 (5.39 MB/s) - `500mb.bin' saved [524288000/524288000]
 

Conclusion: Amended
So, the 50/5 service really is 50mbps afterall. It just looks like the it takes a lot of work to get TCP to perform at that level. Given the amount of tweaking I had to do to the TCP stack on my Linux box to achieve these results, I can pretty much guarantee that 95+% of Internet servers are not going to be tweaked to provide this level of performance.

Note: these tweaks need to be performed at the SERVER side of the connection; changes on the client side alone will be ineffective at bringing these results!

The most critical components are enabling Appropriate Byte Counting (with aggressive 2*MSS window scaling) to allow the window to scale based on the number of bytes acknowledged with each TCP ACK instead of just based on the number of TCP ACKs, increasing the TCP write memory space for outbound windowing, and changing the TCP Congestion Control Algorithm from 'bic' or 'reno' to 'cubic'.


jbob
Reach Out and Touch Someone
Premium
join:2004-04-26
Little Rock, AR

Re: Yet another DOCSIS 3.0 install

More interesting stuff there Eric. So for now this would seem more marketing hype rather than actual throughput speeds. At least until more servers are tweaked. So rather than paying for 50mbps down speeds you're really paying for "Up To" in this case. At least the Upstream is much better.

I wonder however even without all the server tweaks wouldn't you still be able to get to that bandwidth by say downloading from multiple sources of higher bandwidth providers? Example: Downloading a very large file from Microsoft and at the same time pulling some Nix distro, etc. Perhaps the combined downloads would be able to hit the 50mbps mark as long as the sources can push at higher levels.

After all I see the problem as not so much how fast Comcast can provide it's more of how fast the data can be sent that seems more of the limitation for now!


espaeth
Digital Plumber
Premium,MVM
join:2001-04-21
Minneapolis, MN
kudos:2
said by jbob:

I wonder however even without all the server tweaks wouldn't you still be able to get to that bandwidth by say downloading from multiple sources of higher bandwidth providers?
Yeah, that works just fine -- it's why I was initially sure that it was per-flow balancing across the channels. If I fired up one session it would hit 2.2MB/sec and never get higher, but I could open up another that would hit 2.2MB/sec, and a 3rd that would hit 1.8-2.2MB/sec. Hitting 50mbps across multiple TCP connections has been quite easy, actually, if I choose the right sources.


Chiyo
Save Me Konata-Chan
Premium
join:2003-02-20
Charlotte, NC
kudos:1
reply to espaeth
Awesome work!

The only time I"ve ever gotten close to 50megs constant was using DAP even for files located in Minneapolis. or at speedtest.net

either way still kind of sucks and really makes you wonder if Comcast is going to address this problem otherwise a lot of people are going to be upset they aren't getting their speeds.
--
My Blog:
»abanzai.animeblogger.net/