dslreports logo
site
 
    All Forums Hot Topics Gallery
spc

spacer




how-to block ads


Search Topic:
uniqs
8443
share rss forum feed


telcodad

join:2011-09-16
Lincroft, NJ
kudos:6

1 edit

[IPv6] Comcast Pulls Back On IPv6 Rollout

A report tonight that Comcast has temporarily rolled back deployment of IPv6 protocol in parts of its network, due to a problem reported with the Netgear CMD31T cable modem:

Comcast Pulls Back On IPv6 Rollout, Citing Netgear Modem Glitch
MSO Says Retail Cable Modem Has a 'Critical IPv6-Related Defect'
Multichannel News - March 28, 2012
»www.multichannel.com/article/482···itch.php

"Comcast said it has temporarily rolled back deployment of the next-generation IPv6 protocol in parts of its network, with the MSO citing a compatibility glitch in a retail DOCSIS 3.0 cable modem from Netgear that knocked some customers offline.

"We recently identified that the retail Netgear CMD31T device ships with and runs an uncertified version of firmware that exacerbates a critical IPv6-related defect," the cable operator said in a statement. "To ensure Comcast customers with these devices will continue to have uninterrupted Internet service, we have rolled back IPv6 temporarily in some parts of our network to give Netgear more time to address the issue."

Comcast said it expects Netgear "will soon address the issue for their retail devices, which we will test and deploy on an emergency basis."

Netgear, asked to comment, said that the company last week became aware of the problem with the CMD31T and was "immediately in touch with Comcast regarding the issue and within a day corrective action was taken... We are working closely with Comcast to ensure that customers do not face Internet disruptions for any such future upgrades to the system."

Comcast, in an update on its website (»mydeviceinfo.comcast.net/device.···&sc=1097) listing approved DOCSIS devices, says the Netgear CMD31T device has "a known IPv6-related software defect. This prevents the device from being provisioned on the Comcast network."

The issue with the Netgear CMD31T modems on Comcast networks was brought to light Tuesday by Mike Demler, a Comcast customer in California who writes a blog called EE Daily News (»www.eedailynews.com/2012/03/comc···-of.html).

According to a source familiar with Comcast's IPv6 initiative, the cable operator has known about the Netgear CMD31T's IPv6 compatibility problem for two years -- and that Netgear has been aware of the issue for that long as well.

Comcast is in the process of deploying IPv6 nationally. The move to IPv6 for Internet service providers around the world has become urgent as the pool of 4.3 billion available addresses in the current IPv4 is nearing total depletion.

IPv6 -- which provides an astronomically large number of unique addresses -- and IPv4 not compatible, so without some kind of translation layer websites and other Internet services hosted on IPv4 are invisible to users with IPv6-only end devices (and vice versa).

Comcast is among the ISPs participating in World IPv6 Launch day on June 6, 2012, along with Time Warner Cable and AT&T. Web content companies that plan to participate include Facebook, Google, Microsoft Bing and Yahoo, which will enable IPv6 on their main websites permanently beginning June 6.

In addition, two home networking equipment manufacturers -- Cisco Systems and D-Link -- have said they will enable IPv6 by default across their home router products by June 6."



Mike Wolf

join:2009-05-24
Beachwood, NJ
kudos:3

1 edit

Cool article but can't Comcast just isolate the accounts using those Netgear modems and continue pushing forward with the rollout? Then they can come back and work on the Netgear problem. This way the rest of us don't get screwed over with having to be delayed? This is a serious question and I'm not trying to be a jerk to the Netgear users.



telcodad

join:2011-09-16
Lincroft, NJ
kudos:6

1 edit

According to the EE Daily News article (»www.eedailynews.com/2012/03/comc···-of.html), only "selected" areas, such as the San Francisco Bay Area, are affected by this:

"From Netgear's PR firm, via email:

"Last week Comcast upgraded their Cable Internet head-end systems in select cities which caused the Internet connection failure for a few customers who had the NETGEAR CMD31T DOCSIS 3.0 Cable Modem. We were immediately in touch with Comcast regarding the issue and within a day corrective action was taken. Comcast had rolled back to the older system and all customers were online.

We are working closely with Comcast to ensure that customers do not face Internet disruptions for any such future upgrades to the system."



RR Conductor
NWP RR Inc.,serving NW CA
Premium
join:2002-04-02
Redwood Valley, CA
kudos:1

2 edits

said by telcodad:

According to the EE Daily News article (»www.eedailynews.com/2012/03/comc···-of.html), only "selected" areas, such as the San Francisco Bay Area, are affected by this:

"From Netgear's PR firm, via email:

"Last week Comcast upgraded their Cable Internet head-end systems in select cities which caused the Internet connection failure for a few customers who had the NETGEAR CMD31T DOCSIS 3.0 Cable Modem. We were immediately in touch with Comcast regarding the issue and within a day corrective action was taken. Comcast had rolled back to the older system and all customers were online.

We are working closely with Comcast to ensure that customers do not face Internet disruptions for any such future upgrades to the system."

I wonder if our area is affected? We're well north of the Bay Area, but our Comcast system in Mendocino County is part of their North Bay Network, and everything from this county goes down through Sonoma County, then into the East Bay and then over to Sacramento to get to the iBone (haha, I love saying that lol).

Tracing route to any-fp3-real.wa1.b.yahoo.com [72.30.38.140]
over a maximum of 30 hops:

1 1 ms 1 ms 1 ms 192.168.1.1
2 74 ms 21 ms 29 ms bu-10-ubr03.ukiah.ca.sfba.comcast.net [67.180.40
.1]
3 11 ms 17 ms 9 ms te-3-2-sr01.ukiah.ca.sfba.comcast.net [68.86.249
.21]
4 23 ms 11 ms 11 ms te-8-1-ur06.rohnertpr.ca.sfba.comcast.net [68.87
.226.229]
5 17 ms 15 ms 15 ms te-0-1-0-7-ar01.oakland.ca.sfba.comcast.net [68.
87.192.162]
6 21 ms 19 ms 19 ms pos-2-3-0-0-cr01.sacramento.ca.ibone.comcast.net
[68.86.91.225]
7 22 ms 19 ms 18 ms pos-0-8-0-0-cr01.sanjose.ca.ibone.comcast.net [6
8.86.85.78]
8 18 ms 29 ms 16 ms xe-10-3-0.edge1.SanJose1.Level3.net [4.71.118.5]

9 17 ms 18 ms 17 ms ae-33-80.car3.SanJose1.Level3.net [4.69.152.133]

10 18 ms 19 ms 18 ms YAHOO-INC.car3.SanJose1.Level3.net [4.71.112.14]

11 20 ms 22 ms 21 ms ae-0-d161.msr1.sp1.yahoo.com [216.115.107.59]
12 19 ms 18 ms 18 ms et-17-1.fab1-1-gdc.sp2.yahoo.com [67.195.128.65]

13 21 ms 20 ms 19 ms po-10.bas1-3-prd.sp2.yahoo.com [76.13.244.7]
14 93 ms 83 ms 67 ms ir1.fp.vip.sp2.yahoo.com [72.30.38.140]

Trace complete.

Edit-Our CMTS is in Ukiah, 9 miles south of me. FYI Redwood Valley is 120 miles northwest of SF.


jjmb

join:2009-12-01
USA

1 edit
reply to Mike Wolf

Defects have varying scopes, in some scenarios what you describe may be possible however this is not always the case. We are working very hard to limit the impact to our customers while we collaborate with Netgear to arrive at a viable long term solution.



telcodad

join:2011-09-16
Lincroft, NJ
kudos:6

1 edit
reply to telcodad

This statement is on Comcast's IPv6 Information Center page this morning:

Statement Regarding NetGear CMD31T Devices
Thursday, March 29, 2012


Comcast is in the process of deploying IPv6 nationally, as noted on this site in great detail. We recently identified that the retail NetGear CMD31T device ships with and runs an uncertified version of firmware that exacerbates a critical IPv6-related defect. To ensure Comcast customers with these devices will continue to have uninterrupted Internet service, we have rolled back IPv6 temporarily in some parts of our network to give NetGear more time to address the issue. Comcast anticipates NetGear will soon address the issue for their retail devices, which we will test and deploy on an emergency basis.



RR Conductor
NWP RR Inc.,serving NW CA
Premium
join:2002-04-02
Redwood Valley, CA
kudos:1

1 edit

This is on the device info page for that modem as well-

"The retail NetGear CMD31T device has a known IPv6-related software defect. This prevents the device from being provisioned on the Comcast network. NetGear is aware of this defect and is working with us to test a fix. If you have purchased this device and it does not appear to work on our network, please contact NetGear or the store where you purchased the device."



Mike Wolf

join:2009-05-24
Beachwood, NJ
kudos:3

What Comcast and Netgear should have done was notify retail outlets that if any customer comes in to return or exchange, to tell them that the issue is only temporary with the modem. Now there are demands that Comcast and/or Netgear reimburse customers who jumped the gun and purchased new modems from other manufacturers.



tshirt
Premium,MVM
join:2004-07-11
Snohomish, WA
kudos:4
Reviews:
·Comcast

1 recommendation

Maybe.
But ComCast has no idea how long it will take Netgear to fix the problem, so what are those subs supposed to do in the meantime?
This is Netgear failing to deal with a known problem over several years, hardly ComCast's fault, yet it effects CC's business and upgrade schedule and their relationship with their customers.
One of the big advantages of renting a modem is that it would be on ComCast to replace the modem with a fully IPv6 modem (likely a different brand) When you choose to buy your own ALL the risks of poor design, manufacturing defects, or inadequate support fall on you and it becomes your responsiblity to persue claims against the manufacter.

CC can't hold up their upgrade process for long just because of NetGear or any other single manufacter failed to assure a quality product.
I don't believe CC knew how severe the problem would be, but it would be risk to interfere in the netgear customer relationship... posting "Don't buy their crap" anywhere would certainly get you sued, even if it proved correct.



telcodad

join:2011-09-16
Lincroft, NJ
kudos:6

The thing is, the statements from Comcast say that the Netgear issue "exacerbates a critical IPv6-related defect."

This sounds like there is also a problem on Comcast's side, with their IPv6 hardware and/or software, that the issue with Netgear's modem brought to light.



tshirt
Premium,MVM
join:2004-07-11
Snohomish, WA
kudos:4
Reviews:
·Comcast
reply to telcodad

"says the Netgear CMD31T device has "a known IPv6-related software defect. This prevents the device from being provisioned on the Comcast network." and the other statement "are working with Netgear to find a longterm solution".
Imply that it's a NetGear problem, BUT CC is effected if it knocks their customers offline.
so they give Netgear a few days/weeks to make a statement/ design a recall* (so CC doesn't cut subs off) and if Netgear doesn't CC bans Netgear from their network (big money!!)replaces all rental modems and tries to help those owning them find a better modem at a reasonable cost.

* If netgear hasn't been able/tried to fix it in 2 years, it probably can't, remember CC is leading the cable IPv6 effort and what they experience tody will effect other cable systems when they make the same changes
Netgear MUST make this right, or face a hard (er) future in the modem and networking business.



telcodad

join:2011-09-16
Lincroft, NJ
kudos:6

OK, so then the "critical IPv6-related defect" is something that was lurking inside the Netgear modem and it did not cause any problems until it was used with Comcast's IPv6-upgraded head-end CMTSs.

Damn English language! It's not always clear which words refer to what!


earletp

join:2004-02-03
PDX

said by telcodad:

OK, so then the "critical IPv6-related defect" is something that was lurking inside the Netgear modem and it did not cause any problems until it was used with Comcast's IPv6-upgraded head-end CMTSs.

Damn English language! It's not always clear which words refer to what!

That's the way I read it, plus when you add in the fact it's been a known issue for two years and it was never certified as IPv6 on the Comcast approved modem list, it would tend to back that up.


Mike Wolf

join:2009-05-24
Beachwood, NJ
kudos:3
reply to tshirt

From a story I read, Comcast returned service to the affected customers within a day or so by rolling back the CMTS.



Mike Wolf

join:2009-05-24
Beachwood, NJ
kudos:3

1 edit
reply to earletp

and yet somehow Netgear's own product page says that it supports IPv6. This isn't the first time that Netgear exaggerated IPv6 support on their products. Last year an independent company tested IPv6 support on Netgear routers only to find that when given a /56 address the router would burst into flames in a metaphorical sense and was unable to properly operate. What the issue was was that Netgear certified their routers with a /64 address and since everything worked fine in the lab, they assumed everything else would work.
An excert from the article:
"In our IPv6 trial we hand out a /56 to each router. When I discovered that the PC attached to the Netgear router didn't have an IPv6 address, a little poking around revealed that the router was attempting to perform SLAAC with the full /56, rather than select a /64 out of the delegated prefix. In compliance with IETF standards, the PC wasn't getting an IPv6 address. I can only speculate, but it appears that in its testing Netgear was only handing out a /64 to each router, which likely would have resulted in a successful test"

»www.networkworld.com/news/2011/0···ers.html



tshirt
Premium,MVM
join:2004-07-11
Snohomish, WA
kudos:4
Reviews:
·Comcast

1 edit
reply to Mike Wolf

said by Mike Wolf:

From a story I read, Comcast returned service to the affected customers within a day or so by rolling back the CMTS.

Yes, they backed out of the upgrade (problem avoided, not solved)
so at first glance, some will say "it comcast's fault"....but it 's only this model with a certain "uncertified firmware/software" on it that failed. a model with a known problem. and apparent the software it current ships with.


tshirt
Premium,MVM
join:2004-07-11
Snohomish, WA
kudos:4
Reviews:
·Comcast

1 recommendation

reply to Mike Wolf

said by Mike Wolf:

and yet somehow Netgear's own product page says that it supports IPv6.

CC's extensive test (equipment certs, in house, and Beta) group has uncovered at least a couple incompatible pieces of consumer equipment during the IPv6 and other tests, no matter what the box says. this is a network engineers nightmare to allow all equipment that meets the minmum standard (based on carefully designed tests be the Mfg.) knowing that some (hopefully small) percentage will fail in the real world.


PeteC2
Got Mouse?
Premium,MVM
join:2002-01-20
Bristol, CT
kudos:6
Reviews:
·Comcast

2 recommendations

reply to telcodad

Well, I do not think that there should be any large sense of surprise here. I never expected IPv6 implementation to be seamless, nor should have anyone else.

How long did it take for 802.11n to go from a "draft" version to certified...something like 5 to 7 years? In fact, even today, not all current 11n adapters and routers play along nicely!
--
Deeds, not words


earletp

join:2004-02-03
PDX

1 recommendation

reply to Mike Wolf

said by Mike Wolf:

and yet somehow Netgear's own product page says that it supports IPv6. This isn't the first time that Netgear exaggerated IPv6 support on their products. Last year an independent company tested IPv6 support on Netgear routers only to find that when given a /56 address the router would burst into flames in a metaphorical sense and was unable to properly operate. What the issue was was that Netgear certified their routers with a /64 address and since everything worked fine in the lab, they assumed everything else would work.
An excert from the article:
"In our IPv6 trial we hand out a /56 to each router. When I discovered that the PC attached to the Netgear router didn't have an IPv6 address, a little poking around revealed that the router was attempting to perform SLAAC with the full /56, rather than select a /64 out of the delegated prefix. In compliance with IETF standards, the PC wasn't getting an IPv6 address. I can only speculate, but it appears that in its testing Netgear was only handing out a /64 to each router, which likely would have resulted in a successful test"

»www.networkworld.com/news/2011/0···ers.html

I had seen that review, but it was from over a year ago, and have not found anything more current. So that issue may or may not still be present.

At that time Cisco/Linksys had no routers with IPv6 support, for example.


Mike Wolf

join:2009-05-24
Beachwood, NJ
kudos:3

1 recommendation

reply to tshirt

Based on my experience with my Motorola modems, Comcast replaces the shipped firmware with a compatibe version of its own, even when the shipped firmware is newer. I would think this same practice takes place with other brands as well like Netgear. I also would think that any changes would be tested internaly every step of the way well before it would be released into the public production network.



Mike Wolf

join:2009-05-24
Beachwood, NJ
kudos:3
reply to earletp

mmm actually thats not true because I purchased my E4200 back in Janurary of 2011. It's oriignal firmware 1.0.00 supported IPv4 to IPv6 tunnelling back in November of 2010 until it was disabled in the next firmware release March of 2011 numbered 1.0.01 only to be added back June of 2011 with firmware version 1.0.02.


earletp

join:2004-02-03
PDX

Not to argue, but from that same article you linked to...

quote:
Some, such as Cisco's Linksys consumer routers, don't have IPv6 yet at all, although Cisco has promised to add IPv6 to its new routers by mid-2011.
And further, from a link in that article
»www.networkworld.com/news/2011/0···pv6.html
quote:
It's hard to fathom why Cisco hasn't added IPv6 to its Linksys consumer routers yet, but the company has promised support will come this spring.

It's 2011, IPv4 addresses are officially exhausted, and the world's largest router maker, Cisco, still doesn't support IPv6 in its best-selling line of Linksys wireless routers. This is true even for the new E4200 router released just last month (priced at $180). The company has promised to have IPv6 support for the Linksys line by the spring but has not been specific.


Mike Wolf

join:2009-05-24
Beachwood, NJ
kudos:3

That's because like I said the IPv6 to IPv4 tunnelling option found in the original firmware was not officially recognized as an IPv6 function by Cisco and was turned off in the following firmware version and because it lacked the ability to use native IPv6 connectivity.

Official support for IPv6 was not brought to the E4200, E3200, and E2500 until September with the IPv6 Gold logo certification notation in its firmware release notes.



tshirt
Premium,MVM
join:2004-07-11
Snohomish, WA
kudos:4
Reviews:
·Comcast
reply to Mike Wolf

said by Mike Wolf:

I also would think that any changes would be tested internaly every step of the way well before it would be released into the public production network.

If you put 10 engineers in a room, and told them "Make it PERFECT"
we'd still be on sneakernet.

I'm sure they are thoroughly tested (to death if needed, but some minor idiosyncrasies* must be allowed to exist)
but some effects/defects don't show up until stress tested** which is the point of the limit rollout area wise, and system wise***

*Brandcentric "FEATURES"
** exposed to the billions (really!) of possible hardware/software/equipment/usage combinations possible when big iron touches small systems. ( I think will call it "Internet")
*** I believe they are still in the ~~direct connect to a limited group of modems phase (this modem was known to be noncompliant, but not known to be incompatible)


PeteC2
Got Mouse?
Premium,MVM
join:2002-01-20
Bristol, CT
kudos:6
Reviews:
·Comcast
reply to earletp

said by earletp:

It's hard to fathom why Cisco hasn't added IPv6 to its Linksys consumer routers yet, but the company has promised support will come this spring.

It's 2011, IPv4 addresses are officially exhausted, and the world's largest router maker, Cisco, still doesn't support IPv6 in its best-selling line of Linksys wireless routers. This is true even for the new E4200 router released just last month (priced at $180). The company has promised to have IPv6 support for the Linksys line by the spring but has not been specific.


Cisco has added IPv6 support for a number of its routers to include at least several of the Linksys E series, and the just released EA series routers have IPv6.

I have the new Linksys EA4500 which has IPv6 (so do the EA3500 and EA2700). The E4200 v2 had IPv6 from the release...the E4200 has been out for over a year, and is one of the E series which has a firmware upgrade to IPv6.

--
Deeds, not words


telcodad

join:2011-09-16
Lincroft, NJ
kudos:6
reply to telcodad

Latest status from the Multichannel News site:

Netgear Working On IPv6 Modem Update For Comcast
Users of Vendor's CMD31T Lost Connectivity After MSO's IPv6 Deployment
Multichannel News - March 29, 2012

"Netgear said it will provide Comcast a firmware update for one of its DOCSIS 3.0 cable modems to make the devices compatible with IPv6, after several hundred of the MSO's customers lost Internet connectivity last week when Comcast put the next-generation protocol into operation in several markets.

"Recently, some Comcast customers lost their Internet connection due to an IPv6-compatibility issue we have with the Netgear CMD31T high-speed cable modem," Netgear said in a statement Thursday. "We are providing a firmware upgrade to Comcast for certification to make these devices IPv6 capable."

Comcast on Wednesday said it was rolling back its IPv6 deployment in some markets because of the glitch in the Netgear cable modems. The issue affected fewer than 1,000 subscribers, according to a source close to Comcast.

Once Comcast has certified the upgrade, the MSO will push it out across their network.

"Resolving this issue for customers in a timely fashion is our number one priority and we are working closely with Comcast on the solution," Netgear said. "We apologize for any inconvenience we have caused our customers."

According to Comcast, the Netgear CMD31T device currently "runs an uncertified version of firmware that exacerbates a critical IPv6-related defect," which prevents the device from being provisioned on the Comcast network. The cable operator is in the process of deploying IPv6 nationally, in a "dual-stack" implementation in which both IPv6 and IPv4 are supported on the client.

Comcast provides an "IPv6-readiness" check at »test-ipv6.comcast.net for any Internet user to check for IPv6 compatibility problems."


earletp

join:2004-02-03
PDX
reply to PeteC2

You do realize I wasn't the one that said that, correct?

(edited to add)
To make it easier to follow, click here ... »Re: [IPv6] Comcast Pulls Back On IPv6 Rollout



jlivingood
Premium,VIP
join:2007-10-28
Philadelphia, PA
kudos:2
reply to telcodad

said by telcodad:

The thing is, the statements from Comcast say that the Netgear issue "exacerbates a critical IPv6-related defect."

This sounds like there is also a problem on Comcast's side, with their IPv6 hardware and/or software, that the issue with Netgear's modem brought to light.

It is not a problem with the network...
--
JL
Comcast


Mike Wolf

join:2009-05-24
Beachwood, NJ
kudos:3

hi Jason We missed ya.



RR Conductor
NWP RR Inc.,serving NW CA
Premium
join:2002-04-02
Redwood Valley, CA
kudos:1

said by Mike Wolf:

hi Jason We missed ya.

Mike, I changed routers, I sent the Netgear back and went with the Asus RT-N66U. Anyway, I am having trouble getting IPv6 to work, I can get an IPv6 address using Comcast's 6to4, but every IPv6 test fails, any ideas? I tried HE's 6in4 too, same story. Thanks!