Not to keep kicking the dead horse, but... Either way they go, there will be issues. No updates mean there will be buggy modems out there. Doing the updates means some of them are going to fail -- yes, with a sample size as large as an ISP, there. will. be. failures. (I've been there, signed the PO for the t-shirts.) If you screw up a modem, you'll have to replace it. This makes for a very pissed customer -- esp. if they owned the modem. Angry customers are very vocal customers.
DOCSIS 1.1 introduced a number of security processes for firmware updates. The process is specifically engineered to only come from the network, so no manufacturer even has a channel to get a new firmware to end users.
Given that, it follows that the firmware on any given modem will either be a "GA" release loaded from the factory, or an ISP specific firmware they loaded on it. The operator has to authenticate every modem on the network. So they (should, can, etc.) know the manufacturer, model, serial number, and firmware version
of every device out there. Proper testing means having one of everything
in your testlab. How the 6120 bug didn't happen in their lab leaves me bald. This wasn't some random Chinese modem; it was a very popular, and approved, modem they themselves used, and they were coming from the factory with the old/buggy firmware. It makes no sense that they wouldn't have *one* 6120 with that firmware on it in the lab. Or that they wouldn't've retrieved one from the field as soon as this started to put it in the lab to see *why* it's failing. (I'm also assuming motorola didn't have it striped down so far it wouldn't log why it was rebooting. "invalid field" doesn't help much if it doesn't say which
field; that doesn't mean you cannot find it, just that the engineers will have to earn their massive paycheck to find it.)
I've been there and worked through many a panic. The test lab cannot simulate an entire universe. Bugs are going to be missed; this is a fact. The 6120 issue should've been seen in the lab, but it wasn't. (we'll come back to that when we're looking for someone to fire.) This is why it's suicide to do any rapid, large scale roll out. When an issue surfaces -- and it will -- you now face a network-wide problem instead of localized problem. Rolling back an entire network is, well, more of mess than the upgrade. "Staged Upgrades", say it with me class.
Given the size of TWC (Comcast, etc.) network, they should never be finished with an upgrade -- they should never have the same version on every system... there should be several versions rolling across the network like ripples in pond. </rant>
[Edit: I've worked for and with several ISPs over the decades, but TWC isn't one of them. I have no idea what they test, or how -- and it's never come up in conversation with the people I know there.
There are pictures of someone's test lab over in the bbphotos forum