Cox → [Modem] SB6190 is a terrible modem - Intel Puma 6 / MaxLinear mistake

Im getting further into the problem. It looks intractable and its the underlying tech which is patented.

((** I have been corrected that the entire issue may indeed be mostly the Intel Puma6 chip that is the issue NOT the Max Linear technology. I cant isolate the 2 so I cant verify that. But it seems likely. So the below might be wrong **))

Max Linear has patents that make for MUCH cheaper chips to do multichannel receivers. Rather then use individual tuners for each channel. They just capture the entire spectrum at once and then do math on the WHOLE SPECTRUM and remove the signals mathematically. The tuners are mathematical rather then physical. This is VERY different from previous designs. Its cheaper to make, lower power, less part count ( cheaper to make ) and allows any number of channels to be tuned at the same time across the entire spectrum.

HOWEVER.. Doing all this serious math in a DC-1.2Ghz wide signal with thousands of signals on it + noise requires massive computational power. This math has to be done at a stunning rate per second. Then the resulting signals get decoded and fed to another chip.

This whole process, patents galore, has a down side obviously. It takes time to do it all. LATENCY.. This time to do it all varies widely each time it does FSC sample. So this varies widely per second. As the signals and noise change, the decoding time changes as it is harder/easier to find, demux, clean and decode signals. This introduces jitter.

So.. There is no way to lower the latency of this process without using incredible processing power and tons of power. It would also become horridly expensive. So there is no way around a huge increase in latency. That appears to be what im seeing in the above measurements.

Same thing for latency jitter. A cleaner signal with less distortion and noise will lower jitter from this technology, but it will ALWAYS be present.

Soooooo....

((** I have been corrected that it may indeed be mostly the Intel Puma6 chip thats the issue NOT the Max Linear technology. I cant isolate the 2 so I cant verify that. But it seems likely. So the below might be wrong **))

The entire technology these chips are based on is flawed for home modem use and will introduce massive latency increases and jitter. You gain more profit as the box is cheaper to make and it draws less power. it can also handle a huge number of bonded channels at once. So you can boast speed.

This technology should be avoided for home browsing and gaming use. Its fine for one way applications like sat rcvrs and cable TV boxes. Its a bad idea for 2 way boxes that are latency and jitter sensitive.

I have reached out to Max Linear in email about this issue. But I think its safe to say that products based on "Full-Spectrum Capture" will inherently add unwanted and significant latency and jitter in ways no previous technology has.

Full-Spectrum Capture

I don't seem to have the 250ms spikes like you, my ping sticks right around 70ms, which is higher than I remember in game. I did also notice a delay in normal web browsing after I upgraded from my SB6141 to this 6190. Was almost considering trying to sell this and switch to a Netgear CM600.

Do you have any experience with that one?

get the SB6183 and a good router. Use a wired connection. The 6183 is the best way to interface to cable.

Also the 250ms spikes are so short they will only effect a single keystroke. So unless you use good testing like I did you wont normally see them. BUT you will feel them as when you press a key, it might take 1/4 second longer for only that one keystroke. Its random and pretty serious cuz its sneaky.

((** I have been corrected that it may indeed be mostly the Intel Puma6 chip thats the issue NOT the Max Linear technology. I cant isolate the 2 so I cant verify that. But it seems likely. So the below might be wrong **)) The issue is real, but it looks more like blaming Intel is the proper course.

MaxLinear Full Spectrum Capture technology used in this modem is TERRIBLE for gaming..

Im not joking This technology is HORRENDOUS for gaming. It produces random 200ms+ jumps in latency for single keystrokes.

Welcome to the Puma 6 users club. Nice to see someone else take a serious look at this. That latency is typical of Puma 6/6-MG modems and gateways, and this particular chipset is used by a number of manufacturers and ISPs as a result. Your SB6190 has an Intel Puma 6 chipset, as seen by the "DHCE2604 11L543C679SR299" imprinting. Thats the first time that I've seen the 2604 designation, but that appears to fit in with the Puma 6 line. Its paired with the MaxLinear MXL268 tuner, which is also common:

»www.maxlinear.com/maxlin ··· eceiver/

Have a look at the following post which will hopefully provide enough details to flush out your theories. Fwiw, the latency is due to the packet processing thru the Puma 6 CPU, instead of the Hardware Accelerator/Processor:

»communityforums.rogers.c ··· 6#M36956

Also fwiw, ping the modem's Default Gateway Address and the CMTS address which is the first address to show up in a trace to anywhere when you have the modem connected directly to the pc. Back off on the ping rate, one ping per second or two is sufficient to see this. Using multiple pings per second might result in rate limiting by the CMTS which will cloud the issue and inject packet loss which is due to the rate limiting, not to any connection issue between the modem and CMTS. There are two issues on the go here, the latency to the modem which you can see by pinging the modem, and the latency thru the modem, which you can see by pinging the CMTS. There are other issues with a Puma 6/6MG chipset, but those are probably the worst for the average user.

I doubt that MaxLinear knows anything about this. All they do is supply the tuner. What happens after the tuner, they probably wouldn't care about. By now I would say that Intel certainly does know about this.

@chris_c21, your SB6141 is a Puma 5 chipset modem. The 6190 is a Puma 6 modem. It would probably be strange to see the additional latency by just web browsing. Gaming or running some other latency intolerant application would see the difference. Several people have switched to the CM600 and reported that they were very satisfied with the reduction in latency with the CM600, as in back to normal.

Good to know im not crazy...

Well then im sorry to the MaxLinear guys I will correct my post.

So its a Intel issue.

As my pings have never been rate limited to the CMTS over 8 years, at 3 per seconds, I think im OK. I can do 10 per second and they still come out OK but that starts to stress out the computer and multiping. You actually really do need this fine temporal detail to see the fine 250ms+ spikes in latency. You miss them with 1 sec pings.

This puma6 stuff is very bad. I will do more research. Thanks.

I set multiping to the smallest packet size and 0 delay between pings. You must type in .3 or .1 into the drop box to get below 1 sec on multiping. And yep anyone wanting to reproduce what im doing you do a tracert from a cmd prompt to someplace like google.com and the first stop out that responds ( besides yourself ) is the CMTS.

I would caution tho that Cox might get angry if you ping them at this rate for too long. So. Be nice and only ping at this rate when doing testing.

i have edited all my above posts to more blame Intel Puma6. I will need to find the person directly responsibe for this Puma6 chip program and email them the link to this thread

Datalink, so the Max Linear is just a digital filter chip ? it seems like its doing more ? If its just a wideband digital filter,, then this is all Intel's fault. I want to be sure to assign blame for this as its really bad..

Why companies like Arris have gotten in bed with this clearly dysfunctional chip needs to be addressed.

Very interesting sets of stats. Thanks for sharing!

Just another point to add - Not only home, but BUSINESS use. Don't forget that coax can (and does) also service businesses!

I bought one of these, blindly, off the shelf at a local store thinking "it's an 'almost Moto,' and it has the higher channel count in case we get 24-32 channels here..."

After some research (and posting here!), I returned it for the 6183, as that was also on the "supported" list of Business Service modems.

You know what? ...We're actually "just fine" with that 6183 unit.
We have a few hundred users. The thing actually keeps up, and keeps up quite well, I must say. Had thought about the Netgear 24ch. modem (CM600) but it wasn't on "the list" so I stuck with a 16ch. model.That is a very interesting approach, but you'd be correct in saying that it's one which would require a rather impressive chip behind it to do the heavy lifting. Maybe, just maybe, someday, a really good ASIC could do this, but why bother? It's quite obvious that a "dedicated" tuner has already pretty much been proven to be "the way" to do this, right? Even with 3.1, it would probably still be "the way" to do it.

For some users, perhaps they won't notice or care. It does technically work, just not as well as 99% of other 3.0 modems out there. For a non-technical person, who doesn't know any better about any of it, if it's working, it's working.
Still, they really should can this model and let the stock run out.

It's too bad Cablelabs doesn't need to take pictures or tests for DOCSIS certification. One of the first things I do when buying a router is looking at the internal FCC pictures. Consumer devices are designed to be flashy and get the users attention, but the real goods are inside. If the inside of the device looks like shit, chances are the device is shit.

@xymox1, consider this as food for thought.

»www.theinquirer.net/inqu ··· and-chip

That article dates back to May 2012. Ballparking the dates, that would put development of the Puma 6 / 6-MG chipset to somewhere around 2010, with sampling around early 2011. The ealiest date that I can find for a Puma 6 product is the Arris TG1672G in 2012. That is a 16x4 product. Maybe there were other products out sooner than that?

Consider the date on the Maxlinear announcement for the 32 channel receiver:

»www.maxlinear.com/maxlin ··· eceiver/

Sept 2014. That would put development finishing around late 2012, sampling in 2013, and production around 2014.

So, the Puma 6 / 6MG chipset is apparently designed to accept different MaxLinear receivers, starting initially with a 16 channel receiver in 2011/2012. If my speculation on the dates is anywhere near correct, that means that a 32 channel receiver wasn't available for testing for another year, perhaps even two years. I would also speculate that a 32 channel CMTS might not have been available either.

Initially with 16 channels running, the CPU probably has enough horsepower to get the job done without showing any latency. I suspect that someone, somewhere, made the decision not to pursue any development of the firmware that would be required to route the data packets through the Hardware Accelerator/Processor. There probably didn't appear to be any need to do that at the time so it was either never pursued, or it was dropped. Why spend resources, money and time on something that isn't required?

Well, fast forward two to three years, and now the 32 channel receiver arrives on scene, doubling the required processing. And guess what, now you see the latency that occurs when you process the packets thru a CPU based software process. Now is probably the time that the ability to route through the Hardware Accelerator/Processor is required, and ...... its not available.

I would speculate that the SDK that the various modem companies use is probably the same, which would explain why all of these modems suffer from the same symptons, no matter who manufactures them. So, further development is required to get the job done. Now however the product is a mature product, so to speak. Intel has already developed the Puma 7 chipset and its in production. Maybe the Puma 8 & 9 chipsets are in development? Looking back at the 2010/2011 timeframe and the personel that developed the chipset, I wonder if those individuals are even still with Intel. If they are, they have most likely moved on, so now someone else would have to go in, learn how the chipset operates and develop whatever firmware is required. Having seen that in operation before, I know that its not that easy to do.

So, who's to blame? Probably no one. The chipset was developed and tested with the receivers and CMTS equipment that were available at that time, and at that time, maybe it ran perfectly. Fast forward to today, and end users can see the latency issue with the 32 channel receiver onboard. Did anyone go back and test the chipset with that receiver onboard? Thats the interesting question. Cable Labs and the Wifi Alliance test the modems and certify them for correct operation in their specific areas, but, from what I can see, there doesn't appear to be any test agency that will put a device through its paces, testing all of the end user IPV4 and IPV6 protocols through the device, looking for standards adherence and throughput capability and issues. If that was done, I think this would be been detected long ago. Derived from that testing would be a report, which would show the test results. With that on hand, it would be pretty simple to determine which modems do it right, and with respectable throughput, and which one's don't. Cablelabs would probably be the ideal place to do this as you need the CMTS test capability as well. But, that's just a personal opinion.

One final point to remember, its not just Arris. Hitron and Cisco also use these chipsets and no doubt there are probably more. There's at least a dozen ISPs if not more across North and South America and Europe that use the final products in their many different variations. So, that adds up to a lot of unhappy end users as well as a good number of users who aren't techie enough to understand that they have a problem, whether they know it or not.

As I said at the top, food for thought.......

How does one switch easily between modems to A/B test? Do you call Cox and supply MAC address every time you swap?

I have an SB6190 that replaced an SB6182. Only 8x4 but curious to see results.

Interesting reading. I guess I am left with the question, is this something that can be fixed or alleviated with the firmware or not? I am guessing not based on what I'm reading.

Yes and No. Yes: a firmware change has been shown to drastically improve IPv4 ICMP ping times. No: IPV6 ICMP, IPv4 & IPv6 TCP/IP, UDP, FTP, and every other protocol has yet to been seen. Hopefully just a matter of time. Unfortunately this can't be done in one single revision. Its turning out to be a step by step process. Have a look at the following post for the before and after firmware change comparison for IPv4 ICMP. This is for a Hitron CGNM-3552 32x8 modem running in 32x3 at the moment.

»communityforums.rogers.c ··· 6#M36956

"How does one switch easily between modems to A/B test? Do you call Cox and supply MAC address every time you swap?"

Yep. Thats exactly what you do.. Cox is fine with doing this they tell me. They keep them in the system so its a pretty much a single click for support.

" is this something that can be fixed or alleviated with the firmware or not? "

Im guessing not. Or even if so, I dont think they would want to go thru the expense ? Maybe ?

Datalink.... Wow... i have so much respect for someone who knows what they are doing.

A do think someone is to blame. In my case Arris. The box alone is cause for concern. 1.4Gbps ? Not possible in any way with a 1Gpbs ethernet cable. I blame them right away for misleading statements. Surely they have ways to test latency and latency jitter in house ? I can do it with free software.

"1.4Gbps" I suspect thats the max throughput including overhead. Take away the overhead and the max that you might see is 1 Gb/s. More likely somewhere around 940 to 960 Mb/s.

Just FYI, swapping between a stand alone modem and anything that has telephone function (eMTA or eMTA gateway) is more complicated. The devices will have a telephone and data occurrence, and removing the telephone occurrence from a eMTA will sometimes (most times?) delete everything related to your phone system (voicemail, call history, etc). I only mention this because the TG2472 is a popular phone/internet gateway and I have seen users have issues swapping between it and other modems. CSR's seem to always forget to re-add the telephone occurrence.

Also, is it possible the issues with the SB6190 is because it has the ability to use so many downstream channels? It's like adding a bigger antenna for OTA, yea you get more signal but you also get more noise. I think the chip inside is supposed to help with that, but doesn't. So IMO the issue is general ingress issues on those channels between 16 and 32 and a firmware issue that causes latency gaps while the modem experiences issues on one of it's downstream channels. I think that is why only certain people in some areas have the problem with the SB6190.

32 bonded is 1372 IE 1.4 Gbps. . Never mind the ethernet port... Or overhead.. They should just add the 32 down and 8 up together ! 1.6 Gbps !!!

We do need modems and routers with SFP+ ports tho.. Its that or 10Gb Ethernet. At least dual bondable 1Gbps ethernet ports.

Sir Wobbler. Yea more channels = more noise, more processing, more decoding, more everything. More CPU use..

" I think that is why only certain people in some areas have the problem with the SB6190."

I would argue they all have this issue to some degree. But few have really looked correctly. I think Multiping to the first hop out and set for smallest packets and a rate of .3 will expose that all these have that issue to some degree. You are 100% right tho, the issue will for sure get worse with channels times noise and all sorts of other factors. When the CPU has to work harder. For example, if you put one on a 16x4 system maybe its almost no issue. I think Datalink really nailed it. The CPU is old and key features were not enabled and now we see the CPU struggling and causing latency and jitter. A Puma 7 would have been a better choice. I see a SB6193 in our future.

"swapping between a stand alone modem and anything that has telephone function (eMTA or eMTA gateway) is more complicated. "

I would get a modem separate from the emta. This is a legal and supported option. Being stuck under a emta sucks. You have far less choices. You just buy a modem or router and call Cox and tell them you want both at once. You will need a coax splitter and short coax cables. I know 4 people who have done this and cant live any other way now.

Wow. Good work.

So essentially the "fix" is moving the data packet processing from software down to the hardware accelerator? If so I'm curious why they didn't opt to use it from the start.

My guess is that there didn't appear to be any need to do that at the time. If there was any latency that could be seen, maybe it was considered to be negligible. Just guessing. It is however a very good question. That leads into the Puma 7 which is in production. That SOC also has a Hardware Accelerator. I'm wondering if the same mistake has been made?

I would recommend this too, but sometimes signal strength is a issue. See my recent thread for example. Using two modems means splitting the coaxial signal to both. Also, by not having data provisioned to the eMTA, the web access is disabled, stopping anyone from checking it's diagnostics or resetting the device. This is useful since some still have batteries and the reset button is hard to get to if the modem is out of the way. However my main warning was for people switching back and forth between a SB6190 and a eMTA for testing purposes. If you have any important voicemails, I would suggest saving them to your computer before calling Cox to do any swap.

I have the SB6190 and just recently went from 12d to 24d. I have noticed a little bit more spiking on ping tracks but haven't heard any complaints from the kid that is a heavy gamer. I can definitely tell something is odd glad to see someone get to the bottom of it though! Kinda crappy that I passed up the CM600 since this one had 32d, was trying to future proof myself a bit. Now I'm stuck trying to sell this modem and spend more money on a CM600. Damn technology! Wish they would just fix the issue.

There's a lot more info on this here:
»Ping spikes - DG2470; SB6190; Requesting metrics for troubleshooting

It sounded like they were working on updated firmware months ago? Still not resolved? That would make me worry that it's not fixable in firmware.

Fortunately I saw this post BEFORE I bought my 6183, coming from a DG1670A with a similar latency spike problem. It made the choice to save money and not pay extra to future-proof real easy.

Just to make it clear, there are two issues going on in that thread, the LAN side ping issue, which is the ping time to the modem, and the WAN side issue, the ping time thru the modem which xymox1 has shown in his pingplotter results. Understand one thing however, its not just ping times thru the modem that are affected, its everything, ICMP, TCP/IP, UDP, etc, etc. The update that was issued for the 6190 fixes the ping time to the web server, 192.168.100.1. That does not fix the ping time to the modem's gateway IP address or the processing latency thru the modem.

I admit im proud to have brought this to the forefront. i know that MaxLinear, Arris and Broadcom have all looked at this thread. It helps that i sent links to the right people. My intent with the post and emails was to call attention to this issue because we cant let S P E E D be the blinding only factor in marketing set top boxes. latency and latency jitter are also serious, and maybe even more serious concerns. It could even be a new way to differentiate a product.

Ive done more real life testing. This effects DNS. Thats what causing simple browsing to judder in its page flipping. Randonmly a DNS look up that would take 20ms from Cox is taking 200ms. You can percieve a 1/4 second. If you get 2 or 3 of these spikes in a cluster on one page load its gonna stutter looking up those 3 DNS hits. Its also effecting TCP setup times too.

Hyper low latency does wonders for DNS as it does just establishing TCP connections to the gateway. These effects are real and can be easily seen by just browsing the web. Once cached the DNS look ups and TCP become a non issue. So its real obvious on first time page loads.

Casually I would say that for a normal user experience, page load speed is way more important then 1.4Gbps speeds. Latency is what to focus on FIRST. For a normal user they will never use 300Mbs let alone 700 or 1400Mbs. My router has usage charts and keeps track of peaks. In 6 months ive never used more then 27Mbs at any time. But I am a single person in the apt. Speed is just a meaningless marketing number that now means nothing.

Routers have meaningless number like this too. Home router say they do gigabit. Ummm, no... With NAT on and say Statefull Inspection firewall they will be really lucky to do 200Mbs with 64 byte packets. A LOT of traffic is 64Byte traffic.

Even with a real router with 36 cores doing 1.2Ghz on the cpu you barely get past 1.5Gbps with 64 byte packets, NAT and some firewall rules. I could claim my router does 27Gbps tho. It depends on the test. See the bottom of this page. There are real, standardized, performance tests for routers, see bottom of that same page. But consumer router mfgrs avoid this testing at all costs and claim the biggest number they can fabricate. Like the 1.4Gbps on the of the Arris SB6190 box. This fixation on SPEED above everything else, real life performance be damned, needs to be addressed.

For example what spped can the SB6190 really do ? Well,,, for sure not 1.4Gbps, ahahahah..

I am glad to bring these issues to the attention of management at Arris, MaxLinear and Broadcom. Intel does not seem to care. So. Im hopeless there. But Arris might care. So they can maybe bring pressure on Intel.

So what options do we have? This isn't the first post calling out the Intel Puma6/6190 for latency issues that I've seen here.
Netgear CM600/700? I'd really prefer not to give Netgear another penny due to their business practices, regardless of what chipset they use.

Take out the 6190, and the netgears - yes the 6183 is out there - is that the best STABLE modem available these days?

Nice investigating! I finally gave up on my 6190 after 8 months of headaches... I just switched to a CM600... I've always used Motorola/Aris modems for the past 13 years... 6190 was such a huge headache and disappointment, that I moved on. Cox has 32 channels active and I want to use it, but no other modem offers 32 channels.

Yup, I've been posting about this issue for a few weeks now as have others. The problem with determining the issue is the lack of contact with the engineering community of your ISP. If you're fortunate to have that, you can obtain some straight answers to the problem, to the point where this makes sense, and to the point where you can see the results yourself by involving yourself in firmware testing. The front end tech support staff and field techs, as well meaning as they might be, can't help, as this is an engineering issue that requires sorting out. Hats off to xymox1 for posting the results of his testing on the 6190. I absolutely agree that Latency should be a major consideration, along with throughput. It would be great if an organization such as CableLabs expanded its testing beyond DOCSIS certification. What about the other side of the modem?

The best option? Maybe wait for the next generation Broadcom chip to show up in a modem:

»www.cedmagazine.com/news ··· -chipset

Thats from last year, so I expect these to start showing up in modems fairly soon. Its possible that ISPs have these on hand for inhouse testing at the present time.

»www.broadcom.com/press/r ··· =s951482

»www.fudzilla.com/news/39 ··· is-ready