Sandvine: Netflix, Google ISP Rankings 'Conflicting,' Inaccurate
Network gear manufacturer Sandvine apparently isn't a big fan of both Netflix's and YouTube's new ISP streaming performance rankings, insisting that the data collected by both is unreliable and conflicting. In a blog post
, Sandvine points out that ISPs deemed "HD Verified" by Google's new ISP ranking (discussed by us here
) are sometimes categorized as under-performers in Netflix's rankings, and vice-versa:
Google is essentially saying Rogers’ customers who use YouTube are capable of regularly experiencing HD streams, while Netflix is saying Rogers’ subscribers are experiencing the worst quality of Netflix streaming in the country. At the same time Netflix is saying Bell Canada’s DSL subscribers are capable of experiencing HD streams when using Netflix, but Google is saying Bell non-Fibe (DSL) are not YouTube HD-verified and “should be able to watch YouTube videos in Standard Definition (at least 360p) with moderate load times."
I've already pointed out numerous times
that Netflix's streaming rankings are made less valuable by the fact that Netflix CDN partners do better in the rankings than those who refuse to participate in the free service. Similarly I've noted Google and YouTube's new ISP rankings need some work as well, as navigation is clunky and some ISPs are listed as HD verified
even in markets they don't even provide service.
With that said, it's worth noting that the lion's share of Sandvine's money comes courtesy of the nation's largest ISPs, and as such Sandvine has traditionally been reluctant to admit fault on the ISP end of the equation. The company recently proclaimed that YouTube streaming issues that have plagued most broadband users are entirely YouTube's fault
, even if data tends to suggest there's plenty of blame to go around.
It also seems obvious that ranking YouTube and Netflix streaming performance would be "conflicting," given those rankings are testing entirely different content services, taking entirely different routes to the end user. Routes can be even more different now that companies like Netflix are (begrudgingly) striking direct interconnection deals with ISPs like Comcast and Verizon.
While both Google and Netflix efforts need work (and will hopefully improve), they're at least attempting to shed a little more light on the buffering problems that have plagued the industry the last few years
. That's more than can be said for most ISPs, which have long hidden performance data (unless it's to cherry pick it to justify policies like usage caps, deregulation, etc.). If ISPs want to contest YouTube and Netflix's findings, they could simply release raw performance data of their own
At a time when transit companies are claiming these slowdowns are intentional to make an extra buck
, it seems like every shred of additional performance data is helpful if we're interested in the truth.
Verizon Verizon is HD verified with Google but performs poorly with Netflix. Verizon also uses Google Global Cache...
said by rebus9:And that's a direct result of VZ refusing to update their peering links with Level 3, NTT, Tata, Cogent, TeliaSonera, and virtually everybody else
And it doesn't address peering-specific deficiencies. My FIOS connection runs great-- except for certain things like Netflix, and that's a direct result of VZ refusing to upgrade their peering links with Cogent.
Re: Verizon Peering is a two-, or more, party arrangement
| |fuziwuziNot born yesterdayPremium
Really? Sandvine, the obvious shill for the ISPs that pay it.
Teabaggers: Destroying America is Priority #1
Re: It makes sense
said by PapaMidnight:I agree with most of your points. It's not even that easy, however, as most of Comcast's peering paths are congested.
Considering that either company may have partnered with individual ISPs to directly peer with them, and/or that subscribers to any of the ISPs on that list could take any number of routes to reach either YouTube or Netflix, it indeed makes sense that the data each has could be conflicting.
Sandvine Confirms ISPs Not Treating Traffic Equally So, how is that for an alternate headline?
Seems like Sandvine stuck their foot in it this time.
Re: Sandvine Confirms ISPs Not Treating Traffic Equally Or maybe even "YouTube and Netflix not treating pay and free customers equally by blending peering traffic to optimize routes"
Not quite as good of a ring to it, but just as true.
Re: Sandvine Confirms ISPs Not Treating Traffic Equally With as large a content company as Netflix is, and with their ability to gather enough metrics to see exactly where congestion is happening (or that is at least something they say they can do with all this new fangled isp blaming) then I would say they certainly could lend a hand coordinating their services, aside from their CDN and open peering options, which I don't consider to be anything short of band-aid or a scapegoat on their obligation to buy the transit needed to operate their business.
Any kind of peer blending past what they do now certainly couldn't hurt. Obviosly they are only using a couple of teir1 peers if their traffic outbound is enough to overload peers at the largest of providers that never had issues before. Id say they have balancing issues of their own that could be fix not only with these "evil isp peering arrangements" but with some plain ole "stop picking one cheap outbound provider to put all your crap on". Yeah that costs money, but maybe they can raise the cost of netflix to fix their issues,
TL:DR : When your content company is so big that you are a significantly measurable amount of the entire internet's daily traffic, you have an obligation to make sure your product gets to your viewers the best way possible, with something more effort wise than blaming everyone else for the problems you caused.
Re: Sandvine Confirms ISPs Not Treating Traffic Equally Oh no, I fully believe Netfix / Youtube / Anyone else should be paying any ISP to interconnect with them if said content companies wants to use an ISP as a peer.
The absolute is content companies have to pay for their transit, as they have nothing to "give" to balance out into a peering agreement. Any ISP could care less unless their small fries if any content companies traffic makes it into their network. Assuming "better access to x content company's content" doesn't count as something to give, thats something to be expected if content companies want to stay in business.
The thing people of you fail to realize is agreements like this are in no way nor never can be classified as "a failure of net neutrality". Net Neutrality has squat to do with backbone transit. You can't keep looking at Netflix, Youtube, and the huge providers of content like you do DSLReports, or Reddit, or whatever small fries website in comparison to big content company's traffic.
As an example, Netflix puts outbound over 1,000,000 mbit/s of traffic ( Link to back that up). That is more than most Teir 2 transit companies touch bi-directionally per second to all of their customers combined. That is a single content company, creating more traffic than a company that provides backbone access for several thousand small companies. Or as another example, thats 10x more than the amount of traffic handled by popular Canadian ISP TekSavvy (Link again)
Its easy for a content company this size to overwhelm a single peering point of any sized ISP. And with a content provider playing pick and choose instead of spreading load, all this traffic never sees half the other peer points of an ISP.
Should the ISP upgrade the peer? Hell no! They have tens of peering points in that location already that handle just fine aside from this one, hence why you see all these reports of EVERY OTHER SERVICE BESIDES NETFLIX working fine simultaneously to Netflix bogging down.
If Netflix were to say "hey we need to diversify and spread all this traffic into several Teir1 peering points, with people such as Level3, Cogent, XO, Global Crossing, HE, etc" at their peering points, then you'd probably never see this happen with any ISP, at all. Netflix's traffic would travel over multiple peering points as it hit the internet, and would have several routes of priority into any ISP with their already established multiple links. That takes money though, and from the ISP's eyes they already have multiple peers to handle this for every other company not sending out traffic in the order of magnitude Netflix does.
You don't see this in smaller ISP's networks because they only have one or two peering points, or may not even be multi homed at all. When they saturate, they saturate for everything. This is why smaller ISPs that can actually attach to a Netflix POP tend to do this "open connect" agreement. They get to not have to upgrade their 1 or 2 backbone connections to the world if they can hook a large line for free into a Netflix POP. Generally this ends up not being free either, as the ISP has to build to a pop, or upgrade a backbone to a POP, but at least they don't have to pay for DIA on that part of the connection, as Netflix is giving ti away.
This Netflix fiasco of linking with providers directly is one of many ways Netflix, Youtube, or any other big player could fix this congestion. Netflix could have chosen to scale back their peering with who they use now, and instead open peer point with the 10's of other Teir1 providers they don't use at all their pop's, and this would fix for everyone. Instead the biggest of the ISP's with the largest of capacity problems to Netflix have chosen to go into agreement with each other. This sets a standard for every other provider that doesn't also peer to Verizon of Comcast to need to do the same to fix their issue with Netflix.
| |IowaCowboyIowa nativePremium
Conflict of interest Sandvine's biggest customers are the big telcos and the cablecos.
Re: In research forums too
said by brucecr0ft :Yes, they badmouthed Netflix non stop at the Atlanta NANOG conference pretending it was fact based research. Disgusting indeed.
I've seen them try to pass off their cable/telco inspired "research" as facts at multiple tech forums like NANOG and IETF. It is absolutely disgusting. Karl is right to call them out.