AT&T Calls Netflix's Request for Equitable Peering 'Arrogant'

Who is getting rich?
Damn sure is not me.

I wonder what the cost is to any ISP to expand peering? Without peering, we don't have internet, we just have net. Maybe isolate offending ISP's from internet?

Part of me, feels like, we pay the cost, no matter what happens in these types of issues, so why should I care?

Also, it's hard to support the idea, that the government should get involved with these types of issues, where its really two or more "private" companies with a dispute.

I also have a hard time swallowing that these types of issues should remain private between the companies involved. At the end of the day, its our money making ALL of this commerce possible.

Maybe everyone would stop being dicks if the rest of us knew what was really going on.

We should care because the ISPs' dream is to make the Internet into Cable TV 2.0. Namely, you would buy a "Internet Package" that included the ISP's services as well as some big name sites that paid good money to get in the default listing. All other sites would be accessible but so slow that they would be unusable. If you wanted these sites to be sped up then you would need to purchase additional "ISP packages" and the content providers would need to pay the ISPs to be included in those packages. Cost for the Internet access AND cost for the Internet-based services would go up and innovation would be stifled as only the big boys would be able to afford entry into the packages.Ideally, the government wouldn't have to get involved. In an ideal world, ISPs would have enough competition that doing something like slowing Netflix down to extort money out of them would cause customers to flee that ISP to a competing one. Unfortunately, that's not the market we have. Many people (myself included) only have one option for broadband ISP. The ISPs (mostly cable companies) know this and realize they can exploit their monopoly position for maximum profit.Sadly, what's going on is that the cable company execs see their TV revenue being threatened. At the same time, they see Internet companies making money from the cable company's subscribers without the cable company getting a cut. So they resort to "that's a nice website you have... it'd be a shame if it slowed down to a crawl" tactics.

Sad part is I have 2 providers to choose from (Comcast & FIOS) but both where doing it so it didn't matter if I switched. Verizon kneecapped our third providers and they left the consumer market.

And that is the exact issue with monopolies/duopolies and why they are usually illegal.

Yes I really feel sorry that you have to choose between FTTH and DOCSIS.

This is one of the most insightful posts I've ever read on this site, and it absolutely hits the nail right on the head.

This is EXACTLY what a lack of competition results in, and it will only get worse if the Comcast merger is allowed.

EDIT: btw I'm referring to Jason's post.

To me Net Neutrality would mean that ALL traffic is treated the same as all other traffic. That being said Netflix is asking the ISPs to violate the very policy they are crying that ISPs are not following.

Netflix's demand that ISPs build out their networks so that Netflix's traffic can reach the end user with less congestion seems to me that they want special treatment of their traffic.

That is an interesting way to look at it. I prefer to view it as Netflix is making the argument to my ISP in favor of me getting the bitrates sold to me at reasonable latency figures. Streaming video is sensitive to high latency and transfer rates, so it is sort of nice to have someone with a lot more money arguing that I'm not getting what I'm paying to receive.

Because, it isn't *just* Netflix that is affected by congested peering points. The data of many other sites flows through those interconnections, and they don't have the weight to complain about it.

Netflix had little choice but to pay Comcast because Cogent wasn't going to budge from its position. I am stuck dealing with Cogent's peering pissing matches for a customer of mine, and it is one of the many reasons Cogent is a cheap transit provider. But Netflix has opened up a Pandora's box for everyone, and the last mile ISPs are salivating over the prospects it brings.

The real problem is that it is Netflix demanding it. If everyone* wasn't subject to a broadband monopoly/duopoly it would be the customers demanding it (or making it irrelevant by switching providers).
It is abuse of monopoly power, plain and simple.

* Pretty much everyone, except for the very lucky few.

Customers would demand Netflix be given free peering? I think not.

Not exactly, but if there were a choice between several providers and they all sold 20 Mbps down tier, those providers that could provide 20 Mbps down when Netfilx hosted the content would have an advantage over those that today struggle to let Netfilx content flow at 2 Mbps (to their 20 Mbps customers).

Is netflix an ISP or backbone network provider that I dont know about that has free peering with anyone?

Netflix pays an awful lot of money to move their bits. Those bits are being requested by the ISP's consumers whom are paying those ISP's quite a bit of money.

If they ISP's don't like the amount of traffic coming into their network compared to what is going out thus causing their peering points to "cost" them, then I guess they should find a way to send more bits the other way making it an "even" exchange.

The stupidity of your statement that anyone is getting anything free is ridiculous.

And what was Netflix's recent proposal Mr. Skippy? Did you read what Mr. Hastings actually said, how Netflix should be granted settlement-free peering or did you just read what was posted on DSLR and take it as the whole story?

And there's the rub... They're actually paying much less than they should. The Cogent's and Level3's of the world under-sold their bandwidth to Netflix, and now cannot afford to upgrade the peering links with other providers (Comcast, Verizon, AT&T, etc.) to handle the massive streaming rates necessary.

Put another way, yes Netflix is paying their providers to deliver their bits. However, the cost of delivering those bits far exceeds what the providers are charging. Instead of facing reality, those providers turn to the media and whine about net neutrality and monopoly abuses. Comcast, et. al., are perfectly within their rights to refuse to shoulder the burden (to the tune of millions) of upgrades so their competitors can be profitable.

At the very end, Karl finally gets to the real point... Once Netflix is a direct customer of the various "last mile" ISPs, they will have significantly greater power to jack up prices.

What is this, a guess? Or are you taking AT&T's word for it (LOL?)

From what I understand Netflix's providers established their peering agreements based on Pre-Netflix traffic in a time when their relationships were closer to 1:1. Then after they took on Netflix the ratios changed greatly but the compensation did not.

BTW it's almost unheard of for at&t to enter into settlement free peering agreements. I think they generally only do this with other Tier1 providers which is less than a dozen companies.

I'm talking about Netflix's providers (Level3, Cogent, etc.) -- you know, the one's crying that no one will give them fatter pipes without buying them. I've explained this before... why should (eg) Verizon spend millions to increase capacity to Cogent due to a single one of Cogent's customers (Netflix), when Netflix can (and does) change providers leading to the same problem with a different peer, meanwhile leaving significant excess capacity to Cogent. Bottom line: Cogent sold something they cannot deliver at the price they agreed to.

(In a world with competition, the last-mile ISPs would be much more interested in private peering with Netflix, to the point of initiating the conversation.)

I dont think you understand the point of peering in this situation.

Cogent, L3 and them don't need to buy a fatter pipe to ISPs. It is not their network that is congesting these ISP's. It is the ISP's lack of network with them that is causing the issue as that ISP's consumer want more data and the ISPs are unwilling to upgrade to provide it. It doesn't matter if it is coming from Netflix, Amazon, or DSLReports.com. The bottom line is, the ISP's consumers want data and the ISP is choosing to let the node saturate.

You claiming that Cogent, L3 and all other backbone providers should bend over for the ISP's consumer sake is silly and I think you have that backassward.

Exactly. The ISPs are selling a service to the customer, and it is their job to actually deliver that product.

By that same logic... Netflix is paying Cogent (etc.) to get their traffic to people (Comcast, Verizon, etc.), but they don't have the capacity to do it, and aren't willing to pay to increase that capacity (even for just their own side) -- because it would make their contract with Netflix unprofitable (and they knew this from the start.)

This is exactly where peering belongs. Netflix wants it's traffic to reach Comcast, and Comcast (should) want it's customers to have an acceptable experience getting to Netflix (read: a service they want to use.) Even 20 years ago, that would be an issue to be hashed out between Netflix and Comcast; not a never ending bitch-fest about "net neutrality" and transit provider peering problems. When an ISP's upstream link(s) are saturated with average traffic, then it's time to buy a bigger pipe. When that upstream is flooded due to one f***ing service, it's better to work out exchanging traffic directly with them, lest you have them change ISPs and then flood a different uplink.

No it is not by the same logic.

Netflix pays Cogent to assist in hosting their data as Netflix does not have a backbone of fiber connected to every ISP or other backbone providers that connect to said ISPs. They have no choice but to use someone else to pass the data. Even with direct peering they are relying on someone's network that is not their own.

To the ISP, Cogent is a backbone provider into that ISP and it is not their job to make sure that the door into the ISP is big enough to accommodate all of the data that the ISP's consumers are requesting. That is and always has been the job of the ISP as the service they provide to their paying customers. Who that data is coming from and why is not at all relevant, but the ISPs are trying to make it that way because then they can target a specific person for payment instead of the entire random world that may reach inside through a request by their own subscribers.

Customers would demand that the video stream that they requested work. If that requires their ISP to upgrade their network, then that is what they would have to do... or go out of business.

That is their argument. Netflix may take up 40% of their bandwidth during prime, so if the exchanges get saturated it is up to the bandwidth providers to either pay for additional transit or for Netflix to look toward more direct jump points.

This board is littered with the bs that these arrangements are settlement-free peering. IT IS NOT. No f**king way. I haven't seen the contracts, but I know damn well this is not the case, and whomever propagates this stuff has little knowledge of transit cost structures.

It looks like Netflix is looking to more direct jump points which they can control more directly, and PAY for.

It is NOT clear that today Netflix is paying more for Comcast than to other providers (L3, Cogent, etc).

Most people on this board assume they are under balanced-trade peering which is simply NOT the case. They are most assuredly paying for transit bandwidth, plain and simple. It may be the case of L3 not wanting to add bandwidth to take away from their margin..

I think what will eventually happen is that Netflix starts adding their boxes in-net and then these connection issues mitigate OR they start doing direct deals w/ the operators. or both.

Netflix per se is a virtual CDN... They own no networks. All their services run on AWS and Openconnect CDN servers for providers that have the. So in a virtual world, you have to PAY for connectivity from AWS to the end user.

If you look at it these guys are consolidating so many are already tier1 providers with national networks, so it simply makes sense to have direct transit relationships with these guys and while we trumpet net neutrality the reality is that this is really the case if my operator (Verizon) is NOT traffic shaping my connection.

Now I DO think that these operators are acting like baby bells and letting these exchanges points explode, because they have the end-user advantage and are taking that to the max. Who wouldn't do that? You guys think CIX is still relevant, cmon. But a 10GB connection is a 10GB connection in one direction. If the 10GB downlink is saturated, and the uplink is at 5% (which it probably is) that is because video is a HIGHLY asymmetric profile.

Also the board is confusing net neutrality (which has to do with prioritization and traffic shaping -- that is in FULL force up in Canada) versus one single provider (Netflix) who is not managing their data needs correctly and is being pushed to the brink by the operators.

Long story short, eventually the end customer is going to have to pay for all these SuperHD streams and really pay for 4K. Ever notice how HD PAYG costs more than SD? Well it's partly profit, but it costs MORE to deliver.

I agree pretty much with your explanation of it, but this again falls on the ISPs not being able to handle what they promise to their customers.

"Sure, we'll sell you 50meg DL at 60 bucks a month and you will see that speed all the time...as long as its on said ISPs network."

The ISP needs to cover the traffic that THEIR customers are requesting from netflix but they dont want to foot the bill to upgrade it. So you get what Comcast is doing, letting links saturate, moving traffic to overcongested links etc. then the customer is left in the middle.

Customer complains to Netflix, Netflix can upgrade all of their connections to any tier 1 provider or whatnot, but the issue remains getting onto Comcast network. Netflix tells the customers there isn't anything they can do (short of a direct connect to the ISP, which is where we are going now) and to contact Comcast.

Comcast tells the customer that they only can guarantee speed on their network, not out to the "internet", even if the problem lies with the connection between Comcast and Cogent/Tata/Level3

Comcast then turns around and when called out on it by Netflix they take the stand of Netflix is sending Comcast too much traffic? BS...Comcast customers are requesting it, its not like Netflix is dumping traffic onto Comcast to reach Verizon or something (this is why comparing this to cogent/L3 peering dispute that another poster did is idiotic, not referring to elefante72 at all).

While i disagree with this deal between Netflix and Comcast for the long haul of the internet, because the only people that are going to have to spend more money will be customers, it was really the only thing left for Netflix to succumb to.

"I agree pretty much with your explanation of it, but this again falls on the ISPs not being able to handle what they promise to their customers."

No, it falls on the ISP AND Netflix. They need to both work this problem out to take care of their mutual customer instead of pointing fingers back and fourth.

So according to you at what point does a website or service become popular enough that THEY then have to then work with every ISP to get prioritized traffic delivered in a manor that is not disruptive to their consumer base? 10GB, 50GB, 3TB a month?

Come on, lets get a specific amount of traffic that ALL companies large and small must abide by to start the Intertube process so it is black and white and we can all decide is fair and equal to all.

I disagree. It's the ISP's issue. I'm a Charter customer and have NO problem receiving Netflix feeds in 1080P/5:1 surround 24/7/365 via PS3

Netflix had plenty of options, like using 3rd party CDNs as many of their competitors do. Netflix used to and things worked well. Then they went about this game of trying to do everything themselves and things broke down so they blamed everyone else.

You do realize that even using a 3rd party CDN does not resolve the issue unless that CDN is physically INSIDE of the ISP's network through physical server placement or direct peering right?

So netflix could have a 1 petabyte connection to the internet and be on a CDN that has a 1 server for every 100 people in every zipcode and that still will not resolve the peering problem if the ISP's don't let the traffic enter their network through a non-congested node.

You want to know when this is a Netflix issue? When their servers are constantly spiked at 100% and/or when their actual direct connection to the internet is constantly spiked at 100%. That is when Netflix needs to upgrade to improve their services to their customers. Anything in between falls on the ISP and their peers.

So you have absolutely no idea but you know for certain?

I can't say for certain but I have seen dozens of CDN contracts and none of them have been the settlement-free peering version that has been spewing about in the press.

I have walked into transit centers. It's pretty amusing. You may have Level 3, Cogent have a a switch or two sitting in a rack and in the same rack is a switch for Verizon, Comcast, etc and all it takes is a line card and a 2m optical cable and poof the issue goes away. Interestingly enough (and I didn't know this), lots of these links are non-redundant and if one port goes down or a switch some serious shit will start hitting the fan. It's not like the data has to go thousands of miles to get onto an operators network...It's like 2m

So the actual congestion is not an issue, it really boils down to (IMHO) the operators trying to slow the growth of their network UTILIZATION AND make money on both ends (caps on one end, transit on the other) to simply monetize it. And on the Netflix side, they just want to pay a flat rate, and then unleash SuperHD on the world and then expect the marginal cost to not go up. Well it does.

Plain and simple it's a balancing act, and this costs money. Even if it costs Netflix 1c to deliver SD and 2c to deliver SuperHD, multiply that by millions of videos a day and you can see how they are trying to minimize cost. When you get to that scale, fractions of a cent MATTER. If Netflix takes up 40% of Comcast prime traffic, do you not think they need to build infrastructure to keep up the pace? Even if transit handoffs aren't the bottleneck, they are still going to need to invest hundreds of millions in their infrastructure to expand intranet capacity.

Where is the congestion occurring? To me it seems like it's occurring at the transit/peering end, which is something that does not require significant capital to upgrade (compared to middle/last-mile infrastructure).

If Netflix is asking to peer settlement-free at a neutral facility where an ISP already has a presence, what's the build-out cost exactly?

Netflix isn't demanding they do anything.

Netflix is simply pointing out that they are an ISP and as an ISP they have consumers that are trying to acquire content of another company (in this case Netflix) through the normal and intended use of the internet. However, because said ISP is not maintaining their peering points properly the consumers of said ISP are getting poor service.

It is the ISP's responsibility to maintain the internet experience of their consumers and to upgrade their network as needed to do so. It doesn't matter if the consumers of the network are trying to get packets from Netflix, YouTube, Pandora or this site. If they are not maintaining their network in a way that adequately supplies the bandwidth needed at the request their consumers are making then they need to do 1 of 2 things: 1.) Reduce the size of their subscriber base by using whatever method they need to do so (price increase or speed decrease) 2.) Invest in their network to improve the services to their current subscriber base.

The ISPs should build their network on as as-needed basis to handle ALL traffic that their customers want. It just happens Netflix is far larger than just about anything else.

WTF?! AT&T and ISPs are the postal service of information signals. The absurdity of their suggestion is that they want both parties to pay, as if I paid postage to send, and the other person payed COD to pick up. Only, both parties pay the full postage.. Great idea if you are aiming for a shameless money grab. My mind balks at how asinine his statement of "Mr. Hastings’ arrogant proposition is that everyone else should pay but Netflix" is. Besides the statement being a flat out lie. Netflix DOES pay for their content to reach their customers over the internet. They just don't pay you. Statements that AT&T is leveraging their customers to extort money from services like Netflix are more credible, which is reinforced by false and misleading statements. The whole thing should be a non-issue though since AT&T internet customers are paying... AT&T to get to content and deliver it from Netflix. They aren't paying AT&T because they want AT&T, but because they want Netflix and the stuff on the rest of the internet. If ANYTHING, AT&T should be paying Netflix, because it's what AT&T's customers are requesting. This is exactly why Internet service providers should be completely seperate from content companies. The company dislikes their own customer when the customer prefers content from anyone else, and the is too big of a motivation to disrupt or otherwise deliver less than optimal service that a customer they already have is paying for.

Funny thing most last mile ISP's have explicit rules in their TOSes stating that end users can't serve content. Yet ISP's are complaining that peering for free isn't fair and equitable because of the amount of consumption. Yet they make it impossible to go the other direction.

AT&T is right that it takes money to add the hardware/bandwidth to support this. However, didn't Netflix offer to provide all of this "for free" to ISPs who would partner with them? AT&T is in the position that they don't really need to spend that money to make large profits. Being super profitable doesn't change the rules, they aren't required to spend money if they don't want to. If they can legally force people to pay them money, then they're going to do that.

The only way to fix this is if people have a few alternatives for the internet access to reach Netflix. Then AT&T would be shooting themself in the foot, and Netflix can work with a different internet provider.

From what I understand, Netflix is paying enough to it's networks to have sufficient bandwidth. The networks Netflix buys from have to make sure it is available at THEIR interconnection points, not Netflix. If they fail to achieve this then Netflix should find other transit routes. Having many ways to get to Verizon is an acceptable thing. They should NOT have to pay Verizon directly. The same goes with Comcast, etc.

Now, as far as the customer goes, if you pay to have 25Mb/s then you should have 25Mb/s speed. If certain routes are a problem, then the customer needs to let his provider know it is a problem. If the provider says it is not their problem and the consumer disagrees, they can contact their local officials and the FCC, who will probably do nothing, but you can sure give them a headache (recommended).

Some day, the BS about bandwidth and availability will be a non-issue. That day will be when we go back to the open Internet where content is not the bait. We really only had it a very short while though. In the beginning ISPs had to pay for preferred transit to the CIX and other major hubs. There was no way around it. It went back that way it seems. Regression is not always a good thing.

I'll tell you what's arrogant. It's AT&T or any large ISP thinking that they should be able to charge the sender and the receiver for the same data. I pay a whopping $60/month for a 15M/1M internet connection. Whether I use that data for Netflix or email or online gaming, I expect my data to get equal treatment. For the ISP to turn around and threaten Netflix with degraded speed just because they are a popular destination is ridiculous.

Yes since they are giving you access to the Internet, not being a backbone, they need to do just that. I have been in the ISP business before. When a customer paid for a circuit, it better deliver. We made sure it did. Those days are pretty much gone, so now we just do Internet services like website hosting and the like. If you are in a rural area ISPs can still survive by wireless delivery, etc, but not in the New York market where we have been displaced by the big cable/telco gang. It is sad that we built it, then they took it right out from under us. They will burn themselves soon. We all look forward to that day.

For me, I'll wait and see. I have a low-end $7.99 Netflix account that is perfect for me. Now, should Netflix decide they need to add an ISP surcharge at some point, I may review my relationship with them. The problem is Netflix is agreeing to pay ransom. I already pay my ISP $60/month for 18/5 service. If they decide to start demanding that Netflix pay to reach me, which in turns means I will be paying twice to get content, then the FCC is going to have to step in.

But the FCC is now run by a lobbyist monster who pretends to support us. In other words they hold all the cards.

Anyone who still thinks Comcast "extorted" tribute from Netflix needs to read this:

Deal with the devil: why Netflix broke its own rules on net neutrality

If this article is to be believed, it was Netflix that got the better end of the deal:

"it seems more likely that Netflix was finally able to extract terms from Comcast it found favorable enough to agree to."

The thing that these days calls itself "at&t" accusing somebody else of being arrogant is about the funniest thing I've read in quite some time

That is a very good point.

Netflix sends so much data in one direction that balanced peering is anything but balanced.

"What appears to be happening is that Comcast, AT&T and Verizon are eager to kill off free, balanced peering exchanges and make more money by growing their own direct transit and interconnection businesses."

I agree, that appears to be true for better or worse - the part I disagree with is "free, balanced peering exchanges". Your typical transit providers (Level 3, NTT, Cogent) are anything but "free".

So I pay $X a month for a speed of Y Mb/s and Z GB usage. I don't believe it should matter what I do with my connection since I am paying for a certain speed (legalities aside), and a certain amount of usage. All these under the board fees do would be to increase what Netflix charges, all so the ISP can avoid raising their rates directly, but the end result is the same, their customers pay more for a service that should be accessible already, and the ISP makes more money for no change in service levels.

>AT&T hoping to convince you

What happened to "all-you-can-eat?" AT&T speaks with forked tongue...

Tom Evslin - He conceived, launched, and ran AT&T's first ISP, AT&T
WorldNet Service. WorldNet popularized all-you-can-eat flatrate
monthly pricing for Internet access and forced the rest of the
industry, including AOL and MSN, to follow suit. Tom has been blamed
and praised for this ever since. He is unrepentant.

»blog.tomevslin.com/about.html