DTuttle
@verizon.net
| reply to drake
Re: Verizon FiOS FAQ - let's work together on it!
How can Verizon provide internet access that fast?
Some people, like myself, wondered how they could provide internet access as fast as they do. Sure they can get it down the fiber really fast, but how can they afford to get that much data to their local offices to distribute to you? In other words, how do you know you can actually get advertised throughput?
There are two main businesses responsible for this. The first is BBN. You might have heard of an ISP called BBNPlanet, particularly if you lived in Boston. This was the company that invented the router, the core piece of technology that made the internet possible. These are the guys who, literally, put the @ in your email address. BBN was bought by GTE, which was bought by Verizon. Verizon spun off the ISP/Hosting portion of the business but kept a lot of the network infrastructure. Behind the scenes each autonomous network on the internet is given a number called an AS number that is used to advertise groups of networks among special types of routers. BBN's number was AS-1, and Verizon owns this.
The second important part is a company called UUNet. If you were working with the internet in the 1990s you probably remember them. They were created at the time when the internet was first starting to be commercialized. They created their own separate network called AlterNet that didn't fall under the regulations of the US Government who, at the time, still controlled much of the internet. (isn't it ironic that the government is trying to regulate a network they controlled and practically gave away!) They were the early player. UUNet got bought by MFSNet who got bought by WorldCom who merged with MCI (who in the past had already been a major cross country network), who then bought Compuserve and ANS from AOL (who had previously bought up much of the infrastructure when the government was busy deregulating). In 2006 Verizon acquired MCI. They use AS-701 which was historically UUNet's (as well as several others).
Anyone who has vague recollections of the early days of the commercial internet will recognize most of these networks. They were big players at one time or another. So what does that mean for you, the FIOS user?
There are 9 Tier One Internet providers. Many will complain with the definition I use but it's mostly because they want to call themselves tier one and they aren't. A Tier One internet provider does not pay anyone to connect to their network, they connect to all other tier one providers directly and they do not rely on anyone else for a full route of the internet. These companies are ATDN (AOL), AT&T, Global Crossing, Level 3, NTT Communications (Verio), Qwest, Savvis, Sprint, Telia and Verizon. If you're not getting your internet access from one of these companies the company you get your access from is depending on other companies in turn to provide access to other networks. For example Cogent depends on Verio to reach ATDN.
Additionally each tier one company connects to different numbers of other networks. At a minimum a new Tier One company might only have to peer with the nine existing Tier One companies and exchange enough data with each of them that they don't have to pay for the connection (because it's a benefit to both companies). On the other hand a company like Verizon connects to most of the networks in the world and has, by far, the highest degree. If you look at this map
»www.caida.org/analysis/topology/···2005.png
This map is a little dated (2005) but the situation is essentially the same.
You will see that they are closest to the center of the internet and additionally have several other networks (702, 703) near the middle.
What does this mean for me? It means they don't have to pay another internet company to get any of their internet connections. Not one cent ever. Let's say I wanted to start Bob's ISP. The first thing I'd have to do is buy bandwidth from a service provider, or more likely, multiple service providers. This is a huge recurring expense. For every Mbit/sec of bandwidth I need I'll likely end up paying about $20 to $25 a month. In turn I will depend on those network providers to make sure I have internet access for my customers. Brighthouse Networks in the Tampa area buys internet access from Level 3 and ATDN. That's an expense for them. To offer me higher speed internet access they must not only upgrade their equipment but must also pay for that bandwidth as well. For Verizon they only have to have the equipment and, since they have the largest IP network already, they have that sort of equipment available to them. On top of that every website you connect to that is hosted in a Verizon data center, served from a customer who's using Verizon's business network or (most likely) served from a data center that connects to Verizon's network Verizon is charging that business a fee to send the traffic over their network and reducing the amount of traffic they have to send to other Tier 1 providers. They are increasing their revenues on the other side of their business and reducing their need to transit the data to another tier-1 provider for delivery.
Their costs right now are the costs of building out the network, building the customer support system (and fixing it) where as the cable companies have the infrastructure in place but they have to pay for their internet access.
Having said all that there's two technical details that affect the speed you get, one good and one bad.
The first is response time. Most people should notice very fast response times. Mine were nearly cut in half going from Road Runner to FIOS. The reason is that Verizon doesn't have to send the data across multiple networks to get the data from one end to the other. They also have interconnects in so many parts of the country that they can get from their network to the one they are sending it too very quickly. This is important because the longer the delay the slower the connection will be because the server must wait for you to acknowledge that you received data before it will send more. Less delay equals faster throughput.
The second, and bad, part is that most web servers aren't configured to handle the kind of throughput possible with FIOS. The web server has an amount of memory reserved for each connection that's created. This amount of memory is limited because of you allocated a large amount of memory and you had a lot of individual connections you'd run out of memory quickly. There have been standard buffer sizes used in servers for quite some time which provide ample buffer space without using too much memory and everything just worked. The problem is with a high speed low delay network (like FIOS) the buffers are empty before your computer sends back the packet that acknowledges all the data it's received. I found for downloading a file with HTTP this limited me to about 800KB/sec or 6.4 Mbit/sec. I'm paying for 20Mbit/sec! by tuning the buffers on my own web server which is hosted in a Tier One facility I was able to get full throughput, but it's easy for me because my server doesn't handle a lot of connections and has more than enough memory available to create buffers with. In heavily loaded web servers there are a lot of connections and the web server and applications may need most of the available memory. Point being you may have 20 or 30 Mbit/sec available but many web servers are only going to be able to send you 6 Mbit/sec of data however at the same time you're downloading that large file you can always start downloading two more. |
