how-to block ads
|
|
Uniqs:
6033 |
Share Topic
 |
 |
|
|
|
GregOP
join:2012-06-19
Beaverton, OR
1 edit | [Internet] Frontier FIOS - Latency and QoS - Where they fail 
So I've got a huge beef and I want to have a conversation on this topic. I've seen plenty of people post latency results from their FIOS connection for no load and load scenarios that perfectly match up to what I see. What I've seen at home, and what I've seen others post, is that under network load, ping skyrockets, sometimes in the 800ms+ range and packet loss occurs. Stop the download, ping drops back to normal and packet loss stops. I've sen this on 35/35 and 15/5 FIOS connections where I was getting the actual line speed advertised (as is almost always the case with FIOS unless they messed up provisioning).
So everyone, here is the crux of my issue. I was with comcast for 6 years. When I left comcast, we had their 30/5 service and I was getting consistent speeds above advertise for upload and at the advertised for download - sustained over large download sizes - during peak and off peak hours. My wife and I were able to play games like L4D, CSS (latency sensitive games) while downloading at the full 30Mbit speeds - not using torrents which tax your router- and we did not have any negative impact to the games. Pings were steady, and this showed up on the pings and tracert where the avg latency would increase by perhaps 10ms when downloading at full speed for sustained periods of time (>45m).
Fast forward to FIOS. I'm extremely excited to finally have FTTH/FTTP. I'm excited to see a change in latency, and, for the most part I get slightly better ping then comcast - say 25 instead of 30… Not much. That's fine, no beef there. Download speeds sustained and upload speed sustained are as advertised with the exception of 4.6Mb up instead of 5Mb as advertised. Fine, no beef there, perhaps they use some of the bandwidth for QoS.
However, the second you start a full speed download (any decent site can provide full speed 15Mb) and the ping skyrockets to 500ms or more consistently until the download is over. This is 100% reproducible at 2am, 12pm, 7pm, any day of the week. In games, this effect produces unplayable results. You cannot download any file, stream any 1080p youtube videos, watch hulu, ANYTHING that requires any significant portion of your download while gaming or doing something latency dependant (VOIP etc). The tracert shows that the lag starts as soon as I hit their first route. This is 100% Frontiers equipment failure.
I've been a network engineer, systems engineer, and now I work in security. I am fully aware of how networks operate, and used to work in a datacenter on FC/FCoE/10GbE/40GbE networks. The most likely scenario I see occuring is that:
FIOS QoS is non-existent or highly mis-configured on their network hardware. There is a known amount of bandwidth available. There is a demand for some high speed HTTP traffic, NNTP, SSL, you name it, along with some quick firing UDP gaming packets. The network should prioritize the small UDP packets over the large packet sized NNTP/HTTP/SSL etc traffic. This does not happen on FIOS. The traffic appears to be sent/received first come first serve and since the high bandwidth traffic is the majority, it starves the connection and the UDP packets must wait in line to be sent along.
Now why does this not occur with comcast? I assume it is because comcast has either better network equipment then frontier, or more likely, better configured network equipment with proper QoS priorities and traffic shaping. I am highly considering dropping FIOS and going back to stable, low latency, comcast networks.
But being the networking person I am, I figured I would troubleshoot something that should 'just work' in this day and age. First of all, I am running on cat 5e ethernet from the ONT direct to a router(dd-wrt linksys wrt54gl). From there, I've got a semi-managed 8-port netgear GS108T. In troubleshooting, I was direct connected to the 4-port built in switch on the dd-wrt router. I set QoS up properly on both the supplied FIOS router, and, my own dd-wrt router. I found I was able to do the best QoS on the dd-wrt, but, to some extent, the FIOS router's QoS worked too.
When QoS was setup, NNTP/HTTP/FTP traffic was set to bulk priority and UDP over the specific ports of a few games were set to highest priority. In doing so, the pings (which are classsified as normal traffic - above the bulk) were better too. Under full saturation of the download pipe (minus the 100Kb I used to buffer the QoS downstream and about 200Kb for upstream - the bottleneck has to be at the router for QoS to work there) latency increased by about 5-10ms. The average went from 35ms to one server with no load, to 45ms to the same server with full load. Turn off QoS, rinse, repeat and the results are 35ms no load, 500ms+ full load. Great, so QoS helps tremendously as expected.
Some of you may say - "Ok what is your problem then? - You figured it out". Setting up QoS for every single game that I play or have the fancy of playing, is unrealistic. I have a library of about 30 steam games, and a ton of BF series, F2P games, etc. I work all day. I don't want to come home have to spend a ton of timing ensuring my QoS is full up to date for every game, before I can relax and PLAY the game. Comcast, did this. Comcast let me play my games without the need to micromanage my outgoing network. They did QoS. FIOS, appears to flat FIFO the packets with no consideration to priority what so ever.
This sucks. End users who are not savvy would be driven insane - or their kids or roommates or spouses will - if they play games or do any latency sensitive application work. Also, my QoS is not as good as a high end $50-100,000 dollar FC/FCoE switch or router with proprietary high end hardware. I can never do as good of QoS as they are capable of doing in the datacenters.
So can someone, please, add intelligent conversation to this topic and explain their views and opinions on this? I see the internet littered with examples of people attempting to do QoS with the bad router that FIOS provides - obviously some people are starting to understand the problem. Can someone explain why this is the way frontier does business? I can't for the life of me understand why they would do little or no QoS on their end. Is it lack of understanding and know-how? Is it some misguided attempt to give their end users freedom but instead inducing frustration? Is it laziness? Why has no publicity been given to this topic on the internet or local news?
Look forward to hearing from you. | |
Reviews:
 ·Frontier Communi..
| Re: [Internet] Frontier FIOS - Latency, QoS, and the ball has be Umm,
Question: Why are you this upset about this? Every internet connection I've ever worked on (from dial up modems to DS-3s) has skyrocketing pings when the downstream (or upstream) is saturated. It's the nature of the FIFO queuing beast. It's nice if Comcast is queuing traffic in a more sophisticated manner than FIFO but it's certainly not an obligation of an ISP to do so.
With that out of the way, a few suggestions are in order:
1) Why do you have to classify EACH game in your QoS rules? Can't you just classify HTTP/FTP/NNTP/etc as "best effort", a few specific things (Skype) as "real time/highest priority" and everything else in between? This should get you the gaming performance you need without having to do too much classification.
2) Failing the above, is your router sophisticated enough to classify traffic via the originating IP? If it does, assign two IP addresses to your workstation, use one for bulk transfers and the other for gaming. Most web browsers/usenet clients have the option to specify a different IP to bind to.
3) Failing that, have you considered using a different router? I've used the traffic shaping tools in Linux with great success over the years. You can categorize the traffic any way you want with iptables and the HTB queuing discipline. I put our traffic at work into the following categories:
VOIP
LOW-LATENCY (ntp, dns, TCP syns, small ssh packets, etc.)
TCP ACKs
VPN
ADMINISTRATION
OTHER DEPARTMENTS
BEST EFFORT/LOWEST
I'd be happy to share my configuration with you for Option #3.
4) Is shaving a lousy 100KB off the downstream really enough to ensure all the packets queue at your router rather than the ISP? I've had to shave as much as 25% off my downstream to ensure acceptable results, particularly when you've got a lot of users behind the router and/or applications that make a lot of TCP connections (e.g., bittorrent). YMMV with fewer users and your faster connection of course. | | |
|
Reviews:
·Frontier Communi..
 ·WildBlue
| reply to GregOP
I would love to see FrontierTim and Smith6612 weigh in on this.
I understand the basic of of how QOS and the networl topology works in an ISP but I've never seen a conversation this in depth.
this sort of conversation can only help Frontier get better if they listen in.... | |
GregOP
join:2012-06-19
Beaverton, OR | reply to GregOP
Re: [Internet] Frontier FIOS - Latency and QoS - Where they fail Thanks for the reply. To be honest, I got cozy with Comcast providing great QoS and not having to think about it. I actually valued it, without realizing how far ahead they were compared to other companies. That fact alone really surprises me - that Comcast would be ahead o the industry. While I'm capable of all this work, I'd really prefer to not deal with it. After all, kids, wife, house, errands, repairs, all drive my attention and I have less and less time to fiddle with the network, however much I enjoy doing so.
That aside, I can see your point that they aren't required to provide QoS. I do think, however, that philosophically they SHOULD since the average user has no clue about QoS and just wants a connection that works most of the time and in most situations.
My router is a bit old at this point. It does run custom firmware that allows it much much higher QoS settings than most people have available. In fact, I can and do run iptables for firewall and HTB Queuing discipline on my router. The router has a command line interface and script auto-execution on startup in order to implement these things. I believe my downstream is actually limited by about 800Kb and my upstream is limited by about 300-400Kb. I can verify when I'm home.
I need to check again, but, I think I've got HTTP/FTP/NNTP/SSL as bulk/best effort. I think I did UDP with specific ports for classification because I didn't want all UDP traffic to be prioritized. However, it might be wise to just give all UDP/TCP SYN/DNS full priority and specify other protocols as bulk. Recently, my wife was complaining during a gaming session (we were both gaming, no downloads were going, no network activity was occurring on our side other then the games) about how she was getting 100-250ms spikes and it was unplayable. I think after the initial issues we had, her opinion on FIOS is down the gutter since none of these advanced 'crazy techie husband magic' methods were needed. And my wife is actually pretty knowledgeable with computers, I just, blow her away in that respect so she feels out of the loop. Anyway, the point I was making is that she as a standard internet user is frustrated with the service because of the hoops we have to jump through in order for it to be as good as comcast. People just want their service to work.
If it were just me at the house, I think I wouldn't be so concerned. I'd annoyed but not actively speaking out about it. But obviously you, like me, are a technical person and we are able to cope with and fix these issues much better than the average internet user. I imagine most people in this forum are pretty technical in nature. But if you take off your techie hat for a second, and put yourself in the shoes of a standard internet user, a spouse, or other person using the network, streaming some youtube or downloading a MS patch while gaming or talking on skype, would you be happy with a company that required you to set up an advanced QoS at home in order to use their service at the same level as another company (Comcast)? I think most people would get fed up and leave FIOS. I for one, don't want to do that. But there are other forces in the universe then my own.
On a side note, feel free to share your QoS settings. I'd like to see them and perhaps gain some insight. | |
 Ben JPremium
join:2011-09-16
Elk Grove, CA
kudos:3 | You've basically hit the nail on the head already. FiOS provides simple FIFO queuing on the subscriber egress interfaces, while the Comcast CMTS system is more fair queue based. Fair queuing effectively means low-bandwidth flows are given some amount of priority over high-bandwidth flows and thus simultaneous connections are not as easily delayed/starved out. There are pros/cons to fair queuing, but it is sufficient to say that the FiOS infrastructure does not support flow-based fair-queuing in its current configuration.
Also, FiOS does not "hard" police/rate-limit your provisioned speed. FiOS attempts to shape before dropping. This will cause extra delay to be introduced in individual packets when you start bouncing against your speed limit. The egress scheduler will attempt to delay exceeding packets rather than immediately drop them, looking for a gap/slowdown in the data rate to be able to send them. So instead of no ping reply, you get a higher latency response. This behavior is obviously kinder to TCP flows than it is latency-sensitive UDP flows when your downstream is fully saturated.
Note that anytime an ISP changes to *any* packet treatment other than FIFO, complaints and regulatory scrutiny is likely to immediately follow. I don't know if it's entirely accurate to say that Comcast is ahead of Frontier in this regard, as much as they seem far more willing at the moment to take on regulatory scrutiny on neutrality issues. As the FCC continues to clarify the rules, ISPs become more comfortable making changes.
said by GregOP:That aside, I can see your point that they aren't required to provide QoS. I do think, however, that philosophically they SHOULD since the average user has no clue about QoS and just wants a connection that works most of the time and in most situations.
Your point here is well taken.
--
Transparency Disclosure and Disclaimer: I am a Frontier employee posting in my own personal capacity. The opinions and positions expressed are my own and do not necessarily reflect those of Frontier. | |
Reviews:
·Frontier Communi..
| reply to GregOP
Shoot me a PM with your e-mail address and I'll e-mail you my Linux QoS configuration; it's too lengthy to post here.
I have seen TWC use QoS in my old hometown; you could be online in the evening when the DOCSIS node was maxed out (downloads topped out at 2mbit/s on my so-called 10mbit/s connection) and get the same ping times as you would at 3am.
Perhaps the shared last mile is the key difference? Cable companies have to contend with that limitation and FIFO queuing with a shared last mile would suck; they presumably make an effort to prioritize smaller packets over larger ones, hence the reason why ping times remain good and interactive protocols like ssh are responsive even during peak hours.
When I had a 10mbit/s Verizon DSL connection they seemed to use a much simpler QoS. When the backhaul network was congested I got an equal share of whatever was available (usually I'd top out at 5.5-7mbit/s); if I didn't max out my share I got great pings but once I used it all the pings went to hell. My guess is they do some sort of fair queuing based on destination IP address which doesn't take into account the actual content of the packets.
Personally I don't bother to shape my downstream at home. We have a 6/1 DSL connection and we download large files/stream video so rarely that it isn't worth the effort. I _do_ shape the downstream at my employer. That's a consequence of having to share a 3.0mbit/s connection with 60 employees. We've got classrooms that need to stream video, a business office that needs a responsive VPN for RDP, remote users using VOIP and employees that need responsive web browsing for research. Doing all of that with a 3.0mbit/s connection would be impossible without downstream shaping.
The downside to downstream shaping is that it can be somewhat wasteful. All you can do is delay or drop packets after they've been sent across the wire. If you wind up dropping packets because of full buffers they'll eventually have to be retransmitted and will consume additional downstream bandwidth. At work I've found downstream shaping to be effective for normal activities but it completely falls flat when you start dealing with protocols that make lots of simultaneous connections (e.g., bittorrent). Each new connection maxes out the downstream until the TCP congestion avoidance protocol can catch up. I only allow through 75% of our downstream bandwidth and even that isn't enough to prevent the downstream utilization from reaching 100% if I try to download a well seeded torrent.
Don't let me deter you; downstream shaping _can_ be useful, you've just got to understand the limitations.
In any case this is an interesting discussion to be having. I've rarely run into other people that are familiar with traffic shaping/QoS. I'd be interested in hearing your review of my Linux shaping setup once I'm able to send it to you; it's the end result of five years of tweaking and works very well for my employer. We are so far out in the boonies that T-1s and satellite are our only options. For the longest time we had a single T-1 and I had to make 1.5mbit/s work for 60 users. Now we have bonded T-1s and double the bandwidth but 3.0mbit/s obviously isn't a lot in this day and age; making it work effectively for 60 users has been quite the challenge. | |
 Smith6612Premium,MVM
join:2008-02-01
North Tonawanda, NY
kudos:22
·Verizon Online DSL
·Frontier Communi..
| reply to GregOP
I guess I can chime in a bit. I've already checked over this thread and it seems to be well covered as far as why connections experience issues with latency when downloading and uploading. It is true, it is a QoS configuration issue not only on the CPE end, with both the networking gear and even the device in use (buffering settings, receive windows), but also with the provider end as to how they manage traffic coming into their network towards the customer line. This is pretty typical for residential grade products where QoS isn't exactly a concern for the company. Business circuits (Fiber, OC-*, DS*, T*, etc) are the exception here and those are obviously held up to standards. The rule of thumb for a saturated line is no more than 15ms. Any more is excessive and prone to performance issues.
I find that with Verizon and Frontier, the kind of latency you pull depends on the gear you are attached to. For example, on Verizon attached to an old Redback/Ericsson router, your idle ping may be 10ms. Fire up a download and you might see that latency jump 80 to 100ms. If you have a small receive window on your system for example, or if you use QoS that is set up to prevent provider-end queuing of packets, you'll find you might be able to score latency to the ISP anywhere between 20ms and 80ms, but by no means would it be a guaranteed or solid elevated latency.
You'll then find that on Verizon lines with Juniper ERX routers, which obviously have a different configuration as far as Quality of Service goes, that your idle latency will remain the same, however downloading shoots your latency up into the range of 600+ms. This can be resolved by of course, user-end QoS or TCP/IP configurations, but it isn't too useful especially if you have other devices on the network that may receive data. The same goes with lowering your receive window on your system to a setting that does not cause throughput to decrease but does decrease buffering on the provider end.
Then, if I were to take my Frontier line, that connection has an idle ping of 15ms. When downloading, that goes up to 40-50ms when there is no congestion. If the network is congested where I am, that can go between 40ms and 200ms with inconsistent speeds. Uploads however are an entirely different story, but often CPE-based QoS can fix this problem with a throughput hit (most common way of doing it on a typical SOHO device).
But most definitely, it's something Frontier needs to consider and address. Their equipment may in fact be storing data in a FIFO buffer which is causing a large amount of latency for incoming data. It's best to keep these as minimal as possible, but not to remove them entirely.
In case anyone wishes to see how a connection should operate, I'm currently at work which is a Datacenter. I'm attached to a 100Mbps port currently with one of my computers, but the same results happen on Gigabit. Pings are on a lower priority on outgoing requests here, so what you're seeing as far as jitter goes is due to QoS in effect on the datacenter network. It is also the result of multipath routing with the multiple providers coming in and the nature of the Internet itself.
PING www.l.google.com (74.125.226.210): 56 data bytes
64 bytes from 74.125.226.210: icmp_seq=0 ttl=56 time=11.578 ms
64 bytes from 74.125.226.210: icmp_seq=1 ttl=56 time=12.127 ms
64 bytes from 74.125.226.210: icmp_seq=2 ttl=56 time=11.739 ms
64 bytes from 74.125.226.210: icmp_seq=3 ttl=56 time=9.660 ms
64 bytes from 74.125.226.210: icmp_seq=4 ttl=56 time=11.854 ms
64 bytes from 74.125.226.210: icmp_seq=5 ttl=56 time=9.952 ms
64 bytes from 74.125.226.210: icmp_seq=6 ttl=56 time=9.661 ms
64 bytes from 74.125.226.210: icmp_seq=7 ttl=56 time=11.673 ms
64 bytes from 74.125.226.210: icmp_seq=8 ttl=56 time=9.808 ms
64 bytes from 74.125.226.210: icmp_seq=9 ttl=56 time=11.818 ms
64 bytes from 74.125.226.210: icmp_seq=10 ttl=56 time=9.768 ms
64 bytes from 74.125.226.210: icmp_seq=11 ttl=56 time=11.950 ms
64 bytes from 74.125.226.210: icmp_seq=12 ttl=56 time=11.603 ms
64 bytes from 74.125.226.210: icmp_seq=13 ttl=56 time=11.509 ms
64 bytes from 74.125.226.210: icmp_seq=14 ttl=56 time=9.349 ms
64 bytes from 74.125.226.210: icmp_seq=15 ttl=56 time=11.659 ms
64 bytes from 74.125.226.210: icmp_seq=16 ttl=56 time=9.833 ms
64 bytes from 74.125.226.210: icmp_seq=17 ttl=56 time=9.618 ms
64 bytes from 74.125.226.210: icmp_seq=18 ttl=56 time=9.590 ms
64 bytes from 74.125.226.210: icmp_seq=19 ttl=56 time=11.882 ms
64 bytes from 74.125.226.210: icmp_seq=20 ttl=56 time=11.751 ms
Running my network port maxed in both directions, here is what I get as a ping result.
4 bytes from 74.125.226.211: icmp_seq=28 ttl=56 time=37.256 ms
64 bytes from 74.125.226.211: icmp_seq=29 ttl=56 time=31.582 ms
64 bytes from 74.125.226.211: icmp_seq=30 ttl=56 time=21.715 ms
64 bytes from 74.125.226.211: icmp_seq=31 ttl=56 time=38.200 ms
64 bytes from 74.125.226.211: icmp_seq=32 ttl=56 time=24.890 ms
64 bytes from 74.125.226.211: icmp_seq=33 ttl=56 time=30.898 ms
64 bytes from 74.125.226.211: icmp_seq=34 ttl=56 time=33.808 ms
64 bytes from 74.125.226.211: icmp_seq=35 ttl=56 time=34.333 ms
64 bytes from 74.125.226.211: icmp_seq=36 ttl=56 time=33.813 ms
64 bytes from 74.125.226.211: icmp_seq=37 ttl=56 time=33.281 ms
64 bytes from 74.125.226.211: icmp_seq=38 ttl=56 time=32.551 ms
64 bytes from 74.125.226.211: icmp_seq=39 ttl=56 time=33.819 ms
64 bytes from 74.125.226.211: icmp_seq=40 ttl=56 time=33.156 ms
64 bytes from 74.125.226.211: icmp_seq=41 ttl=56 time=28.781 ms
64 bytes from 74.125.226.211: icmp_seq=42 ttl=56 time=14.866 ms
Ultimately, it's it's a two sided problem that first needs to be addressed by Frontier (downstream). Upstream issues generally need to be dealt with on the CPE end especially with servers having significantly higher receiving speeds over standard home connections. Best bet right now is to work QoS at home and see what can be done to avoid those latency spikes. | |
 spewakR.I.P DadkinsPremium
join:2001-08-07
Elk Grove, CA
kudos:1 | reply to GregOP
Reading this post and following discussions is like Networking porn! I love it. 
--
The weekend is here, grab a can of beer!
| |
GregOP
join:2012-06-19
Beaverton, OR | reply to GregOP
Re: [Internet] Frontier FIOS - Latency and QoS - Where they fail Great conversation guys. Glad to put this out in the open and to get affirmation of my initial assumptions. On the one hand, it's nice knowing that if I do my QoS right, for the most part, network lag should be minimal even under full download saturation (my upload is never saturated when downloading and I rarely upload large files - youtube is an exception). On the other hand, I'd really just prefer that it work without all the QoS on the CPE. Comcast in my area has been extremely stable. I always got the speeds advertised (non-speedboost speeds) and had no complaints with the service.
I do have the potential for much much higher upload which is why I personally want to stick with FIOS. The option to have 35Mbit up is tantalizing. Do frontier reps ever frequent these boards and take suggestions to their internal teams? It'd be interesting to do a survey to a large amount of FIOS customers and determine how many of their more technical people have notice high latency during streaming/saturated downloads. I'm not sure how many people realize it's happening.
I can tell you, for anyone switching to Frontier FIOS, it'd be nice to have something like this in a FAQ stating that frontier does not do QoS and that saturated download or upload will cause extremely high latency. It certainly wasn't something I expected when I s witched and gave me an instantaneous headache and bad opinion about their service (Wife saying WTF this FIOS sucks. 1000 ping when we download. Comcast never did that. I want to switch back. - That could be avoided) | |
Smith6612Premium,MVM
join:2008-02-01
North Tonawanda, NY
kudos:22 Reviews:
·Verizon Online DSL
·Frontier Communi..
| said by GregOP:I can tell you, for anyone switching to Frontier FIOS, it'd be nice to have something like this in a FAQ stating that frontier does not do QoS and that saturated download or upload will cause extremely high latency. It certainly wasn't something I expected when I s witched and gave me an instantaneous headache and bad opinion about their service (Wife saying WTF this FIOS sucks. 1000 ping when we download. Comcast never did that. I want to switch back. Noted. If someone with Frontier FiOS and a Wired connection wants to let me Teamviewer in someday to get some characteristics off of the line, or if someone wants to run some tests for me, that would be great. A few lines to test against would be great, if anyone wants to offer something up. I have seen similar characteristics with some cable connections in my area as well way back in the day.
There are Frontier reps who visit these forums. Some of them I haven't seen for a while but they are here. I'm sure Higher Up also browses these forums as Anonymous users too. | |
TJ_665
join:2001-07-04
Fairview, OR | reply to GregOP
Since I noticed you're in OR, thought I would throw this link out:
»www.frontierforhome.com/oregon/leadership/
Perhaps if voiced your concerns to someone high up (GM for OR), the wheels might get going on this. Couldn't hurt IMO. | |
Ben JPremium
join:2011-09-16
Elk Grove, CA
kudos:3 | said by TJ_665:Perhaps if voiced your concerns to someone high up (GM for OR), the wheels might get going on this. Couldn't hurt IMO.
For something technical like this, I think the correct people are right here on the forums. I've challenged the the equipment vendor to come up with a solution here, and incorporated this as a requirement for the FiOS Engineering team to include in the next set of subscriber profile configurations. We'll see what they come up with. All goes well, we could conceivably trial this later this year in Washington with some other network upgrades we're planning, and get it broadly deployed soon thereafter.
--
Transparency Disclosure and Disclaimer: I am a Frontier employee posting in my own personal capacity. The opinions and positions expressed are my own and do not necessarily reflect those of Frontier. | |
Smith6612Premium,MVM
join:2008-02-01
North Tonawanda, NY
kudos:22 Reviews:
·Verizon Online DSL
·Frontier Communi..
| said by Ben J:For something technical like this, I think the correct people are right here on the forums. I've challenged the the equipment vendor to come up with a solution here, and incorporated this as a requirement for the FiOS Engineering team to include in the next set of subscriber profile configurations. We'll see what they come up with. All goes well, we could conceivably trial this later this year in Washington with some other network upgrades we're planning, and get it broadly deployed soon thereafter.
+1 | |
GregOP
join:2012-06-19
Beaverton, OR | reply to Ben J
That is great to hear. I think in the long run, for most consumers, this will be a big improvement. Keep us updated on the status of this project. Thanks for getting it kicked off. | |
 cdruGo ColtsPremium,MVM
join:2003-05-14
Fort Wayne, IN
kudos:7 | reply to GregOP
said by GregOP:It certainly wasn't something I expected when I s witched and gave me an instantaneous headache and bad opinion about their service (Wife saying WTF this FIOS sucks. 1000 ping when we download. Comcast never did that. I want to switch back. I find it somewhat ironic that we complain when ISPs muck with our connections by throttling connections, blocking ports and protocols, etc... for "the greater good of everyone". But now we're complaining because they don't muck with connections "for the greater good of everyone". | |
Reviews:
·Frontier Communi..
·WildBlue
| said by cdru:I find it somewhat ironic that we complain when ISPs muck with our connections by throttling connections, blocking ports and protocols, etc... for "the greater good of everyone". But now we're complaining because they don't muck with connections "for the greater good of everyone".
Note quite. What we are complaining about is that the connection sharing for everyone INSIDE THE HOUSEHOLD is not working the way it should. Just because you are downloading something it should not give all available bandwidth to that download ONLY. As more things are downloaded simultaneously the available bandwidth should balanced between all items being downloaded. This is what QOS is supposed to do and it is not enabled properly.
As an example, if 100mbs is available and I start a download I can use all of it, but if my wife starts a download she should get 1/2 of the total available. That is not happening my download is still using the entire connection and she gets maybe 1% which is why pings are going through the roof.
Frontier DSL does not seem to have this issue. When I am doing a large download pings only go up to around 250ms from the usual 50ms and she is not badly affected, her youtube video does not stop etc. People on Frontier FIOS are going to 10ms to over 1000ms and other people sharing the connection are being affected. | |
Ben JPremium
join:2011-09-16
Elk Grove, CA
kudos:3 | said by cdru:I find it somewhat ironic that we complain when ISPs muck with our connections by throttling connections, blocking ports and protocols, etc... for "the greater good of everyone". But now we're complaining because they don't muck with connections "for the greater good of everyone".
Trust me, the irony is not lost on us. Implementing QoS becomes an issue of whack-a-mole policies. How do you determine what to prioritize/shape/drop and when? Going by flows works fine until someone turns on BT, then BT will force everything else out because it uses so many flows. What happens when I try to get Gaming to perform better than say, BT, and end up leaving behind World of Warcraft because my network equipment can't tell the difference based on simple packet headers/sizing? What happens when Game/Application Y figures out my policy and rewrites their protocol so their traffic is miscategorized as something else, and gets preference over their competitor's site/application? Do I then have to deploy heavy-handed and expensive DPI engines to identify and handle all these corner cases? Somewhere there's going to be scatterings of 100 or 1000 customers who use some protocol we didn't test, or have some odd problem. Am I in for an administrative nightmare by deploying this? How quickly can I train my CSRs and NOC techs to recognize an issue on the network is due to QoS interactions now and not something else? This isn't in the realm of "simple" networking skills anymore. Will this cause outages to take longer to resolve?
And suppose I do actually get a policy that works for 99% of users out there, this forum will still be littered with posts from various power users who complain about how it messes up their obscure firewall/NAT/QoS/app/VPN/etc. and wish we'd have just left it the way it was. Look at the posts about not wanting to use an ISP's modem/router because it "sucks" and is "not geared to power users", even though it's perfectly fine for 99% of the subscriber base.
Having been working on various strategies and proof of concepts in this area for the better part of 2 years, and all I can say about it is it's a minefield wrapped up in pandora's box and tossed into a pressure cooker on high. It's going to be impossible to please everyone at best, and going to piss just about everyone off at worst.
said by Aranarth:Note quite. What we are complaining about is that the connection sharing for everyone INSIDE THE HOUSEHOLD is not working the way it should. Just because you are downloading something it should not give all available bandwidth to that download ONLY. As more things are downloaded simultaneously the available bandwidth should balanced between all items being downloaded. This is what QOS is supposed to do and it is not enabled properly.
QoS is not enabled at all. The TCP protocol is well designed and by itself, balances out connections quite well. Four TCP downloads will "figure it out" and balance themselves naturally to ~25% each of the link speed. When you start mixing non-linear UDP flows in there, TCP gets confused. The UDP retransmissions don't follow the same "back off" methodology (if the protocol retransmits at all). This, to the TCP algorithm, is like trying to calculate the speed of a link with randomly changing bandwidth. TCP can't stabilize on a specific window size. Depending on aggressively TCP is tuned in that particular IP stack, one will start to force the other out (like you're seeing), and/or TCP will start to "wave" (burst then slow, burst then slow, etc.) which just wrecks more havoc on both. In the end, one of the two are going to suffer disproportionately. The more you try to fix this across the breadth of protocols out there, the higher the level of intelligence the QoS classifier has to have (and thus the processing power to do this across thousands of subscriber queues). Usually you end up breaking more stuff than you fix.
Enterprise QoS engineers don't typically have this problem, because they know very specifically what traffic they want to handle differently (e.g., they don't classify "voice" generically, but know it's "W protocol running X codec running at Y bps with Z packet sizes, destined to these hosts, etc." and it's easy to cherry-pick out that traffic).
Frontier DSL does not seem to have this issue. When I am doing a large download pings only go up to around 250ms from the usual 50ms and she is not badly affected, her youtube video does not stop etc. People on Frontier FIOS are going to 10ms to over 1000ms and other people sharing the connection are being affected.
A quick but "crude" fix would be to switch from shaped to policed with a dramatically smaller queue length. That doesn't solve the problem, but would make FiOS act more like a DSL connection in that regards.
Ideally, there would be a remotely applied QoS policy that a subscriber is able to self-configure. I've looked at a handful of potential solutions here and have yet to find one that's cost-effective over supporting a large subscriber base. So instead we'd probably take this in a different direction and give subscribers a choice of a simple and standardized/well-known generic policy by default, while leaving the option to call into a CSR/tech and revert to a simple FIFO queue for power users who just want the untouched pipe or for people who have issues. Much in the same way today we interleave on DSL by default, but power users can still get fast path if they want it.
--
Transparency Disclosure and Disclaimer: I am a Frontier employee posting in my own personal capacity. The opinions and positions expressed are my own and do not necessarily reflect those of Frontier. | |
Smith6612Premium,MVM
join:2008-02-01
North Tonawanda, NY
kudos:22 Reviews:
·Verizon Online DSL
·Frontier Communi..
| said by Ben J:Ideally, there would be a remotely applied QoS policy that a subscriber is able to self-configure. I've looked at a handful of potential solutions here and have yet to find one that's cost-effective over supporting a large subscriber base. So instead we'd probably take this in a different direction and give subscribers a choice of a simple and standardized/well-known generic policy by default, while leaving the option to call into a CSR/tech and revert to a simple FIFO queue for power users who just want the untouched pipe or for people who have issues. Much in the same way today we interleave on DSL by default, but power users can still get fast path if they want it.
I would love something like this if you guys can find a way to implement it effectively. It's sort of like a similar idea I've had in my mind on allowing folks to dynamically adjust their speed profile based on a "cost per megabit" and do a "Build your own speed using what's available" sort of setup. | |
GregOP
join:2012-06-19
Beaverton, OR | reply to Ben J
said by Ben J:Ideally, there would be a remotely applied QoS policy that a subscriber is able to self-configure. I've looked at a handful of potential solutions here and have yet to find one that's cost-effective over supporting a large subscriber base. So instead we'd probably take this in a different direction and give subscribers a choice of a simple and standardized/well-known generic policy by default, while leaving the option to call into a CSR/tech and revert to a simple FIFO queue for power users who just want the untouched pipe or for people who have issues. Much in the same way today we interleave on DSL by default, but power users can still get fast path if they want it.
I imagine that it is stated somewhere, but, if I had known from the get go that FIOS was running FIFO, I could have turned on my own QoS before the wife noticed and got upset about it. Personally, I am not against a flat open pipe where the user has to manage it. That is something that is hard to find these days for sure. And with more and more attempts at internet censorship, government watchdogs, etc, I'm glad I've got an ISP out there that isn't going the heavy handed approach.
That said, there probably should be a generic QoS policy for the majority of users and then have some sort of statement showing power users that if they wish to have a FIFO line, they may do so at the request of a CSR. Now that I've set my QoS up properly, I've not had any issues with slow downs. I have however lost connection at the outside box 3 times since I got the service, but that is a different issue altogether.
I've talked to other users in my area who are 'power' users, and they too like the option to have a FIFO line. We are the minority though. I do appreciate that people are paying attention to the minority for once though  Normally we get shunned out the back door and off the cliff. | |
|
