dslreports logo
site
 
    All Forums Hot Topics Gallery
spc

spacer




how-to block ads


Search Topic:
uniqs
5
share rss forum feed

pfsmith

join:2006-11-16
Lafayette, IN
Reviews:
·Comcast
reply to JigglyWiggly

Re: [Speed] ICSI Netalyzr Results - "Excess Buffering"

said by JigglyWiggly:

if you setup QOS then my excess buffering problem went away on pfsense.

OFC now that I have 50 megabits, I don't care.

I have a linksys E4200 with up to date firmware.

When I setup QOS of any kind it slows things down.

So, it's off during the above tests... but I'm alone on my network, too, so it should not matter.

My speedtest are fine, btw. But, I can tell somethings not right...


JigglyWiggly

join:2009-07-12
Pleasanton, CA
excess buffering is only a problem if you are maxing your connection and then trying to do other stuff while maxing your connection


graysonf
Premium,MVM
join:1999-07-16
Fort Lauderdale, FL
kudos:2
reply to pfsmith
said by pfsmith:

When I setup QOS of any kind it slows things down.

This is the price of using QOS. In order for it to work you have to surrender a bit of your bandwidth to it. The implementation I use takes 5%.

pfsmith

join:2006-11-16
Lafayette, IN
Reviews:
·Comcast
reply to JigglyWiggly
I think maybe there's some confusion...

My router didn't used to do this. I didn't change anything.

I'm alone on my connection. Wired. No wireless.

Nothing using the network except the test when I ran it.

It just started a couple of weeks ago. It's really effecting everything, but especialy XBOX gaming.

I think it's something Comcast is doing... how do I make it stop?


jlivingood
Premium,VIP
join:2007-10-28
Philadelphia, PA
kudos:2

2 recommendations

said by pfsmith:

I think it's something Comcast is doing... how do I make it stop?

Don't think that is the case. Take a few minutes and watch this tech talk:

»www.youtube.com/watch?v=-D-cJNtKwuw

--
JL
Comcast


tshirt
Premium,MVM
join:2004-07-11
Snohomish, WA
kudos:5
Reviews:
·Comcast

1 recommendation

Excellent linkage there, Jason.
That is the quintescental missing piece of most peoples "how to use broadband networks" training manual.

Yes it's 20 minutes,
yes his speaking style is less than exciting
but if you watch (at least once) and UNDERSTAND his point, you begin to see why many things people CHOOSE to do on their LAN actually limits the usablity for some apps, and the push for MORE/MAXIMUM bandwidth NOW, actually pushes some ISP's to make some of the same mistakes.

Should be required watching.


NetFixer
Freedom is NOT Free
Premium
join:2004-06-24
The Boro
Reviews:
·Cingular Wireless
·Comcast Business..
·Vonage

2 edits
reply to pfsmith
said by pfsmith:

said by JigglyWiggly:

if you setup QOS then my excess buffering problem went away on pfsense.

OFC now that I have 50 megabits, I don't care.

I have a linksys E4200 with up to date firmware.

When I setup QOS of any kind it slows things down.

So, it's off during the above tests... but I'm alone on my network, too, so it should not matter.

My speedtest are fine, btw. But, I can tell somethings not right...

Yes, as others have pointed out QoS will reduce your overall bandwidth (that is the nature of the beast). However, the benefits can override the slight bandwidth loss (especially for latency sensitive applications such as gaming and VoIP).


Netalyzer test without QoS



Netalyzer test with QoS



Comcast speed test without QoS



Comcast speed test with QoS


FYI, I am on a 12/2 mbps speed tier. For me the tradeoff between a reliably responsive connection and seeing the warm fuzzy PowerBoost induced speed test results is an easy choice. Of course, YMMV.
--
We can never have enough of nature.
We need to witness our own limits transgressed, and some life pasturing freely where we never wander.

pfsmith

join:2006-11-16
Lafayette, IN
Reviews:
·Comcast
reply to jlivingood
I watched the video. I don't think this is the issue...

I'm alone on my network. One Win7 PC, one XBOX.

Both wired through Linky E4200.

No QOS. (disabled on my Linky)

Nothing running in the background. Completely idle connection. XBOX is turned OFF.

I run the ICSI Netalyzr and get the "excess buffering".

I'm going to try hooking my PC up directly - without the router. Will post results in the morning.

Also, when I tried QOS for my XBOX it made things worse.

I've tried DMZ for the XBOX and that didn't help, either.

I have 2-4M upload - confirmed by multiple speedtests - but, my XBOX gives a warning about slow upload speeds. How is this possible? 2M is *plenty* fast. Yet, my games are *horrible* - about half are unplayable.

Browsing on my PC - with XBOX off - is very laggy. Yet, I have ~25M down according to multiple speed tests. How is this possible? I can't load two *simple* pages at the same time without it taking 10+ seconds (for exampe, dslreports & drudgereport. These are quick - mostly text - webpages).

My speedtests indicate about 70ms to Chicago. It feels more like 500ms, or worse.


NetFixer
Freedom is NOT Free
Premium
join:2004-06-24
The Boro
Reviews:
·Cingular Wireless
·Comcast Business..
·Vonage

1 edit
said by pfsmith:

I watched the video. I don't think this is the issue...

I'm alone on my network. One Win7 PC, one XBOX.

Both wired through Linky E4200.

No QOS. (disabled on my Linky)

Nothing running in the background. Completely idle connection. XBOX is turned OFF.

I run the ICSI Netalyzr and get the "excess buffering".

I'm going to try hooking my PC up directly - without the router. Will post results in the morning.

Also, when I tried QOS for my XBOX it made things worse.

I've tried DMZ for the XBOX and that didn't help, either.

I have 2-4M upload - confirmed by multiple speedtests - but, my XBOX gives a warning about slow upload speeds. How is this possible? 2M is *plenty* fast. Yet, my games are *horrible* - about half are unplayable.

Browsing on my PC - with XBOX off - is very laggy. Yet, I have ~25M down according to multiple speed tests. How is this possible? I can't load two *simple* pages at the same time without it taking 10+ seconds (for exampe, dslreports & drudgereport. These are quick - mostly text - webpages).

My speedtests indicate about 70ms to Chicago. It feels more like 500ms, or worse.

Look at the test results I posted here: »Re: [Speed] ICSI Netalyzr Results - "Excess Buffering"

There was nothing else significant going on when I did the Netalyzer tests and the Comcast speed tests either. The only difference is that with QoS enabled in my Netgear switch, I don't have the buffer bloat effect, but without QoS, I do have the buffer bloat effect. I use minimal QoS settings, just enough to limit my connection to its real sustained speed and that eliminates the effects of PowerBoost and buffer bloat for me. Of course, if you can't live without PowerBoost...
--
We can never have enough of nature.
We need to witness our own limits transgressed, and some life pasturing freely where we never wander.


jlivingood
Premium,VIP
join:2007-10-28
Philadelphia, PA
kudos:2
reply to pfsmith
said by pfsmith:

I watched the video. I don't think this is the issue...
I run the ICSI Netalyzr and get the "excess buffering".

The point is that the buffer issue exists in lots of equipment and operating systems.
--
JL
Comcast

pfsmith

join:2006-11-16
Lafayette, IN
Reviews:
·Comcast
said by jlivingood:

said by pfsmith:

I watched the video. I don't think this is the issue...
I run the ICSI Netalyzr and get the "excess buffering".

The point is that the buffer issue exists in lots of equipment and operating systems.

»n1.netalyzr.icsi.berkeley.edu/re···-ac51/rd

Here's my latest ICSI test with my Win7 PC hooked up directly to the Comcast modem.

The "excess buffering" persists. Browsing still kinda sucks.

Is this normal? Really?

It seems like it only started recently, maybe about 2 months ago. Before that everything was VERY snappy. Gaming was good, too.

Actually, the only thing that changed was my 1 year pricing expired with Comcast. Would that have changed my connection somehow? My "speeds" didn't change. Just this buffering as near as I can tell...


tshirt
Premium,MVM
join:2004-07-11
Snohomish, WA
kudos:5
Reviews:
·Comcast
Other possibilites include malware of some kind, The "slow clock" warning can also be a sign of malware (unless you have choosen to use a different time standard) I also see warnings there about possible DNS errors.
check all those things would probably be a good idea.


NetFixer
Freedom is NOT Free
Premium
join:2004-06-24
The Boro
Reviews:
·Cingular Wireless
·Comcast Business..
·Vonage
said by tshirt:

Other possibilites include malware of some kind, The "slow clock" warning can also be a sign of malware (unless you have choosen to use a different time standard) I also see warnings there about possible DNS errors.
check all those things would probably be a good idea.

The "slow clock" is a common symptom on Windows PCs because the default settings that MS uses for time sync is to say the least not very effective.

The DNS warnings are because the OP uses a hosts file to block certain Google hostnames (apparently Google provided some funding for the Netalyzer project).

If the OP turned on minimal QoS in the Cisco/Linksys router to limit bandwidth to the sustained rate, the buffer bloat symptom would probably disappear.
--
We can never have enough of nature.
We need to witness our own limits transgressed, and some life pasturing freely where we never wander.


tshirt
Premium,MVM
join:2004-07-11
Snohomish, WA
kudos:5
Reviews:
·Comcast
I haven't found that to be a problem on Win7 64 bit, but hardware does vary.

Malware isn't the only reason but it can cause a noticable time loss effect.
The reason I mentioned it is that it was 339 seconds off in the first test posted, and 346 in the latest test, A decay rate of 7 seconds in 2 days is not/should not be considered normal.
also an inacurate real time clock can lead to false test results.

I do have clients who have choosen not to use internet time because a school district work or local TV time has a different offset and is more important to them, but this clock seems to be drifting fast.

pfsmith

join:2006-11-16
Lafayette, IN
Reviews:
·Comcast
reply to NetFixer
said by NetFixer:

The "slow clock" is a common symptom on Windows PCs because the default settings that MS uses for time sync is to say the least not very effective.

The DNS warnings are because the OP uses a hosts file to block certain Google hostnames (apparently Google provided some funding for the Netalyzer project).

If the OP turned on minimal QoS in the Cisco/Linksys router to limit bandwidth to the sustained rate, the buffer bloat symptom would probably disappear.

My PC has always had a clock problem. Dunno why. It predates having a browsing/network/buffer problem by several *years*.

DNS: I redirect many advertising URLS to localhost. Got the hosts file from someplace, don't remember where. This is not the issue.

I run mal-ware and virus scans regularly. I've never been infected.

WHY does turning on QOS fix this problem? If I'm directly connected to my modem - no router - why do I still have a problem? Is my Comcast connection being throttled? I'm telling you, this has only begun recently and it's NOT because of my home router's buffers. I should be able to use my connection at full speed, right? Should everything I do feel like it's 500ms lag... I might as well get a sattelite connection! It would probably *feel* faster!


NetFixer
Freedom is NOT Free
Premium
join:2004-06-24
The Boro
Reviews:
·Cingular Wireless
·Comcast Business..
·Vonage

2 edits
said by pfsmith:

said by NetFixer:

The "slow clock" is a common symptom on Windows PCs because the default settings that MS uses for time sync is to say the least not very effective.

The DNS warnings are because the OP uses a hosts file to block certain Google hostnames (apparently Google provided some funding for the Netalyzer project).

If the OP turned on minimal QoS in the Cisco/Linksys router to limit bandwidth to the sustained rate, the buffer bloat symptom would probably disappear.

My PC has always had a clock problem. Dunno why. It predates having a browsing/network/buffer problem by several *years*.

DNS: I redirect many advertising URLS to localhost. Got the hosts file from someplace, don't remember where. This is not the issue.

I run mal-ware and virus scans regularly. I've never been infected.

WHY does turning on QOS fix this problem? If I'm directly connected to my modem - no router - why do I still have a problem? Is my Comcast connection being throttled? I'm telling you, this has only begun recently and it's NOT because of my home router's buffers. I should be able to use my connection at full speed, right? Should everything I do feel like it's 500ms lag... I might as well get a sattelite connection! It would probably *feel* faster!

Did you view the video posted by jlivingood See Profile in this post: »Re: [Speed] ICSI Netalyzr Results - "Excess Buffering" ? If so, and you don't understand the problem, then I am sorry, but I don't think I can explain it any simpler than the author of that video. But FYI (just in case you have not actually looked at the above mentioned video) the problem is in buffers located at various places on the Internet, not buffering in your router (which is why you still see the symptom when you bypass the router). As for why you did not used to see the symptom, but now you do...that is just the dynamic nature of the Internet.

FWIW, the test results that I posted here: »Re: [Speed] ICSI Netalyzr Results - "Excess Buffering" that demonstrated how enabling QoS eliminated the buffer bloat symptom detected by the Netalyzer test, were done on my Windows Server box which is for all intents directly connected to my "modem". I have a business class static IP account, and that box has a public static IP, so logically it is directly connected to the Internet. However, there is a Netgear switch between the server and the SMCD3G which can do rate limiting QoS, and when I enable that rate limiting QoS, I do not see the buffer bloat symptoms.

Shown below is my DSLR line monitor for my Comcast connection. The high ping spikes you see prior to late Monday evening, are a symptom of buffer bloat. The clean portion of that chart (since late Monday evening) is a result of enabling minimal rate limiting in my Netgear switch. I actually would have done this sooner, but it was a PITA to rewire my network to put a VLAN segment in my Netgear switch between the SMCD3G (which has no QoS settings) and the rest of my network.




Shown below is some additional information taken from my SamKnows box for Monday. This shows the actual (minimal) bandwidth reduction resulting from enabling QoS rate limiting (the times differ from the DSLR line monitor because SamKnows uses a different time zone). The data below shows that after a bit of fiddling with the QoS rate limiting settings, my throughput is now limited to just below my official 12/2 mbps sustained connection speed. For me, this setting works quite well because I am not interested in Comcast's PowerBoost anyway. As always, YMMV.







Shown below is a diagram of my local network, just in case seeing how I am connected helps you to understand where I did my QoS. The VLAN segment labeled VWAN is where I have enabled rate limiting QoS in my Netgear switch (it only effects traffic to/from my SMCD3G). In your case, enabling minimal QoS rate limiting in your own Cisco/Linksys router would probably help with your situation...a managed switch is not really required (I used to do the same thing in my now retired Cisco/Linksys RV082).




All I can guarantee that it works for me...YMMV.
--
We can never have enough of nature.
We need to witness our own limits transgressed, and some life pasturing freely where we never wander.

pfsmith

join:2006-11-16
Lafayette, IN
Reviews:
·Comcast

1 edit
Click for full size
No, I guess I still don't get it... but, what I think you don't get is that everything was fine for *years* until a couple of months ago when the symptoms started...

However, I will try some things... maybe you can help me:

My E4200 only has one QOS setting as far as I can tell - it can limit upstream bandwidth. Since I don't care about Powerboost, either, and I'm supposedly on a 25/2 type speed, I've tried setting it to "2Mbps". Anything more than that I should be doing?

»stage.results.speedtest.comcast.···0062.png

»netalyzr.icsi.berkeley.edu/resto···d4f-8abd

I still get the buffering on the *upload* - actually, I think it's worse, IIRC.

As near as I can tell the E4200 doesn't have a way to QOS the download.

I will use this setting for today and see how it feels.

Thanks!

Update: I've tried various settings - 1-4Mbps, 2500 & 2200Kbps, and "Auto". 2Mbps seems to reduce downlink buffering to ~110ms. Uplink buffering remains ~500-1000ms.

Symptoms remain. We'll see about gaming tonight.


NetFixer
Freedom is NOT Free
Premium
join:2004-06-24
The Boro
Reviews:
·Cingular Wireless
·Comcast Business..
·Vonage
said by pfsmith:

No, I guess I still don't get it... but, what I think you don't get is that everything was fine for *years* until a couple of months ago when the symptoms started...

However, I will try some things... maybe you can help me:

My E4200 only has one QOS setting as far as I can tell - it can limit upstream bandwidth. Since I don't care about Powerboost, either, and I'm supposedly on a 25/2 type speed, I've tried setting it to "2Mbps". Anything more than that I should be doing?...

As near as I can tell the E4200 doesn't have a way to QOS the download...

Update: I've tried various settings - 1-4Mbps, 2500 & 2200Kbps, and "Auto". 2Mbps seems to reduce downlink buffering to ~110ms. Uplink buffering remains ~500-1000ms.

Symptoms remain. We'll see about gaming tonight.

I do understand that this is a new symptom for you, but as I previously noted, the Internet is dynamic (it is subject to frequent changes).

Sorry to hear that your router has such a poor rate limiting implementation. My old RV082 had overall up and down rate limiting as well as individual up/down rate limiting settings for specific attached devices and/or services, and I incorrectly assumed that your E4200 had similar capability. I won't suggest that you invest in a more advanced router or a managed switch since there is no way I can guarantee that would fix your symptoms (but if your symptoms persist and continue to cause noticeable problems, you may want to consider that option).
--
We can never have enough of nature.
We need to witness our own limits transgressed, and some life pasturing freely where we never wander.

pfsmith

join:2006-11-16
Lafayette, IN
Reviews:
·Comcast
»netalyzr.icsi.berkeley.edu/resto···287-af22

Today it's 710ms uplink buffer and 1700ms download buffer.

Despite still having it set at the 2Mbps QOS uplink speed. The "buffering" seems to fluctuate.

I'm not buying this buffering explanation. This is not "daddy bogs down the house" like the video talks about. This is me, alone, on my own connection - one PC - and having a problem even if I take out the router.

It seems to me what your saying is that if I *throttle* *myself* I won't notice that Comcast cannot actually give me what I'm paying for.

Right?

I really don't mean to be difficult. But it makes no sense to me that I cannot use the speed my connection claims to have available. It makes no sense that this just started up one day and now my connection doesn't work very well and *I* have to figure out what's wrong with it even though I haven't changed anything??


tshirt
Premium,MVM
join:2004-07-11
Snohomish, WA
kudos:5
Reviews:
·Comcast
I think Jason was pointing out it's not JUST your computer and your router, and comast's various routers or the other routes outside and on the return path.
It's like a firehose filling your bathtub, and you opening the drain, needing to empty it to get to the freshest water
It's each step adding it's slight delay, (and sometimes a longer delay, when EVERYBODY needs something NOW. It's the cumulative effect which many of the commerial routers handle ok load up the return path and the bottle neck being the fairly constrained equipment on your LAN (router first) can be overwhelmed by the que depth.

We have created enough layers of big buffers to keep your connection FULL for 1700ms down and 710 up so even your ACKs may appear to be lost to the server,causing resends, and keeping the que (over)full at all times.


NetFixer
Freedom is NOT Free
Premium
join:2004-06-24
The Boro
Reviews:
·Cingular Wireless
·Comcast Business..
·Vonage

1 recommendation

reply to pfsmith
said by pfsmith:

»netalyzr.icsi.berkeley.edu/resto···287-af22

Today it's 710ms uplink buffer and 1700ms download buffer.

Despite still having it set at the 2Mbps QOS uplink speed. The "buffering" seems to fluctuate.

I'm not buying this buffering explanation. This is not "daddy bogs down the house" like the video talks about. This is me, alone, on my own connection - one PC - and having a problem even if I take out the router.

It seems to me what your saying is that if I *throttle* *myself* I won't notice that Comcast cannot actually give me what I'm paying for.

Right?

I really don't mean to be difficult. But it makes no sense to me that I cannot use the speed my connection claims to have available. It makes no sense that this just started up one day and now my connection doesn't work very well and *I* have to figure out what's wrong with it even though I haven't changed anything??

As you have pointed out, your router only allows you to do outbound rate limiting. You need to do rate limiting in both directions in order for it to have a chance to alleviate the buffer bloat symptom.

You can "buy" the buffer bloat explanation or not; that is your decision (I suspect that your personal "daddy bogs down the house" interpretation of the video that was provided by jlivingood See Profile is why you probably don't "buy" it). The symptoms you are describing however, are classic symptoms of buffer bloat.

It has already been pointed out that the changes are occurring in devices on the internet that are beyond your direct control. Just because you did not personally make any usage or hardware changes on your own personal piece of the Internet does not mean that you can't/won't be effected by what happens on the other side of your cable modem.

All I can tell you (again) is that when I do bidirectional rate limiting to eliminate the effects of PowerBoost, and only see my sustained speed (which I don't personally consider to be getting less than I am paying for), I no longer see the effects of buffer bloat.

Good luck on finding a solution, and please let us know what you finally do to alleviate your symptoms.
--
We can never have enough of nature.
We need to witness our own limits transgressed, and some life pasturing freely where we never wander.