dslreports logo
 
    All Forums Hot Topics Gallery
spc
Search similar:


uniqs
3410
BBL
join:2013-03-13
Raleigh, NC

BBL

Member

High latency from 8-11 PM every night in Raleigh, NC

Click for full size
For several months now, I've been having a problem where my latency becomes incredibly high every night from around 7:30 to 11:30 PM EST. Above is a 48 hour graph where the problem window is very visible, and here is a direct link: »i.imgur.com/nLdoecB.png

I've been monitoring it with this PingPlotter program for a few weeks now, and every night it's the same thing. On Mon/Tue/Wed/Thur it usually only occurs from around 8 - 11, but on Fri/Sat/Sun it will begin about 30 minutes earlier and end 30 minutes later. Due to this, and the fact that it is happening during prime time, I'm fairly confident that the issue must be on TWC's end, but I've tried several different troubleshooting methods to be more certain:

• The same thing happens on two completely different modem/router combo units (though they will report the lag at different hops, which I don't completely understand...).
• The lag happens on three different PCs that are all hard connected. I turned off all other PCs while testing this, and disabled all wireless functions.
• I've run many different virus scanners, and temporarily disabled Windows firewall, all to no avail. (Though a virus seems unlikely, unless all 3 machines are infected.)
• I have pinged several different sites, such as Google/Yahoo/RR.com, but the lag clearly occurs on one of TWC's servers, so this seems unnecessary.
• I have even had a new cable line installed from the street, which made no noticeable difference.

I have contacted TWC customer support countless times, including being escalated to tier 3 and a tier 3 supervisor, but they have repeatedly told me that they see nothing on their end. This is incredibly frustrating for me, since I have exhausted all of my troubleshooting ideas, and I can only be left with the conclusion that it must be something on their end.

It is mostly my latency that is affected during these lag windows; usually a speed test will still return normal results for bandwidth. But the high latency makes all gaming completely impossible during the hours I usually want to play. I also like to use a streaming program that will consume about 3.5 of my 5 Mb upload, and even outside of this lag window, my ping becomes slightly high and unstable (which can be somewhat seen on the far left of the graph), and this was not happening several months ago.

So my question, to you fine folks here:
1) Does it seem reasonable to assume that this issue must be on TWC's end?
2) If so, what can I communicate to them to get them to actually acknowledge and fix the problem? If not, what else can I do to troubleshoot on my end?

Thank you for your time!
Radisc359
join:2010-10-02
Fremont, NE

Radisc359

Member

This looks like what happened to the routers here after they released Docsis 3 tiers here then released the Standard 15Mbps the next week.

1 1 ms 1 ms 1 ms 192.168.1.1
2 51 ms 15 ms 23 ms cpe-76-84-96-1.neb.res.rr.com [76.84.96.1]
3 9 ms 9 ms 9 ms gig2-35.frmnne01-rtr1.neb.rr.com [76.85.222.5]
4 10 ms 10 ms 9 ms tge10-0-0.lnclne03-mx91.neb.rr.com [76.85.221.12
9]
5 12 ms 9 ms 9 ms ae1.lnclne03-mx92.neb.rr.com [76.85.221.114]
6 10 ms 27 ms 10 ms ae0.lnclne04-mx91.neb.rr.com [76.85.221.118]
7 99 ms 98 ms 99 ms ae1.lnclne04-mx92.neb.rr.com [76.85.221.122]
8 101 ms 102 ms 146 ms ae0.lnclne00-mx41.neb.rr.com [76.85.221.126]
9 113 ms 114 ms 110 ms ae9.chcgill3-rtr1.kc.rr.com [76.85.220.137]
10 113 ms 132 ms 119 ms ae-5-1.cr0.chi30.tbone.rr.com [66.109.6.112]
11 118 ms 111 ms 111 ms ae-0-0.cr0.chi10.tbone.rr.com [66.109.6.20]
12 112 ms 114 ms 114 ms 107.14.17.194
13 145 ms 138 ms 144 ms 74.125.48.109
14 116 ms 114 ms 114 ms 209.85.254.130
15 134 ms 122 ms 143 ms 72.14.237.130
16 127 ms 125 ms 123 ms 72.14.239.15
17 134 ms 134 ms 133 ms 216.239.46.59
18 * * * Request timed out.
19 136 ms 137 ms 168 ms ob-in-f105.1e100.net [173.194.77.105]

Does this look similar? I called in after it happened two days in a row during peak times and Tier 3 got it to the right people and it has been fixed since then. If the 2nd hop's ping is fine, but one of the routers after that hop has high ping, then it is TWC's router malfunctioning or congestion on the router. But if its the 2nd hop which is the cmts, you could have node congestion. Also which model gateways have you tried so far?
BBL
join:2013-03-13
Raleigh, NC

1 edit

BBL

Member

I'm not sure if I can say that your tracert looks similar, but how you described it definitely does.

I've tried a Motorola SBG6580 (purchased on my own) and an Arris DG860A (provided by TWC). I actually purchased the Motorola since it had some of the best reviews and I was hoping it might fix (or somewhat alleviate these issues), but no luck. The Arris seems slightly better even, but both have the same issue.

Here is a graph of my 2nd and 4th hops: »i.imgur.com/jibQ4Hh.png

The 2nd hop shows some crazy spikes that I don't fully understand, but they happen 24/7 and don't really correspond to any kind of performance hits that I can feel while gaming, so I haven't been too concerned with it.

The 4th hop is where the performance hits are noticeably visible and do correspond directly to gaming performance. But as I mentioned before, the different modems will report them at different hops some reason (maybe just a flaw in the PingPlotter software?). On the Motorola, the nightly spike was not visible at hop 4, but it was at hop 3. The Arris reported lag at hop 3 three nights ago, but it has reported it at hop 4 for the last two nights.
Radisc359
join:2010-10-02
Fremont, NE

Radisc359

Member

Of course my tracert is different(I'm in Nebraska), but you should see a large jump of ping on one of the routers. Your modems are Docsis 3, so they aren't the issue unless you see high pings at the 2nd hop.

From your picture I can see that your third hop is fine with ping but your problem is on the 4th hop, and this would be TWC's fault.

In ping plotter you should be able to see past tracroutes to google(on top), by clicking where it shows the high pings it should show exactly where the ping is going up that high, but as I'm suspecting its hop 4.

EDIT: You should post your signals, they should be located here: »192.168.100.1/

Jabbu
Premium Member
join:2002-03-06

Jabbu to BBL

Premium Member

to BBL
Tracert is misused daily here.

You need to worry about the destination ping. If your destination is 150ms, most likely there is a problem and going bottom to top you would likely see a 130ms to 30ms bump for example.

Please post tracerts during those hours, as the ones posted don't have a issue.

If you destination is 50ms, any router reporting 50+ is just from that ICMP packet being put on the back burner while the router deals with real traffic. Disregard any ping above the destination ping!!!
BBL
join:2013-03-13
Raleigh, NC

BBL

Member

Thanks for the replies, guys!

I setup PingPlotter to show pings for hop 4 (the first hop that shows lag) and hop 13 (the final hop at Google). Here is a snapshot of the data taken during the middle of last night's lag window: »i.imgur.com/VTdc8oY.png

Also, here are my signals: »i.imgur.com/285VYIj.png (though I just took this, outside of my lag window -- I can post them again during tonight's lag, if you suspect they'd be any different)

And the reason why I wasn't displaying the destination ping before, was that TWC has said in the past that they can't control anything beyond their routers, so I wanted to just highlight the fact that it appears to be happening on their hops. But good to know.

RaleighNC
@rr.com

RaleighNC to BBL

Anon

to BBL
Hello BBL,
I have been experiencing the same issue as you are seeing. It certainly seems to be concurrent with congestion, and while it's been fairly easy to track, as you've discovered it is unfortunately difficult to get traction on.

I am curious, are you currently seeing high latency this afternoon, Sunday, March 24th?

I have a voice server in New Jersey that I am currently seeing a response time between 150-190ms. Typical non-congested response times are in the 30-40ms range.

1 1 ms 1 ms 1 ms 192.168.1.1
2 * * * Request timed out.
3 20 ms 19 ms 12 ms 66.26.45.181
4 36 ms 18 ms 38 ms ae19.chrlncpop-rtr1.southeast.rr.com [24.93.64.2]
5 164 ms 183 ms 168 ms 107.14.19.46
6 163 ms 179 ms 180 ms ae-0-0.pr0.atl20.tbone.rr.com [66.109.6.171]
7 159 ms 197 ms 161 ms te1-6.bbr01.tl01.atl01.networklayer.com [66.109.11.118]
8 195 ms 209 ms 179 ms ae7.bbr01.tl01.atl01.networklayer.com [173.192.18.172]
9 48 ms 29 ms 42 ms ae0.bbr01.eq01.wdc02.networklayer.com [173.192.18.152]
10 154 ms 176 ms 177 ms ae0.dar01.sr01.wdc01.networklayer.com [173.192.18.197]
11 180 ms 163 ms 157 ms po1.fcr03.sr02.wdc01.networklayer.com [208.43.118.157]
12 160 ms 160 ms 159 ms xe0-0-1.cr1.wdc01.light-speed.com [184.173.191.35]
13 160 ms 164 ms 160 ms 174.36.193.148-static.reverse.softlayer.com [174.36.193.148]

Trace complete.

I can repeat this and get the same results for other destinations as well. I'm hopeful that perhaps a combination of folks reporting this will perhaps spur some resolution. I know speed issues are always a bit troublesome to troubleshoot in the first place and first line support typically do not have the tools/knowledge to be able to effectively do so. This isn't always the case of course and sometimes their hands are tied, but hopefully this will get kicked up the chain and addressed soon. The VPN disconnects I've been getting due to this have been troublesome.
BBL
join:2013-03-13
Raleigh, NC

BBL

Member

Click for full size
Heya,

I'm sorry to hear that you're having to deal with this frustrating issue as well, but I'm also glad to know that I'm not the only one experiencing it.

And yes, today, my lag began at around 1:30 PM and lasted all the way until 11:15 PM. Graph here: »i.imgur.com/91fTXty.png I have never seen it begin this early, so this is quite concerning.

Now that I have confirmation that I'm not the only one with this issue, I might try to be a little more persistent in getting them to fix it. Before, I wasn't 100% sure that the issue was on their end, and I didn't want to look like a fool if I repeatedly harassed them for something that wasn't their fault.

Let me know if you make any progress in getting them to acknowledge/fix this. Thanks!
dude34221
join:2002-06-13
Athens, OH

1 edit

dude34221 to BBL

Member

to BBL
said by BBL:

• The same thing happens on two completely different modem/router combo units (though they will report the lag at different hops, which I don't completely understand...).

this could be due to dif mac ids, getting dif ip addresses, of which are routed differenetly.

your snr is almost to the level of concern (35db) but not quite. i have two tests you need to run.

»www.hostmycalls.com/tools/ - linemonitor, not well known, HIGHLY useful. set it for two days, it will email you results when done. very helpful in diagnosing. example: »www.hostmycalls.com/ispt ··· me=feb23

»EarthLink DSL FAQ »Where are the TCP/Web100 Network Diagnostic Tool Locations? - choose the web100 test closest to you and paste results here

also, have you contacted direct support here?
BBL
join:2013-03-13
Raleigh, NC

BBL

Member

Thanks for the reply.

I started a 48 hour test on the HostMyCalls tool, but I did not receive an email within 5 minutes as they mentioned I should (and it didn't go to my spam folder). So I'm not sure if it's actually running?

Here are the results for the Web100 tool: »i.imgur.com/zQKMVpH.png -- Though this was taken outside of the lag window. I can run it again tomorrow during the lag.

I have tried contacting direct TWC support on DSLReports, and I was impressed at how quickly they responded. They just told me that a local person would get in touch with me, but everyone I have spoken to locally has just repeatedly said that they see nothing wrong on their end.
Radisc359
join:2010-10-02
Fremont, NE

Radisc359

Member

BBL, Your signal looks very good to me, your SNR(37-38dB) matches mine almost and your power levels are all within spec. So I am doubting your issue lies there.

Usually when its congestion the speeds will fall, like in my case I had 4Mbps on my speedtests with the high ping at one point. »www.speedtest.net/ Run a speedtest from that site or run one of dude's linked tests or both(preferably) during the high ping times and the results here.

Well you could give them the ip of the server that is having issues, that is why I did twice and on the second time I think I got a Tier 3 person with extra stuff and she went to the right people and it got fixed fast once they were made aware of it, never to happen again.
BBL
join:2013-03-13
Raleigh, NC

BBL

Member

Thanks. I'm trying to reach out to them again (via the direct support forum). They said someone should be in touch with me soon; hopefully it will be someone that can understand this problem.
BBL

BBL

Member

Hey guys,

I have good news. I received a call today saying they finally identified the problem, and are working to fix it. They don't have an ETA in place, but I got the impression that it might be a few weeks or so. I'm just glad that it's finally being sorted out.

Thanks for all the help!

kilrathi
Premium Member
join:2005-04-22
Rockaway Park, NY

kilrathi to BBL

Premium Member

to BBL
Try asking them what exactly the problem was, that way if it happens again in future you can point them in right direction.
BBL
join:2013-03-13
Raleigh, NC

BBL

Member

They said it was a "utilization issue", which just means overprovisioning I think.
Radisc359
join:2010-10-02
Fremont, NE

Radisc359

Member

It means congestion, but that is indeed good to hear.

F100
join:2013-01-15
Durham, NC
Alcatel-Lucent G-010G-A
(Software) pfSense
Pace 5268AC

F100 to BBL

Member

to BBL
When the networking director here at the University uses the term "utilization issue" it is usually related to one of the core campus routers. The CPU utilization on the router spikes and causes latency. This can happen for lots of different reasons but probably explains why it will take a few weeks to sort out. They may have to put in a more powerful router or find what is causing the issue.

In the RTP area you can also ping or test to ndt.ncren.net which is at MCNC in RTP. The 10gig uplinks for Duke, UNC, NC State all hit here to access the internet backbone for one of their campus connections. They hit the NC Research and Education network in a different diverse path over in Charlotte as well. This is one of the closest, fastest places in the Raleigh, Durham area to test to. If there is latency to NCREN, it is almost certainly TWC.

I have TWC Business Class but you can see that I hit the same TWC router at hop 4 even thought it hits a TWCBC router at hop 8 to ncren. I have the Ubee DDW3611 in bridge mode to my router. Everything is gigabit but my plan is only 7x768k. I have the Sam Knows router and it does show high latency at times. I have trouble using VOIP.ms at times and this may be the reason.

Tracing route to ndt.ncren.net [128.109.178.83]
over a maximum of 30 hops:

1 1 ms 1 ms 1 ms 192.168.1.1
2 7 ms 7 ms 5 ms 10.131.224.1
3 8 ms 9 ms 7 ms gig0-0-3.rlghnca-rtr1.nc.rr.com [24.25.20.74]
4 8 ms 7 ms 5 ms ae19.rlghncpop-rtr1.southeast.rr.com [24.93.64.0]
5 18 ms 29 ms 14 ms ae14.chrlncpop-rtr1.southeast.rr.com [24.93.64.25]
6 11 ms 13 ms 17 ms ten1-3.chrlncsa-p-rtr01.southeast.rr.com [24.93.73.57]
7 15 ms 13 ms 15 ms por100.twcc.rlghnc-a-c2701.nc.rr.com [24.27.255.253]
8 13 ms 13 ms 11 ms rrcs-24-172-64-46.midsouth.biz.rr.com [24.172.64.46]
9 15 ms 17 ms 16 ms rtp7600-gw-to-chlt7600-gw.ncren.net [128.109.9.18]
10 34 ms 19 ms 15 ms dc6500-1.dcs.mcnc.org [128.109.191.74]
11 18 ms 17 ms 16 ms ndt-03.ncren.net [128.109.178.83]

Trace complete.

Anon123
@rr.com

Anon123 to BBL

Anon

to BBL
I'm in Goldsboro, having the same issues. During morning/afternoon hours I see normal latency (5-30ms to Virginia, 30-40 Florida etc) but right around 7-8 every night for a few hours it spikes horribly to 140+ Virginia 180+ Florida. If you find a solution, or hear anything from TWC please update us!
blargoner
join:2013-03-02
New York, NY

blargoner to BBL

Member

to BBL
I like the thorough testing that you did.

I had a similar issue with speed, and also ran automated tests, as documented in this thread: »[TWC] TWC NYC (Manhattan) slow speeds during peak hours
BBL
join:2013-03-13
Raleigh, NC

BBL to Anon123

Member

to Anon123
Will do!

RaleighNC
@rr.com

RaleighNC

Anon

Any traction on this? I'm still seeing peak time latency, as well as issues streaming, well, anything at peak time. This doesn't even include any work related issues.
Decko37
join:2013-03-28
Goldsboro, NC

Decko37

Member

I'm in Goldsboro, was also seeing the unusually high latency at peak hours but it seems to have subsided for me since last week.

TWC_GSOuser to RaleighNC

Anon

to RaleighNC
As for traceroutes, Jabbu is correct. Only the destination latency truly matters. To add to that, however, when you begin to see latency go up, then continue to the end of the trace, there "could" be a congestion issues on a link where the latency becomes constant to the end.

The one thing that everyone must remember is that ICMP packets (those used in traceroute and ping in Windows-based computers) are treated by ISP routers with the absolute LOWEST priority. During "peak" times, those routers are processing heavy loads of traffic and it can seem like there are "issues" when there really are none. Google, Yahoo, and other search providers host their websites "in the cloud" and load balance across many different servers and/or routers in various places. This explains why everyone gets a different IP address for Google.com and Yahoo.com.

These steps should always be followed in order:
1) do a continuous ping on the URL/domain you are trying to get to (ping -t)
2) do a continuous ping on the DNS servers you are using
2) do a traceroute/PingPlotter once ping times begin to have constant latency. Does it take a different route?
3) Flush your DNS resolver cache (ipconfig /flushdns) and try again. Does the issue persist/continue?

It may well be due to over-utilization within your respective local areas. Typically it's caused by others doing heavy uploads (a.k.a BitTorrent) within your local area or there could be noise issues as well. The best thing is to do is to call tech support when it's happening as it's extremely hard to troubleshoot something you cannot "see" or duplicate in a "live" environment.