 pflogBueller? Bueller?Premium,MVM
join:2001-09-01
El Dorado Hills, CA
kudos:3 | [Connectivity] Route to company's VPN took a diveNormally my route to my employer's VPN is quite good. Today for some reason, everything is routing over ALTER.NET and all the way cross country and back.
I'm not sure how to determine if it's something on Comcast's end, my employer's end or the peering or what. All I know is 500+ ms latency and going cross-country and back (at least assuming the dns is accurate, which I know can not be the case) is not the norm.
How can I find out whether to call my IT people or Comcast or if there's just nothing I can do about it (other than ask my IT people or Comcast to complain to some transit/peering provider or whatever).
Packets Pings
Host Loss% Snt Last Avg Best Wrst StDev
1. 173-14-73-190-Sacramento.hfc.comcastbusiness.net 0.0% 3 0.6 0.6 0.5 0.6 0.0
2. ???
3. te-9-2-ur01.placerville.ca.sacra.comcast.net 0.0% 3 8.3 8.1 7.1 8.9 0.9
4. te-8-1-ar02.sacramento.ca.sacra.comcast.net 0.0% 3 10.0 9.0 8.1 10.0 0.9
5. pos-0-3-0-0-cr01.sacramento.ca.ibone.comcast.net 0.0% 3 11.1 12.4 11.1 13.5 1.2
68.86.90.21
6. pos-1-14-0-0-cr01.losangeles.ca.ibone.comcast.net 0.0% 3 20.7 27.6 20.1 42.0 12.5
7. pos-0-14-0-0-cr01.dallas.tx.ibone.comcast.net 0.0% 3 50.7 51.6 50.7 52.1 0.8
8. pos-0-12-0-0-cr01.atlanta.ga.ibone.comcast.net 0.0% 3 73.9 73.3 72.7 73.9 0.6
9. pos-4-8-0-0-cr01.ashburn.va.ibone.comcast.net 0.0% 3 87.0 86.7 86.0 87.1 0.6
10. pos-0-9-0-0-cr01.newyork.ny.ibone.comcast.net 0.0% 3 94.8 95.7 94.8 97.5 1.6
11. pos-0-3-0-0-pe01.111eighthave.ny.ibone.comcast.net 0.0% 3 93.1 92.9 92.7 93.1 0.2
12. Vlan570.icore1.NTO-NewYork.as6453.net 0.0% 3 132.0 107.1 92.3 132.0 21.7
13. 0.ae20.BR2.NYC4.ALTER.NET 0.0% 3 584.9 582.4 579.9 584.9 3.5
14. 0.ae2.XT1.NYC4.ALTER.NET 0.0% 3 578.9 577.5 576.0 578.9 2.1
15. 0.so-5-1-0.XT1.SAC1.ALTER.NET 0.0% 3 584.0 582.4 580.7 584.0 2.3
16. POS6-0.GW4.SAC1.ALTER.NET 0.0% 3 601.5 601.5 601.5 601.5 0.0
17. intel-gw.customer.alter.net 0.0% 3 572.2 577.0 572.2 581.7 6.7
18. ???
--
"Women. Can't live with 'em, pass the beer nuts." -Norm |
|
pflogBueller? Bueller?Premium,MVM
join:2001-09-01
El Dorado Hills, CA
kudos:3 | Tried a different VPN endpoint in Oregon and it's just as crappy. Looks like an ALTER problem?
Packets Pings
Host Loss% Snt Last Avg Best Wrst StDev
1. 173-14-73-190-Sacramento.hfc.comcastbusiness.net 0.0% 3 0.6 0.6 0.6 0.8 0.1
2. ???
3. te-9-2-ur01.placerville.ca.sacra.comcast.net 0.0% 2 7.6 7.6 7.6 7.6 0.0
4. te-8-1-ar02.sacramento.ca.sacra.comcast.net 0.0% 2 8.3 9.5 8.3 10.7 1.8
5. pos-0-3-0-0-cr01.sacramento.ca.ibone.comcast.net 0.0% 2 21.9 16.3 10.6 21.9 8.0
6. pos-1-13-0-0-cr01.losangeles.ca.ibone.comcast.net 0.0% 2 19.9 19.9 19.9 19.9 0.0
7. pos-0-12-0-0-cr01.dallas.tx.ibone.comcast.net 0.0% 2 50.6 78.2 50.6 105.9 39.1
8. pos-0-11-0-0-cr01.atlanta.ga.ibone.comcast.net 0.0% 2 74.1 75.1 74.1 76.2 1.5
9. pos-4-8-0-0-cr01.ashburn.va.ibone.comcast.net 0.0% 2 86.0 87.2 86.0 88.4 1.7
10. pos-0-8-0-0-cr01.newyork.ny.ibone.comcast.net 0.0% 2 94.8 95.8 94.8 96.7 1.4
11. pos-0-4-0-0-pe01.111eighthave.ny.ibone.comcast.net 0.0% 2 93.7 94.3 93.7 94.8 0.8
12. Vlan569.icore1.NTO-NewYork.as6453.net 0.0% 2 93.7 93.7 93.7 93.7 0.0
13. 0.ae20.BR2.NYC4.ALTER.NET 0.0% 2 492.2 492.7 492.2 493.1 0.7
14. 0.ae2.XT2.NYC4.ALTER.NET 0.0% 2 514.0 512.9 511.7 514.0 1.6
15. ???
16. POS7-0-0.GW9.POR3.ALTER.NET 0.0% 2 515.0 515.0 515.0 515.0 0.0
17. intel-gw.customer.alter.net 0.0% 2 515.3 515.3 515.3 515.3 0.0
18. ???
--
"Women. Can't live with 'em, pass the beer nuts." -Norm |
|
hhahn
join:2011-06-25
Marlton, NJ | I'm having the same problem with my company's VPN. The main office is on a FiOS business line, and the clients on Comcast connections are struggling.
I began getting alerts last night lasting from about 12 AM through 2 AM, and again from 4 AM through 6 AM. At around 2:30 today, the latency went from about 25 ms to 100 ms, and it has been that way since.
Here's the route from the main office to one of the satellite locations:
2 <1 ms <1 ms <1 ms 172.20.0.1
3 4 ms 3 ms 5 ms L100.CMDNNJ-VFTTP-35.verizon-gni.net [98.110.91.1]
4 9 ms 6 ms 6 ms G0-3-4-3.CMDNNJ-LCR-22.verizon-gni.net [130.81.182.74]
5 * * * Request timed out.
6 12 ms 11 ms 14 ms so-10-1-0-0.RES-BB-RTR2.verizon-gni.net [130.81.17.3]
7 13 ms 11 ms 12 ms 0.ae2.BR2.IAD8.ALTER.NET [152.63.34.73]
8 15 ms 13 ms 15 ms ix-20-0.tcore2.AEQ-Ashburn.as6453.net [216.6.87.33]
9 81 ms 79 ms 79 ms te-0-14-0-1-pe01.ashburn.va.ibone.comcast.net [66.208.233.49]
10 378 ms 385 ms 391 ms pos-3-11-0-0-cr01.ashburn.va.ibone.comcast.net [68.86.86.145]
11 87 ms 86 ms 85 ms so-1-0-0-0-ar03.audubon.nj.panjde.comcast.net [68.86.95.158]
12 301 ms 302 ms 301 ms xe-0-0-0-0-sur01.mtlaurel.nj.panjde.comcast.net[68.85.62.114]
13 166 ms 173 ms 164 ms 68.86.192.214
14 95 ms 95 ms 98 ms -remote office-
And here's the route from the remote location to the main office:
2 1 ms <1 ms <1 ms 10.10.1.1
3 28 ms 14 ms 11 ms c-68-36-136-1.hsd1.nj.comcast.net [68.36.136.1]
4 7 ms 9 ms 9 ms xe-2-0-0-0-sur01.mtlaurel.nj.panjde.comcast.net[68.86.192.213]
5 8 ms 8 ms 9 ms xe-3-1-0-0-ar03.audubon.nj.panjde.comcast.net [68.85.62.113]
6 13 ms 14 ms 13 ms pos-3-9-0-0-cr01.ashburn.va.ibone.comcast.net [68.86.95.157]
7 20 ms 19 ms 19 ms pos-0-15-0-0-cr01.newyork.ny.ibone.comcast.net [68.86.87.198]
8 32 ms 17 ms 72 ms pos-1-6-0-0-pe01.111eighthave.ny.ibone.comcast.net [68.86.87.110]
9 21 ms * 16 ms vlan552.icore1.nto-newyork.as6453.net [209.58.26.85]
10 81 ms 80 ms 81 ms vlan590.icore1.nto-newyork.as6453.net [209.58.26.94]
11 80 ms 81 ms 81 ms 0.ae3.ny325-bb-rtr2.verizon-gni.net [152.63.16.50]
12 91 ms 92 ms 91 ms p9-0-0.cmdnnj-lcr-01.verizon-gni.net [130.81.29.33]
13 91 ms 91 ms 92 ms p12-0-0.cmdnnj-lcr-03.verizon-gni.net [130.81.27.221]
14 93 ms 93 ms 93 ms p0-12-2-0.cmdnnj-lcr-21.verizon-gni.net [130.81.27.89]
15 * * * Request timed out.
16 95 ms 97 ms 96 ms -main office-
I have no idea what to infer from this, since each traceroute tells a different story. |
|
 owlynPremium,MVM
join:2004-06-05
Newtown, PA | No problem on my getting to my company's VPN. 15 hops, all in the teens(or less)/ms. |
|
| reply to pflog
Does your normal web surfing work ok if so then call your IT guys. |
|
bdnhsv
join:2012-01-20
Huntsville, AL | reply to hhahn
It looks to me like it's just the route that Comcast's PE router thinks is the best cost. maybe they have recently made some changes in their BGP config's or changed some peering agreements with some other providers. |
|
andyrossPremium,MVM
join:2003-05-04
Schaumburg, IL | reply to pflog
Looks like it's the hope between as6453.net and alter.net. Basically, it's after it leaves Comcast.
A good idea with a VPN, when it's working normally, is to do a traceroute and keep a copy. That way, when things to go funky, you can compare what changed.
I have one VPN that is on Integra.net. They have their own line between Chicago and Denver, and normally, data goes directly through that. Every once in awhile, my latency skyrockets and I see that it's routing via Minneapolis, Montana, Utah, then Denver. This is after it leaves Comcast. |
|
pflogBueller? Bueller?Premium,MVM
join:2001-09-01
El Dorado Hills, CA
kudos:3 | Yeah, I hadn't captured a route, but I knew ALTER.NET was not normal. It seems to be back to normal this morning and indeed it's going through sprintlink instead now (the norm):
Packets Pings
Host Loss% Snt Last Avg Best Wrst StDev
1. 173-14-73-190-Sacramento.hfc.comcastbusiness.net 0.0% 2 0.6 0.7 0.6 0.8 0.1
2. ???
3. te-9-2-ur01.placerville.ca.sacra.comcast.net 0.0% 1 8.0 8.0 8.0 8.0 0.0
4. te-8-1-ar02.sacramento.ca.sacra.comcast.net 0.0% 1 12.9 12.9 12.9 12.9 0.0
5. 68.86.90.21 0.0% 1 10.2 10.2 10.2 10.2 0.0
6. pos-0-6-0-0-cr01.sanjose.ca.ibone.comcast.net 0.0% 1 13.7 13.7 13.7 13.7 0.0
7. pos-0-4-0-0-pe01.11greatoaks.ca.ibone.comcast.net 0.0% 1 15.2 15.2 15.2 15.2 0.0
8. 173-167-56-254-static.hfc.comcastbusiness.net 0.0% 1 15.8 15.8 15.8 15.8 0.0
9. sl-crs1-sj-0-4-0-2.sprintlink.net 0.0% 1 16.3 16.3 16.3 16.3 0.0
10. sl-crs1-stk-0-0-0-2.sprintlink.net 0.0% 1 59.7 59.7 59.7 59.7 0.0
11. 144.232.19.122 0.0% 1 19.2 19.2 19.2 19.2 0.0
12. sl-intel10-363131-0.sprintlink.net 0.0% 1 21.0 21.0 21.0 21.0 0.0
13. ???
Guess it was a temporary issue or re-route due to a problem or congestion or something.
--
"Women. Can't live with 'em, pass the beer nuts." -Norm |
|
|
|
hhahn
join:2011-06-25
Marlton, NJ | reply to bdnhsv
The problem got resolved around 3 AM today.
Here's the route currently from the main office to the satellite office:
2 <1 ms <1 ms <1 ms 172.20.0.1
3 5 ms 3 ms 4 ms L100.CMDNNJ-VFTTP-35.verizon-gni.net [98.110.91.1]
4 6 ms 9 ms 6 ms G0-3-4-3.CMDNNJ-LCR-22.verizon-gni.net [130.81.182.74]
5 * * * Request timed out.
6 20 ms 11 ms 11 ms so-10-1-0-0.RES-BB-RTR2.verizon-gni.net [130.81.17.3]
7 16 ms 13 ms 11 ms 0.ae2.BR2.IAD8.ALTER.NET [152.63.34.73]
8 15 ms 14 ms 13 ms ix-20-0.tcore2.AEQ-Ashburn.as6453.net [216.6.87.33]
9 12 ms 12 ms 14 ms te-0-14-0-1-pe01.ashburn.va.ibone.comcast.net [66.208.233.49]
10 16 ms 13 ms 14 ms pos-3-11-0-0-cr01.ashburn.va.ibone.comcast.net [68.86.86.145]
11 17 ms 18 ms 19 ms so-1-0-0-0-ar03.audubon.nj.panjde.comcast.net [68.86.95.158]
12 18 ms 17 ms 18 ms xe-0-0-0-0-sur01.mtlaurel.nj.panjde.comcast.net[68.85.62.114]
13 26 ms 35 ms 22 ms 68.86.192.214
14 30 ms 26 ms 31 ms -remote office-
And here's the route from the satellite office to the main office:
2 c-68-36-136-1.hsd1.nj.comcast.net (68.36.136.1) 23.590 ms 24.014 ms 9.711 ms
3 xe-2-0-0-0-sur01.mtlaurel.nj.panjde.comcast.net (68.86.192.213) 9.229 ms 7.060 ms 7.205 ms
4 xe-3-1-0-0-ar03.audubon.nj.panjde.comcast.net (68.85.62.113) 39.966 ms 8.034 ms 10.086 ms
5 pos-3-6-0-0-cr01.ashburn.va.ibone.comcast.net (68.86.92.161) 13.854 ms 13.694 ms
pos-3-9-0-0-cr01.ashburn.va.ibone.comcast.net (68.86.95.157) 14.824 ms
6 pos-0-3-0-0-pe01.ashburn.va.ibone.comcast.net (68.86.86.142) 14.146 ms 14.674 ms 13.563 ms
7 66.208.233.38 (66.208.233.38) 13.964 ms 13.818 ms 14.200 ms
8 216.6.87.34 (216.6.87.34) 14.598 ms 13.327 ms 14.334 ms
9 0.ae2.RES-BB-RTR2.verizon-gni.net (152.63.34.74) 36.822 ms 18.940 ms 14.583 ms
10 P9-0-0.CMDNNJ-LCR-02.verizon-gni.net (130.81.29.35) 21.601 ms 22.558 ms 21.331 ms
11 P12-0-0.CMDNNJ-LCR-04.verizon-gni.net (130.81.27.197) 23.089 ms 24.061 ms 21.830 ms
12 P0-12-2-0.CMDNNJ-LCR-22.verizon-gni.net (130.81.27.91) 26.328 ms 23.674 ms 23.456 ms
13 * * *
14 -main office- 27.430 ms 25.759 ms 27.455 ms
Graph of latency over time:
|
|
andyrossPremium,MVM
join:2003-05-04
Schaumburg, IL | reply to pflog
said by pflog:Yeah, I hadn't captured a route, but I knew ALTER.NET was not normal. It seems to be back to normal this morning and indeed it's going through sprintlink instead now (the norm):
Guess it was a temporary issue or re-route due to a problem or congestion or something. On the original, Comcast was sending it to NY, where presumably alter.net was sending it back to CA. So, it could have been a Comcast issue with a router out or misconfigured, so it was routed to the next down the line until it found one that knew how to handle the destination IP.
Another issue is that your return path could and probably is totally different. On my VPN again, integra.net would connect to Comcast in Dallas (which then went to Atlanta and Indianapolis, then Chicago) instead of sending it directly to Chicago. It was nearly a year before they finally started routing it over their own return to Chicago and connecting to Comcast there. |
|
 espaethDigital PlumberPremium,MVM
join:2001-04-21
Minneapolis, MN
kudos:2 | I believe it was a TATA routing issue -- we saw the same pattern reaching both CenturyLink and Verizon from Comcast, and both of those are reachable through TATA (as6453, formerly Teleglobe). |
|
