scooby
Premium
join:2001-05-01
Schaumburg, IL
2 edits | Wacky routing?
Can someone pass this to an AT&T network engineer please? Latency is not all bad but sometimes there is very bad packetloss on NTT.
Why is my traffic to downtown Chicago being routed across the US? Looks like AT&T/SBC sends it to Virginia, then dumps it on NTT which goes Virginia to New York City to Chicago. Outbound from Internap looks fine. All traffic stays in Chicago.
Reason I post is that my CEO's wife recently had a child so he is working from home four days a week for the next month or two. He has SBC/AT&T adsl as do I and he is experiencing this terrible routing which makes vpn and other work impossible at times.
Please help!
Scooby2
my dsl to internap (AT&T/SBC controls this route)
traceroute to 74.63.64.1 (74.63.64.1), 64 hops max, 40 byte packets
1 192.168.1.1 (192.168.1.1) 0.771 ms 2.559 ms 0.630 ms
2 adsl-99-141-239-XXX.dsl.emhril.sbcglobal.net (99.141.239.XXX) 7.650 ms 8.211 ms 7.344 ms
3 rback2-g1-1.emhril.sbcglobal.net (68.250.251.2) 8.341 ms 8.378 ms 7.297 ms
4 bb2-g9-0.emhril.sbcglobal.net (151.164.94.164) 7.641 ms 7.771 ms 8.173 ms
5 ex1-p1-1.eqabva.sbcglobal.net (151.164.95.101) 26.445 ms 26.571 ms 25.807 ms
6 asn4766-koreantelecom.eqlaca.sbcglobal.net (151.164.250.206) 26.855 ms 26.993 ms 26.048 ms
7 ae-2.r21.asbnva01.us.bb.gin.ntt.net (129.250.2.86) 27.386 ms 27.364 ms 27.277 ms
8 as-0.r21.nycmny01.us.bb.gin.ntt.net (129.250.2.8) 31.966 ms 32.716 ms 33.146 ms
9 p64-2-0-0.r20.chcgil09.us.bb.gin.ntt.net (129.250.5.4) 59.350 ms 59.720 ms 58.479 ms
10 xe-3-1.r01.chcgil09.us.bb.gin.ntt.net (129.250.3.206) 57.249 ms 57.780 ms 58.070 ms
11 xe-3-3.r01.chcgil09.us.ce.gin.ntt.net (129.250.208.6) 34.057 ms 33.961 ms 35.061 ms
12 border5.pc1-bbnet1.chg.pnap.net (64.94.32.10) 34.553 ms 34.526 ms 35.271 ms
13 66.90.127.205 (66.90.127.205) 35.326 ms 34.911 ms 34.116 ms
14 66.90.127.138 (66.90.127.138) 164.267 ms * *
From Internap to my dsl:
TL LFT trace to adsl-99-141-239-XXX.dsl.emhril.sbcglobal.net (99.141.239.XXX)/icmp
1 . (67.159.44.1) 0.4ms
2 66.90.127.137 (66.90.127.137) 0.4ms
3 border8.tenge3-3.fdcs-1.chg.pnap.net (64.74.106.77) 123.5ms
4 core1.te2-1-bbnet1.chg.pnap.net (64.94.32.1) 0.5ms
5 cr1.chg003.inappnet-42.core1.chg.internap.net (66.79.150.129) 0.4ms
6 cr2-cr1.chg003.internap.net (66.79.146.194) 0.5ms
7 64.127.128.253 0.5ms
** [neglected] no reply packets received from TTL 8
9 ex2-p1-0.eqchil.sbcglobal.net (151.164.191.250) 22.1ms
10 bb2-p2-1.emhril.ameritech.net (151.164.42.138) 120.4ms
11 dist1-g2-3.emhril.sbcglobal.net (151.164.94.165) 23.1ms
12 se14-g5-2.emhril.sbcglobal.net (68.250.251.44) 24.2ms
13 adsl-99-141-239-XXX.dsl.emhril.sbcglobal.net (99.141.239.XXX) 26.1ms
--
Ron Paul 2008 - Hope For America - »www.ronpaul2008.com/
|
|
RadioDoc
58ef2c0
Premium,ExMod 2000-03
join:2000-05-11 | That's usually the responsibility of the server's network to advertise routing to it. Who is your ISP?
--
Toolmaster of La Grange. |
|
scooby
Premium
join:2001-05-01
Schaumburg, IL
2 edits | said by RadioDoc  :That's usually the responsibility of the server's network to advertise routing to it. Who is your ISP? How would it be the responsibility of the server's network on the route it takes to get there? Internap has zero say on how AT&T routes things. It's the responsibility of AT&T/SBC on the route it gets there. It's usually done with weighting which connection is fastest but it can be done to use which connection is cheapest.
Internap provides the route back. Internap has connections to 7 networks in Chicago. Level3, Verizon Business/UUnet, Verio/NTT, Sprint, AT&T (not SBC), Global Crossing, and Savvis. Thus why they are mucho expensive and known as one of the best ISPs (if not the best) in the country.
Our old datacenter is on pure Level3/Broadwing bandwidth in Chicago and traceroutes go straight to Internap via Level3. So Internap is advertising the ip space out Level3. There is no reason AT&T/SBC should not be sending the traffic via Level3 unless they are trying to save some money or something is mis-configured on their end. Traceroutes included...
From home dsl to old colo:
Tracing route to www.XXX.XXX [66.147.15X.XXX]
1 1 ms 1 ms 1 ms 192.168.1.1
2 20 ms 9 ms 39 ms adsl-99-141-239-XXX.dsl.emhril.sbcglobal.net [99.141.239.XXX]
3 27 ms * 8 ms dist1-vlan60.emhril.ameritech.net [68.22.72.130]
4 48 ms 32 ms 44 ms bb1-g0-0.emhril.ameritech.net [151.164.43.82]
5 37 ms 15 ms 9 ms 151.164.94.41
6 184 ms 228 ms 227 ms Te-3-2.Chicago1.Level3.net [4.68.110.197]
7 25 ms 39 ms 21 ms ae-14-51.car4.Chicago1.Level3.net [4.68.101.8]
8 10 ms 34 ms 26 ms te-4-1-73.rp0-5.chcgilca.Level3.net [4.68.63.14]
9 46 ms 9 ms 10 ms 216.140.15.157
10 47 ms 76 ms 41 ms g1-2-0.core1.chi.foc.broadwing.net [216.140.14.134]
11 * * * Request timed out.
12 37 ms 53 ms 23 ms www.XXX.XXX [66.147.15X.XXX]
From old colo to Internap:
traceroute to 74.63.64.1 (74.63.64.1), 30 hops max, 40 byte packets
1 66.147.15X.XXX 0.374 ms 0.289 ms 0.323 ms
2 216.140.15.145 1.650 ms 1.648 ms 1.727 ms
3 G1-1.rp0.chcg.broadwing.net (216.140.14.162) 1.686 ms 1.754 ms G2-3.rp0.chcg.broadwing.net (216.140.15.18) 1.832 ms
4 te-8-3-73.car4.Chicago1.Level3.net (4.68.63.13) 1.792 ms 1.703 ms 1.855 ms
5 ge-9-2.ipcolo1.Chicago1.Level3.net (4.68.101.138) 1.938 ms 1.777 ms 1.981 ms
6 unknown.Level3.net (209.247.34.166) 2.190 ms 2.153 ms 2.059 ms
7 border5.pc2-bbnet2.chg.pnap.net (64.94.32.74) 20.128 ms 18.278 ms 6.586 ms
8 66.90.127.205 (66.90.127.205) 2.709 ms 2.637 ms 2.223 ms
9 66.90.127.138 (66.90.127.138) 2.556 ms 2.623 ms 2.431 ms
From Internap to old Colo:
TTL LFT trace to www.XXX.XXX (66.147.15X.XXX)/icmp
1 . (67.159.44.1) 0.4/0.4ms
2 66.90.127.137 (66.90.127.137) 0.2/0.5ms
3 border8.tenge3-3.fdcs-1.chg.pnap.net (64.74.106.77) 0.5/0.5ms
4 core5.te6-1-bbnet1.chg.pnap.net (64.94.32.14) 0.4/0.5ms
5 ge-8-0-519.ipcolo1.Chicago1.Level3.net (209.247.34.165) 0.5/0.5ms
6 ae-24-54.car4.Chicago1.Level3.net (4.68.101.104) 0.9/0.9ms
7 te-4-1-73.rp0-5.chcgilca.Level3.net (4.68.63.14) 1.0/0.7ms
8 216.140.15.157 1.0/1.0ms
9 g1-2-0.core1.chi.foc.broadwing.net (216.140.14.134) 1.2/1.2ms
** [neglected] no reply packets received from TTL 10
11 [target] www.XXX.XXX (66.147.15X.XXX) 2.6/2.6ms
--
Ron Paul 2008 - Hope For America - »www.ronpaul2008.com/
|
|
Dennis
Premium,Mod
join:2001-01-26
Algonquin, IL
 ·AT&T Yahoo
Host:
Chicago
Users Find Hot Deals
Users find Hot Dea..
Requests for Hot D..
Home Repair & Impr..
| reply to scooby
I think the proble is that, this router:
7 28 ms 27 ms 27 ms asn4766-koreantelecom.eqlaca.sbcglobal.net [151.164.250.206]
Is broadcasting ownership of the /22 via BGP....now the question is who owns it? The IP is AT&T...but I can't access it. Somebody who is smarter than I might be able to figure out if my hunch is right, but you'll have to wait till Monday.
--
My Blog. Because I desperately need the acknowledgement of others.
Mainegirl and my Beer Review's |
|
scooby
Premium
join:2001-05-01
Schaumburg, IL | Thanks Dennis! |
|
rolande
Certifiable
Premium,Mod
join:2002-05-24
Powell, OH
clubs:
Host:
Linksys
AT&T Midwest
| reply to Dennis
Either way, look at the incremental latency increase through each hop. There is no way that traffic is actually physically going to Virginia, then New York and back to Chicago. Likely those are just router hops within Equinix and they happen to be named after the primary transit path they provide egress for. Round trip is ~34msecs to 66.90.127.205. The big latency jump is to the next hop 66.90.127.138.
You can't compare forward and reverse path routing. Especially with InterNAP they are almost always going to be asymmetric.
--
Ignorance is temporary...stupidity lasts forever!
»www.thewaystation.com/
»blog.thewaystation.com/ |
|
scooby
Premium
join:2001-05-01
Schaumburg, IL
1 edit | I highly doubt both SBC/AT&T and NTT/Verio have ips misnamed in Chicago. The latency is right on for going to virginia, then new york, and back. If the traffic was staying local, the latency would be in the 10-20 ms range with 20ms being extremely high.
Pinging my personal colo box which is in the same building as the Internap colo (350 E Cermak, Chicago IL) gives me 8-9 ms ping times from my SBC/AT&T DSL. Traffic leaves the sbcglobal network on Level3 and then hits my personal colo provider. Return traffic runs the same way. 8-9ms is what I am looking for. Pinging the network of our parent company which is in New York and New Jersey gives 25ms ping times roundtrip from SBC/AT&T dsl. So yes it is very possible the traffic is asynchronously going to VA, NYC, Chicago to get there but then stays in Chicago on the way back like my traceroutes above show.
--
Ron Paul 2008 - Hope For America - »www.ronpaul2008.com/
|
|
Dennis
Premium,Mod
join:2001-01-26
Algonquin, IL
·AT&T Yahoo
Host:
Chicago
Users Find Hot Deals
Users find Hot Dea..
Requests for Hot D..
Home Repair & Impr..
| I think his point is just that the DNS isn't always 100% reliable for location, and I agree. The odds that your going from Virginia to California to Virginia to New York and back to IL with that low latency are pretty low.
Either way, people broadcasting ownership of blocks erroniously happens all the time. Not much you can do about it.
--
My Blog. Because I desperately need the acknowledgement of others.
Mainegirl and my Beer Review's |
|
Frohike
Premium
join:2000-07-23
Waxahachie, TX
| reply to scooby
said by scooby :Can someone pass this to an AT&T network engineer please? Latency is not all bad but sometimes there is very bad packetloss on NTT. Why is my traffic to downtown Chicago being routed across the US? Looks like AT&T/SBC sends it to Virginia, then dumps it on NTT which goes Virginia to New York City to Chicago. Outbound from Internap looks fine. All traffic stays in Chicago. Reason I post is that my CEO's wife recently had a child so he is working from home four days a week for the next month or two. He has SBC/AT&T adsl as do I and he is experiencing this terrible routing which makes vpn and other work impossible at times. Please help! Scooby2 my dsl to internap (AT&T/SBC controls this route)traceroute to 74.63.64.1 (74.63.64.1), 64 hops max, 40 byte packets 1 192.168.1.1 (192.168.1.1) 0.771 ms 2.559 ms 0.630 ms 2 adsl-99-141-239-XXX.dsl.emhril.sbcglobal.net (99.141.239.XXX) 7.650 ms 8.211 ms 7.344 ms 3 rback2-g1-1.emhril.sbcglobal.net (68.250.251.2) 8.341 ms 8.378 ms 7.297 ms 4 bb2-g9-0.emhril.sbcglobal.net (151.164.94.164) 7.641 ms 7.771 ms 8.173 ms 5 ex1-p1-1. eqabva.sbcglobal.net (151.164.95.101) 26.445 ms 26.571 ms 25.807 ms 6 asn4766-koreantelecom.eqlaca.sbcglobal.net (151.164.250.206) 26.855 ms 26.993 ms 26.048 ms 7 ae-2.r21. asbnva01.us.bb.gin.ntt.net (129.250.2.86) 27.386 ms 27.364 ms 27.277 ms 8 as-0.r21. nycmny01.us.bb.gin.ntt.net (129.250.2.8) 31.966 ms 32.716 ms 33.146 ms 9 p64-2-0-0.r20.chcgil09.us.bb.gin.ntt.net (129.250.5.4) 59.350 ms 59.720 ms 58.479 ms 10 xe-3-1.r01.chcgil09.us.bb.gin.ntt.net (129.250.3.206) 57.249 ms 57.780 ms 58.070 ms 11 xe-3-3.r01.chcgil09.us.ce.gin.ntt.net (129.250.208.6) 34.057 ms 33.961 ms 35.061 ms 12 border5.pc1-bbnet1.chg.pnap.net (64.94.32.10) 34.553 ms 34.526 ms 35.271 ms 13 66.90.127.205 (66.90.127.205) 35.326 ms 34.911 ms 34.116 ms 14 66.90.127.138 (66.90.127.138) 164.267 ms * * Your path never left Illinois. Hops 4 - 6 share the same 151.164.x.x and then its passed to the IP 129.250.x.x set which reside in Illinois. |
|
Dennis
Premium,Mod
join:2001-01-26
Algonquin, IL
·AT&T Yahoo
Host:
Chicago
Users Find Hot Deals
Users find Hot Dea..
Requests for Hot D..
Home Repair & Impr..
| reply to scooby
You know....I didn't mention this at the time, but even though there are 14 hops, it really shouldn't impact VPN at all.
The latency is pretty low....I really don't see why it's a problem honestly. Is the VPN client using UDP?
After talking to somebody else, here are the points I was explained.
1. VA is a large peering point, and probably the closest to ntt. They must not directly peer with AT&T
2. The DNS is fubar'd a bit....
Either way you look at it though, it's outside of AT&T's control. They are just responding to whats being advertised from koreantelecom. It's the magical world of the internet.
--
My Blog. Because I desperately need the acknowledgement of others.
Mainegirl and my Beer Review's |
|
