Anyone in FL have higher pings today and longer routes?
Generally all FL traffic goes to Miami and from that point Atlanta if going out of state. The two backbones it generally rides are ibone and Level 3.
Today all my traffic goes to Miami and from that to Dallas, TX. Even Level 3 with a destination of my own city...
Here is ibone:
3 10 ms 10 ms 10 ms te-5-2-ur01.nworange.fl.lakecnty.comcast.net [68
4 10 ms 11 ms 11 ms te-1-1-ur01.southorange.fl.lakecnty.comcast.net
5 11 ms 10 ms 12 ms te-1-4-ur01.osceolawest.fl.lakecnty.comcast.net
6 14 ms 23 ms 23 ms te-0-9-0-3-ar03.bonitasprngs.fl.naples.comcast.n
7 21 ms 22 ms 19 ms pos-1-10-0-0-cr01.miami.fl.ibone.comcast.net [68
8 53 ms 50 ms 50 ms pos-3-14-0-0-cr01.dallas.tx.ibone.comcast.net [6
9 65 ms 63 ms 67 ms pos-2-14-0-0-cr01.denver.co.ibone.comcast.net [6
10 69 ms 71 ms 71 ms he-5-12-0-0-cr01.350ecermak.il.ibone.comcast.net
11 69 ms 67 ms 67 ms be-15-pe03.350ecermak.il.ibone.comcast.net [68.8
12 85 ms 71 ms 71 ms as4436.350ecermak.il.ibone.comcast.net [173.167.
13 67 ms 66 ms 69 ms ae1-60g.cr2.ord1.us.nlayer.net [220.127.116.11]
Here is Level 3, and this route use to go to Miami and ride level 3 straight to Orlando.
Initially I thought this was a level 3 issue but the above ibone doing the same...
3 11 ms 10 ms 9 ms te-5-2-ur01.nworange.fl.lakecnty.comcast.net [68
4 11 ms 9 ms 10 ms te-1-1-ur01.southorange.fl.lakecnty.comcast.net
5 11 ms 11 ms 10 ms te-1-4-ur01.osceolawest.fl.lakecnty.comcast.net
6 26 ms 22 ms 23 ms te-0-9-0-3-ar03.bonitasprngs.fl.naples.comcast.n
7 20 ms 23 ms 23 ms pos-1-11-0-0-cr01.miami.fl.ibone.comcast.net [68
8 48 ms 47 ms 49 ms xe-10-0-0.edge2.Miami1.Level3.net [18.104.22.168
9 62 ms 56 ms 67 ms ae-31-51.ebr1.Miami1.Level3.net [22.214.171.124]
10 59 ms 58 ms 58 ms ae-2-2.ebr1.Dallas1.Level3.net [126.96.36.199]
11 57 ms 56 ms 56 ms ae-71-71.csw2.Dallas1.Level3.net [188.8.131.52]
12 58 ms 58 ms 58 ms ae-73-73.ebr3.Dallas1.Level3.net [184.108.40.206]
13 57 ms 57 ms 58 ms ae-7-7.ebr3.Atlanta2.Level3.net [220.127.116.11]
14 56 ms 58 ms 57 ms ae-63-63.ebr1.Atlanta2.Level3.net [18.104.22.168]
15 57 ms 77 ms 57 ms ae-1-8.bar1.Orlando1.Level3.net [22.214.171.124]
16 58 ms 57 ms 56 ms ae-6-6.car1.Orlando1.Level3.net [126.96.36.199]
17 53 ms 52 ms 53 ms ATLANTIC.NE.car1.Orlando1.Level3.net [63.209.98.
18 51 ms 53 ms 51 ms andc-abr-1-ten2-1.atlantic.net [188.8.131.52]
NormanSI gave her time to steal my mind awayPremium,MVMReviews:
San Jose, CA
·Pacific Bell - SBC
AFAIK, Comcast keeps traffic on their transit network to as close to the destination as possible before handing off.
Your first trace shows a direct handoff from Comcast to nLayer Communications (now a part of GTT) in Chicago. This fits with keeping traffic on Comcast transit to near the destination.
Your second trace shows a handoff to Level 3 in Miami. This looks like the closest to Orlando that Comcast goes to meet with Level 3. This is also consistent with carrying data on Comcast as close to the destination as possible. The peripatetic routing from Miami to Dallas to Atlanta to Orlando is caused by Level 3, and beyond the control of Comcast.
Based on a trace I just ran, it appears that Orlando-bound traffic is carried first to Dallas, even from the West Coast. Just a Level 3 thing.
~Oh Lord, why have you come
~To Konnyu, with the Lion and the Drum
|reply to AnonMan |
Your first traceroute is actually to a nLayer Chicago server. Normally this would transition quickly to the nLayer backbone with hot-potato routing, but perhaps nLayer wants to use Comcast's backbone $$ vs their own and announces routes inconsistently making Comcast cold-potato the traffic.
Each of these look like non-Comcast issues
That first one they are on the ibone backbone, that last hop is directly on it.
That said the entire last 2+ years that exact same route use to go
Central FL to Miam, to Atlanta to Chicago ALL on ibone...
This is something going on in Miami.
I also know all providers keep traffic on net as long as possible to keep peering down but this traffic is all abnormal. The second one use to go to Miami and back to Orlando. The first one use to go directly to Chicago only touching Atlanta first.
Here is a good example of ibone going from Miami to Atlanta before this odd routing issue
The ping naturally is double due to traveling to TX first now, also bandwidth is slower.
I am just confused is all how would BOTH Level 3 & ibone have issues on same day unless was just one huge fiber mess with multiple providers unless it's a Comcast router issue or link issue and this is the re-route. I really wish they had a real network status page to show network outages or temporary re-routes etc.
|reply to AnonMan |
1 1 ms 1 ms 1 ms 172.xx.xx.xx
2 38 ms 15 ms 22 ms 98.244.xxx.x
3 11 ms 11 ms 9 ms te-5-2-ur01.nworange.fl.lakecnty.comcast.net [68
4 11 ms 10 ms 11 ms te-1-1-ur01.southorange.fl.lakecnty.comcast.net
5 11 ms 11 ms 11 ms te-1-4-ur01.osceolawest.fl.lakecnty.comcast.net
6 22 ms 20 ms 23 ms te-0-9-0-3-ar03.bonitasprngs.fl.naples.comcast.n
7 23 ms 22 ms 23 ms pos-1-11-0-0-cr01.miami.fl.ibone.comcast.net [68
8 60 ms 59 ms 58 ms xe-10-1-0.edge2.Miami1.Level3.net [184.108.40.206]
Similar to »Comcast - Level 3 Miami issue?
except today seems to be all comcast peer, even traces in and out using looking glass for different backbone providers.
Anyway notice how the latency on the above is lower vs. the link but it's still high.
Look at CUBS_FAN trace. He is going from another state to Miami and gets a ping around 60. I get +40 just going from miami comcast ibone to miami level 3 lol.
Oh well, just figured would see if anyone here heard or saw issues too.
|reply to AnonMan |
Of course, I reach out to a datacenter I do some work with to have them go bug some upstream providers as the issue was repeatable from them also and 30 min later almost all except for comcast ibone is back to normal.
The issue impacted any connection to Comcast that met in Miami from Level 3 and also Cogentco which is another peering provider Comcast hooks to.
All my traces and latency is back down now except for some odd high ones in the middle of Level 3's traces though endpoint is fine and also ibone is still going through TX first. Hopefully the get it all back to normal. Guess I will stop posting these threads as I seem to be only one in the area to catch these issues and that does the type of work that causes me to notice the issues lol
I verified all the issues via 2 different providers in the data center and also via looking glass on both providers from various points to Miami. So I really wasn't making this all up! lol
End of Spam post...