theo2
@mit.edu
| [Connectivity] Traceroutes through comcast's network
I am confused as to what Comcast is doing with my traceroutes through their network.
For a UDP-based traceroutes between two boxes all the hops send a TTL expired message. However, with a TCP-based traceroute between the same two boxes, some of the hops consistently do not send a TTL expired messages. This is not a transient behavior either, but has been persistent for long periods of time.
---------------------------------------------------------
UDP-based traceroute
---------------------------------------------------------
root# traceroute 65.96.238.168
traceroute to 65.96.238.168 (65.96.238.168), 30 hops max, 40 byte packets
1 legacy26-0.default.csail.mit.edu (18.26.0.1) 0.543 ms 0.364 ms 0.370 ms
2 kalgan.trantor.csail.mit.edu (128.30.0.245) 0.352 ms 0.388 ms 0.403 ms
3 B24-RTR-2-CSAIL.MIT.EDU (18.4.7.1) 0.436 ms 0.426 ms 0.423 ms
4 EXTERNAL-RTR-1-BACKBONE.MIT.EDU (18.168.0.18) 336.441 ms 336.468 ms 336.505 ms
5 EXTERNAL-RTR-2-BACKBONE.MIT.EDU (18.168.0.27) 6.812 ms 6.777 ms 7.555 ms
6 ge-1-2-ar02.woburn.ma.boston.comcast.net (24.218.0.193) 7.818 ms 7.849 ms 7.849 ms
7 po-12-ar02.needham.ma.boston.comcast.net (68.87.146.45) 8.256 ms 8.311 ms 8.330 ms
8 te-9-1-ur01.needham.ma.boston.comcast.net (68.87.145.162) 8.328 ms 8.355 ms 8.363 ms
9 ge-0-1-ubr06.needham.ma.boston.comcast.net (68.87.157.170) 8.292 ms 8.169 ms 8.124 ms
10 c-65-96-238-168.hsd1.ma.comcast.net (65.96.238.168) 12.961 ms 16.931 ms 9.403 ms
---------------------------------------------------------
TCP-based traceroute
---------------------------------------------------------
root# tcptraceroute 65.96.238.168
1 legacy26-0.default.csail.mit.edu (18.26.0.1) 0.371 ms 0.324 ms 0.322 ms
2 kalgan.trantor.csail.mit.edu (128.30.0.245) 0.370 ms 0.361 ms 0.366 ms
3 B24-RTR-2-CSAIL.MIT.EDU (18.4.7.1) 0.359 ms 0.385 ms 0.398 ms
4 EXTERNAL-RTR-1-BACKBONE.MIT.EDU (18.168.0.18) 0.656 ms 0.795 ms 0.791 ms
5 EXTERNAL-RTR-2-BACKBONE.MIT.EDU (18.168.0.27) 0.487 ms 0.534 ms 0.551 ms
6 * * *
7 * * *
8 * * *
9 ge-0-1-ubr06.needham.ma.boston.comcast.net (68.87.157.170) 1.766 ms 1.749 ms 1.592 ms
10 c-65-96-238-168.hsd1.ma.comcast.net (65.96.238.168) 7.492 ms 30.183 ms 30.149 ms
Did they configure their routers to send a TTL expired message for UDP packets but not TCP packets? Is it even possible? If so, what is the operational motivation? Or are they routing UDP and TCP packets through different routers? |
|
gar187er
Premium Alcoholic
join:2006-06-24
Dover, DE | only a person from MIT would ponder this and make a post about it.... |
|
K Patterson
Premium,MVM
join:2006-03-12
Columbus, OH
 ·RoadRunner Cable
| reply to theo2
Many Comcast routers (and those of other ISP's and networks) are configured to drop some or all traceroute packets. Traceroute packets that expire at a router take CPU time, as compared to packets that are routed which do not impact the CPU. I've always understood that they were just prioritizing the CPU workload. |
|
theo2
@mit.edu | What is curious is why they are configured to respond
*differently* for UDP and TCP packets. TTL expiring is a
network layer event and prima faci has nothing to do with
whether the rest of the packet is TCP or UDP... |
|
Tacrep181
@comcast.net
| reply to theo2
TCP traceroute is a far less commonly used version of traceroute - so they may have actually turned off the routers responding to the TCP packets with a ttl=1. This would be a good security practice to protect your networks against DDOS attacks on the infrastructure itself - the same reason you would rate-limit standard ICMP/UDP ttl=1 responses. |
|
theo2
@mit.edu
| reply to theo2
Hmm... I am skeptical of your claim that this is done as a DoS mitigation mechanism. Sending a TTL expired message is equally time consuming for a router whether the original packet is a TCP, UDP, or ICMP packet... it is all just data past the IP header.
An attacker can DoS the infrastructure with any of this types of packets... so why would limiting *just* TCP be a "good security practice"? Strikes me as irrelevant to DoS security. Or am I missing something? |
|
espaeth
Misanthrope
Premium
join:2001-04-21
Minneapolis, MN
 ·voip.ms
·Callcentric
·VoiceStick
·ViaTalk
·Comcast
 ·Embarq
| Considering all of the traffic shaping features Comcast is leveraging (ie, Powerboost, aka delayed shaping) they are more than likely handling that in packet shaping appliances external to the infrastructure routers. It would be my speculation that they are doing this via policy-based routing to define a next hop of the packet shaping appliances for TCP-based flows (since TCP is really the only protocol you can shape). This would give them the benefit of cleaner failover than having a device in-line, as once the connected packet shaper's port goes dead the connected subnet will go away in the routing table. Since PBR won't forward to an invalid next hop the traffic would follow the standard routing path and thus survive an outage or alternately allow for easy bypass.
Otherwise, on the Cisco 12000s and 6500s I am unaware of any option to differentiate on TTL=0 traffic based on protocol type for any of the various supervisor / code configurations.
-Eric |
|
