Endorphine
Endorphine
Premium
join:2002-09-30
Bremerton, WA
| reply to koitsu
Re: Packet Loss/Latency Reports
Is the Seattle POP outage/problems related to the blaster worm, or the other reason you didn't tell us yet?  |
|
Yourself
join:2000-08-10
Lynnwood, WA | reply to KatOak
Kat-
Any update on the Seattle POP?
Self |
|
koitsu
Premium
join:2002-07-16
Mountain View, CA
| reply to KatOak
SE SFO, 09/08 |
The "huge red blotch" you see from about 16:12 until 17:00 or so is Verio. I verified this since PP logs route changes. Why are BGP route changes at InterNAP affecting my DSL gateway? This makes absolutely no sense to me what-so-ever.
The "smaller packet loss" lines shown are probably caused by MSBlast and SoBig.F. That I can justify.
But I still don't understand why latency shoots through the roof from 09:00 until midnight. That has absolutely nothing to do with MSBlast or SoBig.F, that's for sure. Plus, you can see a point in the graph above (a little after 11:00) where the latency dips. Unless all of these people (and there would have to be a LOT meeting this condition) using the 'net through SE don't leave their PCs from 00:01 until 08:59, and simultaneously suffer from MSBlast.
Tomorrow is 09/09. MSBlast stops working after that point in time. If what's being shown here (from everyone) continues past that, my vote is that it has absolutely nothing to do with MSBlast.
EDIT: Typos.
--
Making life hard for others since 1977.
[text was edited by author 2003-09-08 21:00:26] |
|
bhan261
join:2001-02-12
New York, NY | Yes...today is the day that we'll find out whether SE's engineers were correct or simply living in hope. Anyone want to make any bets? |
|
Chicago_DSL6
join:2003-08-04
Palatine, IL |  I'll bet koitsu will find out before SE! |
|
koitsu
Premium
join:2002-07-16
Mountain View, CA
| Hahaha. Thanks for the enthusiasm. 
Probably not though, since I work today (Tue) and tomorrow. I usually close Ping Plotter while using Remote Desktop, but of course the week before last was an exception since I absolutely needed to prove that something strange was afoot at the Circle K.
I'm hoping that the occasional packet loss shown in our graphs goes away. As far as Seattle goes, hoo boy, I'm really crossing my fingers on that one. Just when I thought it was back via SFO, SEA comes along and kicks me to the curb!
--
Making life hard for others since 1977. |
|
GamingGirl
@speakeasy.n
| I am out of the LAX pop and I am seeing continuous (throughout the day) packet loss... anywhere from 1% to 28%. I have the Gamer package and have never seen packet loss on my line before last week... obviously this is affecting my gaming =( and although I've contacted support, I'm still waiting for contact from advanced support (it's been three days now). If this truly was related to the worms going around, why is it that nobody else I game with is experiencing any problems?
And where do you get that ping plotter? |
|
bhan261
join:2001-02-12
New York, NY
| www.pingplotter.com
Also, yesterday was the first day in a month that I didn't have several spikes of P/L. So maybe it was MSBlaster...I'll keep my fingers crossed. |
|
Yourself
join:2000-08-10
Lynnwood, WA
| reply to KatOak
I have a feeling that my issue is not worm related, as I am on the Seattle POP. This morning(5:45 am)I am still showing up to 20-30% PL on the 2nd hop of my connection. In my opinion it sounds like a NOC(Internap) issue for us Seattle POP folks...I just wish we weren't still in the dark.
Self |
|
paulhaskew
Unoffical Dominos Spokesman
join:2002-01-10
Vancouver, WA
clubs: | reply to KatOak
heh, my modem just lost sync for the third time in two days... hmmm |
|
sporkme
drop the crantini and move it, sister
Premium,MVM
join:2000-07-01
Morristown, NJ
 ·Optimum Online
| reply to koitsu
said by koitsu  :
I realise SE doesn't filter, but I strongly urge reconsideration -- filtering TCP src port 135, or TCP dst port 135 at the border routers. Border routers are specifically deployed for this purpose, and it takes less processing time to drop a packet on a Cisco or Juniper than it does to route it.
A few problems with that... Blaster (or the next worm, or NACHI) doesn't really care if it's being filtered upstream. It will still blast away out your pc, out your modem, out the dslam, out Covad's ATM network, out SE's backhaul. SE does not have any real breathing room for this, so they get slammed. Unlike the cable guys, they can't filter back at your CPE to save their network.
And depending on IOS, filtering is much more CPU intensive. On 12.2T(mumble), adding 135 filters gobbled up 20% more CPU, and that is with CEF enabled.
But I don't think SE has Cisco or Juniper boxes, just a Redback and an FE or GigE handoff to InterNAP. And from what I've seen, the Redback doesn't get too fancy with filtering...
Another thing with Redback is that they prioritize ICMP replies *really low*, so pointing pingplotter at the router gives a less accurate picture than say pointing it to a DNS or game server at your pop. That also gives you more leverage if their network guy knows that pinging routers gives flawed results.
said by koitsu :
This is a route change, which is EXTREMELY prominent with InterNAP (I've seen them change routing tables every 30 minutes -- I have no idea what criteria they use to decide when to drop BGP with someone and switch, but it seems almost random).
While their PR stance is that they have all these transit connections for "robust connectivity", I think a big part of their decisions are based on closely meeting minimums with each provider to meet certain price points. Since they went in the crapper, you see almost 0% of traffic going out "expensive" paths. I think they put the rest of their money into automating route selection based on economic constraints rather than "optimum routing". No one likes to dis InterNap, but they really aren't that "premium" anymore. Sorry.
said by koitsu :
What I want to know is why the packet loss is induced upon my DSL gateway depending upon which route packets are taking at InterNAP.
Coincidence? Or the bgp scanner process is sucking up cpu after a change? Another reason not to ping a router, but a host at the POP instead.
--
just a minute |
|
koitsu
Premium
join:2002-07-16
Mountain View, CA
| Where did you conclude I was "pinging a router and not a host at a POP?" Not a single graph of mine that I've posted here in the past 2 months has done that -- I test from endpoint to endpoint.
--
Making life hard for others since 1977. |
|
sh0V3L
join:2002-02-16
San Carlos, CA
|  reply to KatOak
Thats all fine and dandy I think speakeasy could care less about our ping plotter charts they know thier system is below par internap sucks routing is just rediculius i can see how spiky the network is i have a 768/768 sdsl circuit i run a game server ok im in sancarlos ca guys in sanfrancisco with a speakeasy dsl connection going through the same pop have a ping of 50 to 80 thats just way to high its 25 miles away yet a tracert reveals 11 or 12 hops internap just flat sux i trace a game server in sanjose again just 25 miles away 12 friggin hops abd a ping of 40 hell it shoould be 6 hops a ping of 15 to 20 max you know that i know that and SE Damn well knows it too i think the only reason they say send in ping plotter logs is to give you something to do so your not constantly calling sorry if this offends anyone its not ment as a flame its reality dont let them fool ya we must demand a cure and this is not a worm issue as i have been trying to rectify this for months as koitsu has to .
Speakeasy u know the cure apply it now please |
|
ykrad
join:2001-08-23
Petaluma, CA
| off topic but; hooooly crap. that is one HELL of a run on!
on topic; yeah, pl sucks, especially for gamers. they're by far the most critical of broadband analyzers. catskinner, no offense but I suggest you post traceroutes with your comments so they're at least semi-understandable.
I also go through the Seattle POP and have experienced the pl but haven't been significantly impacted by it. My roomate plays SW:Galaxies and hasn't expressed any grief with it - except when SE was dropping altogether the other day(s)...
while I agree it is a pain, I still think SE is doing the best they can. They wouldn't be the #1 independant bandwidth reseller if they had a bunch of no-nothing network engineer monkeys on staff. Lets just have a little more faith. |
|
koitsu
Premium
join:2002-07-16
Mountain View, CA
| He and I go through the same Speakeasy POP in SFO. See my PP graphs throughout all my Speakeasy threads (click on my username and look at my recent posts); I can guarantee you what he's seeing is identical to what I am.
--
Making life hard for others since 1977. |
|
GamingGirl
@speakeasy.n
 from:
koitsu
GamingGirl
| I'm not as technically savvy as most of you are but here are some traceroutes I did last night... I'm too tired to wait for them to complete tonight but this will show you that something wacky is going on as I've seen this consistantly for over a week now...
Tracing route to dslreports.com [209.123.109.175]
over a maximum of 30 hops:
1 11 ms 11 ms 11 ms dsl081-084-001.lax1.dsl.speakeasy.net [64.81.84.
1]
2 11 ms 13 ms 11 ms border12.fe4-0.speakeasy-32.lax.pnap.net [63.251
.223.91]
3 12 ms 13 ms 13 ms core1.ge2-0-bbnet1.lax.pnap.net [216.52.255.1]
4 14 ms 13 ms 13 ms sl-st20-la-10-0.sprintlink.net [144.232.154.205]
5 * * * Request timed out.
6 * * * Request timed out.
7 * * * Request timed out.
8 * * * Request timed out.
9 * * * Request timed out.
10 * * * Request timed out.
11 * * * Request timed out.
12 * * * Request timed out.
13 * * * Request timed out.
14 * * * Request timed out.
15 * * * Request timed out.
Trace complete.
Here's a trace to a website I visit based on the east coast...
Tracing route to www.iglnet.com [66.151.180.190]
over a maximum of 30 hops:
1 10 ms 11 ms 11 ms dsl081-084-001.lax1.dsl.speakeasy.net [64.81.84.
1]
2 * 11 ms 11 ms border12.fe4-0.speakeasy-32.lax.pnap.net [63.251
.223.91]
3 12 ms 15 ms 11 ms core1.ge2-0-bbnet1.lax.pnap.net [216.52.255.1]
4 10 ms 13 ms 11 ms gigabitethernet5-1-180.ipcolo2.LosAngeles1.Level
3.net [63.208.234.133]
5 13 ms 11 ms 11 ms unknown.Level3.net [209.244.10.245]
6 * * * Request timed out.
7 * * * Request timed out.
8 * * * Request timed out.
9 * * * Request timed out.
10 * * * Request timed out.
11 * * * Request timed out.
12 * * * Request timed out.
13 * * * Request timed out.
14 * * * Request timed out.
15 * * * Request timed out.
Trace complete.
Here is to Speakeasy...
Tracing route to speakeasy.net [216.254.0.95]
over a maximum of 30 hops:
1 11 ms 11 ms 11 ms dsl081-084-001.lax1.dsl.speakeasy.net [64.81.84.
1]
2 390 ms 105 ms 337 ms border12.fe4-0.speakeasy-32.lax.pnap.net [63.251
.223.91]
3 11 ms 11 ms 11 ms core1.ge3-0-bbnet2.lax.pnap.net [216.52.255.65]
4 11 ms 11 ms 11 ms gigabitethernet5-1-180.ipcolo2.LosAngeles1.Level
3.net [63.208.234.133]
5 13 ms 13 ms 13 ms gigabitethernet5-0.core2.LosAngeles1.Level3.net
[209.244.10.45]
6 * * * Request timed out.
7 * * * Request timed out.
8 * * * Request timed out.
9 * * * Request timed out.
10 * * * Request timed out.
11 * * * Request timed out.
12 *
Obviously I stopped it before I had to go all the way to 30 hops but I think you get the picture.
I get spikes of up to 28% packet loss in game (at least that I've seen with my own eyes) no matter what time of day that I play. The Blaster worm is expired but yet I saw no difference. It's been five days since I opened my service ticket and I STILL haven't heard from advanced support. Is everyone off this week? Do any of you know what could cause this?
Oh well, I'm going to bed, I bet in the morning nothing will have changed =(. |
|
GamingGirl
@speakeasy.n | reply to KatOak
I just got that ping plotter program and I'm wondering... what size packets should I use to accurately show the way my line reacts when playing online games (Unreal Tournament to be exact)? |
|
koitsu
Premium
join:2002-07-16
Mountain View, CA
| The default (56 bytes) probably isn't going to be good, but don't pick something absurdly large either (most routers will ignore large windowed ICMP packets). Pick something like 1020 bytes, or maybe 516.
--
Making life hard for others since 1977. |
|
Yourself
join:2000-08-10
Lynnwood, WA | reply to KatOak
FYI....
I got a call last night from SE(yes they called me) to inform me that there would be a hardware upgrade/changeout taking place next Wednesday that should take care of my issues on the Seattle POP. I will update....
Self |
|
sh0V3L
join:2002-02-16
San Carlos, CA
| reply to GamingGirl
i finaly got them to aknowledge the fact the sfo pops are borked fubar and then some lol.i recieved an email stating a temp fix in the next few days then a permanent one in 2 weeks as they do some upgrades i realized se was violating the tos for sdsl premium routing as it states 2hops to backbone.for 2 months or whatever it is since they did the last duct tape and tie wire fix at sfo its been minimum 3 hops so i had them by the...... heh you get the picture lol so we will se if they do as said.
said by GamingGirl:
I just got that ping plotter program and I'm wondering... what size packets should I use to accurately show the way my line reacts when playing online games (Unreal Tournament to be exact)?
hi gaming girl im an ut nut too well now ut2003 i ran ut servers for 3 yrs and now ut2003 for 1 yr or more.
i dont think the packet really matters, why? because ut is all udp as far as actual game play server to client. the ping is icmp and the initial handshake is tcp so the routers will handle udp different than the icmp ping.
my feelings are this se knows its got problems ok but if they can buy a lil time by having you do the ping ploter thing they will play that game with you as they did me till i called their bluff. just tell them you cannot reproduce the symptoms and document them because its udp connectionless service when i told them thats when things started going my way so give that a shot
frag ya l8r gg |
|
