http://www.dslreports.com/forum/wow W.O.W. forum current topics en Copyright 2007, dslreports.com Tue, 07 Feb 2012 22:55:42 EDT Tue, 07 Feb 2012 22:55:42 EDT http://i.dslr.net/bbrdisc1.gif http://www.dslreports.com 19 18 bbr disc http://www.dslreports.com/forum/remark,26729759 http://www.dslreports.com/forum/remark,26729759 2012-01-04 23:00:16 http://www.dslreports.com/forum/remark,26848826 http://www.dslreports.com/forum/remark,26848826 2012-02-03 09:55:31 http://www.dslreports.com/forum/remark,26863788 http://www.dslreports.com/forum/remark,26863788 2012-02-07 10:46:21 http://www.dslreports.com/forum/remark,26474792 http://www.dslreports.com/forum/remark,26474792 2011-10-25 21:02:19 http://www.dslreports.com/forum/remark,26861708 http://www.dslreports.com/forum/remark,26861708 2012-02-06 20:03:38 http://www.dslreports.com/forum/remark,26679199 http://www.dslreports.com/forum/remark,26679199 2011-12-20 16:29:06 http://www.dslreports.com/forum/remark,26845801 http://www.dslreports.com/forum/remark,26845801 2012-02-02 15:41:28 http://www.dslreports.com/forum/remark,26855810 http://www.dslreports.com/forum/remark,26855810 2012-02-05 07:25:36 http://www.dslreports.com/forum/remark,26826415 http://www.dslreports.com/forum/remark,26826415 2012-01-28 23:11:31 http://www.dslreports.com/forum/remark,26828800 http://www.dslreports.com/forum/remark,26828800 2012-01-29 20:50:34 http://www.dslreports.com/forum/remark,26837409 http://www.dslreports.com/forum/remark,26837409 2012-01-31 22:14:44 http://www.dslreports.com/forum/remark,26814667 http://www.dslreports.com/forum/remark,26814667 2012-01-26 01:16:29 http://www.dslreports.com/forum/remark,26781522 http://www.dslreports.com/forum/remark,26781522 2012-01-17 21:08:28 http://www.dslreports.com/forum/remark,26826309 http://www.dslreports.com/forum/remark,26826309 2012-01-28 22:30:02 http://www.dslreports.com/forum/remark,26821984 http://www.dslreports.com/forum/remark,26821984 2012-01-27 16:28:56 http://www.dslreports.com/forum/remark,26826019 http://www.dslreports.com/forum/remark,26826019 2012-01-28 20:42:55 http://www.dslreports.com/forum/remark,26826638 http://www.dslreports.com/forum/remark,26826638 2012-01-29 01:17:14 http://www.dslreports.com/forum/remark,26824955 http://www.dslreports.com/forum/remark,26824955 2012-01-28 14:11:54 http://www.dslreports.com/forum/remark,26793640 AS14745 direct, but maybe BGP sees something I don't. But do you know how else I know it's NOT my modem? Because rebooting my modem doesn't fix it. In fact, if I just let it sit, it starts working again. This seems to now happen any time I go over 50-100PPS sustained. Oh, and completely at random. In fact, I can actually look at graphs and see that any time I exceed 100PPS for more than a few minutes, regardless of bandwidth utilization, an upstream hop starts stalling out frequently and will randomly just quit passing packets. I know it's upstream, because the IPv4 gateway is still reachable, still reports up, still reports zero packet loss. Latency spikes and disconnects are confirmed by over 30 other people (including 4 known to be routed with XO and 7 via L3) to be just me. I'm the only one on WOW. I also know it's not my modem because I'm also seeing random latency spikes regardless of bandwidth and PPS rates. These spikes are not reflected at the IPv4 gateway heartbeat either. Signal levels? Nope. Downstream -7.6dBmV @ 36.7dB, Upstream ranging 38-47.4dBmV based on weather. (Not worth a truck roll - it's weather related. Always has been, always will be.) Speedtest is showing strange behavior - down at 16.06Mb/s (up from 15) and upload has dropped to absolutely not possible. Here's some ping and smoke tests to one of those two IPs that WOW isn't actively blocking from performing quality tests, shall we? Ping test which shows packet loss: https://secure.dslreports.com/pingtest/97c8c46289b6/2869210 SmokeTest (ongoing) to 144.1: https://secure.dslreports.com/r3/smokeping.cgi?target=network.454a5fcc6274241535f75a2e9502e68a SmokeTest (ongoing) to 144.10: https://secure.dslreports.com/r3/smokeping.cgi?target=network.020781e2f554968af7e57610aa37e2a2 By the way, initial results are pretty embarrassing - 50ms+ average from all points, significant packet loss and extreme jitter over the past 24 hours. I honestly don't care what it is. I don't care about excuses or proclamations of "innocence" or "investigating further." At this point, I apparently know the network better than the people supposedly responsible for it. There is no innocence here - there is deliberate blocking of ICMP traffic and there is clear evidence of a specific issue. There is no investigating further - there is a specific problem, most likely with the WS-C3750V2 or immediate next hop, which requires work be performed to at least the minimum required by an entry level position in IT which includes opening a TAC case. Frankly, I am beyond tired of this at this point. Yes, AT&T ENS blew up their network last night - I am painfully well aware. This is not AT&T ENS. This is not XO. This is not Level3. Hell, for all I know, XO and Level3 are conspiring to slap you around for passing your 10net traffic right out the public facing interface and chewing their CPU cycles filtering it. And YES. YOU ARE DOING IT AND IT IS BOTH WRONG AND STUPID. Your 10net is COMPLETELY wrong. Hell, maybe someone should also tell LW463 to not let his underlings assume people are idiots. Especially people who have worked on geographically diverse DOCSIS systems many times WOW's size before MPLS existed. Or maybe just start by getting rid of the guy who thinks it's "D slam" and lists "Frontpage" as a PROGRAMMING LANGUAGE. Here's a hint: it's an abbreviation for Digital Subscriber Line Access Multiplexer and Copper Mountain was only deployed for very small scale G.991 SDSL and IDSL by minor CLECs; I had some going to Copre DSUs passing V.35. Baby Bells primarily ran Redbacks in mixed-mode with Radius auth'd PPPoA for interLATA and resale and PPPoE for local IP term. I think there might have been a few Nortel NMS deploys in GTE territory since they favored DMS, but don't hold me to it. Huh, turns out old rootwyrm might just have a bit of experience in the regulated communications space, huh? (Oh, and I'm pretty sure you got some 24 and a 64 networks I used to manage DNS for.) Either way, I am beyond tired of this. And I don't mean metaphorically. I mean physically. You see, every time this happens, and I'm asleep? I GET WOKEN UP BY SERVICE ALARMS. I already had to turn off every single live/dead ping alarm because of the ICMP blocking, which is just childish on their part at best, and if being implemented via BOOTP on modem, hysterically incompetent in light of the other much more significant problems present in the config. Blah blah, home internet, yeahNO. I get woken up by alarms because the automated system I had to write to power cycle the modem via AC outlet control gets tripped, sees the heartbeat gateway is just fine, and says it's external. Meantime, the opposite end of rsync waits 120 seconds, tests remote host, finds it completely unreachable, and fires off an alarm because it thinks the host has died. My NAS sync AND backups just flat out break, and I've been forced to totally disable it, because the outages are just that right length that it exceeds retry count and breaks the mirror. Oh, and then there's the time I've spent troubleshooting a problem which is rather explicitly and clearly WOW's network and responsibility to detect and fix. Did I mention it's happening outside of normal change window too? Yeah. That's another indicator that something's rotten in Denmark, by the by. So yes, at this point, I would say I am justifiably furious. Especially since I've had to be the one to do all the work, the problems I have found are not only beyond amateurish in nature AND quality of work, but also extremely hazardous and just flat out stupid. I mean seriously, how can this be going on and WOW have absolutely NO awareness of things? How is it DOCSIS 3 managed to get rolled out on this, forget DVRs even working HALF the time? (And that's the LEAST of the potential woes.) Did your network engineers negotiate for "no on call ever"? Just completely ignore their pagers and cell phones after 4:30PM? Do they have no concept of end-to-end heartbeat? Do they just make absolutely no effort whatsoever to monitor service quality or even baseline function? Pretend that 15 minute MRTG sampling is adequate monitoring? I cannot find any rational or logical explanation for why this has been going on for several days with WOW apparently blissfully unaware the whole time. I can tell you for fact that when I had root on the equipment in that very same building - and yes, I mean that EXACT SAME HEADEND right down to the dishes - I got paged the very SECOND anything went down for ANY reason. Frankly, I don't think it's in the slightest unreasonable to expect the same level of professionalism and service from WOW's staff that most IT professionals consider standard. Like I said; I don't care what the exact problem is, and I'm tired of excuses from people who apparently decided padding their resume was a good idea. Again; no offense intended directly or indirectly, Dan. Not saying you're guilty of it - you definitely are not. I know you're just the messenger here, and I really do appreciate the help. Not your fault OR responsibility that they're putting you in this position, and I don't blame you. But I am tired of these morons putting you in this position with their technical incompetence. All I want to know at this point is 1) WHEN is this all going to be fixed? 2) WHAT procedural steps will WOW be taking to ensure this does not ever happen again? ]]> http://www.dslreports.com/forum/remark,26793640 2012-01-20 16:23:36 http://www.dslreports.com/forum/remark,26804784 http://www.dslreports.com/forum/remark,26804784 2012-01-23 18:46:12