 
Boricua65
join:2002-01-26
Puerto Rico | Who would've thought... a NIC would cause all that trouble? A $5.00 to $10.00 hardware, wow! | |
|
 | TheTony
join:2006-11-02
Minneapolis, MN | Re: Who would've thought... Apparently not the network administration...  | |
|
| 
KrK
Heavy Artillery For The Little Guy
Premium
join:2000-01-17
Tulsa, OK | Heh, Someone probably installed a P2P app on the PC and clogged the network! 
J/K'ing.... I HOPE. | |
|
tightwad10000
@comcast.net | budgeting maybe they will bump their budget to $15 - 20 now....? | |
|
| Bytezboy
join:2001-05-17
New York, NY | Re: budgeting Why don't they use real servers? @_@ | |
|
|
| plat2on1
join:2002-08-21
Hopewell Junction, NY
clubs: | Re: Who's the source ?? reporting first is more important than accuracy today | |
|
| | 
bjl
Premium
join:2002-05-02
Newport Beach, CA | Re: Who's the source ?? especially for the LA times | |
|
| bmn
? ? ?
Premium,ExMod 2003-06
join:2001-03-15
hiatus
1 edit | Thank you... I was wondering the same thing when I read the article. The explanation seemed just technical enough to blow the average newspaper readers mind despite it smelling like an executive or manager stringing together a bunch of computer terms (think PHB) to someone who works with networks.
--
Prove it...
Save the Internet Time (NTP) service, use the pool. | |
|
81399672
Premium
join:2006-05-17
Los Angeles, CA | The real truth of the metdown I would not be surprised if real truth is that they got hacked
--
i am not a lawyer but I do play one on the internet | |
|
| 
morbo
Complete Your Transaction
join:2002-01-22
00000
clubs: | Re: The real truth of the metdown "national security won't let us report that we are completely incompetent." sound familiar? | |
|
| |
81399672
Premium
join:2006-05-17
Los Angeles, CA
| Re: The real truth of the metdown said by morbo  :"national security won't let us report that we are completely incompetent." sound familiar? Unfortunately yes it sounds really familiar | |
|
| 
phattieg
join:2001-04-29
Winter Park, FL
 ·Verizon Wireless B..
·Sprint Mobile Broa..
| said by 81399672 :I would not be surprised if real truth is that they got hacked I know this was suppose to be a joke, but I deal with hard headed people everyday who don't believe their NIC card isn't bad just because they can browse but it's slow, or just because they are getting a "pc activity" light. Their problems include unable to aquire an ip, severe packet loss when pinging websites AND their modem ip (192.168.100.1), or even their router in cases where that applies. I have also seen situations where the pc light works, the winsocket files are intact, and they still get a 169 ip, but when the modem is connected the DHCP get request doesn't hit the modem, even after swapping the modem. They connect thru USB, and just like magic, it works fine. NIC's are just as prone to failure as anything else. I haven't had my default NIC fail in the 8 years I've had it, but I have had 2 of my secondary ones fail, and I've had the ethernet port on my Cisco ATA telephony adapter refuse a public ip, and only connect to a 192 based ip address. If I switch the ip range to, oh lets say 169 based, or even a public ip, no go, it won't even acknowledge the connection, nor will it send a DHCP get message. I know that last one may be a number of things, but the other problems have been confirmed in some cases as being a bad NIC because the person in question buys a new NIC, or uses USB connections to their modem or router, and it fixes the problem. I wish there was a device sold at every local store that NIC's are sold at that allows a person to test their NIC... 
--
SIPPhone/Gizmo # 17476200648 / PIMPNET Chatline / Ran by Asterisk & Slackware 10.1. | |
|
Stumbles
join:2002-12-17
Port Saint Lucie, FL | And we use Ataris as servers I think this is the first time I have heard anyone trying to blame a "network meltdown" on a single NIC.... on a PC. Yeah right. Maybe if it were on a server. | |
|
| |
| | Kearnstd
Elf Wizard
Premium
join:2002-01-22
Mullica Hill, NJ
| Re: And we use Ataris as servers said by KrK :said by Stumbles :I think this is the first time I have heard anyone trying to blame a "network meltdown" on a single NIC.... on a PC. Yeah right. Maybe if it were on a server. No, if they used Atari's or C-64's the system wouldn't have gone down  arent FAA systems a few generations behind the Atari? i seem to remember hearing about them having tubes in some airport systems still.
--
[65 Arcanist]Filan(High Elf) Zone: Broadband Reports | |
|
| raythompsontn
join:2001-01-11
Oliver Springs, TN
| said by Stumbles :I think this is the first time I have heard anyone trying to blame a "network meltdown" on a single NIC.... on a PC. Yeah right. Maybe if it were on a server. It is entirely possible that the NIC shorted, locked up the hub, and this in turn prevented other systems from communicating. A switch should have been used which would not suffer from a single port failure, at least an intelligent switch.
This is the federal government running the system, lowest bid gets the job. A hub would be much cheaper than a switch, a few bucks here, a few bucks there, and you shave enough to win the contract.
Remember that large systems have failed because of a small part. Remember the shuttle accidents. | |
|
| | xo
join:2007-06-15
Perry, FL | Re: And we use Ataris as servers "..lowest bid gets the job."
like halliburton? | |
|
| | | 
dogma
Premium
join:2002-08-15
Boulder City, NV | Re: And we use Ataris as servers Big difference between competitive bid, where the lowest bidder probably gets the contract...
...and no bid, where Halliburton always gets the contract. | |
|
| | | | 
FiL
Premium
join:2005-08-16
Silver Spring, MD | Re: And we use Ataris as servers Got that one right on the money... | |
|
| | | 
RR Conductor
RailRoadDude
Premium
join:2002-04-02
Redwood Valley, CA
 ·Comcast
| said by xo :"..lowest bid gets the job." like halliburton? Lol I thought the same thing when I read his post! | |
|
| 
CyBrChRsT
join:2003-02-28
Kansas City, MO
| Server has nothing to do with it. How naive for you to think so. Any port on any network device has the potential to halt a complete network, interconnected by switches/hubs/routers or not. I worked at a hospital that my boss swore by Dell switches were the same as Cisco but so much cheaper (remember you get what you pay for) (mind this, he cost the hospital thousands of dollars in lost productivity in the hour it took him to find the damn problem) and swore that they were just rebranded Cisco by Dell. BS, it was quite obvious how stupid he was in the knowledge of IT equipment when a single ATA device (eh hem... 3COM) hooked up to a fax ended up sending a broadcast storm across the network of 8 switches 200 VoiP phones and 200 Workstations and 9 servers and 3 routers and a firebox! If the dumbass had been using Cisco to start he could have configured the switches to recognize such a threat on all of the ports and shut down the port from the afending device! Sad thing is, if he had the network protected with APC ProtectNet hardware it would have saved the device to start; from the storm that caused it over the weekend. Not naming names but eh hem "The Headache and Pain Center / Doctors Specialty Hospital (now known as Doctors Hospital" sorry but find this the opportunity to slam. -> First week I was there I ran security tests and was able to hack the admin password in 2 seconds with a dictionary attack and get the same password over a phoneline out in the country only connected at like 22k in 20 min using Brutus against the smtp server! Did he do anything after I warned him of this? NO! Why you ask? Excuse = "I tried getting the time approved to secure the network more but they wouldn't" yeah uh then it cost them a day outtage after the exchange server had been hacked into by a virus. So not spending that extra money in approved time costs them way more in downtime down the road. Then the fucker had the nerve to fire me after I got food poisoning twice the next year. If said dumbass would have had any idea what he was doing at the time of setting up the network it wouldn't have taken any more than usual at all to just do the damn job right the first time. Oh and the wifi network for the Dr.'s was never secured, anyone with a laptop sitting in the parking lot could tap right in. I bet HIPPA would have loved to have known that. Should have turned his ass in. ok I'm done | |
|
| nguyen27
join:2003-05-14
Quincy, MA
1 edit | said by Stumbles :I think this is the first time I have heard anyone trying to blame a "network meltdown" on a single NIC.... on a PC. Yeah right. Maybe if it were on a server. It is possible that the NIC can cause that kind of problem, if some how the NIC cause conflict with the MAC address of the subnet router, it will bring the whole subnet down tho. I saw this with my own eyes when some external vendor came into our office and plug their laptop into our active datajack, but network like LAX should have appliance prevents such problem. | |
|
evilghost
Premium
join:2003-11-22
Springville, AL
 ·Windstream
| ARP flood? I've heard it said, but never experienced it first-hand, that a bad NIC can generate a substantial amount of ARP requests. It could have been poorly configured switches and an ARP broadcast storm that brought the network to it's knees, especially if it were machines replying to a "who-has/tell" ARP flood.
This is highly possible, regardless if it was a 'server' or a 'workstation'. | |
|
| 
RARPSL
join:1999-12-08
Suffern, NY
| Re: ARP flood? said by evilghost :I've heard it said, but never experienced it first-hand, that a bad NIC can generate a substantial amount of ARP requests. It could have been poorly configured switches and an ARP broadcast storm that brought the network to it's knees, especially if it were machines replying to a "who-has/tell" ARP flood. This is highly possible, regardless if it was a 'server' or a 'workstation'. Think back a few weeks ago when a few iPhones were initially blamed for taking down a University's network due to an "ARP" storm that was finally discovered to be due to a bug in the Cisco Routers. | |
|
| | 
packetscan
Premium
join:2004-10-19
Bridgeport, CT
clubs: | Re: ARP flood? This bug might also happen on switches.. thus the switch melt down. Could this card have caused the same condition.
--
Reach out and Tap someone! | |
|
NOCMan
Verizon Fios User
Premium
join:2004-09-30
Flower Mound, TX
| Sounds like a broadcast storm to me Really sounds like a broadcast storm. Perhaps a switch went bad where it crapped on the spanning tree in such a way to cause a broadcast storm.
I've seen it happen even when all the preventing measures are implemented. Seems when a card decides to go crazy it takes the entire switch with it.
--
Mac Chatter
»www.macchatter.net | |
|
| Techie714
join:2005-08-02
Anaheim, CA | Re: Sounds like a broadcast storm to me I agree it does sound like a broadcast storm. But even then they should have fixed the problem with in just a few hours. | |
|
| | 
manfmmd
Premium
join:2003-01-14
Earth
clubs: | Re: Sounds like a broadcast storm to me Sometimes it can be difficult to track down especially if you don't have the correct monitoring software/hardware. | |
|
| 
murfster
Under Siege
Premium
join:2001-03-04
Mcdonough, GA
clubs:
| I'm gonna bet this is the answer ... broadcast storm via spanning tree.
said by NOCMan :Really sounds like a broadcast storm. Perhaps a switch went bad where it crapped on the spanning tree in such a way to cause a broadcast storm. I've seen it happen even when all the preventing measures are implemented. Seems when a card decides to go crazy it takes the entire switch with it.
--
GMC Forums
UnitedGamerz | |
|
smcallah
join:2004-08-05
Home
| Uh... quote:
No, that makes no sense to us, either. An entire network that relies on a single NIC from a desktop PC? A network engineering feat, indeed!
They never said the entire network relies on a single NIC.
A single NIC can easily kill a network if it is not configured for broadcast storm protection.
Nothing says the network had to be "relying" on the port in the one PC that went bad.
I mean, it even says in the article that a "domino effect occurred" that brought down other network cards. Why try to sensationalize it? | |
|
|
manfmmd
Premium
join:2003-01-14
Earth
clubs:
| Re: Uh... said by smcallah : quote:
No, that makes no sense to us, either. An entire network that relies on a single NIC from a desktop PC? A network engineering feat, indeed!
They never said the entire network relies on a single NIC. A single NIC can easily kill a network if it is not configured for broadcast storm protection. Nothing says the network had to be "relying" on the port in the one PC that went bad. I mean, it even says in the article that a "domino effect occurred" that brought down other network cards. Why try to sensationalize it? I think you nailed it. We had an "unplugged" network interface on a Catalyst 4506 bring down an entire network a couple of years ago..."UNPLUGGED" | |
|
cdbma
join:2003-01-19
Bolton, MA
| don't blame the NIC or the server The failure was human-related, not hardware-related. Don't they have anyone there with half a brain that can figure out how to dual-rail, have a hot backup or, god-forbid, a SPARE NIC? Heck, anyone could have gone down to the local Best Buy or Fry's, picked up a new card and popped it in. We all know that hardware fails and part of a good design is to plan for the eventual failure of any key components, especially those in the critical path.
This is classic "cause/effect" excuse behavior. We are supposed to accept this as a valid reason - maybe even feel sorry for the poor FAA and move on. I bet the stranded passengers are not buying it. Akin to "I'm late because I missed the bus." it's standard bureaucratic BS. I think they take a course as part of their training.
Homeland Security is a farce and an oxymoron. The TSA is a joke and I don't think we're any safer today than we were 6 years ago. Unfortunately, the government equates huge process and excessive spending with security. Geez - we're totally broke here. | |
|
|
See 7 replies to this post |
|
Bobcat
Volvo sucks donkey balls
Premium
join:2001-02-04
Bedminster, NJ | Just be happy... ...that they don't build bridges. | |
|
PolarBear
The bear formerly known as aaron8301
Premium
join:2005-01-03
·CableOne
| No flying for me This is yet another reason why, if I have to travel anywhere withing 2,000 miles of home, that I will just drive. The price would be about the same, and I have the luxury of stopping whenever and where ever I want, eating whenever I want, listening to loud music if I wish, using my cell if I need to, and seeing some sights.
If I had to travel over 2,000 miles, say to the East Coast from my West Coast state, Amtrak. Amtrak puts flying to shame.
--
A computer lets you make more mistakes faster than any other invention, with the possible exceptions of handguns and Tequilla. -- Mitch Ratcliffe | |
|
|
See 17 replies to this post |
|
Test99
Premium
join:2003-04-24
San Jose, CA
·DSL EXTREME
 ·InPhonex
| Packet Storms The card, which allows computers to connect to a local area network, experienced a partial failure that started about 12:50 This was a real problem with some of the earliest Ethernets. In the early 1980s, before the hubs/switches and star configurations that are standard now, computers on a local network were daisy-chained together. Each node retransmitted all traffic that was just passing through. So a single failure could bring down the whole network. I spent lots of time following network cables under floors to find the one bad node.
These days, there's a good chance that a hub/switch will detect a malfunctioning NIC and disconnect it from the rest of the network. But if the NIC fails in such a way that it generates a packet storm, such as the ARP flood that evilghost suggests, the hub/switch probably won't notice that anything is wrong. The network will just grind to a halt under the traffic overload.
--
50775@fwd.pulver.com | |
|
| 
milnoc
join:2001-03-05
H2Z
·TekSavvy Solutions..
1 edit | Re: Packet Storms said by Test99 :This was a real problem with some of the earliest Ethernets. In the early 1980s, before the hubs/switches and star configurations that are standard now, computers on a local network were daisy-chained together. Each node retransmitted all traffic that was just passing through. So a single failure could bring down the whole network. I spent lots of time following network cables under floors to find the one bad node. What are the chances their setup is so old that they're still using a token-ring network? Yikes!  | |
|
| | dougrem
Premium
join:2007-08-14
Corona, CA
| Re: Packet Storms When I read the story that a single NIC brought down the network that was the first thing I thought of... They're probably still running Token Ring (or as it was nicknamed; Broken Ring).
Now days everyone immediately thinks of a NIC as being Ethernet. There's still some Token Ring out there... Hell there's probably still some ArcNet and GNet too. | |
|
| | | 
alanhdsl
Premium
join:1999-10-09
Phoenix, AZ | Re: Packet Storms I'll bet they're still using token ring, too. I've seen rings brought down by burst errors caused by a single NIC. IMHO, token ring was great in theory, but too complex in practice to be reliable. | |
|
AnonProxy
Proxy of Anon
Premium
join:2001-05-12
ß | I love when you can start a rumor and it... shows up as national news.
It was a complete server, not the network card. The SABRE gateway server died due to a hard drive crash but it sounded cooler to say "the NIC went south". | |
|
| 
halfband
Premium
join:2002-06-01
Huntsville, AL | Re: I love when you can start a rumor and it... A SABRE gateway that does not use a fully redundant drive array? That does not sound plausible.
--
Registered Bandwidth Offender #40812 | |
|
| |
AnonProxy
Proxy of Anon
Premium
join:2001-05-12
ß | Re: I love when you can start a rumor and it... You don't know the SABRE system then. | |
|
| | | cmaenginsb
Premium,MVM
join:2001-03-19
Palmdale, CA
| Re: I love when you can start a rumor and it... Why is Customs using Sabre for entering processing information? (at least the only Sabre I know of is an airline reservations system)
The more reliable reports I have is that it was a problem with the LAN.
--
CCNA, Comtrain Certified Tower Climber | |
|
| | | |
Starbucks
@spcsdns.net |  network
| |
|
|
Doubter
@swbell.net | You are saying the LA Times lied and we need to believe you? | |
|
shashinka
join:2000-09-16
West Boylston, MA
| bad NIC can cause problems If there was a bad nic and all the computers were on the same broadcast domain then yes it could cause this problem. And of course if a server that was very important was having NIC problems then that of course would cause slowdowns even on separate broadcast domains. | |
|
| Kearnstd
Elf Wizard
Premium
join:2002-01-22
Mullica Hill, NJ | Re: bad NIC can cause problems but a single desktop workstation shouldnt be able to bring down the network just because of HW failure.
--
[65 Arcanist]Filan(High Elf) Zone: Broadband Reports | |
|
| | ydoucare
join:2003-03-12
Rensselaer, IN | Re: bad NIC can cause problems Apparently you're not a very good reader. | |
|
rkreider
join:2007-08-13
New Philadelphia, OH
| oh well I have a feeling this has to do with global warming somehow, some way. If TWC were involved, their packet shaping would have helped to relieve the "thundering hurd" problems, and even _possibly_ prevented this failure!
I love BP storms, mixed with DUP storms and sprinkled with a dash of ARP floods...
Either way, I'll look forward to my flight in September to California (John-Wayne). Maybe I'll be around when this happens again and I can help out. =) | |
|
TScheisskopf
World News Trust
join:2005-02-13
Belvidere, NJ | How interesting. Dreamhost, a hosting company I use, just opened a facility out by LAX. Last weekend they had problems with that facility, in that same time frame.
For the want of a nail... | |
|
Asmodeus
join:2004-05-26
Spring Valley, CA | mission critical components... this is why components like this must be mission critical certified... if it was, i seriously doubt that it would have failed if ever and in the case of the airline industry, everything should be mission critical... just my 2 cents... | |
|
RickM.Net
join:2000-09-01
Louisville, KY
clubs:
·Insight Communicat..
| Seen this before I've had this happen before. The circumstances were when a laptop user with a 10mb Xircom pcmcia card unplugged and then replugged his dongal, it would start broadcasting 1000's of broadcast packets. Every computer on that subnet would have to stop, examine that packet to see if it wanted it, then for some damn reason, broadcast the fact that it did not want it. Not amusing. | |
|
|
| 
tcharp
T C
Premium
join:2002-10-23
Lubbock, TX
·NTS Communications
·Suddenlink
·Vonage
| Re: Chatty NICs suck said by AlexNYC :Chatty NICs happen even to good admins sometimes (Except most modern managed switches would see this kind of crazy spike in traffic and a capable admin should be able to unplug the darn patch cable long before it crashes the entire network) Keyword: capable
--
"It's hard to leave when you can't find the door." - Joe Walsh | |
|
| | Exothermicus
join:2007-05-24
Denton, TX
| Re: Chatty NICs suck I would believe their LAN is so old it still uses unmanaged 10 base hubs (all computers connected on one wire), or some kind of token ring setup. Those old 3270 style terminals were originally daisy chained in groups of as many as 127 on one SDLC link.
But I have seen corrosion cause a switch port to go crazy showing heavy traffic when nothing was connected, but it did not traverse past that one link.
Exo | |
|
|
|
|
LAX Meltdown Thanks To Single NIC
Chatty NICs suck