 | |
Lolwow, looks like someone made too big of a purchase, hehe  I hope they have backups haha(yea i they do, duh). | |
|
 virtbizchrisVIRTBIZ Chris
Premium Member
join:2007-03-01
Dallas, TX |
6 hours and countingIt's unthinkable that they did not have a DR plan, but it's been 6 hours since we have been able to process a card. I see ping response off their website now and looks like they're re-routing. I hope they keep backups. We've already started the application process with another gateway. I remember when Authorize.net was the big daddy. Now this just illustrates how far things have slid there. | |
|
 | | |
jackiebrown
Anon
2009-Jul-3 11:20 am
Re: 6 hours and countingwhich gateway have you picked? we may want to port to a new one as well.
thx | |
|
| | |
| | jamesh00
join:2009-07-03
Egg Harbor City, NJ |
to jackiebrown
Re: 6 hours and countingHello everyone, we have a liquor store and are in TROUBLE! Every wants liqour for the holiday weekend. We got in touch with goEmerchant.com and they were VERY helpful and are setting us up with a new gateway!! | |
|
| |
Gjaros
Anon
2009-Jul-3 11:00 am
Thanks for the info...Thanks for the information. Our clients are getting antsy, glad to know it's out of our hands | |
|
| |
Alan J
Anon
2009-Jul-3 11:01 am
Authorize.netWow! Thanks for the information in your post.
I could not figure out what was going on and could not find any information on it.
I understand that this may be a perfect storm, but it is totally unacceptable.....a company of their size is suppose to have back up systems
If anyone has any information on this situation or more detail about the fire, etc, please post. It will help ease my nerves and anger.
Thanks | |
|
 DC DSLThere's a reason I'm Command.
Premium Member
join:2000-07-30
Washington, DC
|
DC DSL
Premium Member
2009-Jul-3 11:03 am
Abso-frickin-lutely UnbelievableThe same stupidity that can't be bothered implementing/enforcing strict data retention and security compliance policies on merchants also can't be bothered with ensuring there is a viable and thoroughly tested diaster prep/continuity plan in place.
I hope they get blown off the map for this. We can't afford having critical components of the financial system that are run by incompetents. Disaster planning is really not that hard...there are lots of us consultants who offer that expertise and guidance. No, it just takes committed funding and resources than most modern businesses refuse to accept the value-benefits of.
Tar and feather the moron who didn't make sure they had an alternate continuity site that could be hot in less than 90 mnutes. | |
|
| | |
viperlmw
Premium Member
2009-Jul-3 11:10 am
Re: Abso-frickin-lutely Unbelievablesaid by DC DSL:~snip~ ...it just takes committed funding and resources than most modern businesses refuse to accept the value-benefits of. DING DING DING DING DING...We have a WINNER!!! | |
|
|  SLD
Premium Member
join:2002-04-17
San Francisco, CA |
SLD to DC DSL
Premium Member
2009-Jul-3 11:24 am
to DC DSL
said by DC DSL:I hope they get blown off the map for this. They are a great gateway, with a fantastic API and uptime. I'd like them to survive this and improve their redundancy. | |
|
|  kewlkeedGrouch
Premium Member
join:2005-02-05
Knowlton, QC |
to DC DSL
said by DC DSL:Disaster planning is really not that hard...there are lots of us consultants who offer that expertise and guidance. No, it just takes committed funding and resources than most modern businesses refuse to accept the value-benefits of. Not a direct hit towards you, but a good majority of so called consultants aren't worth their weight. Granted a company like this has zero/zilch excuse for what has happened... Perhaps they hired one of these wannabe consultants. But more than likely, the top levels pinched the mighty dollar and refused to spend boatloads on something that might never happen. "It's better to have it and not need it, than to need it and not have it." So how's that workin' for ya Authorize.net? | |
|
|  j1349705
Premium Member
join:2006-04-15
Holly Springs, NC
4 edits |
to DC DSL
said by DC DSL:No, it just takes committed funding and resources than most modern businesses refuse to accept the value-benefits of. I'm going to refrain from talking about this situation too much more until we know more about what happened, but you do have a good point. Most of the time, businesses are warned before things go wrong. Sometimes, they decide to take the risk. | |
|
| | DC DSLThere's a reason I'm Command.
Premium Member
join:2000-07-30
Washington, DC Actiontec GT784WN
|
DC DSL
Premium Member
2009-Jul-3 12:13 pm
Re: Abso-frickin-lutely Unbelievablesaid by j1349705:I'm going to refrain from talking about this situation too much more until we know more about what happened, but you do have a good point. Most of the time, businesses are warned before things go wrong. Sometimes, they decide to take the risk. Whatever the specifics about the situation, what is absolutely crystal clear is their procedures and BDC weren't capable of accommodating realtime failover or going hot fast enough to avoid a catastrophic outage. There should be full-scale tests of catastrophic scenarios at least once a year, and follow-up on and verification of any corrective measures recommendations no more than 60 days after a test. They should be testing less-criticial failure scenarios a few times a year. If anything, this makes sure that everyone knows what they're supposed to do, and correct any weaknesses. Something like this can't be excused or explained away. They're probably going to get spanked pretty hard by the logos...the inability to complete transactions in a timely manner costs the logos and the issuing institutions real money, and bad PR points. They'll probably get wallopped with lawsuits from merchants as well. | |
|
| | |  Dolgan
Premium Member
join:2005-10-01
Madison, WI |
Dolgan
Premium Member
2009-Jul-3 1:21 pm
Each business needs its own backup planIt is also unbelievable that businesses do not have their own backup plans. All walk-in businesses should have knucklebusters and carbon reciepts so they can run charges the old fashioned way, and call them in/input the info into the web later. Saved my butt at several restaurants when the phone lines went down, or the powere went out. Furthermore, if their phone lines are not down they can manually call in each credit card transaction...this is not rocket science, and the workarounds are there for each business to utilize if they so desire. The only exception may be web based merchants who would not have access to the 2 workarounds provided. It does suck that this happened, but there are temporary, albeit inconvenient, workarounds. | |
|
| | | | DC DSLThere's a reason I'm Command.
Premium Member
join:2000-07-30
Washington, DC Actiontec GT784WN
|
DC DSL
Premium Member
2009-Jul-3 1:32 pm
Re: Each business needs its own backup planThe logos don't want paper or manual call-in...their beancounters don't want the expense of supplies and staffing that requires. Statistically "significant" or widespread network outages are uncommon, so when contrasted to the cost of supporting manual auth, it's not justifiable in their eyes. Unfortunately, with all the consolidation in financial services over the last few years, while there are fewer points of failure there is far greater likelihood that any single failure could quickly cascade to a national or global scale...as this situation makes quite apparent.
If anything, people should be smart and heed those warnings to keep a stash of cash on hand for emergencies. It doesn't take a natural disaster or terrorist attack to muck things up...a fire or a backhoe could just as easily ruin a lot of people's days. | |
|
| | | | | |
to Dolgan
said by Dolgan:All walk-in businesses should have knucklebusters and carbon reciepts so they can run charges the old fashioned way, and call them in/input the info into the web later. ... Furthermore, if their phone lines are not down they can manually call in each credit card transaction...this is not rocket science, and the workarounds are there for each business to utilize if they so desire. The only exception may be web based merchants who would not have access to the 2 workarounds provided. Exactly. said by DC DSL:The logos don't want paper or manual call-in Mastercard and Visa *require* merchants to have imprinters and card slips to use in case of an outage--be it small scale or large scale. Yes, it creates more work/costs, but they'd rather get that type of transaction than NO transaction. | |
|
| | | | |  Crazy DaveFormerly known as Seph83
Premium Member
join:2004-04-29
Homer, GA
Ubiquiti UDM-US
|
Re: Each business needs its own backup planMy retail company uses First Data for it's authorizations, and when they go down or our Fractional T1 goes down, we ask for the person using the Credit/Debit Card's ID, and verify that it's their card, and then manually override and accept the transaction through the POS. Our servers store the transaction locally, until we are back online, and it sends it in for processing. This way, the customer gets their products, which keeps them happy, it's simple and easy, and if there is a problem with their card, then First Data or their Bank/Credit Company has to deal with it. It's win-win.
Besides, I'll be damned if I'd let someone "rack-slide" my credit or debit card. Yes, lets just give your full credit card info to some store peon so they can max it out for you... I think not... | |
|
| |
newt
Anon
2009-Jul-3 11:05 am
Where is the backup plan?I really can't believe they have been down this long! I am surprised that authorize.net is "supposed" to be one of the big dogs and we get this kind of service. | |
|
| |
Ergoman
Anon
2009-Jul-3 11:12 am
Don't Forget to Thank Cybersource.comCybersource doesn't even have anything posted on their site which since they are the company company would have been nice. Of all types of businesses that should have had a secondary data center DR plan, credit cards is one. Way to go!!!! | |
|
| |
Alan J
Anon
2009-Jul-3 11:26 am
Apparently they set up a Twitter account today....see below | |
|
|  vipermCarpe Diem
Premium Member
join:2002-07-09
Winchester, CA |
viperm
Premium Member
2009-Jul-3 11:29 am
Re: Apparently they set up a Twitter account today....see belowDamn! that sucks we are just about to submit an application to them for processing of all our wireless hotspot gateways.
What other options are there we have to work with either with Paypal or Authorize.net with our wireless portal system www.wirelessorbit.com | |
|
| |
1 edit |
Killa200
Premium Member
2009-Jul-3 11:42 am
Re: Apparently they set up a Twitter account today....see belowi don't know any off the top of my hand... but a few payment gateways do authorize.net emulation on their end so you can use the authorize.net AIM format in various shopping carts to process payments with these other companies.
::EDIT:: Actually going though my old company email i found out the processor we used to use, CDGcommerce, has authorize.net emulation. I can give them a thumbs up on my end as i never had issues with them. They are about 20 a month cheaper than authorize.net as well i believe. | |
|
| | | vipermCarpe Diem
Premium Member
join:2002-07-09
Winchester, CA |
viperm
Premium Member
2009-Jul-3 11:53 am
Re: Apparently they set up a Twitter account today....see belowWow this sucks this is a qoute from thier twitter posting
@gotwww The backup data center was impacted too. Don't have info as to why. The team is solely focused on getting us back up for now.
6 minutes ago from web in reply to gotwww | |
|
| | | | DC DSLThere's a reason I'm Command.
Premium Member
join:2000-07-30
Washington, DC Actiontec GT784WN
|
DC DSL
Premium Member
2009-Jul-3 12:00 pm
Re: Apparently they set up a Twitter account today....see belowsaid by viperm:Wow this sucks this is a qoute from thier twitter posting @gotwww The backup data center was impacted too. Don't have info as to why. The team is solely focused on getting us back up for now. 6 minutes ago from web in reply to gotwww Sounds to me like their BDC was neither hot nor go-ready. Probably some dumbheaded nonsense like not having point-of-failure recovery, or even a sufficiently current image to come up on. | |
|
| | | |  cabanaDepartment of Adjustments
Mod
join:2000-07-07
New York, NY
1 edit |
to viperm
1) cannot even fathom a company of this magnitude not having a back up plan much less not having 100% redundancy so that worse case scenerio would have been a hiccup while the made the switch. 2) its a peek at what happens in general when any e-commerce company goes down. Sometimes having some good ole cash under the mattress not such a bad idea  3) would love to be a fly on the wall in their OPs room  Wonder what that first call was like... "hey sam" ... take a look at this.... followed by ... (fill in the blank) On a serious note - I hope this does not hurt the businesses that depend on their services too much  (edit speeling) | |
|
| | | | |  FFH5
Premium Member
join:2002-03-03
Tavistock NJ
1 recommendation |
FFH5
Premium Member
2009-Jul-3 12:22 pm
Re: Apparently they set up a Twitter account today....see belowsaid by cabana:1) cannot even fathom a company of this magnitude not having a back up plan much less not having 100% redundancy so that worse case scenerio would have been a hiccup while the made the switch. I am sure they had a backup plan. It just hit problems when they had to pull the trigger. A friend of mine does "disaster recovery" consulting and the biggest problem companies have is that they never bite the bullet and actually test on a scheduled basis that their recovery plan will work when needed. They do "simulated" recoveries because they are a 24x7 business. Top mgt just refuses to schedule a disruptive test of a recovery plan. And that is the only way to know if it will work. The alternative, and this gets very expensive and is used only by the biggest companies, is to run a load balanced dual or more system where if one data center goes down, all transactions are automatically processed by the other data centers. And those data centers have the spare processing power to handle the load. | |
|
| | | | | |  Noah VailOh God please no.
Premium Member
join:2004-12-10
SouthAmerica |
Re: Apparently they set up a Twitter account today....see belowsaid by FFH5:A friend of mine does "disaster recovery" consulting and the biggest problem companies have is that they never bite the bullet and actually test on a scheduled basis that their recovery plan will work when needed. They do "simulated" recoveries because they are a 24x7 business. Top mgt just refuses to schedule a disruptive test of a recovery plan. And that is the only way to know if it will work. That's very true. My scale of employees/site is only one or two hundred and it's no different here. I can't even disconnect main power to check the life of battery backups. No one wants the disruption during the day or to pay me to come in overnight. It's pretty sad, considering an outage costs them 50k+/hr in lost revenue + expenses to put everything right again. NV | |
|
| | | | | DC DSLThere's a reason I'm Command.
Premium Member
join:2000-07-30
Washington, DC Actiontec GT784WN
1 recommendation |
to cabana
said by cabana:On a serious note - I hope this does not hurt the businesses that depend on their services too much Oh, it's gonna hurt a lot of businesses...especially mom and pop ones. I just went downstairs to pick up some dry cleaning and a loaf of bread from the little supermarket in my building. Neither of them can process card transactions. There was a woman ahead of me at the valet who was freaking out because she didn't have any cash and was on her way to the airport. Fortunately for her, the owner knows her and gave her her stuff. The folks behind me couldn't get theirs, though. At the market, it's also cash-only. Same at the liquor store and three of the 4 restaurants across the street. Part of the problem is the credit card companies' doing. It used to be transactions could be processed manually by running the card through a draft imprinter, and floor limits that didn't require authorization, or calling in for authorization. No more of that for all but special arrangements (like big-dollar charges using an Amex). | |
|
| | | | |
2 recommendations |
Chester to cabana
Anon
2009-Jul-3 12:41 pm
to cabana
said by cabana:On a serious note - I hope this does not hurt the businesses that depend on their services too much Legit companies - me too. If you ask me, Karma finally swung around to Authorize.Net for being near accomplices with the RBN as the criminal's processor of choice. » Ebook websites, fraud charges, Devbill/DigitalAge/Pluto | |
|
| | | | | |
1 edit
2 recommendations |
Major accomplices of the Russian mob.i'm with Chester.
These folks deserve to go down. They are one of the largest processors of fraudulent transactions on the 'net. | |
|
| | | | | | |  pcdebbbirdbrain
Premium Member
join:2000-12-03
Brandon, FL
|
pcdebb
Premium Member
2009-Jul-4 2:54 am
Re: Major accomplices of the Russian mob.said by K Patterson:i'm with Chester. These folks deserve to go down. They are one of the largest processors of fraudulent transactions on the 'net. this is probably a blessing nobody else is seeing like we do. | |
|
| | | | |
JBeck to Killa200
Anon
2009-Jul-3 2:38 pm
to Killa200
I know GoEmerchant.com has an Authorize.net Emulator as well. I recommend using them because I've had great experience with their customer support. | |
|
| |
tmyers
Anon
2009-Jul-3 12:10 pm
authorize.netWe have been unable to contact our rep so it is good to know someone can. | |
|
JJV
Premium Member
join:2001-04-25
Seattle, WA |
JJV
Premium Member
2009-Jul-3 1:00 pm
Breaking NEWS!!Its Fisher Plaza that had the fire.
My KOMO TV local station. Fisher Broadcasting is out.
So is their web page.
www.komotv.com
From Seattletimes.com
Fire disrupts stations at Seattle's Fisher Plaza
A fire at Fisher Plaza disrupted stations that broadcast from the building near the Seattle Center.
SEATTLE —
A fire at Fisher Plaza disrupted stations that broadcast from the building near the Seattle Center.
KOMO Radio reports the fire in an electrical vault about 11 p.m. Thursday forced the evacuation of the building.
KOMO Radio is broadcasting Friday from a building at its transmitter on Queen Anne hill in Seattle, and KOMO TV station is broadcasting morning segments from a truck in a park. | |
|
| | JJV |
JJV
Premium Member
2009-Jul-3 1:53 pm
Re: Breaking NEWS!!KOMO Radio is reporting the fire started in the parking garage and the lobby area.
Safety features have shut off back up power.
They confirm the fire has disrupted credit card processing. | |
|
| | |
| |
Susan E
Anon
2009-Jul-3 1:29 pm
Authorize.netThanks, Authorize.net, for sending your customers an email to let them know there was a system-wide failure. Our office is closed today for the holiday. Fortunately, our e-commerce provider advised us what was going on.
The Authorize.net seal that appears throughout our website was causing our pages to load very slowly. To solve this problem, I disabled the seal. As far as the other problems go, I updated my payment settings so a credit card authorization or capture isn't automatic. So far this seems to be working. We'll just have to manually charge transactions through the gateway... when it's finally up and running. | |
|
 screavic4
Premium Member
join:2006-08-11
Paron, AR |
I Knew..I knew there was a reason I left them... oh that and the high monthly fee considering I'm a startup with very little business. So far I have used » www.cdgcommerce.com/ and I love it, the customer service is great and I usually get a response in less than 8 hours. I get alot for $10 a month compared to the $40 I was paying at Authorize.net cause I had to find a "reseller" | |
|
1 edit |
45071419 (banned)
Member
2009-Jul-3 1:46 pm
Doh!Just got an on call page about 10 min ago: "Users can not access Authorize.net, all other site working fine". LoL. Thanks for the heads up DSLR. Me: "TEh wesite is down "Thanks for the Twitter link too. EDIT: It's funny as well that this isn't even front page news on msnbc.com, where I get my news. Their "breaking news" is some shit about Wimbledon. No wonder traditional media is fading into obscurity. DSLR/blogging ftw, traditional media: fail. If someone uses the term "blogosphere" though, I will have to punch you directly in the head. | |
|
 ggma1126GGMA1126
Premium Member
join:2008-08-30
Claymont, DE |
ggma1126
Premium Member
2009-Jul-3 3:09 pm
site is up now but............Site seems to be up as I was able to get into my merchant account (tho it was slow loading) so hopefully its working again | |
|
|
FFH5
Premium Member
join:2002-03-03
Tavistock NJ
4 edits |
FFH5
Premium Member
2009-Jul-3 4:00 pm
News item says Seattle fire hit MS's Bing Travel svc too» news.cnet.com/8301-1023_ ··· 1_3-0-20A fire at downtown Seattle's Fisher Plaza has interrupted service at some top Web sites, including Microsoft's Bing Travel and Authorize.net.
It's a little unclear exactly when the fire at the Fisher Plaza data center started--some reports say Thursday night, others say just after midnight Friday. A message on Bing Travel pins the problems on a blown transformer.
"The blown transformer knocked out power to the entire building, which is home to the Bing Travel servers," a message on the site says. "This is isolated to Bing Travel only, and there is no impact to any other aspect of Bing."
Verizon Communications spokesman Jon Davies said the fire also temporarily disrupted Verizon's Seattle-area DSL service. About 50,000 customers in Oregon and Washington lost Internet connectivity, Davies told TechFlash.
Ongoing updates on status can be found here: » www.techflash.com/micros ··· commentsSee here for a list of web sites affected by this fire: » blog.kylemulka.com/2009/ ··· er-fire/ | |
|
| |
spleen
Anon
2009-Jul-3 4:04 pm
I love a good outageIt brings out all the "I could do better, look at me, look at me" types.
The only fact we have is that there was a fire in the building, and service seems to be out in many areas.
That's it.
And already we have all the wannabe "experts" crawling out of the woodwork accusing the company of not spending enough money on disaster recovery planning, not having proper backup links, etc, etc!
Sorry folks, but it only shows how naive you are! (speaking only to those guilty of it here - you know who you are)
A.) You've just proven that you're the type that is so fixated on being a star that you'll make assumptions to fill in facts that you need to get before proceeding. That means you're dangerous during any kind of emergency, and the type that needs to be kicked right off any disaster team.
B.) You speak in absolutes such as this "never should have happened" which only shows you're too limited in experience to understand that, while planning for a disaster, there is no such thing as "never should have happened" - contradicting the very mistake you're accusing the company of!!!!
Bottom line - until you KNOW what happened, stop trashing Authorized.net just to make you feel good about yourself! It does no good, and just makes you look foolish. | |
|
Lazlow
join:2006-08-07
Saint Louis, MO |
Lazlow
Member
2009-Jul-3 6:06 pm
Potential targetCan you imagine the chaos that would occur if someone organized an attack on the top three competitors in this market? I asked the checkers at Walmart and the grocery store today, both said that most purchases right now are on credit cards(not debit). | |
|
 tshirt
Premium Member
join:2004-07-11
Snohomish, WA
2 edits |
tshirt
Premium Member
2009-Jul-4 12:44 am
Local story here.... eletrical fire in 1 tranformer bank, damage the transfer switch area, AND the sprinkers flooded/damaged the backup generators/ controls. Fisher Plaza IS actually a recent state of the art facility (a little scary, where the state of the art has ended up ) the building house a major data center AND KOMO-4 tv/broadcasting (ABC)(Fisher is the parent company). Once we saw this story, we switched to KOMO to watch the funniest new broadcast ever.. Hosted on the roof garden/heliport deck, (not enough portable power for studio lighting/cooling) anchors complaining "it's too hot/bright out here" which was explained by the weather guy, with a big poster paper/easel/magic marker forecast. . also obiviousley using backup equipment/ cameras/etc. colors were way off, some items were stretched or squashed to fit the widescreen, wrong video multiple times. Overall more interesting then the news itself. the 11 pm newscast should be equally good/bad. BTW KOMO website stayed up/worked as normal, obviiousley not single hosted there, if hosted there at all. ( komo-TV guys take note ; DON"T PUT ALL YOUR EQUIPMENT IN ONE BUILDING no matter who's name is on it ) BTW2~~ this is the second fire causing outages there in the last year | |
|
|
••• |
Stumbles
join:2002-12-17
Port Saint Lucie, FL |
Have some baloney.Oh that's just a complete load of bull. Blaming the "frailaty of the internet" on a company's lack of a disaster recovery plan. That's so last century. In this "enlightened" age of information management, and a company does not have that kind of plan in place puts the blame on the company. Droll author he is. | |
|
|
|
I hope they have backups haha(yea i they do, duh).
