dslreports logo
site
 
    All Forums Hot Topics Gallery
spc

spacer




how-to block ads


Search Topic:
uniqs
589
share rss forum feed


battleop

join:2005-09-28
00000

[OT] Juniper Password Recovery

I know this is a Cisco forum but this is probably the best place to find others who have Juniper experience.

I've got a situation where a customer's has been partially down for 48 hours. They have called us in to help get their network back up. Everyone that has looked at it all agree that the problem is with their Juniper firewall. My understanding is that the only person with the password is either refusing to give it up or they don't have it.

I can't find any procedure online that would get me into the box even on site that does not require a complete wipe of the network. The former admins have done a very poor job of documenting the network which is at least a /16 or larger with at least a dozen or more remote sites.

Is there anyway to get back into the box or is this like Sonicwall where they give a disgruntled admin the upper hand by not allowing any kind of recovery procedure?
--
I do not, have not, and will not work for AT&T/Comcast/Verizon/Charter or similar sized company.

cramer
Premium
join:2007-04-10
Raleigh, NC
kudos:9

1 recommendation

The SRX can have the root password reset. [LINK]

Netscreens and the like can only be erased.


battleop

join:2005-09-28
00000
"Netscreens and the like can only be erased."

What the hell were they thinking? How hard would it be to create a challenge response system that cold be used after establishing that the request was valid? Even with the best documentation its still easy for a disgruntled admin to hold a company hostage. Simply delete the backups, change the password and leave.
--
I do not, have not, and will not work for AT&T/Comcast/Verizon/Charter or similar sized company.


TomS_
Git-r-done
Premium,MVM
join:2002-07-19
London, UK
kudos:5
Thats why youre meant to have regular backups, backed up on to more permanent media, stored in off-site locations. Something like a daily, weekly, and monthly backup so at the very worst you need to recover data that is up to 1 month old.

Of course, what you should do and what you actually do are two completely different things.

cramer
Premium
join:2007-04-10
Raleigh, NC
kudos:9
reply to battleop
They're thinking, CORRECTLY, that it's a security device. It's not very secure if some random person can get at its secrets -- certificates, passwords, service information, etc., etc.

If you put all of your eggs in one basket, you should not be upset when a MAC truck backs over it. It is a World Standard Practice to not put your network in a position where it can be held hostage by a single admin. That would mean having more than one person with some measure of understanding of the network, more than one person with the "keys" to the network, (multiple) archives of current and past configurations. All of this is Disaster Recovery 101... always have enough information at hand to rebuild your network should (1) a pissed admin change the passwords and remove everyone's access, or (2) an "act of God" (fire, lightning, flood, etc.) damages and/or destroys equipment, or (3) thieves walk off with your hardware.


battleop

join:2005-09-28
00000
It's apparent that you did not read or understand this not my network. I was just now called in to help get their network backup so I had no control over any policies prior to the phone call I got yesterday.

"It's not very secure if some random person can get at its secrets -- certificates, passwords, service information, etc., etc."

You are making the assumption that they can just stroll into to a data center, into a locked cage, and then a locked cabinet and then walk right back out. A challenge / response system that can be used after it's been established that the person requesting a reset is legit would prevent Joe Schmoe from gaining such access.

"All of this is Disaster Recovery 101"

Very true in the perfect IT world where a company motto isn't "Do more, for less, with less."
--
I do not, have not, and will not work for AT&T/Comcast/Verizon/Charter or similar sized company.


battleop

join:2005-09-28
00000
reply to TomS_
We keep an onsite backup, local (same city) backup, and out of town 150 miles away backup. I really like our Rancid / CVS servers that polls all of our core equipment and remote equipment once an hour. It's saved my ass more than once.

The restoration time on any device in my network is the amount of time it takes to get someone on site. All someone has to do is put a very bare minimum config into the box and I can have it back up and running in a few minutes.
--
I do not, have not, and will not work for AT&T/Comcast/Verizon/Charter or similar sized company.

cramer
Premium
join:2007-04-10
Raleigh, NC
kudos:9

1 edit
reply to battleop
So, you've been called in to repair a problem that "cannot" be fixed. As a consultant, you should love this as it's a huge pile of billable hours.

You're making the assumption no one can get to the device (or that it won't end up on eBay with your secrets still in it.) Juniper is not making that mistake.

If you think things don't walk out of "secured" data centers, you'd be dead wrong. Look up the incident at CI Hosting in Chicago... thieves cut right through the wall. (I've not seen a colo cage that could stop the janitor much less a determined thief.) This is all moot here as we aren't talking about a high security area, or a firewall with any serious secrets. However, they're using a device from people who take security seriously. How, exactly, is Juniper supposed to authenticate you as someone authorized to recover the device? (don't answer that)

Saving a copy of the config somewhere, at least once, isn't a million dollar IT process. (heck, I have routers that archive changes automatically, even if I don't save them.) If you only have one admin, then you are forever at his mercy. What do you do when he gets hit by a bus?

[Also, this back'n'forth isn't going to get you anywhere. It is the way it is... no password, no config. Go whine to Juniper, but don't expect them to change. BTW, cisco is the same way if you turn off password recovery. (however, I can get past that with a soldering iron)]


battleop

join:2005-09-28
00000
"is Juniper suppose to authenticate you as someone authorized to recover the device?"

There would have to be a paper trail from Juniper to the End user and then an officer in the company would have to prove it was their router to begin with. Maybe you do it as part of a smartnet type contract. There are ways to do this.
--
I do not, have not, and will not work for AT&T/Comcast/Verizon/Charter or similar sized company.

HELLFIRE
Premium
join:2009-11-25
kudos:19
reply to battleop
To inject my 00000010bits into this :

Honestly anytime I see any enterprise level gear with a "hardware reset" button, alittle part of me dies inside
thinking "how hard is it to come in with a paper clip or #2 pencil and screw someone over?" But as rightly pointed
out already, that's a DESIGN issue to take up with the vendor.

As for the challenge-response system, to be fair the caveat is HOW to PERFECTLY prove you're who you say you are.
And given paperwork is the LAST thing that's up to date with ANY equipment your company has... I've lost count the
number of times I've come across a serial # that didn't have an entitlement issue SOMEwhere along the line, and
I'm sure all of us have stories of serial # / entitlement issues with any vendor.

Take it for what it is. Back to your situation battleop See Profile, where's things stand on restoring the client's infrastructure?
Did you get it all sorted out in the end?

Regards


battleop

join:2005-09-28
00000
The problems are much much deeper than just the firewall. I started with an extremely out of date spread sheet and access to their domain controller. From there I've spent a lot of time working with nmap and wireshark. It's taken the better part of the weekend breaking into devices and building a network map of what they have.

I was told that somewhere in the network there are backups to the Juniper but no one knows where. There are about 50 VMs out there and a large SAN. It could be on a desktop or a VM.

I think what may have happened is that they had a couple of in house IT guys who were either laid off or fired or left for a better job during bad economic times. They then outsourced everything to a small IT shop and they got lucky in that they never had any real issues come up. Fast forward a year or two and a real outage happened and there was no one around who knew about the network.

At least they are on the north side of Atlanta so it's not that bad of a drive to get there.
--
I do not, have not, and will not work for AT&T/Comcast/Verizon/Charter or similar sized company.

HELLFIRE
Premium
join:2009-11-25
kudos:19
reply to battleop
So a case of Murphy's Law, plus "translates roughly as 'for some other poor schmuck to keep up to date'" (read:documentation),
plus a healthy "feed IT as little as possible and hope it goes away" attitude from upper management, mixed in with "if it's still
green, okay". Gotta love that...

....so how many extra zeros on the bill are you adding on?

Regards


TomS_
Git-r-done
Premium,MVM
join:2002-07-19
London, UK
kudos:5
Zeros? Why not nines?

Network Guy
Premium
join:2000-08-25
New York
kudos:2
reply to battleop
This sounds like an excellent upcoming payday for you.

Have fun and good luck


battleop

join:2005-09-28
00000
reply to HELLFIRE
"feed IT as little as possible and hope it goes away" attitude from upper management,

You have no idea how common this is. The problem is that good techs in non IT companies are rarely compensated fairly. If they are good at their job and everything works like it should management thinks they are not doing anything. Eventually when they don't feel that they are doing anything they get replaced by some slick talking sales guy who convinced management that they are paying their IT staff too much and would save money by outsourcing. When this happens the good tech leaves along with the fine details of the network and the tech from the new company starts without a full view of everything.

It's funny how much more secure a poor tech's job is because management thinks that's how things work and they couldn't imagine what it would be like without the tech on site.
--
I do not, have not, and will not work for AT&T/Comcast/Verizon/Charter or similar sized company.