Cisco → 1841 hangs loading IOS 15.1.4M adv. IP services

c1841 platform with 393216 Kbytes of main memory
System Bootstrap, Version 12.4(13r)T5, RELEASE SOFTWARE (fc1)

I am currently running
Cisco IOS Software, 1841 Software (C1841-ADVENTERPRISEK9-M), Version 12.4(25g), RELEASE SOFTWARE ()

but want to upg to IOS 15:
c1841-advipservicesk9-mz.151-4.M5.bin
and the run CCPExpress Admin... but that's another issue.

I have a newly formatted 64M Cisco CF card, formatted by the 1841. I put the IOS bin file on it, and did a #reload to get the new version.

The router starts up properly, but then hangs just near the point I expect to see "Press enter to start" on the console. At that point, no console input is accepted, ssh to the router doesn't work etc.

Then I put in my old 256M San Disk CF with the older IOS, and everything works great.

Below is a cut/past from my console session while trying to upgrade IOS.

The one difference I notice between this and a working IOS, is with the working IOS I get an "[OK]" at the end when it's looking up a dns name.

The config has not changed between the two, but I'm thinking that something in the configuration is cause IOS 15 to choke. Any suggestions?

Router>enable
Router#dir
Directory of flash:/

1 -rw- 45716732 Dec 30 2012 17:56:36 -08:00 c1841-advipservicesk9-mz.151-4.M5.bin

63864832 bytes total (18145280 bytes free)
Router#reload
Proceed with reload? [confirm]

System Bootstrap, Version 12.4(13r)T5, RELEASE SOFTWARE (fc1)
Technical Support: »www.cisco.com/techsupport
Copyright (c) 2007 by cisco Systems, Inc.
PLD version 0x10
GIO ASIC version 0x127
c1841 platform with 393216 Kbytes of main memory
Main memory is configured to 64 bit mode with parity disabled

Upgrade ROMMON initialized
program load complete, entry point: 0x8000f000, size: 0xcb80
program load complete, entry point: 0x8000f000, size: 0xcb80

program load complete, entry point: 0x8000f000, size: 0x2b9935c
Self decompressing the image : ###################################################################]

IOMEM set to: 15
PMem allocated: 341835776 bytes; IOMem allocated: 60817408 bytes

Restricted Rights Legend

Use, duplication, or disclosure by the Government is
subject to restrictions as set forth in subparagraph
(c) of the Commercial Computer Software - Restricted
Rights clause at FAR sec. 52.227-19 and subparagraph
(c) (1) (ii) of the Rights in Technical Data and Computer
Software clause at DFARS sec. 252.227-7013.

cisco Systems, Inc.
170 West Tasman Drive
San Jose, California 95134-1706

Cisco IOS Software, 1841 Software (C1841-ADVIPSERVICESK9-M), Version 15.1(4)M5, RELEASE SOFTWARE ()
Technical Support: »www.cisco.com/techsupport
Copyright (c) 1986-2012 by Cisco Systems, Inc.
Compiled Tue 04-Sep-12 15:18 by prod_rel_team
This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
»www.cisco.com/wwl/export ··· qrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

Installed image archive
Cisco 1841 (revision 7.0) with 333824K/59392K bytes of memory.
Processor board ID
2 FastEthernet interfaces
1 Virtual Private Network (VPN) Module
DRAM configuration is 64 bits wide with parity disabled.
191K bytes of NVRAM.
62592K bytes of ATA CompactFlash (Read/Write)

Logging of %SNMP-3-AUTHFAIL is enabled

Translating "members.dyndns.org"...domain server (8.8.8.8)

--- end of console ---
Normally after that "Translaing..." text I see [OK] after the IP address, and I get the "Press enter to start".

Wait for the DNS lookup to fail, or remove the dyndns configuration. It looks like it's trying DDNS before any links are up.

Change your config register to 0x2142 to bypass the running config.

Also I make it a habit of doing a "verify flash:[file] /md5" BEFORE doing any sort of router reboot.
It's a good sanity check to make sure the IOS image went over cleanly BEFORE you find out you bricked it,
though based on your provided output, doesn't look like the IOS copied over badly.

Dumb question, does your config have a "no ip domain lookup" in it? Almost willing to bet it doesn't
based on the following :

Translating "members.dyndns.org"...domain server (8.8.8.8)
 

Regards

Terminal in using 9600/8-N-1 with no flow control
Back-up your current configuration
Set config-reg to 0x2102
Do a 'write erase' then 'reload' the router while on the working IOS
After the reload, upgrade to the new IOS
If you get the same symptoms as before use a different IOS

Thanks everybody, for the suggestions.... Sorry I was so slow getting back... the "somebody added a comment" notices went to my junk folder :-(

I'll try these in the next few days and let you know how it worked out.

Cheers.

Yes, but the lookup never timed out... I waited 20 minutes or more to see if it would continue at some point.

And, the previous IOS had no issue... so I'm thinking either the new IOS is flakey, or there's something in my config that IOS 15.1.4 doesn't like, and can't reject nicely and continue on. (Either way, sounds like an IOS bug to me

Solved (or worked around)

This setting was in effect in my config file...

scheduler heapcheck process memory processor io


Turning that off (#no scheduler ...) solved the problem/masked the symptom. The router came up fine with IOS 15.1.4...

If Cisco agrees this is a bug, I'll post the bug number.

In one of the earlier replies, there was an implied suggestion to turn off ip domain lookup... but if I do that, don't I have to code the actual IP address in my config file for ntp servers,members.dyns.org, etc? Granted those may not change often, but the whole point of DNS is that unless the name changes, I don't have to give it another thought.

But, I also read something about being able to do some sort of telnet reverse dns attack... I'd favor security over dns/ip convenience.

I'm going to go through my whole config file and see if I find anything "interesting" :-)

Thanks for the suggestions and help...
Cheers

IMO this is NOT a bug.
Why did you even enable the heapcheck command in the first place?

As per Cisco....."This feature should be used only as a last resort after all other possible methods to detect processes that are corrupting the chunks have failed".

1) So what exactly were you troubleshooting to warrant enabling the command.
2)I would consider it a bug if after enabling the command the required output was not generated by the router, but NOT because you could NOT successfully upgrade
3)As per my previous post write, erase & reload if having difficulty loading a 'problem' IOS

All this command does is any CLI input it can't parse as an IOS command, it assumes it's a DNS name and
tries to resolve it accordingly. I usually turn this off and never have had an issue with it.

If you can, post up your config, minus IP addresses, passwords, etc. if you want a 2nd set of eyes on this.

Regards

Actually (at least in 12.4), it stops the router from doing *any* DNS lookups -- command line, configuration, and even it's own internal DNS server. (why you'd want to use IOS as a DNS server is left to your own insanity.)