871 stops working

[sebastiasebas]

We have a Cisco 871W working as PIX, controlling external VPN connections to our private network.

The configuration is very similar to the one described here - "Cisco Router as a Remote VPN Server using SDM Configuration Example" :

>>> http://www.cisco.com/en/US/products/sw/secursw/ps5318/products_configuration_example09186a00806ad10e.shtml

About once a week, it stops working, without aparent external intervention.

The ping to the external IP stops answering, and the internal IP stops answering pings also.

The solution is power-off and power-on, and it starts working again ...

What is the correct way to debug this situation ?

I can connect a HyperTerminal using direct cable to the 871, but dont know the relevant commands to debug this situation.

Is there any LOG I can have a look into ? Yes, I am quite new to Cisco world ...

Any sugestions ?

By the way : "show version" says ...

Cisco IOS Software, C870 Software (C870-ADVSECURITYK9-M), Version 12.4(15)T7, RELEASE SOFTWARE (fc3)

Sebastian.

[cadet alain]

Hi,

I would use a syslog server( like Kiwi or tftpd) and log everything to it:

enable

configure terminal

logging x.x.x.x  where x.x.x.x is ip address of syslog server

Can you also post running config:

enable

show run

then copy-paste and post as a file

Regards.

Alain.

[sebastiasebas]

Hi, Alain. Thanks for your support.

I will try to setup an external Log Server, but in my opinion, when the error comes up, all IP traffic gets blocked, so nothing will reach the remote log ...

Next Monday I promise to paste the running-config, as I am already home and a large weekend in front of all the family ... (:-))

I was wondering if there is some king of internal Log in the 871w ... and how to configure it and display its contents ... Shall try to use Google anyway.

Allow me to ask for some more guidance ... What commands can I use to display the interface status (and what values shall NOT be there) ?

Or, in other words ... what commands would you use if your 871 were to block once every week (more or less) ?

I am thinking of using the HyperTerminal, as (again) I think the external Putty access will be broken (on error state).

Cheers. Sebastian.

[cadet alain]

I was wondering if there is some king of internal Log in the 871w

yes you can log to console, to terminal or in a buffer: to log everything to buffer just issue log buffered debug command then do a show log to verify it is enabled and sent to buffer.To see the logs just issue the same sh log command.

What commands can I use to display the interface status

show interface x/x and sh controllers x/x

I am thinking of using the HyperTerminal, as (again) I think the external Putty access will be broken (on error state).

You can use Putty for console logging too.

Regards.

Alain.

[garapoglou]

Hi,

Alain is right about the commands showing the interfaces' and controllers' status. When you issue the show interface command you definely get a lot of interesting information along with any errors occured.

I would also check the flash memory, using the show flash: command to check whether there are hardware problems. I've seen similar problems on various routers and when checked the flash memory, I've found lots of crash info files.

Best regards,

Giorgos

[sebastiasebas]

ok - the commands I shall use next time I have problems are ...

*) show interfaces

*) showcontrollers

*) show processes cpu

*) show flash

Thanks !

[hobbe]

Why not use show tech

This is a command that you use when you want help from the Cisco TAC, it gives yuo the output of a lot of different commands.

Sure you will get a lot of extra information but afaik that is not a bad thing.

Good luck

HTH

[sebastiasebas]

Yes, man - a LARGE output .... THANKS ! -> will use it (with care  _

[sebastiasebas]

Giorgos : what is "crash into files" ?

My flash looks like this :

==========================================================

871-403#show flash
28672K bytes of processor board System flash (Intel Strataflash)

Directory of flash:/

    2  -rwx    16417632   --- -- ---- --:--:-- -----  c870-advsecurityk9-mz.124-15.T7.bin
    3  -rwx        3179   Mar 1 2002 00:04:25 +00:00  sdmconfig-8xx.cfg
    4  -rwx      931840   Mar 1 2002 00:04:44 +00:00  es.tar
    5  -rwx     1505280   Mar 1 2002 00:05:08 +00:00  common.tar
    6  -rwx        1038   Mar 1 2002 00:05:19 +00:00  home.shtml
    7  -rwx      112640   Mar 1 2002 00:05:30 +00:00  home.tar
    8  -rwx     2242560   Mar 1 2002 00:06:01 +00:00  wlanui.tar
    9  -rwx         600  Nov 11 2009 12:03:14 +00:00  vlan.dat
   10  -rwx        4849  Nov 11 2009 12:36:53 +00:00  stored-config
   11  -rwx        5649  Sep 20 2010 16:54:56 +00:00  SDM_Backup

27611136 bytes total (6375424 bytes free)

==========================================================

Seems ok to me ...

[garapoglou]

Hi Sebastia,

When a Cisco device crashes valuable information is written in a file called crashinfo in the flash memory. These files contain information and error messages for troubleshooting.

Since your router's flash memory does not contain such files, it probably means there's no data or stack corruption problems.

Giorgos

[sebastiasebas]

Giorgos : what fields are indicating problems ?

  Input errors ?

  Large queue size ??

I am not too sure of what to do if I find "late collision" errors are high .... jejeje

My display is this one :

===========================================================

871-403#show interfaces FastEthernet4
FastEthernet4 is up, line protocol is up
  Hardware is PQUICC_FEC, address is 0025.45e6.686d (bia 0025.45e6.686d)
  Description: $ETH-WAN$
  Internet address is 213.229.144.194/28
  MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, loopback not set
  Keepalive set (10 sec)
  Full-duplex, 100Mb/s, 100BaseTX/FX
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input 00:00:01, output 00:00:00, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 9000 bits/sec, 10 packets/sec
  5 minute output rate 13000 bits/sec, 11 packets/sec
     259854 packets input, 86221296 bytes
     Received 4685 broadcasts, 0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog
     0 input packets with dribble condition detected
     318308 packets output, 47326786 bytes, 0 underruns
     0 output errors, 0 collisions, 2 interface resets
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier
     0 output buffer failures, 0 output buffers swapped out

[garapoglou]

The only issue reported by the sh int fa4 command, are 2 interface resets, caused by the other side's high speed and can't be handled by the device.

Giorgos

[sebastiasebas]

Good evening ! Hope weekend has been sweet to you all ...

Here is the promised "show running-config" output :

========================================================================================

Current configuration : 7833 bytes
!
version 12.4
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname 871-403
!
boot-start-marker
boot-end-marker
!
logging buffered 51200 warnings
enable secret 5 $1$SWJI$iiRtZ8u26/deGCyfMd/nk0
!
aaa new-model
!
!
aaa authentication login default local
aaa authentication login sdm_vpn_xauth_ml_1 local
aaa authentication login sdm_vpn_xauth_ml_2 local
aaa authorization exec default local
aaa authorization network sdm_vpn_group_ml_1 local
aaa authorization network sdm_vpn_group_ml_2 local
!
!
aaa session-id common
!
crypto pki trustpoint TP-self-signed-57375051
enrollment selfsigned
subject-name cn=IOS-Self-Signed-Certificate-57375051
revocation-check none
rsakeypair TP-self-signed-57375051
!
!
crypto pki certificate chain TP-self-signed-57375051
certificate self-signed 01
  30820243 308201AC A0030201 02020101 300D0609 2A864886 F70D0101 04050030
  2F312D30 2B060355 04031324 494F532D 53656C66 2D536967 6E65642D 43657274
  69666963 6174652D 35373337 35303531 301E170D 31303039 32333132 32303236
  5A170D32 30303130 31303030 3030305A 302F312D 302B0603 55040313 24494F53
  2D53656C 662D5369 676E6564 2D436572 74696669 63617465 2D353733 37353035
  3130819F 300D0609 2A864886 F70D0101 01050003 818D0030 81890281 8100B58C
  7314ED46 F411F8BF 2F692F9F 5D33842D 3F2A96BF BD84F16E 0959B0F7 BB69E1E2
  95B8F100 D6DAEB2A 76FE5019 D0098BE1 C7391B8A 1ABC12C6 D74188CD C8E4F405
  B1996A61 20349E80 4768E50B DDB64EDA A12E368C 96025F4C 214B70F3 893F6BF0
  10379F72 DE4BB316 C1A0411F C13281C2 07C6ABFD A7656D78 9CB433C7 02910203
  010001A3 6F306D30 0F060355 1D130101 FF040530 030101FF 301A0603 551D1104
  13301182 0F383731 2D343033 2E626973 632E6573 301F0603 551D2304 18301680
  14736AD3 0AF32212 BEBD37F8 4564A13C 4D1E8DFA 86301D06 03551D0E 04160414
  736AD30A F32212BE BD37F845 64A13C4D 1E8DFA86 300D0609 2A864886 F70D0101
  04050003 81810013 F12C49EE 58DABEB0 CC90BD59 9339960E ECD1B593 9A0B0C25
  F838C525 F14B5813 44E6D3F5 923A6DFE 81BF2897 06B8D392 8A2D860A 1563F01B
  DBD1A77C 150E7AA4 A3A34140 4387BDAB 97EFDF48 002D42F8 B2C66997 48B28D68
  15819E32 AA65D038 4C404002 0AAA1D1E 7C11A14A 8BB9F2FA 361B5F18 2F85974D
  AB3431F7 318891
        quit
dot11 syslog
ip cef
no ip dhcp use vrf connected
ip dhcp excluded-address 10.10.10.1
ip dhcp excluded-address 192.168.78.1 192.168.78.79
ip dhcp excluded-address 192.168.78.100 192.168.78.254
!
ip dhcp pool sdm-pool
   import all
   network 192.168.78.0 255.255.255.0
   default-router 192.168.78.2
   domain-name bisc.es
   dns-server 212.121.128.10 212.121.128.11
   lease 0 2
!
!
no ip domain lookup
ip domain name bisc.es
ip name-server 212.121.128.10
ip name-server 212.121.128.11
!
username sebas privilege 15 secret 5 $1$K0ex$k3kmCHb1YiiqQ.poQ8h.I0
username sebasColt secret 5 $1$0cdZ$oWgTR/SWvOGABK8roNFDF/
!
!
crypto isakmp policy 1
encr 3des
authentication pre-share
group 2
!
crypto isakmp client configuration group BISCtunnelGroup
key mykey
dns 212.121.128.10 212.121.128.11
domain bisc.es
pool SDM_POOL_1
acl 100
split-dns bisc.es
netmask 255.255.255.0
banner ^CXAuth Banner - ^C
crypto isakmp profile sdm-ike-profile-1
   match identity group BISCtunnelGroup
   client authentication list sdm_vpn_xauth_ml_2
   isakmp authorization list sdm_vpn_group_ml_2
   client configuration address respond
   virtual-template 2
!
crypto ipsec transform-set ESP-3DES-SHA esp-3des esp-sha-hmac
crypto ipsec transform-set ESP-3DES-SHA1 esp-3des esp-sha-hmac
!
crypto ipsec profile SDM_Profile1
set transform-set ESP-3DES-SHA1
set isakmp-profile sdm-ike-profile-1
!
!
archive
log config
  hidekeys
!
interface FastEthernet0
!
interface FastEthernet1
!
interface FastEthernet2
!
interface FastEthernet3
!
interface FastEthernet4
description $ETH-WAN$
ip address 213.229.144.194 255.255.255.240
ip nat outside
ip virtual-reassembly
duplex auto
speed auto
!
interface Virtual-Template2 type tunnel
ip unnumbered FastEthernet4
tunnel mode ipsec ipv4
tunnel protection ipsec profile SDM_Profile1
!
interface Dot11Radio0
no ip address
shutdown
speed basic-1.0 basic-2.0 basic-5.5 6.0 9.0 basic-11.0 12.0 18.0 24.0 36.0 48.0 54.0
station-role root
!
interface Vlan1
description $ETH-SW-LAUNCH$$INTF-INFO-HWIC 4ESW$
ip address 192.168.78.2 255.255.0.0
ip access-group 101 in
ip nat inside
ip virtual-reassembly
ip tcp adjust-mss 1452
!
ip local pool SDM_POOL_1 192.168.78.100 192.168.78.110
ip forward-protocol nd
ip route 0.0.0.0 0.0.0.0 213.229.144.193 permanent
!
ip http server
ip http access-class 2
ip http authentication local
ip http secure-server
ip http timeout-policy idle 60 life 86400 requests 10000
ip nat inside source list 1 interface FastEthernet4 overload
!
access-list 1 remark SDM_ACL Category=2
access-list 1 permit 192.168.78.0 0.0.0.255
access-list 1 permit 192.168.83.0 0.0.0.255
access-list 2 remark Auto generated by SDM Management Access feature
access-list 2 remark SDM_ACL Category=1
access-list 2 permit 192.168.0.0 0.0.255.255
access-list 1 permit 192.168.78.0 0.0.0.255
access-list 1 permit 192.168.83.0 0.0.0.255
access-list 2 remark Auto generated by SDM Management Access feature
access-list 2 remark SDM_ACL Category=1
access-list 2 permit 192.168.0.0 0.0.255.255
access-list 100 remark SDM_ACL Category=4
access-list 100 permit ip 192.168.78.0 0.0.0.255 any
access-list 100 permit ip 192.168.83.0 0.0.0.255 any
access-list 101 remark Auto generated by SDM Management Access feature
access-list 101 remark SDM_ACL Category=1
access-list 101 permit tcp 192.168.0.0 0.0.255.255 host 192.168.78.2 eq telnet
access-list 101 permit tcp 192.168.0.0 0.0.255.255 host 192.168.78.2 eq 22
access-list 101 permit tcp 192.168.0.0 0.0.255.255 host 192.168.78.2 eq www
access-list 101 permit tcp 192.168.0.0 0.0.255.255 host 192.168.78.2 eq 443
access-list 101 permit tcp 192.168.0.0 0.0.255.255 host 192.168.78.2 eq cmd
access-list 101 deny   tcp any host 192.168.78.2 eq telnet
access-list 101 deny   tcp any host 192.168.78.2 eq 22
access-list 101 deny   tcp any host 192.168.78.2 eq www
access-list 101 deny   tcp any host 192.168.78.2 eq 443
access-list 101 deny   tcp any host 192.168.78.2 eq cmd
access-list 101 deny   udp any host 192.168.78.2 eq snmp
access-list 101 permit ip any any
access-list 102 remark Auto generated by SDM Management Access feature
access-list 102 remark SDM_ACL Category=1
access-list 102 permit ip 192.168.0.0 0.0.255.255 any
no cdp run
!
control-plane
!
banner login ^C
-----------------------------------------------------------------------
myMOTD -
-----------------------------------------------------------------------
^C
!
line con 0
no modem enable
line aux 0
line vty 0 4
access-class 102 in
privilege level 15
transport input telnet ssh
!
scheduler max-task-time 5000
end

[paolo bevilacqua]

Update IOS.

[sebastiasebas]

It has been planned with Cisco tech for next Thursday .... jejeje

Thanks !