Cannot access UC-520 via IP

[robert]

I have a UC-520 which has be in operation for over a year. Over the weekend the system can no longer be accessed via the vlan1 interface IP.  This has resulted in not being able to access the service provider (SIP). I can connect via console and reviewed the configuration which seems to be correct.  All of the phones can connect and obtain IP and configuration.  I can make calls between internal phones but nothing in or out of the network.

I cannot access the system via telnet, CCA, browser, only console cable.  I have attempted to access it on the default 192.168.10.1 address but no luck there either.  The system has 12.4 software.  There were power issues over the weekend but the UC-520 is connected to an UPS and the Catalyst switch which is in the same rack is fine.

I am at a loss as to how to troubleshoot this and would welcome any suggestions.

Robert

[Steven DiStefano]

Is there a crashinfo file in the root of flash: ?

When you are in console, you are not in rommon since you would notice that, but there was an issue with older IOS where we did experience problems when flash was filled > 90% which caused a crash and then stuck in rommon.  The fix was to power cycle it one more time.   If you are dead in the water try that.  If you are someone operational, then the next maintenance window.

Another thing to check for is to make sure you have a file called vlan.dat on flash.  If its missng, to re-create it, type the following in enable mode

vlan  database

vlan 1

vlan 100

exit

This is serious enough to open a case I think for fastest response time.

[robert]

I did not find a crashinfo file.  The vlan.dat file was there. there is 42MB of free space on the flash card.  Is there a procedure to copy the existing flash to a larger card? I have done a reload twice with no change.  The interface vlan1 reports up and up but I cannot ping to the gateway.

I understand a case is in order but I'm trying to understand as much as I can before opening one.  Communications are usually an issue with TAC.

Robert

[Steven DiStefano]

So the good news is you didnt crash and you didnt loose your vlan database.

We do not offer larger flash on UC520.  CCA does have a Phone Load manager which can reduce the flash usage to only the phones you deploy, but thats not your problem here.

[Chuan Liu]

Hi Robert,

Can you run "debug ip packet detail" while you ping the interface vlan1  ip address?

Cheers.

[David Trad]

Hi Robert,

Firstly, do not wait to open a TAC case, the longer you wait the more you resent it later on, you can learn heaps from the TAC rep, make sure to ask them heaps of questions, and do not be afraid to ask them along the way as he/she is working on the problem... I probe them like as if I was an alien abducting them, I squeeze every bit of information I can out of them before he case is closed of.

Secondly, I am really curious about this issue, I would be very much appreciative if you could post your full running configuration (Remove all sensitive information like usernames and passwords).

I am fairly certain your issue is small in nature but huge on impact (Like the majority are sadly).

I look forward to your response.

Cheers,

David.

[robert]

I should be opening a case today as soon as I get an issue with the SMARTnet contract resolved. I promise to post the resolution once it is obtained.

Fortunately I have not had to resort to TAC for quite some time and pray that the experience will be better than those I have had in the past. It seems that I usually get someone who provides instructions which assumes that I have the same level of knowledge as the TAC engineers (in which case I would not need to seek assistance). The rest of the time I end up trying to figure if a Rosetta Stone course in Hindi would be more valuable to my business than a CCNE certification.

An interesting side note, another tenant in the building had exactly the same indications, phone system (not Cisco) was not communicating with the service provider's (same provider) equipment resulting in no incoming/outgoing service. I felt this was too much of a coincidence so I went to gather some information from them and had their system up and running in about 5 minutes.

Robert

[robert]

After four hours and two TAC engineers' diligent verification of the configuration, which I assured them had worked since 2008, we discovered that the arp table was unable to create a complete entry for MAC addresses but only for Vlan 1. Entries in Vlan 100 were fine and creation of a Vlan 2, with a different subnet was able to register and retain correct entries.

I was not offered any explanation how this was possible and given the option to move everything from Vlan 1 to Vlan 2 or replace the unit.  I requested a replacement. I still do not understand how this is a hardware issue but was happy to move forward instead of watching failed pings hour after hour.

I will reserve comment on my experience with TAC but will say it was consistent with prior experiences.

I will post further developments after I receive the replacement and have it in place.

Robert

[Steven DiStefano]

Yousa!!         I agree on the RMA.  Let us know!

[robert]

Well the problem was not hardware.  I installed the new UC-520 and transferred the configuration, same problem.

Finally, the bridging entries were removed from the following to restore access to vlan 1:

interface Vlan 1

   description $FW_INSIDE$

   ip address x.x.x.x 255.255.255.0

   ip nat inside

   ip virtual-reassembly

  bridge-group 1

   bridge-group 1 spanning-disabled

No one can explain why this entry was causing the problem, why it worked in this configuration since 8/2008 (and suddenly quit following a power outage), nor why it still works in the configuration of the other two UC-520s in this organization's WAN.

I spent 18 hours on the phone with four different Cisco engineers and the client was without phone service for 5 days.

Robert

[David Hornstein]

Hi robert,

Sure looks like from your posting that you combined  a UC520W  configuration  with that of a UC520 (no wireless or bridging )  configuration.  If your system is a UC520  and not a UC520W  the bridging commands in red are not supposed to be there.  Get rid of them.

Sure would be good to see the Incident number, maybe you can post it or email it to me so I can see what 18hrs of TAC support did. My email address is dhornste@cisco.com.

You need to get TCP connectivity to the UC520 otherwise the next steps wont work. I have attached a default configuration of a 8 user UC520  and a 16 user UC520W that you can use as  to default your machines CUE module If needed.  But we'll wait before using them. Firstly I'd like to try the following first.

step 1. Do a search, via the console port  and CLI  in enable mode of the flash for configuration files

show flash | i cfg

Step 2.  If you have one cfg file copy it to the startup configuration.

copy thename.cfg  start

step 3.  Then reload the system with the following command;

reload

if it prompts you to save the configuration, don't take that option and select 'no'

i'm hoping you have a TCP connection after you have reloaded the system.

lets see how this goes.  This is basically a first step or approach.  There are many ohers we can try, but this a possible first step.

regards Dave

[Steven DiStefano]

Yep, I didnt see Dave's reply, when I saw your RSS post to my email, and checked the same thing. 

Thats the first thing that came to mind.  Self inflicted wound?

You want to be sure you use the correct factory default, which matched your SKU....

CCA would put the correct factory default on the box, if you start with it.

Wireless factory config snip

interface Vlan1
no ip address
bridge-group 1
bridge-group 1 spanning-disabled

!
interface Vlan100
no ip address
bridge-group 100
bridge-group 100 spanning-disabled

Non Wireless factoru config snip

interface Vlan1
description $FW_INSIDE$
ip address 192.168.10.1 255.255.255.0
ip access-group 102 in
ip nat inside
ip virtual-reassembly
!
interface Vlan100
description $FW_INSIDE$
ip address 10.1.1.1 255.255.255.0
ip access-group 103 in
ip nat inside
ip virtual-reassembly

[robert]

Guys,

Thanks for the replies but I want to make sure that a few items are clear. First is that the system is now working, the removal of the bridging entries caused the Vlan 1 IP to be able to communicate with the rest of the network.

The unit is a UC520W-16U-4FXO-K9 which has wireless built in, the original configuration had been working since August of 2008, the only significant event I have been able to identify was that the system shutdown due to a power outage and once the power returned the IP assigned to Vlan 1 was not available on the network, resulting in an inability to access the SIP telephone service provider.

While I would like to understand what caused the problem, in the interest of my client I am not going to fix something which is not broken, at this time.  Once the system has proven to be stable I will schedule preventative maintenance such as the latest software pack.

I posted the results of my issue to bring closure should someone else have a similar problem.

Robert