Unable to ping until arp cache cleared

[samirshaikh52]

Hello Experts

I have 3 servers connected to Cisco Catalyst Switch  C2960 and this switch has uplink to one of the access switch and  ultimately this access switch connected to our 2 Core Switch

We are running HSRP and core switches has direct link between them.

Today  I encountered an issue these server are unreachable from vlans other  that its own.  I just cleared arp-cache and it started pinging.

Please can you help since this happening repeadetly.

Thanks

[JohnTylerPearce]

Can you verify that your topology for this issue looks like this?

(3 Servers)<--->(2960)<--->(Access Switch)<---->(the two core switches)

How did you discover that you were unable to ping the servers from outside their local vlan?

[samirshaikh52]

They medical application server and the users from other vlan suddenly were unable to access the application.

The situation is the issue repeatedly appearing and make it stable I kept continous ping from worksation ( other vlan) to these servers, once stop the ping the issue reoccur after 2-5 min.

[Jon Marshall]

What are your arp tables on the L3 switch and your mac address tables on each switch looking like ?

For the mac addresses you need to check each switch.

When you cannot ping the servers from a client can the servers ping  -

1) their default gateway

2) another L3 SVI IP address on the L3 switch

do you have any non standard configuration anywhere eg. port security/DAI/acls etc.

Jon

[samirshaikh52]

Thanks for your help.

The servers cannot ping their default gateway i.e HSRP virutal IP but they can ping the L3 SVI IP address.

I have stopped continous ping to servers the issue should reappear again then I can note the mac addresses but I guess they were look like on access switch and core switches.

[Jon Marshall]

The servers cannot ping their default gateway i.e HSRP virutal IP but they can ping the L3 SVI IP address

That could well be the issue. Can you answer the following -

1) What is "show standby brief" showing on the core switches for that vlan ?

2) how is the access switch connected to the core switches ie. is it as John suggested -

servers -> 2960 -> access switch -> core switches

if so does the access switch connect to both core switches ?

3) is it all devices in the server vlan that cannot ping the VIP ?

4) if you look at the arp table on a server that doesn't work is there an entry for the HSRP VIP ?

we need to understand the topology and switch interconnects to be able to help.

Jon

[samirshaikh52]

The problem has appeared again so I'll answer all possible question

1. What is "show standby brief" showing on the core switches for that vlan ?

Core1

Vl2         2   110  P Active   local           10.1.1.253      10.1.1.1

Core2

Vl2         2   95   P Standby  10.1.1.254      local           10.1.1.1

2. I have already posted on the above comment.

3.is it all devices in the server vlan that cannot ping the VIP ?

No. only these three servers

4.if you look at the arp table on a server that doesn't work is there an entry for the HSRP VIP ?

I can see arp entry on Core 1 but not on core 2

5.Are there any devices on the 2960 that are working ?

Yes there are other 4 servers connected and working

[Jon Marshall]

Can you also answer all the questions being asked eg. you were asked by John if that was the correct topology but we never got an answer.

Are there any redundant paths between switches or is it simply as John has drawn ?

Jon

[samirshaikh52]

Hi,

This is the topology.

(3 Servers)<--->(2960)<--->(Access Switch)

                                          |               |  

                                          |               |

                                     Core 1         Core2

Actually 2960 is not our administration it is working as L2 and its trunked

Access Switch # sh int gi1/0/44 ( port connected to 2960

Operation_Room_SW_1#sh int gi1/0/44 switchport

Name: Gi1/0/44

Switchport: Enabled

Administrative Mode: dynamic auto

Operational Mode: trunk

Administrative Trunking Encapsulation: negotiate

Operational Trunking Encapsulation: dot1q

Negotiation of Trunking: On

Access Mode VLAN: 2

Trunking Native Mode VLAN: 1 (default)

Administrative Native VLAN tagging: enabled

Voice VLAN: none

Administrative private-vlan host-association: none

Administrative private-vlan mapping: none

Administrative private-vlan trunk native VLAN: none

Administrative private-vlan trunk Native VLAN tagging: enabled

Administrative private-vlan trunk encapsulation: dot1q

Administrative private-vlan trunk normal VLANs: none

Administrative private-vlan trunk associations: none

Administrative private-vlan trunk mappings: none

Operational private-vlan: none

Trunking VLANs Enabled: ALL

Pruning VLANs Enabled: 2-1001

Capture Mode Disabled

Capture VLANs Allowed: ALL

Protected: false

Unknown unicast blocked: disabled

Unknown multicast blocked: disabled

Appliance trust: none

[Jon Marshall]

Thanks for that.

Can you answer the other questions as well. We may well need other outputs depending on the answers.

Are there any devices on the 2960 that are working ?

Jon

[samirshaikh52]

The problem has appeared again so I'll answer all possible question

1. What is "show standby brief" showing on the core switches for that vlan ?

Core1

Vl2         2   110  P Active   local           10.1.1.253      10.1.1.1

Core2

Vl2         2   95   P Standby  10.1.1.254      local           10.1.1.1

2. I have already posted on the above comment.

3.is it all devices in the server vlan that cannot ping the VIP ?

No. only these three servers

4.if you look at the arp table on a server that doesn't work is there an entry for the HSRP VIP ?

I can see arp entry on Core 1 but not on core 2

5.Are there any devices on the 2960 that are working ?

Yes there are other 4 servers connected and working

[JohnTylerPearce]

The servers cannot ping their default gateway i.e HSRP virutal IP but they can ping the L3 SVI IP address

Can you paste the configuration of your HSRP for this vlan?

Also, since you can't ping the HSRP VIP, can you ping the actual physical IP Addresses of the routed ports on each router?

For example, I have network 192.168.1.0/24, and R1 has 192.168.1.2 and R2 has 192.168.1.3, you normally set your VIP to .3, In this vlan can you ping .2 and or .3?

[Jon Marshall]

Sorry to be bombarding you with questions but you said you cleared the arp cache. Was this on the servers or the core switches ?

Jon

[samirshaikh52]

On core switches I have done anything on server since I don't have an access to it.

[Jon Marshall]

So you cannot get onto the servers in question ?

If not -

1) are the non working servers in the same vlan as the working servers ?

2) how do you know the servers can't ping the VIP, is someone else doing that for you ? If so can they do an "arp -a" on the server and look for the VIP entry ?

3) can you pick one of the non working servers and trace it's mac address from the core switch ie. on each switch up to the 2960 can you look in the mac address tables and make sure the mac for that server is going out of the right port.

Jon