Packet loss between Cisco 1941 and HP Switch

[mathias.bosshard]

Dear all

We’ve had now two attempts to migrate a customer site in Poland but failed both times because of a 3% packet loss between CPE and Switch (point to point)!

The equipment on the site is:

• Customer switch (HP model HP 5120-48G EI – power over Ethernet)
• IPANEMA: N0005-256MB-F128
• CPE: both Cisco 1941 (c1900-universalk9-mz.SPA.153-3.M5.bin)

Existing network

• At the moment the customer is using IBM to connect the site in Poland to their global network.

Pre test:

• Before we started the testing the HP/IBM/customer tested the connectivity to the site and had 0% packet loss when pinging the switch management port as well as the servers behind.

First migration attempt:

• Power down IBM routers and connect LAN cable to the IP Engines
• We’ve had to configure the LAN port to 100 full (hardcoded) as the auto negotiations on the HP switch tried to connect with 100/half
• Ethernet interfaces came up on both sides but we’ve seen a 3% packet loss between CPE and Switch as well as remote destination and switch /servers (no packet loss when pining the router from a remote destination)

Troublethooting:

Test1:  

• Hard code the Ethernet interface to 1G as this was the setting on the IBM router but still seeing the same amount of packet loss

Test2

• Power down the Ipanema (fail to wire) but still seeing the same amount of packet loss

Test3

• Remove the Ipanema between the switch and the router -> still packet loss

Test4

• Connect the switch to the second router (HVPN) -> still packet loss

Test5

• Connect the cable back to the MPLS router and take a different port on the HP switch -> still packet loss

Test6

• Replace cable between Switch and Cisco router -> still packet loss

Test7

• Connect a PC directly to the MPLS router à no packet loss        (indicated that the CPE hardware is working fine)

Test8

• Connect Switch back to the IBM router and power it up again à no packet loss

After these test we’ve compared the site hardware with an already working site with a similar setup and the only difference we’ve found is that in Geneva the router is running on c1900-universalk9-mz.SPA.153-3.M6.bin and not M5 like in poland. All the other devices having the same firmware/software and hardware version!

At the moment we’ve uploaded the same IOS version to the router to ensure that it’s not related to any compatibility issues between Cisco and HP on Version M5 but that’s kind of the last idea we have.

So if anyone of you have some more ideas on what we could do to sort out the issue please let me know. Please note that we have a second site with the same setup having an issue and there are more such site to come!

Thanks for your support

Regards

Matt

[Amit Goyal]

Hi,

I would recommend to use extended ACL inbound/outbound on both WAN and LAN interface of Cisco router.

ACL should have explicit permit statement of icmp traffic with source ip, destination ip and vice-versa.

permit icmp host <src> host <dst>

permit icmp host <dst> host <src>

ACL name should be different for WAN interface and LAN interface.

In the last you can add "permit ip any any" to permit all traffic.

Initiate 100 ping from remote or from Local LAN side which can match in the permit statement of ACL.

Check the hit count of ACL on WAN interface and LAN interface. Check if all 100 pings entered in the router and leaving the router. Same hits you can check for ICMP reply. It will tell you if the router is actually dropping the packet  or not.

HTH

-Amit