VRF and VLAN Communication Issue

[edgar-gutierrez]

Configuration overview:

Network engineer workstatoin --> access switch(2960)--> Core switch (6509)-->Distro Switch (WS-C4507R+E)-->access switch (WS-C4510R+E) -->printers

Distro switch has been configured with VSI for layer 3 (vrf Ops).

Access switch has been configured with same VLANs as Distro switch.

Problem:

After 10 min idle time printers in this particular building will not be reachable.

Disctro switch ip arp table contains IP and MAC address info for all printers

When the network engineer pings printers from his workstation, the pings are unreachable.

Traceroute from Net Eng workstation indicates last hop is Distro switch.

In order to wake up printers, pinging must be done from the Distro switch. Then pings from Net Eng workstation can ping all printers.

command used: ping vrf Ops <printer IP address>

Any thoughts..

[Julio E. Moisa]

Hi

Do you have a diagram or provide more details like IGP/EGP? have you configured the vrf Ops on the interface facing to the Core 6509 on the C4507R+E.

If you configure the vrf on the interface it will remove the IP and you should set up the ip again.

Regards. 

[edgar-gutierrez]

Julio,

Thank you for replying to my post.  We found out that the problem is with unicast float issue. 

Seems like there is bug in the current IOS image run by 4500 Catalyst.

Applying the changes below fixed printer communication issue.

configure terminal

interface vlan 1410 (Printer's VLAN)

arp timeout 240

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvb78700

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCum08763

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCsm71237/

v/r

Edgar

[Julio E. Moisa]

Great!, good to know :-)

Regards. 

[Richard Burts]

Edgar

Thank you for posting back to the forum and telling us that you have resolved the problem and explaining what the problem was. +5 for this.

It is not uncommon for the difference between the default timer for arp (4 hours) and the default timer for the switch mac address forwarding table to cause issues. By configuring a shorter timer for arp you make the functions have similar timeout. With a longer timeout for arp you can get a situation like this:

- network engineer attempts to access the printer.

- packet gets to the distro switch. there is an arp entry for the address but no entry in the mac forwarding table.

- there is an arp entry so the switch knows what mac address to use in the Ethernet header.

- but there is no entry for that mac in the mac forwarding table. So the switch drops the packet. The switch does not perform an arp request since it already knows the mac address to use.

When you change the timer you can get a situation like this:

- network engineer attempts to access the printer.

- packet gets to the distro switch. there is no arp entry and no entry in the mac forwarding table.

- the switch sends an arp request (broadcast) into the network.

- the printer responds with its mac address.

- the distro switch now has an entry in the arp table and an entry in the mac forwarding table.

- the distro switch is able to forward the packet to the printer and the network engineer is successful in accessing the printer.

HTH

Rick

[jacky.reinbold]

Hey Edgar,

This was a great information to find a workaround, but I'm not sure that the real source of this IOS problem has been analyzed in deep by Cisco, before supposing that it is only a bug.

In my case I could compare as an older HP Core (which never had the issue) was replaced by a VSS Engine (2 x 4500x and Sw. cat4500e-universalk9.SPA.03.08.02.E.152-4.E2.bin), which is not described as faulty in the bug descriptions.

In our case, only Lexmark printers, IGEL Terminals and some print servers are showing the issue into there own VLANs, with the consequence that the traffic stops to cross the layer 3 interface after some time.

The first test was to configure a secondary IP interface on the virtual windows print servers to match the printers L2 VLAN, and this worked in the same broadcast domain as the printers or terminals, but was not satisfying for the customer.

On a Wireshark printer port trace, I've seen that the printers are talking to themselves (source and destination  MAC are the same and the speed turns from 1Gb to 100Mb) after some time.

The Trace shows intermittant 802.3Ethernet DSAP/SSAP framing on the printer NIC, instead of Ethernet II type and zero padding.

And I've found the information that this framing will not cross the L3 boundary on Cisco Switches and that Lexmark NICs possibly allows wrong framing types.

https://supportforums.cisco.com/discussion/12556901/ethernet-8023-vs-ethernet-ii-frame

The main question to this IOS behavior could be:

- is it only an IOS arp timer bug who could be resolved by a new IOS version (not published yet)?

- is this bug sourced by older network devices using wrong ethernet framings ?

- Did Cisco analyze this issue in deep ?

Cheers,

Jacky

[jacky.reinbold]

Let's have a look at my add below in the discussion.

Did you trace a Printer port too ?

Regards,

Jacky