cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
821
Views
0
Helpful
4
Replies

Individual Vlan with performance issue

william.briere
Level 1
Level 1

Hi,

We have a performance problem with a specific Vlan, that began happening after we saw some host flapping that caused the Vlan in question to move to standby and back (preempt is specified). None of the other Vlans have this performance issue. The active router (6506E) has many Vlans on it. The preempt command is specified on about half of the Vlans (Yes, HSRP is being used). The core is a 6506E and the distribution layer contains 4948's trunked. The symptoms have a file copy start out at regular speed and then begin to degrade down to an unbelievably slow level. The links are all Gigabit between the Servers copying and the distribution switches they are connected to. Also there are zero ports errors of any kind being shown on the equipment. This would appear to be some sort of spanning-tree issue, but I am puzzled how spanning-tree could cause what appears to be some sort of queueing issue.

Any help on where to look next would be appreciated.

4 Replies 4

william.briere
Level 1
Level 1

This is the Ciscoworks message that precipitated the issues with this Vlan.

Syslog message generated from device Distribution-4: Jan 17 12:52:47 192.168.51.9 80581: Jan 17 12:50:11 EST: %C4K_EBM-4-HOSTFLAPPING: Host 00:1C:B1:2C:E6:80 in vlan 11 is flapping between port Gi1/35 and port Po2

glen.grant
VIP Alumni
VIP Alumni

   Yeah it looks like you have some ports tied together which is causing a bridging loop .  Check to see where the spanning tree root is for vlan 11 , should be the hsrp active router side for vlan 11 .  Do a sh cdp nei and see if you have any ports in which the  4948's see themselves on different ports.  Do a show spanning  blockedports and see if any ports are blocked that are not supposed to be .  Spanning tree "should"  block looped ports but we have all had instances where it did not do what it is supposed to do .  If 2 ports in the same vlan get physically tied together it can raise all kinds of issues.

Hi,

Thanks for your response.

Probably worth mentioning that the posted Ciscoworks message is where the issue seems to have started. The problem caused the Vlan to fail to it's HSRP standby router, and a short time later it failed back. Since the fail back only one Vlan has begun to experience this transfer issue. In any type of file copy (straight server to server copy, backup, FTP, etc.) the copy now starts normally and fast, and then at a certain point begins to perform very poorly. I know what your thinking, must be the server nic (driver old) or even the driver version of the controller, but it only started happening once this hostflapping briefly happened and also affects every transfer from every server that has an address in this Vlan. There is only gigabit nics on all of the servers in this Vlan and no other Vlan is affected. Also, if this Vlan is in play, being either the source or the destination, the problem exists. If the same devices (6506E and 4948) are used, but different Vlans on those devices are in play, we see no issues. We use VTP and the VTP server is the 6505E and the 4948's participate. I also considered it may be an assemetric routing issue, but not everything adds up to make that the case. Most of this article is true, but the only thing that doesn't make sense is the copy starts fast then part way through begins to degrade and never gets better.

Asymmetric Routing and HSRP (Excessive Flooding of Unicast Traffic in Network with Routers That Run HSRP)

http://www.cisco.com/en/US/tech/tk648/tk362/technologies_tech_note09186a0080094afd.shtml#t8

Thought's?

The sh cdp nei command doesn't show anything out of the ordinary now, as that was an individual event that happened. The thing is more around why after that event thinigs on that Vlan have slowed. We run ipflow captures and that Vlan is not getting saturated, and neither are the others. The CPU on our 6506E sits at a point where a reasonable person would ask if it was even getting used.

Any help would be appreciated.

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Innovations in Cisco Full Stack Observability - A new webinar from Cisco