cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Announcements
Join Customer Connection to register!
1300
Views
0
Helpful
10
Replies

HSRP instability

Hello everyone,

I'm having a problem of hsrp instability. The scenario is two 6500 and one port-channel ( two tengiga ports - 6704). I started to notice hsrp instability and searching for one workaround i´ve found the ip spd command. This command has ended with the flapping, but i´m still looking for the cause of the problem.

Any suggestions ???

best regards,

Mauricio

1 ACCEPTED SOLUTION

Accepted Solutions

Input queue: 0/75/112/0 (size/max/drops/flushes); Total output drops: 0

Q depth is just 75,

try hold-queue 2000 on the vlan interface. It should resolve it.

But, also go through the following documentation.

http://www.cisco.com/en/US/products/hw/routers/ps133/products_tech_note09186a0080094791.shtml#topic2

In case if it does not resolve the issue, we need to find out what kind of traffic is going to the CPU at the time of drops.

Have a great day.

View solution in original post

10 REPLIES 10
Amit Singh
Cisco Employee

Could you Please paste the interface configurations and logs if you have any?

Cheers,

-amit singh

rassoul.ghaznavi
Beginner

Are you using copper or fiber?

Have you checked the negotiationon the link?

Satya Narra
Cisco Employee

Hi,

Since you say that ip spd has solved da issue, it looks like you were having input Q drops on the interface. Can you give us more info like interface config and show interface o/p and da config you put in. I will get back to you with a better answer.

Sent from Cisco Technical Support iPhone App

Satya Narra
Cisco Employee

check out the SPD document

http://www.cisco.com/en/US/products/hw/routers/ps167/products_tech_note09186a008012fb87.shtml

This should explain why SPD change resolved the issue.

Here is the config and some shows :

Ios Version : 12.2(17d)SXB11a

##global##

ip spd queue max-threshold 600

ip spd queue min-threshold 500

###Switch A###

interface TenGigabitEthernetx/y

mtu 9216

no ip address

mls qos trust dscp

switchport

switchport trunk encapsulation dot1q

switchport mode trunk

!

TenGigabitEthernetx/y is up, line protocol is up (connected)

  Hardware is C6k 10000Mb 802.3, address is 0025.842f.322e (bia 0025.842f.322e)

  Description:

  MTU 9216 bytes, BW 10000000 Kbit, DLY 10 usec,

     reliability 255/255, txload 87/255, rxload 125/255

  Encapsulation ARPA, loopback not set

  Full-duplex, 10Gb/s

  input flow-control is off, output flow-control is on

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input 00:00:03, output 00:00:26, output hang never

  Last clearing of "show interface" counters 5d19h

  Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  5 minute input rate 4936424000 bits/sec, 2480996 packets/sec

  5 minute output rate 3440547000 bits/sec, 1880687 packets/sec

     686292835356 packets input, 183937804559174 bytes, 0 no buffer

     Received 179655086 broadcasts, 0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 input packets with dribble condition detected

     520451505701 packets output, 130350475370460 bytes, 0 underruns

     0 output errors, 0 collisions, 0 interface resets

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier

     0 output buffer failures, 0 output buffers swapped out

!

SWITCH_A#sh ip spd

Current mode: normal.

Queue min/max thresholds: 500/600, Headroom: 100

IP normal queue: 0, priority queue: 1.

SPD special drop mode: none

###Switch B###

interface TenGigabitEthernetZ/K

mtu 9216

no ip address

mls qos trust dscp

switchport

switchport trunk encapsulation dot1q

switchport mode trunk

!

TenGigabitEthernetZ/K is up, line protocol is up (connected)

  Hardware is C6k 10000Mb 802.3, address is 0025.45f1.adf2 (bia 0025.45f1.adf2)

  Description: 

  MTU 9216 bytes, BW 10000000 Kbit, DLY 10 usec,

     reliability 255/255, txload 125/255, rxload 87/255

  Encapsulation ARPA, loopback not set

  Full-duplex, 10Gb/s

  input flow-control is off, output flow-control is on

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input 00:00:12, output 00:00:49, output hang never

  Last clearing of "show interface" counters 5d18h

  Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  5 minute input rate 3425071000 bits/sec, 1878092 packets/sec

  5 minute output rate 4921813000 bits/sec, 2482505 packets/sec

     519735420778 packets input, 130210672707506 bytes, 0 no buffer

     Received 88475812 broadcasts, 0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 input packets with dribble condition detected

     685338021183 packets output, 183730030043381 bytes, 0 underruns

     0 output errors, 0 collisions, 0 interface resets

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier

     0 output buffer failures, 0 output buffers swapped out

SWITCH_B#sh ip spd

Current mode: normal.

Queue min/max thresholds: 500/600, Headroom: 100

IP normal queue: 0, priority queue: 0.

SPD special drop mode: none

The show interfaces are very clean. Did you capture these output after the issue has been resolved?

I see that interfaces are just configured as trunk. I am expecting that vlan interfaces have the HSRP configuration, is that right?

Can you check the vlan interfaces for any input Q drops?

This is correct, the output was collected yesterday afternoon. I forgot to mention that when i notice the problem the traffic it was close to 5 giga (peak traffic). Yesterday at night the problem has occurred again even with the spd configuration.

Here are some information about the vlans that i´ve collected rigth now:

###SWITCH_A###

interface Vlan12

ip address x.x.x.2 255.255.255.0

no ip redirects

standby 12 ip x.x.x.1

standby 12 preempt

!

Vlan12 is up, line protocol is up

  Hardware is EtherSVI, address is 0015.2cb6.ca00 (bia 0015.2cb6.ca00)

  Description: Rede GERENCIA II

  Internet address is x.x.x.2

  MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,

     reliability 255/255, txload 25/255, rxload 1/255

  Encapsulation ARPA, loopback not set

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input 00:00:00, output 00:00:01, output hang never

  Last clearing of "show interface" counters 6d11h

  Input queue: 0/75/112/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  5 minute input rate 9000 bits/sec, 16 packets/sec

  5 minute output rate 99224000 bits/sec, 13247 packets/sec

  L2 Switched: ucast: 5136130694 pkt, 6031008158267 bytes - mcast: 10835447 pkt, 1082019516 bytes

  L3 in Switched: ucast: 45764 pkt, 2947309 bytes - mcast: 0 pkt, 0 bytes mcast

  L3 out Switched: ucast: 9275671189 pkt, 7145521811463 bytes mcast: 0 pkt, 0 bytes

     9418387 packets input, 664720408 bytes, 0 no buffer

     Received 9355383 broadcasts, 0 runts, 0 giants, 2 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     9294236023 packets output, 7125481295613 bytes, 0 underruns

     0 output errors, 0 interface resets

     0 output buffer failures, 0 output buffers swapped out

interface Vlan70

mtu 1526

ip address x.x.x.162 255.255.255.240

no ip redirects

no ip unreachables

ip mtu 1500

load-interval 30

standby 70 ip x.x.x.161

standby 70 priority 105

standby 70 preempt

!

Vlan70 is up, line protocol is up

  Hardware is EtherSVI, address is 0015.2cb6.ca00 (bia 0015.2cb6.ca00)

  Description: Vlan - GBoIP_BSC_SNE01

  Internet address is x.x.x.162

  MTU 1526 bytes, BW 1000000 Kbit, DLY 10 usec,

     reliability 255/255, txload 7/255, rxload 1/255

  Encapsulation ARPA, loopback not set

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input 00:00:00, output 00:00:01, output hang never

  Last clearing of "show interface" counters 6d11h

  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  30 second input rate 2000 bits/sec, 5 packets/sec

  30 second output rate 29775000 bits/sec, 8586 packets/sec

  L2 Switched: ucast: 2250492201 pkt, 421466690051 bytes - mcast: 3010411 pkt, 184055808 bytes

  L3 in Switched: ucast: 23908 pkt, 4210410 bytes - mcast: 0 pkt, 0 bytes mcast

  L3 out Switched: ucast: 4801497663 pkt, 2202834662978 bytes mcast: 0 pkt, 0 bytes

     3053897 packets input, 200937411 bytes, 0 no buffer

     Received 3010403 broadcasts, 0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     4805260906 packets output, 2184850651347 bytes, 0 underruns

     0 output errors, 0 interface resets

     0 output buffer failures, 0 output buffers swapped out

!

###SWITCH_B###

interface Vlan12

ip address x.x.x.3 255.255.255.0

no ip redirects

standby 12 ip x.x.x.1

standby 12 priority 105

standby 12 preempt

end

SWITCH_B#sh int vlan 12                                                              

Vlan12 is up, line protocol is up

  Hardware is EtherSVI, address is 0015.2cb6.de00 (bia 0015.2cb6.de00)

  Description:

  Internet address is x.x.x.3/24

  MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,

     reliability 255/255, txload 2/255, rxload 31/255

  Encapsulation ARPA, loopback not set

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input 00:00:00, output 00:00:00, output hang never

  Last clearing of "show interface" counters 6d11h

  Input queue: 0/75/95/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  5 minute input rate 124290000 bits/sec, 15035 packets/sec

  5 minute output rate 9339000 bits/sec, 964 packets/sec

  L2 Switched: ucast: 16960444670 pkt, 15204885675072 bytes - mcast: 10882456 pkt, 1087842971 bytes

  L3 in Switched: ucast: 9958793719 pkt, 10970753270506 bytes - mcast: 0 pkt, 0 bytes mcast

  L3 out Switched: ucast: 744085595 pkt, 984080080452 bytes mcast: 0 pkt, 0 bytes

     9968667882 packets input, 10971596827481 bytes, 0 no buffer

     Received 9400016 broadcasts, 0 runts, 0 giants, 2 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     744718114 packets output, 981254288977 bytes, 0 underruns

     0 output errors, 0 interface resets

     0 output buffer failures, 0 output buffers swapped out

interface Vlan70

mtu 1526

ip address x.x.x.163 255.255.255.240

no ip redirects

no ip unreachables

ip mtu 1500

load-interval 30

standby 70 ip x.x.x.161

standby 70 priority 105

standby 70 preempt

end

SWITCH_B#sh int vlan 70

Vlan70 is up, line protocol is up

  Hardware is EtherSVI, address is 0015.2cb6.de00 (bia 0015.2cb6.de00)

  Description:

  Internet address is x.x.x.163/28

  MTU 1526 bytes, BW 1000000 Kbit, DLY 10 usec,

     reliability 255/255, txload 1/255, rxload 2/255

  Encapsulation ARPA, loopback not set

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input 00:00:00, output 00:00:01, output hang never

  Last clearing of "show interface" counters 6d11h

  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  30 second input rate 7896000 bits/sec, 5937 packets/sec

  30 second output rate 0 bits/sec, 0 packets/sec

  L2 Switched: ucast: 1462911798 pkt, 665826931259 bytes - mcast: 3032920 pkt, 192202566 bytes

  L3 in Switched: ucast: 3083188365 pkt, 567226593864 bytes - mcast: 0 pkt, 0 bytes mcast

  L3 out Switched: ucast: 19347908 pkt, 9452365132 bytes mcast: 0 pkt, 0 bytes

     3086272153 packets input, 567430451135 bytes, 0 no buffer

     Received 3032938 broadcasts, 0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     19561689 packets output, 9393901802 bytes, 0 underruns

     0 output errors, 0 interface resets

     0 output buffer failures, 0 output buffers swapped out

I also notice ospf flaps at the same time of the hsrp flaps (not the same vlans ...rs), i´m suspecting that the peak of traffic it´s really the problem. How can i confirm my suspicious ?

Input queue: 0/75/112/0 (size/max/drops/flushes); Total output drops: 0

Q depth is just 75,

try hold-queue 2000 on the vlan interface. It should resolve it.

But, also go through the following documentation.

http://www.cisco.com/en/US/products/hw/routers/ps133/products_tech_note09186a0080094791.shtml#topic2

In case if it does not resolve the issue, we need to find out what kind of traffic is going to the CPU at the time of drops.

Have a great day.

View solution in original post

I´ve adjust SPT with HSRP and change the queue and solve the problem.

Thanks for the help !