Errors and link flaps on fibre (fiber) trunk - Page 2

Andrew Devine · ‎08-11-2011

Hi,

I have setup a trunk between a WS-X4306-GB line card and a dual-media uplink port on a WS-CBS3020-HPQ blade switch. However, when I enable the port it flaps every minute or so and after the flap reports runts on the interface.

The WS-X4306-GB line card is installed in mod 9 of a 4510R-E running IOS 12.2(46)SG (cat4500e-ENTSERVICESK9-M), a 1000BASE-SX GBIC (WS-G5484) is installed in Gi 9/1
The WS-CBS3020-HPQ blade switch is running 12.2(40)SE2 (CBS30X0-LANBASE-M) and has a 1000BASE-SX SFP (GLC-SX-MM) installed into port Gi 0/17
The fibre connection goes via a patch panel
62.5/125 and 50/125 fibre has been tried but with the same result for both types

Config:-

!

! STP port-priority configured so I can enable new uplink without affecting STP topology, copper uplinks existing

!

4510R-E#sh run int gi 9/1

!

interface GigabitEthernet9/1

switchport mode trunk

udld port aggressive

spanning-tree port-priority 144

!

! Uplinkfast enabled so I can control root port by altering 9/1 STP port priority out of hours, link state group on all uplinks to bring blade NICs down

!

3020#sh run int gi 0/17

!
interface GigabitEthernet0/17

switchport trunk encapsulation dot1q
switchport mode trunk

udld port aggressive
link state group 1 upstream

!

Outputs:-

4510R-E#sh int gi 9/1 contr

GigabitEthernet9/1 is down, line protocol is down (notconnect)

Hardware is Gigabit Ethernet Port, address is 0022.bddd.48b6 (bia 0022.bddd.48b6)

Description: 3020 Gi0/17

MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Full-duplex, Auto-speed, link type is auto, media type is 1000BaseSX

input flow-control is off, output flow-control is off

ARP type: ARPA, ARP Timeout 04:00:00

Last input 21:03:48, output never, output hang never

Last clearing of "show interface" counters never

Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue: 0/40 (size/max)

5 minute input rate 0 bits/sec, 0 packets/sec

5 minute output rate 0 bits/sec, 0 packets/sec

1310747 packets input, 1293130218 bytes, 0 no buffer

Received 4965 broadcasts (4504 multicasts)

2 runts, 0 giants, 0 throttles

2 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

0 input packets with dribble condition detected

5931346 packets output, 1660789456 bytes, 0 underruns

0 output errors, 0 collisions, 0 interface resets

0 babbles, 0 late collision, 0 deferred

0 lost carrier, 0 no carrier

0 output buffer failures, 0 output buffers swapped out

4510R-E#sh int gi 9/1 capabilities

GigabitEthernet9/1

Model: WS-X4306-GB-Gbic

Type: 1000BaseSX

Speed: 1000

Duplex: full

Auto-MDIX: no

Trunk encap. type: 802.1Q

Trunk mode: on,off,desirable,nonegotiate

Channel: yes

Broadcast suppression: percentage(0-100), hw

Multicast suppression: percentage(0-100), hw

Flowcontrol: rx-(off,on,desired),tx-(off,on,desired)

VLAN Membership: static, dynamic

Fast Start: yes

CoS rewrite: yes

ToS rewrite: yes

Inline power: no

SPAN: source/destination

UDLD: yes

Link Debounce: no

Link Debounce Time: no

Port Security: yes

Dot1x: yes

Maximum MTU: 9198 bytes (Jumbo Frames)

Multiple Media Types: no

Diagnostic Monitoring: N/A

Queuing: rx-(N/A), tx-(1p7q1t)

3020#sh int gi 0/17 controller
GigabitEthernet0/17 is administratively down, line protocol is down (disabled)
Hardware is Gigabit Ethernet, address is 0022.91af.9a11 (bia 0022.91af.9a11)
Description: 4510R-E Gi9/1
MTU 1500 bytes, BW 10000 Kbit, DLY 1000 usec,
reliability 254/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive not set
Auto-duplex, Auto-speed, link type is auto, media type is 1000BaseSX SFP
input flow-control is off, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 21:06:47, output 21:06:47, output hang never
Last clearing of "show interface" counters 21:20:17
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
1133530 packets input, 783706915 bytes, 0 no buffer
Received 4791 broadcasts (12215 multicasts)
2166 runts, 0 giants, 0 throttles
2166 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 12215 multicast, 0 pause input
0 input packets with dribble condition detected
302 packets output, 35116 bytes, 0 underruns
0 output errors, 0 collisions, 5 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 PAUSE output
0 output buffer failures, 0 output buffers swapped out

Transmit GigabitEthernet0/17 Receive
1474443247 Bytes 2002996525 Bytes
1316622994 Unicast frames 45790944 Unicast frames
90800633 Multicast frames 1047999 Multicast frames
71657260 Broadcast frames 410590 Broadcast frames
0 Too old frames 1693481132 Unicast bytes
2167069 Deferred frames 236924556 Multicast bytes
0 MTU exceeded frames 45180464 Broadcast bytes
862378 1 collision frames 0 Alignment errors
2398 2 collision frames 0 FCS errors
0 3 collision frames 0 Oversize frames
0 4 collision frames 63932 Undersize frames
0 5 collision frames 0 Collision fragments
0 6 collision frames
0 7 collision frames 12648203 Minimum size frames
0 8 collision frames 4929545 65 to 127 byte frames
0 9 collision frames 5811136 128 to 255 byte frames
0 10 collision frames 342125 256 to 511 byte frames
0 11 collision frames 1252505 512 to 1023 byte frames
0 12 collision frames 22266019 1024 to 1518 byte frames
0 13 collision frames 0 Overrun frames
0 14 collision frames 0 Pause frames
0 15 collision frames
0 Excessive collisions 0 Symbol error frames
163200 Late collisions 0 Invalid frames, too large
0 VLAN discard frames 0 Valid frames, too large
0 Excess defer frames 0 Invalid frames, too small
645753757 64 byte frames 63932 Valid frames, too small
296775985 127 byte frames
135004286 255 byte frames 0 Too old frames
66860049 511 byte frames 0 Valid oversize frames
134507717 1023 byte frames 0 System FCS error frames
200245716 1518 byte frames 0 RxPortFifoFull drop frame
0 Too large frames
862378 Good (1 coll) frames
2398 Good (>1 coll) frames

4510R-Esh int gi 0/17 capabilities
GigabitEthernet0/17
Model: WS-CBS3020-HPQ
Type: 1000BaseSX SFP
Speed: 1000
Duplex: full
Trunk encap. type: 802.1Q,ISL
Trunk mode: on,off,desirable,nonegotiate
Channel: yes
Broadcast suppression: percentage(0-100)
Flowcontrol: rx-(off,on,desired),tx-(none)
Fast Start: yes
QoS scheduling: rx-(not configurable on per port basis),tx-(4q2t)
CoS rewrite: yes
ToS rewrite: yes
UDLD: yes
Inline power: no
SPAN: source/destination
PortSecure: yes
Dot1x: yes
Multiple Media Types: rj45, sfp, auto-select

Aug 10 12:17:02 BST: %LINK-3-UPDOWN: Interface GigabitEthernet0/17, changed state to up

Aug 10 12:17:05 BST: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/17, changed state to up

Aug 10 12:17:41 BST: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/17, changed state to down

Aug 10 12:17:44 BST: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/17, changed state to up

Aug 10 12:17:58 BST: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/17, changed state to down

Aug 10 12:18:01 BST: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/17, changed state to up

Aug 10 12:19:11 BST: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/17, changed state to down

Aug 10 12:19:14 BST: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/17, changed state to up

Aug 10 12:19:41 BST: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/17, changed state to down

Aug 10 12:19:44 BST: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/17, changed state to up

Aug 10 12:22:12 BST: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/17, changed state to down

Aug 10 12:22:15 BST: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/17, changed state to up

Aug 10 12:22:20 BST: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/17, changed state to down

Any help appreciated, I intened to run a direct cable to rule out the patch panel once I've sourced one long enough.

Cheers,

Andrew

Andrew Devine · ‎08-11-2011

Yes, I think I have just seen why:-

http://www.cisco.com/en/US/docs/switches/blades/3020/software/release/12.2_25_sef1/configuration/guide/swint.html#wp1028308

auto-select—The switch dynamically selects the type. When link up is achieved, the switch disables the other type until the active link goes down. When the active link goes down, the switch enables both types until one of them links up. In auto-select mode, the switch configures both types with autonegotiation of speed and duplex (the default). Depending on the type of installed SFP module, the switch might not be able to dynamically select it. For more information, see the information that follows this procedure.

When I change to media-type sfp I can then use the command 'speed nonegotiate'

I'll fix both ends and test

amikat · ‎08-12-2011

Hi,

Please be careful when using "speed nonegotiate" interface command as you can hit the CSCsf98370 caveat as per release notes:

CSCsf98370

Fiber optic small form-factor pluggable (SFP) modules do not establish a link when you configure the Ethernet interfaces with the speed nonegotiate interface configuration command.

The workaround is to remove the speed nonegotiate setting for the Ethernet interface that has the fiber SFP installed. This configuration has to be done on the link partner device as well.

Best regards,

Antonin

Andrew Devine · ‎09-02-2011

Turned out to be an IOS issue on the HP 3020 Blade switches.

Upgraded from 12.2(40)SE2. to 12.2(58)SE1 and the link was stable.