cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1430
Views
0
Helpful
16
Replies
Highlighted
Contributor

mysterious 4500-x to 6500 fiber connection issue??

im having a strange issue with a fiber connection between 4500-x and 6500 , it is a 10 gig connection with both LR sfb on both 4500 and 6500 and a sm cable , the issue is that the port on the 4500-x keep switching between connected and notconnected but the the port on the 6500 stays connected , when i check both interfaces on 4500 and 6500 it have gradually increasing crc errors and input errors on both switches , i tried replacing the cable , the sfb and the port with no avail , i tried switchport non-negotiate also tries setting it to trunk and access and still the same story , even when i connect the 4500-x with the same sfb to 2960-x sfb 10 gig as well there is no issues , also connecting the 2960-x to the 6500 shows no issues using the same 10 gig cable and sfb , please help as i ran out of solutions

16 REPLIES 16
Highlighted
VIP Mentor

have you tried to hardcode

have you tried to hardcode the speed and duplex settings , does it happen on all 10gig connections between the 65 and 45 or just the same interface each time

crc and inputs together usually infer there's a layer 1 issue , how far away are these devices from each other attenuation can be a cause for this as well  , whats the reliability show on the interface when connected is it 255/255

you have ruled out sfp and cable by using different equipment so i would check these next

If there sfps that support DOM you can you can do a show interface transceiver

Highlighted
Contributor

regarding the duplex i think

regarding the duplex i think u cant change it for sfb it is always full-duplex and yes i entered the bandwidth under the interface and it didnt work i even lowered the bandwidth , also yes i tried three ports on both switches and the same issue presist ,regarding cable im using a 1-1.5 meter back to back cable between the two devices , but i havnt checked the reliability. 

Highlighted
VIP Mentor

yes sorry fibre you dont have

yes sorry fibre you dont have duplex ,can  you post the show interface for each side

the exact type of sfps and the ios currently in use in on 45

where are the crcs mostly incrementing on 45 or 65 side

the fact the issue does not exist using same cabling and sfps with 2960 & 6500  would indicate issue is on the 4500 whether its hardware or software support problem

Highlighted
Contributor

the interface and the version

the interface and the version of the 6500 is:

1-version >

 
ROM: System Bootstrap, Version 12.2(17r)S2, RELEASE SOFTWARE (fc1)
 
 
 
 
 
 
 
cisco WS-C6509-E (R7000) processor (revision 1.2) with 458720K/65536K bytes of memory.
Processor board ID SMC0934007G
SR71000 CPU at 600Mhz, Implementation 0x504, Rev 1.2, 512KB L2 Cache
Last reset from power-on
21 Virtual Ethernet interfaces
132 Gigabit Ethernet interfaces
32 Ten Gigabit Ethernet interfaces
1917K bytes of non-volatile configuration memory.
8192K bytes of packet buffer memory.
 
65536K bytes of Flash internal SIMM (Sector size 512K).
Configuration register is 0x2102
 
interface of the 6500 switch>
KIA-Core-Switch(config-if)#do sh int t9/16
TenGigabitEthernet9/16 is up, line protocol is up (connected)
  Hardware is C6k 10000Mb 802.3, address is 6c41.6a60.0f67 (bia 6c41.6a60.0f67)
  MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, loopback not set
  Keepalive set (10 sec)
  Full-duplex, 10Gb/s
  Transport mode LAN (10GBASE-R, 10.3125Gb/s), media type is 10Gbase-LR
  input flow-control is on, output flow-control is off
  Clock mode is auto
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input 00:00:04, output 00:00:17, output hang never
  Last clearing of "show interface" counters 02:10:38
  Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 215000 bits/sec, 98 packets/sec
     216 packets input, 112169 bytes, 0 no buffer
     Received 216 broadcasts (216 multicasts)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog, 0 multicast, 0 pause input
     0 input packets with dribble condition detected
     132090 packets output, 41503844 bytes, 0 underruns
     0 output errors, 0 collisions, 4 interface resets
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier, 0 PAUSE output
     0 output buffer failures, 0 output buffers swapped out
 
 
 
 
version and interface on the 4500:
version>
#sh ver
Cisco IOS Software, IOS-XE Software, Catalyst 4500 L3 Switch  Software (cat4500e
-UNIVERSALK9-M), Version 03.07.01.E RELEASE SOFTWARE (fc3)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2015 by Cisco Systems, Inc.
Compiled Tue 28-Apr-15 12:27 by prod_rel_team
 
Cisco IOS-XE software, Copyright (c) 2005-2015 by cisco Systems, Inc.
All rights reserved.  Certain components of Cisco IOS-XE software are
licensed under the GNU General Public License ("GPL") Version 2.0.  The
software code licensed under GPL Version 2.0 is free software that comes
with ABSOLUTELY NO WARRANTY.  You can redistribute and/or modify such
GPL code under the terms of GPL Version 2.0.
(http://www.gnu.org/licenses/gpl-2.0.html) For more details, see the
documentation or "License Notice" file accompanying the IOS-XE software,
or the applicable URL provided on the flyer accompanying the IOS-XE
software.
 
ROM: 15.0(1r)SG11
 uptime is 3 days, 1 hour, 22 minutes
System returned to ROM by power-on
System image file is "bootflash:cat4500e-universalk9.SPA.03.07.01.E.152-3.E1.bin
"
Jawa Revision 2, Winter Revision 0x0.0x41
Last reload reason: power-on
 
 
interface on 4500 with the issue appearing on the interfaces>
 
#sh int t1/5
TenGigabitEthernet1/5 is up, line protocol is up (connected)
  Hardware is Ten Gigabit Ethernet Port, address is 84b8.0201.fe44 (bia 84b8.020
1.fe44)
  MTU 1500 bytes, BW 10000000 Kbit/sec, DLY 10 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, loopback not set
  Keepalive set (10 sec)
  Full-duplex, 10Gb/s, link type is auto, media type is 10GBase-LR
  input flow-control is on, output flow-control is on
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input 00:00:00, output never, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 183000 bits/sec, 86 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     72635 packets input, 25174291 bytes, 0 no buffer
     Received 10766 broadcasts (2813 multicasts)
     0 runts, 0 giants, 0 throttles
     3 input errors, 3 CRC, 0 frame, 0 overrun, 0 ignored
     0 input packets with dribble condition detected
     167 packets output, 86673 bytes, 0 underruns
     0 output errors, 0 collisions, 2 interface resets
     0 unknown protocol drops
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier
     0 output buffer failures, 0 output buffers swapped out
 
 

4500-switch#sh int t1/5
TenGigabitEthernet1/5 is down, line protocol is down (notconnect)
  Hardware is Ten Gigabit Ethernet Port, address is 84b8.0201.fe44 (bia 84b8.020
1.fe44)
  MTU 1500 bytes, BW 10000000 Kbit/sec, DLY 10 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, loopback not set
  Keepalive set (10 sec)
  Full-duplex, Auto-speed, link type is auto, media type is 10GBase-LR
  input flow-control is off, output flow-control is off
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input 00:00:00, output never, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 185000 bits/sec, 87 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     72966 packets input, 25274041 bytes, 0 no buffer
     Received 10845 broadcasts (2828 multicasts)
     0 runts, 0 giants, 0 throttles
     3 input errors, 3 CRC, 0 frame, 0 overrun, 0 ignored
     0 input packets with dribble condition detected
     168 packets output, 87192 bytes, 0 underruns
     0 output errors, 0 collisions, 2 interface resets
     0 unknown protocol drops
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier
     0 output buffer failures, 0 output buffers swapped out
 
 
 
 
 
 
 
 
 
 
Highlighted
VIP Mentor

They need to be LC-LC

They need to be LC-LC connections or

The ports can be configured with either SFP transceivers for 1-GB operation or SFP+ transceivers for 10-GB operation. Both transceivers use LC-type connectors (optical) or RJ-45 (copper).


Highlighted
Beginner

Our connection is an LC (4500

Our connection is an LC (4500-x) to SC (4900).    We didn't have an issue for the 1st 8 months in production.  Is that due to software\hardware limitations, or is that a documented requirement?

Highlighted
VIP Mentor

That was taken out of a Cisco

That was taken out of a Cisco document for 4500 series switches 10GB requirements, i also came across it in another forum that they must be LC either side to achieve 10GB and SFP+ , you could check it with TAC but it looks to be hardware specific

Highlighted
Beginner

markmalone2008, I don't know

markmalone2008,

 

I don't know where you got this information from, but we have several areas where we connect switches together using 10G, where one end is LC and the other end is SC.  We have done it with multimode fiber and SR optics, as well as with singlemode fiber and LH optics.  It's never been a problem.

Can you post a link to the Cisco document that says this isn't supported?

 

To the OP, can you post your port configurations?  Also, when you say that the port(s) go down, how long do they remain down?  Do they bounce and come back up?

Are you setting your ports to negotiate everything, or are you hard setting them to encapsulation (if it's an option), mode (access or trunk), turn off negotiation?

Have you tried configuring them as Layer 3 ports?  If so, does the problem still occur? 

-rb

Highlighted
Beginner

baselzind, Have you verified

baselzind,

 

Have you verified that your patch cable is certified for 10G operation?

 

Also, since you are using Single Mode optics, you need to be careful with optical power levels.  With a short fiber optic cable, you may be overloading the optical receiver.  In the long run this will burn out your SFP.  In the short run, it may very well be causing the errors you are seeing.

It is very important to calculate an optical power budget for single mode connections, and to use attenuators to make sure that the receive levels are within specifications for the transceivers.  Typically, attenuating it so the receive value is slightly above the middle range for the transceiver will be best.

 

-rb

 

Highlighted
Beginner

Ron, How do you modify the

Ron,

 

How do you modify the optical levels? 

 

Fyi - this case has been escalated within Cisco  and hope to have a game plan in the next day or two

Highlighted
VIP Mentor

I didnt say it was not

I didnt say it was not supported and not the same for different switches just what i found online as the same cabling and sfps work with other switches fine and to check with TAC, this is one of the docs if your running sup7 series in 4500 series you need to use lc-lc

https://supportforums.cisco.com/discussion/11629431/catalyst-4510r-e-10gb-sfp-not-working

http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst4500/hardware/configuration/notes/OL_23144.html#wp153042

 

Catalyst 4500 L3 Switch  Software (cat4500e
-UNIVERSALK9-M) - See more at: https://supportforums.cisco.com/discussion/12505826/mysterious-4500-x-6500-fiber-connection-issue#sthash.UbDWJvxL.dpuf
Highlighted
Beginner

markmalone2008, Neither one

markmalone2008,

 

Neither one of those links says that you need to use lc-lc.  What they say is those particular switches have SFP optics, which by DEFAULT use LC connectors. Therefore, those switches can ONLY have LC connectors for those ports.

Conversely, SC connectors are found on X2 and XENPAK transceivers for 10G optics. So, Catalyst 6500 switch modules such as WS-X6708-10GE (X2) or WS-X6704-10GE (XENPAK) would have SC connectors for 10G.  But as I said earlier, we have several connections using SC-LC, generally connecting an X2 module to a Service Provider 10G connection.  In our cases, the cables are 15-20 meters long, and we still utilize attenuators to pad down the optical level on the receive side of each connection.

-rb

Highlighted
Beginner

We are setting them as trunks

We are setting them as trunks, negotiation was not turned off.  Although, I did just turn it off as it hasn't been tried yet.  We have not attempted to set it us as a Layer 3 port.

The port does down, bounces for a few minutes and typically comes back up after 12-18 minutes.  It then stays up for 5 + plus and occurs again.  Now, this was fine for 8 months and then it started happening every 5 hours.  We even had the 4500-x switches replaced.

Currently, there is nothing on the 4500-x except a trunk link to the 4900 and its still happening.

4500-x trunk config

 

interface TenGigabitEthernet1/1 
 switchport trunk native vlan 999
 switchport trunk allowed vlan 99
 switchport mode trunk
 switchport nonegotiate
 mtu 9198

 

4900 trunk config

interface TenGigabitEthernet3/8
  switchport trunk native vlan 999
 switchport trunk allowed vlan 99
 switchport mode trunk
 switchport nonegotiate
 mtu 9198
 

 

Highlighted
Beginner

ttcroziercisco, How long are

ttcroziercisco,

 

How long are your fiber optic cables?  If they are short, what is the requirement to use singlemode rather than multimode?

I highly suggest calculating an optical power budget for your connection, and pad the levels down as necessary.  Also, combined with padding down the optical levels, test the connection with a different set of SFPs on both sides.  If the problem is related to high optical levels, you may have damaged the SFPs.

-rb

CreatePlease to create content
Content for Community-Ad