40GBASE-SR-BD Compatible Fiber

[CiscoMedMed]

I have multimode fiber running between two cages at our data center to migrate from one to the other. I have a NX6K on one side and NX9K on the other and 40GBASE-SR-BD for the optics. I can't get the link to come up. I haven't yet changed polarity. I'll try that tomorrow. But I wanted to rule out other issues. The fiber I'm using for jumpering is LC to LC Fiber Patch Cable Multimode Duplex - 3m (9.84ft) - 50/125um OM2. But now I'm not seeing that in the guide below table 1. The cages are only a few meters from each other. I don't have the detailed specs for inter-cage run (maybe 20m) other than it's multi-mode. But if I could rule in or out the OM2 patch cables as likely or not to work that would be a good help. Thanks.

https://www.cisco.com/c/en/us/products/collateral/switches/nexus-9000-series-switches/white-paper-c11-729908.html

https://www.amazon.com/gp/product/B00NFR9GLY/ref=ppx_yo_dt_b_asin_title_o04_s00?ie=UTF8&psc=1

[balaji.bandi]

since you mentioned it is less (but still required proper patch lead to work)

Make sure patch connect right way example :

             Light-----------------no Light

SFP1                                                             SFP2

           no light -----------------------Light

Try  speed no nogiation let us know if that works.

here is the distance cisco point of view:

https://www.cisco.com/c/en/us/products/collateral/interfaces-modules/gigabit-ethernet-gbic-sfp-modules/datasheet-c78-366584.html

still issue post config both the sides

show run interface x/x

show interface x/x trans

show interface x/x

[CiscoMedMed]

So I was able to see light in one fiber strand of the pair on each side. Hopeful it was just a polarity issue I swapped A and B on one side. But the SFP still fails top come up.

Nx9K Output:

interface Ethernet1/50                                                        

  switchport                                                                  

  switchport mode trunk                                                       

  switchport trunk allowed vlan 1,102,104,254                                 

  no shutdown   

CORE01# sho interface eth 1/50 transceiver                               

Ethernet1/50                                                                  

    transceiver is present                                                    

    type is QSFP-40G-SR-BD                                                    

    name is CISCO-AVAGO                                                       

    part number is AFBR-79EBPZ-CS2                                            

    revision is 01                                                            

    serial number is AVM2035U1FS                                              

    nominal bitrate is 20600 MBit/sec per channel                             

    Link length supported for 50/125um OM3 fiber is 100 m                     

    cisco id is 13                                                            

    cisco extended id number is 220                                           

    cisco part number is 10-2945-02                                           

    cisco product id is QSFP-40G-SR-BD   

CORE01# sho interface eth 1/50                                                                                        

Ethernet1/50 is down (Link not connected)                                                                                  

admin state is up, Dedicated Interface                                                                                     

  Hardware: 1000/10000/25000/40000/50000/100000 Ethernet, address: cc16.7eba.920                                           

d (bia cc16.7eba.920d)                                                                                                     

  MTU 1500 bytes, BW 100000000 Kbit, DLY 10 usec                                                                           

  reliability 255/255, txload 1/255, rxload 1/255                                                                          

  Encapsulation ARPA, medium is broadcast                                                                                  

  Port mode is trunk                                                                                                       

  auto-duplex, auto-speed, media type is 40G                                                                               

  Beacon is turned off                                                                                                     

  Auto-Negotiation is turned on  FEC mode is Auto                                                                          

  Input flow-control is off, output flow-control is off                                                                    

  Auto-mdix is turned off                                                                                                  

  Rate mode is dedicated                                                                                                   

  Switchport monitor is off                                                                                                

  EtherType is 0x8100                                                                                                      

  EEE (efficient-ethernet) : n/a                                                                                           

  Last link flapped 00:58:25                                                                                               

  Last clearing of "show interface" counters never                                                                         

  1 interface resets                                                                                                       

  30 seconds input rate 0 bits/sec, 0 packets/sec                                                                          

  30 seconds output rate 0 bits/sec, 0 packets/sec                                                                         

  Load-Interval #2: 5 minute (300 seconds)                                                                                 

    input rate 0 bps, 0 pps; output rate 0 bps, 0 pps                                                                      

  RX                                                                                                                       

    0 unicast packets  42 multicast packets  0 broadcast packets                                                           

    42 input packets  5730 bytes                                                                                           

    0 jumbo packets  0 storm suppression bytes                                                                             

    0 runts  0 giants  0 CRC  0 no buffer                                                                                  

    0 input error  0 short frame  0 overrun   0 underrun  0 ignored                                                        

    0 watchdog  0 bad etype drop  0 bad proto drop  0 if down drop                                                         

    0 input with dribble  0 input discard                                                                                  

    0 Rx pause                                                                                                             

  TX                                                                                                                       

    0 unicast packets  18 multicast packets  0 broadcast packets                                                           

    18 output packets  4244 bytes                                                                                          

    0 jumbo packets                                                                                                        

    0 output error  0 collision  0 deferred  0 late collision                                                              

    0 lost carrier  0 no carrier  0 babble  0 output discard                                                               

    0 Tx pause                      

NX6K:

SAC-CORE01# sho inter eth 2/3
Ethernet2/3 is down (Link not connected)
Dedicated Interface
Hardware: 40000 Ethernet, address: 002a.6aa1.4afa (bia 002a.6aa1.4afa)
MTU 1500 bytes, BW 40000000 Kbit, DLY 10 usec
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA
Port mode is trunk
full-duplex, 40 Gb/s, media type is 40G
Beacon is turned off
Input flow-control is off, output flow-control is off
Rate mode is dedicated
Switchport monitor is off
EtherType is 0x8100
Last link flapped 3week(s) 3day(s)
Last clearing of "show interface" counters never
1 interface resets
30 seconds input rate 0 bits/sec, 0 packets/sec
30 seconds output rate 0 bits/sec, 0 packets/sec
Load-Interval #2: 5 minute (300 seconds)
input rate 0 bps, 0 pps; output rate 0 bps, 0 pps
RX
0 unicast packets 2969981 multicast packets 0 broadcast packets
2969981 input packets 3207579440 bytes
0 jumbo packets 0 storm suppression bytes
0 runts 0 giants 0 CRC 0 no buffer
0 input error 0 short frame 0 overrun 0 underrun 0 ignored
0 watchdog 0 bad etype drop 0 bad proto drop 0 if down drop
0 input with dribble 0 input discard
0 Rx pause
TX
0 unicast packets 3179918 multicast packets 0 broadcast packets
3179918 output packets 2086026188 bytes
0 jumbo packets
0 output error 0 collision 0 deferred 0 late collision
0 lost carrier 0 no carrier 0 babble 0 output discard
0 Tx pause

[CiscoMedMed]

It looks like the problem is the fiber run between the cages was ordered as SINGLE mode and the SFP's are for multi-mode.  

Is there a single mode version of 40GBASE-SR-BD? Thank you.

[balaji.bandi · Accepted solution]

rather getting Single-mode - get MMF cable if this is nearby and lay and test(by running cable)

check single-mode SFP

https://www.cisco.com/c/en/us/products/collateral/interfaces-modules/transceiver-modules/data_sheet_c78-660083.html

[CiscoMedMed]

I earlier ran a sanity check between the two 9Ks with multi-mode fiber and those lit up right away. Unfortunately I don't have a long enough patch cable to do the same between cages. But I am getting a quote on running MMF between the cages. I am also getting a quote on 40Gbps single mode SFPs. But on first glance those look very expensive - around $8k or so each?

[balaji.bandi · Accepted solution]

yes that is what my advice, those are not cheap as expected, so running MMF cable grand or 2 will be the maximum, if I were you, I lay my self-ordering cable, if that is my DC, i do many times. (this where sometimes add advantage audit before the order).

[lolamano98901]

Thanks for the tips @balaji.bandi , that will help me for sure!