topic Re: cisco c9300 DAC to HP server in Optics and Optical Networking

cisco c9300 DAC to HP server

AbedB — Mon, 29 Nov 2021 05:52:15 GMT

We installed new Cisco C9300 stack (v16.12.4)

and connected ports Te1/1/1-4 and Te2/1/1-4 to 4 HP DL360 servers (each HP server has two NICs)

So Te1/1/1-4 is connected to all 4 HP servers

And the other Te2/1/1-4 is also connected to the same 4 HP servers but the other NICs

The problem is that only Te2/1/1-4 is going UP. the other Te1/1/1-4 is still Down.

All transceivers vendor is Cisco. So they're compatible.

we crossed between the cables

we tried shut/no shut

we rebooted the servers ...

nothing suspicious appeared in the "show logg"

we consulted with HP and they blame cisco (please don't ask me why)

Not sure what we should do in order to bring the other 4 interfaces UP.

Has anyone came across this problem?

Re: cisco c9300 DAC to HP server

Leo Laohoo — Mon, 29 Nov 2021 06:38:46 GMT

Please stop using 16.12.X.

Re: cisco c9300 DAC to HP server

AbedB — Mon, 29 Nov 2021 06:42:22 GMT

This is what we get from the box

Which version you recommend for this switch?

Re: cisco c9300 DAC to HP server

Leo Laohoo — Mon, 29 Nov 2021 06:47:05 GMT

@AbedB wrote:

Which version you recommend for this switch?

Whatever version but 16.12.X.

Re: cisco c9300 DAC to HP server

AbedB — Mon, 29 Nov 2021 07:51:57 GMT

Done

We upgraded to 17.06.02

But still we're facing the same problem

CORE_SW#sh ip int brie | i Te
Te1/1/1                unassigned      YES unset  down                  down    
Te1/1/2                unassigned      YES unset  down                  down    
Te1/1/3                unassigned      YES unset  down                  down    
Te1/1/4                unassigned      YES unset  down                  down    
Te2/1/1                unassigned      YES unset  up                    up      
Te2/1/2                unassigned      YES unset  up                    up      
Te2/1/3                unassigned      YES unset  up                    up      
Te2/1/4                unassigned      YES unset  up                    up

Switch Ports Model              SW Version        SW Image              Mode   
------ ----- -----              ----------        ----------            ----   
*    1 41    C9300-24T          17.06.02          CAT9K_IOSXE           BUNDLE 
     2 41    C9300-24T          17.06.02          CAT9K_IOSXE           BUNDLE

Re: cisco c9300 DAC to HP server

Leo Laohoo — Mon, 29 Nov 2021 08:01:40 GMT

Post the output to the following commands:

sh inventory
sh log

Re: cisco c9300 DAC to HP server

AbedB — Mon, 29 Nov 2021 08:09:30 GMT

in the show logg you will see nothing but the UP ports:

*Nov 29 07:46:54.453: %LINK-3-UPDOWN: Interface TenGigabitEthernet2/1/1, changed state to up
*Nov 29 07:46:54.461: %LINK-3-UPDOWN: Interface TenGigabitEthernet2/1/2, changed state to up
*Nov 29 07:46:54.468: %LINK-3-UPDOWN: Interface TenGigabitEthernet2/1/3, changed state to up
*Nov 29 07:46:54.476: %LINK-3-UPDOWN: Interface TenGigabitEthernet2/1/4, changed state to up
*Nov 29 07:46:56.455: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/1/1, changed state to up
*Nov 29 07:46:56.463: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/1/2, changed state to up
*Nov 29 07:46:56.470: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/1/3, changed state to up
*Nov 29 07:46:56.477: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/1/4, changed state to up

Nothing there regarding the Down ports. Lokks like the Cisco does not recognize even the SFP transceivers.

In show inventory port 1-4 is not recognized:

#show inventory         
NAME: "c93xx Stack", DESCR: "c93xx Stack"
PID: C9300-24T         , VID: V04  , SN: FOC25012345

NAME: "Switch 1", DESCR: "C9300-24T"
PID: C9300-24T         , VID: V04  , SN: FOC25012345

NAME: "StackPort1/1", DESCR: "StackPort1/1"
PID: STACK-T1-50CM     , VID: V01  , SN: MOC25012345

NAME: "StackPort1/2", DESCR: "StackPort1/2"
PID: STACK-T1-50CM     , VID: V01  , SN: MOC25012345

NAME: "Switch 1 - Power Supply A", DESCR: "Switch 1 - Power Supply A"
PID: PWR-C1-350WAC-P   , VID: V02  , SN: DCC24512345

NAME: "Switch 1 - Power Supply B", DESCR: "Switch 1 - Power Supply B"
PID: PWR-C1-350WAC-P   , VID: V02  , SN: DCC24512345

NAME: "Switch 1 FRU Uplink Module 1", DESCR: "8x10G Uplink Module"
PID: C9300-NM-8X       , VID: V03  , SN: FJZ25012345

NAME: "Te1/1/7", DESCR: "SFP-10GBase-LR"
PID: SFP-10G-LR-S        , VID: V01  , SN: OPM22412345     

NAME: "Te1/1/8", DESCR: "1000BaseLX SFP"
PID:                     , VID:      , SN: AGC14012345

Only 7+8 which they're SM transceivers working fine.

Re: cisco c9300 DAC to HP server

Mark Elsen — Mon, 29 Nov 2021 08:18:38 GMT

>All transceivers vendor is Cisco. So they're compatible.... > Cisco does not recognize even the SFP transceivers.

- The SFP's need to be compatible with the specific-9300 model too ,verify with :

https://tmgmatrix.cisco.com/

Re: cisco c9300 DAC to HP server

AbedB — Mon, 29 Nov 2021 08:22:02 GMT

They are

All 8 twinax SFPs are same vender (Cisco) , all same: SFP-H10GB-CU3M

Only 4 go up and other 4 go down

Anyway, we escalated the issue again to HP TAC, We suspect problem in there NICs

We're waiting for them to respond

I'll update also here for sure

Thanks 🙂

Re: cisco c9300 DAC to HP server

Leo Laohoo — Mon, 29 Nov 2021 08:33:06 GMT

Swap those DAC cables around. Move the DAC from switch 2 to switch 1. Move DAC from switch 2 to switch 1.

Observe if the problem "stays" with switch 1 or moves to switch 2.

Next, when doing this, pay attention to the logs. During the removal or insertion, there should be log entries. We want to see that.

Re: cisco c9300 DAC to HP server

AbedB — Mon, 29 Nov 2021 08:44:42 GMT

So its funny yeah ...

We went to the DataCenter to physically see the problem

And ... Seems like the system engineers and the vendor engineer is not that much in engineering when they can't tell in HP server the difference between fiber-channel port and regular SFP port (attached photo)

And believe me, I just summarized two essays feelings in two lines ...

Problem solved