Nexus 5548UP with 1Gb ether to 10Gb ether on same switch

[RaymondKapperman]

I have a Nexus 5548UP switch with a couple of systems some of which have been configured with 1Gb nic's running various OS's and other that have 10Gb nic's.  When I try to SSH from 1Gb box to a 1Gb box everything is fine, when I SSH from 1Gb to 10Gb box it does not work but 10Gb to 10Gb boxes will work.  The switch is a fresh install and has one VLAN 51 the first 2 ports are setup with LACP and the only differences is speed of 1000 and 10000 for the boxes attached.  Is there some trick or additional configuration that is needed to make 1Gb to 10Gb work for SSH and other applications?  I trying to setup some test boxes now to have a repeatable environment and will update when completed.

!Command: show running-config
!Time: Fri Jan  9 18:30:44 2009

version 5.2(1)N1(4)
hostname ISC5548B

no feature telnet
feature lacp
feature lldp

...... account info removed

ssh key rsa 2048
ip domain-lookup
class-map type qos class-fcoe
class-map type queuing class-fcoe
  match qos-group 1
class-map type queuing class-all-flood
  match qos-group 2
class-map type queuing class-ip-multicast
  match qos-group 2
class-map type network-qos class-fcoe
  match qos-group 1
class-map type network-qos class-all-flood
  match qos-group 2
class-map type network-qos class-ip-multicast
  match qos-group 2
................

vrf context management
  ip route 0.0.0.0/0 192.168.52.1
vlan 1
vlan 51
  name InfoSec
port-profile default max-ports 512

interface port-channel1
  switchport access vlan 51
  speed 1000

interface Ethernet1/1
  lacp port-priority 500
  switchport access vlan 51
  speed 1000
  channel-group 1 mode active

interface Ethernet1/2
  switchport access vlan 51
  speed 1000
  channel-group 1 mode active

interface Ethernet1/3
  switchport access vlan 51
  speed 1000

interface Ethernet1/4
  switchport access vlan 51
  speed 1000

interface Ethernet1/5
  switchport access vlan 51
  speed 1000

interface Ethernet1/6
  switchport access vlan 51
  speed 1000

interface Ethernet1/7
  switchport access vlan 51
  speed 1000

interface Ethernet1/8
  switchport access vlan 51
  speed 1000

interface Ethernet1/9
  switchport access vlan 51
  speed 1000

............

interface Ethernet1/32
  switchport access vlan 51
  speed 1000

interface Ethernet2/1
  switchport access vlan 51

interface Ethernet2/2
  switchport access vlan 51

interface Ethernet2/3
  switchport access vlan 51

interface Ethernet2/4
  switchport access vlan 51

interface Ethernet2/5
  switchport access vlan 51

interface Ethernet2/6
  switchport access vlan 51

interface Ethernet2/7
  switchport access vlan 51

interface Ethernet2/8
  switchport access vlan 51

.....

interface Ethernet2/14
  switchport access vlan 51

interface Ethernet2/15
  switchport access vlan 51

interface Ethernet2/16
  switchport access vlan 51

interface mgmt0
  ip address 192.168.52.4/24
line console
line vty
boot kickstart bootflash:/n5000-uk9-kickstart.5.2.1.N1.4.bin
boot system bootflash:/n5000-uk9.5.2.1.N1.4.bin

ISC5548B# show int brief

--------------------------------------------------------------------------------
Ethernet      VLAN    Type Mode   Status  Reason                   Speed     Port
Interface                                                                    Ch #
--------------------------------------------------------------------------------
Eth1/1        51      eth  access up      none                       1000(D) 1
Eth1/2        51      eth  access up      none                       1000(D) 1
Eth1/3        51      eth  access up      none                       1000(D) --
Eth1/4        51      eth  access up      none                       1000(D) --
Eth1/5        51      eth  access up      none                       1000(D) --
Eth1/6        51      eth  access up      none                       1000(D) --
Eth1/7        51      eth  access up      none                       1000(D) --
Eth1/8        51      eth  access up      none                       1000(D) --
Eth1/9        51      eth  access up      none                       1000(D) --
Eth1/10       51      eth  access up      none                       1000(D) --
Eth1/11       51      eth  access up      none                       1000(D) --
Eth1/12       51      eth  access up      none                       1000(D) --
Eth1/13       51      eth  access down    Link not connected         1000(D) --
Eth1/14       51      eth  access down    Link not connected         1000(D) --
Eth1/15       51      eth  access up      none                       1000(D) --
Eth1/16       51      eth  access down    Link not connected         1000(D) --
Eth1/17       51      eth  access down    SFP not inserted           1000(D) --
Eth1/18       51      eth  access down    SFP not inserted           1000(D) --

........

Eth1/28       51      eth  access down    SFP not inserted           1000(D) --
Eth1/29       51      eth  access down    SFP not inserted           1000(D) --
Eth1/30       51      eth  access down    SFP not inserted           1000(D) --
Eth1/31       51      eth  access down    SFP not inserted           1000(D) --
Eth1/32       51      eth  access down    SFP not inserted           1000(D) --
Eth2/1        51      eth  access up      none                        10G(D) --
Eth2/2        51      eth  access up      none                        10G(D) --
Eth2/3        51      eth  access up      none                        10G(D) --
Eth2/4        51      eth  access up      none                        10G(D) --
Eth2/5        51      eth  access up      none                        10G(D) --
Eth2/6        51      eth  access up      none                        10G(D) --
Eth2/7        51      eth  access down    SFP not inserted            10G(D) --
Eth2/8        51      eth  access down    SFP not inserted            10G(D) --
Eth2/9        51      eth  access down    SFP not inserted            10G(D) --
Eth2/10       51      eth  access down    SFP not inserted            10G(D) --
Eth2/11       51      eth  access down    SFP not inserted            10G(D) --
Eth2/12       51      eth  access down    SFP not inserted            10G(D) --
Eth2/13       51      eth  access down    Link not connected          10G(D) --
Eth2/14       51      eth  access down    SFP not inserted            10G(D) --
Eth2/15       51      eth  access down    Link not connected          10G(D) --
Eth2/16       51      eth  access down    Link not connected          10G(D) --

--------------------------------------------------------------------------------
Port-channel VLAN    Type Mode   Status  Reason                    Speed   Protocol
Interface
--------------------------------------------------------------------------------
Po1          51      eth  access up      none                      a-1000(D)  lacp

--------------------------------------------------------------------------------
Port   VRF          Status IP Address                              Speed    MTU
--------------------------------------------------------------------------------
mgmt0  --           up     192.168.52.4                            100      1500

ISC5548B# show interface ethernet 1/3
Ethernet1/3 is up
 Dedicated Interface
  Hardware: 1000/10000 Ethernet, address: 8c60.4f49.818a (bia 8c60.4f49.818a)
  MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec
  reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA
  Port mode is access
  full-duplex, 1000 Mb/s, media type is 10G
  Beacon is turned off
  Input flow-control is off, output flow-control is off
  Rate mode is dedicated
  Switchport monitor is off
  EtherType is 0x8100
  Last link flapped 1d02h
  Last clearing of "show interface" counters 1d20h
  30 seconds input rate 0 bits/sec, 0 packets/sec
  30 seconds output rate 304 bits/sec, 0 packets/sec
  Load-Interval #2: 5 minute (300 seconds)
    input rate 0 bps, 0 pps; output rate 200 bps, 0 pps
  RX
    0 unicast packets  0 multicast packets  0 broadcast packets
    0 input packets  0 bytes
    0 jumbo packets  0 storm suppression bytes
    0 runts  0 giants  0 CRC  0 no buffer
    0 input error  0 short frame  0 overrun   0 underrun  0 ignored
    0 watchdog  0 bad etype drop  0 bad proto drop  0 if down drop
    0 input with dribble  0 input discard
    0 Rx pause
  TX
    202 unicast packets  87027 multicast packets  1077 broadcast packets
    88306 output packets  7601016 bytes
    0 jumbo packets
    0 output errors  0 collision  0 deferred  0 late collision
    0 lost carrier  0 no carrier  0 babble 0 output discard
    0 Tx pause
  4 interface resets

ISC5548B# show interface ethernet 2/1
Ethernet2/1 is up
 Dedicated Interface
  Hardware: 1000/10000 Ethernet, address: 8c60.4f23.5de0 (bia 8c60.4f23.5de0)
  MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec
  reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA
  Port mode is access
  full-duplex, 10 Gb/s, media type is 10G
  Beacon is turned off
  Input flow-control is off, output flow-control is off
  Rate mode is dedicated
  Switchport monitor is off
  EtherType is 0x8100
  Last link flapped 1d20h
  Last clearing of "show interface" counters never
  30 seconds input rate 0 bits/sec, 0 packets/sec
  30 seconds output rate 312 bits/sec, 0 packets/sec
  Load-Interval #2: 5 minute (300 seconds)
    input rate 0 bps, 0 pps; output rate 200 bps, 0 pps
  RX
    285 unicast packets  0 multicast packets  218 broadcast packets
    503 input packets  32724 bytes
    0 jumbo packets  0 storm suppression bytes
    0 runts  0 giants  0 CRC  0 no buffer
    0 input error  0 short frame  0 overrun   0 underrun  0 ignored
    0 watchdog  0 bad etype drop  0 bad proto drop  0 if down drop
    0 input with dribble  0 input discard
    0 Rx pause
  TX
    406 unicast packets  89109 multicast packets  1077 broadcast packets
    90592 output packets  7827146 bytes
    0 jumbo packets
    0 output errors  0 collision  0 deferred  0 late collision
    0 lost carrier  0 no carrier  0 babble 0 output discard
    0 Tx pause
  1 interface resets

[Reza Sharifi]

What is the output of "sh portch summary" and "sh int po1"?

There isn't special config needed it to go between 1G and 10G ports.

What you say 1G to 10G, are you referring to the application/host that is connected to each port?

HTH

[RaymondKapperman]

I have a laptop connected to a port on the switch and I am currently adding a SUSE 11 SP2 to another port that has a 10G card for a NIC.  Once I finished the install I will post more specific tests using ping and SSH to illustrate. 

Here is the output you requested.

ISC5548B# show port-channel summary
Flags:  D - Down        P - Up in port-channel (members)
        I - Individual  H - Hot-standby (LACP only)
        s - Suspended   r - Module-removed
        S - Switched    R - Routed
        U - Up (port-channel)
        M - Not in use. Min-links not met
--------------------------------------------------------------------------------
Group Port-       Type     Protocol  Member Ports
      Channel
--------------------------------------------------------------------------------
1     Po1(SU)     Eth      LACP      Eth1/1(P)    Eth1/2(P)
ISC5548B# sh int po1
port-channel1 is up
  Hardware: Port-Channel, address: 8c60.4f49.8188 (bia 8c60.4f49.8188)
  MTU 1500 bytes, BW 2000000 Kbit, DLY 10 usec
  reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA
  Port mode is access
  full-duplex, 1000 Mb/s
  Input flow-control is off, output flow-control is off
  Switchport monitor is off
  EtherType is 0x8100
  Members in this channel: Eth1/1, Eth1/2
  Last clearing of "show interface" counters never
  30 seconds input rate 560 bits/sec, 0 packets/sec
  30 seconds output rate 3064 bits/sec, 2 packets/sec
  Load-Interval #2: 5 minute (300 seconds)
    input rate 128 bps, 0 pps; output rate 2.74 Kbps, 2 pps
  RX
    578695 unicast packets  11167 multicast packets  1056 broadcast packets
    590918 input packets  760063851 bytes
    0 jumbo packets  0 storm suppression bytes
    0 runts  0 giants  0 CRC  0 no buffer
    0 input error  0 short frame  0 overrun   0 underrun  0 ignored
    0 watchdog  0 bad etype drop  0 bad proto drop  0 if down drop
    0 input with dribble  0 input discard
    0 Rx pause
  TX
    392417 unicast packets  434693 multicast packets  2984 broadcast packets
    830094 output packets  320257324 bytes
    0 jumbo packets
    0 output errors  0 collision  0 deferred  0 late collision
    0 lost carrier  0 no carrier  0 babble 0 output discard
    0 Tx pause
  1 interface resets

[RaymondKapperman]

OK,  The issue has been resolved.

What was neglected in the first post was that I just got this new switch because we had the same switch with the same basic config but the difference being that we suspected the switch to be bad.  It had indicated failed ports in the 1Gb slots about 7 of 32 were noted as failing to receive a 10Gb SFP but would accept a 1Gb SFP.  So we testing not using the bad ports on the switch and still had the issue.  Nobody here expected the switch to fail in that manner but we ordered a new 5548.  I was still rebuilding the environment today when I posted the question and now that I have rebuilt the network with 2 hosts and tested I was able to go out the 10gb and join the domain, ssh to 10gb hosts, ssh to 1gb hosts and go from 1gb hosts to the 10gb host.  Thanks for the input and confirming that there is no config special to get 1Gb to talk too 10Gb ports.