Nexus 3172T bonding doesn't work

inakiss · ‎05-10-2023

Hello,

We have create a LACP of two ports. These are the instructions to created:

switch# conf t
switch(config)# interface ethernet 1/5
switch(config-if)# channel-group 2 mode active
switch(config-if)#exit
switch# conf t
switch(config)#  interface ethernet 1/6
switch(config-if)# channel-group 2 mode active
switch(config-if)#exit

When we do a summary of port-channel, it display:

switch# show port-channel summary
Flags:  D - Down        P - Up in port-channel (members)
        I - Individual  H - Hot-standby (LACP only)
        s - Suspended   r - Module-removed
        b - BFD Session Wait
        S - Switched    R - Routed
        U - Up (port-channel)
        p - Up in delay-lacp mode (member)
        M - Not in use. Min-links not met
--------------------------------------------------------------------------------
Group Port-       Type     Protocol  Member Ports
      Channel
--------------------------------------------------------------------------------
2     Po2(SU)     Eth      LACP      Eth1/5(P)    Eth1/6(P)

But when we list info of port-channel 2, it display is full-duplex, 10Gb/s. We think that must be 20Gb/s.

switch#  show interface port-channel 2
port-channel2 is up
admin state is up,
  Hardware: Port-Channel, address: e865.492e.c94d (bia e865.492e.c94d)
  MTU 1500 bytes, BW 20000000 Kbit , DLY 10 usec
  reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, medium is broadcast
  Port mode is access
  full-duplex, 10 Gb/s
  Input flow-control is off, output flow-control is off
  Auto-mdix is turned off
  Switchport monitor is off
  EtherType is 0x8100
  Members in this channel: Eth1/5, Eth1/6
  Last clearing of "show interface" counters never
  1 interface resets
  Load-Interval #1: 30 seconds
    30 seconds input rate 30720 bits/sec, 13 packets/sec
    30 seconds output rate 162488 bits/sec, 31 packets/sec
    input rate 30.72 Kbps, 13 pps; output rate 162.49 Kbps, 31 pps
  Load-Interval #2: 5 minute (300 seconds)
    300 seconds input rate 24672 bits/sec, 15 packets/sec
    300 seconds output rate 100688 bits/sec, 28 packets/sec
    input rate 24.67 Kbps, 15 pps; output rate 100.69 Kbps, 28 pps
  RX
    1887370 unicast packets  476 multicast packets  249 broadcast packets
    1888095 input packets  179148007 bytes
    0 jumbo packets  0 storm suppression packets
    0 runts  0 giants  0 CRC  0 no buffer
    0 input error  0 short frame  0 overrun   0 underrun  0 ignored
    0 watchdog  0 bad etype drop  0 bad proto drop  0 if down drop
    0 input with dribble  0 input discard
    0 Rx pause
  TX
    25357213 unicast packets  2588 multicast packets  1817 broadcast packets
    25361618 output packets  38197127309 bytes
    0 jumbo packets
    0 output error  0 collision  0 deferred  0 late collision
    0 lost carrier  0 no carrier  0 babble  0 output discard
    0 Tx pause

Does anyone know how to change it so that it is at 20Gb/s?

Switch info:

Hardware
cisco Nexus 3172T Chassis
Intel(R) Pentium(R) CPU @ 2.00GHz with 4029688 kB of memory.

Software
BIOS: version 5.3.1
NXOS: version 9.3(7)

Thank you,

I

Flavio Miranda · ‎05-10-2023

Hi

switch#  show interface port-channel 2
port-channel2 is up
admin state is up,
  Hardware: Port-Channel, address: e865.492e.c94d (bia e865.492e.c94d)
  MTU 1500 bytes, BW 20000000 Kbit , DLY 10 usec

The important here is the BW of 20Gbps.

You can see more detail on this discussion here

https://community.cisco.com/t5/unified-computing-system-discussions/fabric-interconnect-uplink-port-channeling/td-p/1895155

inakiss · ‎05-10-2023

Hi Flavio,

Thanks to answering. If the bond is configured a 20 Gb/s, why do we have the same transfer speed copying files with one cable as with two?

Thanks,

I

Flavio Miranda · ‎05-10-2023

@inakiss then we need to undestant better. File transfer is often a problem and it may not be related to this port-channel specifically but with the environment as a whole. For example, the source and destination can the the problem and anything in between.

Please, can you describde to me how is the flow and the parts involved? Does it cross any security device? Which kind of file is it?

inakiss · ‎05-10-2023

Sorry Favio, but I'm out of the office. I'll be back next week and then I'll be able to describe better how our network is.

Thank you,

I

Christopher Hart · ‎05-10-2023

Hello!

If you are testing the bandwidth of this port-channel with a single file transfer "session" (such that the flow of traffic is between two specific source and destination IP addresses and the TCP/UDP headers have the same port numbers), then the 10Gbps limitation in bandwidth you are experiencing is most likely due to a phenomenon called polarization.

On my personal blog, I have a post titled "Understanding Load Balancing on Network Devices" that shows a practical example of polarization on a port-channel interface similar to the topology you described. Hopefully, this blog post explains this in more detail and answers any questions you may have.

Thank you!

-Christopher

MHM Cisco World · ‎05-10-2023

What I see is correct'

Speed must match in all port member and port channel

BW of PO is equal to sum of BW of port member.

File transfer speed is not change since the bit will transfer per one port

But how many SW can send is different' now SW can use multi port to send same data.