LACP trunk between BNT layer2/3 switch and Cisco 2960s stack on a bladeCenter H

eric.philippon · ‎03-10-2016

My objective is to replace a Nortel 5520 stack with a stack of Cisco 2960s. The old working configuration was using mlt trunk between 5520 and BNT layer 2/3 switch. I will use LACP for the trunk between the Cisco and BNT switches, and PVST is activated on the Cisco.

To make the configurations of all switches, I followed chapter 7.7 of this document : Nortel Networks L2/3 Ethernet Switch Module for IBM eServer BladeCenter (https://lenovopress.com/redp3586.pdf)

My problem is that bld1 and bld2 aren’t reachable if both Gbesm are powered on, but if only one is up, the configuration is working.

NBT switches :

/* bc-io1          /**** bc-io2 is the same **** /
> /c/sys
>         hprompt ena
> /c/sys/access/user
>         admpw "xxxxx"
> /c/sys/ssnmp
>         name "bc-io1"
>         wcomm "public"
/c/port INT1                 
        pvid 11

/**** INT1 ... INT14 are identical ****/

 /c/port INT14
        pvid 11

/c/port EXT1
        dis

 /**** EXT1 ... EXT4 are identical, EXT5-6 not used ****/

/c/port EXT6
        tag ena
        pvid 11
        tagpvid ena
/c/l2/bpdugrd ena
/c/l2/vlan 1
        def 0
/c/l2/vlan 10
        ena
        name "Admin"
        def INT1 INT2 INT3 INT4 INT5 INT6 INT7 INT8 INT9 INT10 INT11 INT12 INT13 INT14 EXT1 EXT2 EXT3 EXT4 EXT5 EXT6
/c/l2/vlan 11
        ena
        name "PROD1"
        def INT1 INT2 INT3 INT4 INT5 INT6 INT7 INT8 INT9 INT10 INT11 INT12 INT13 INT14 EXT1 EXT2 EXT3 EXT4 EXT5 EXT6
/c/l2/vlan 12
        ena
        name "PROD2"
        def INT1 INT2 INT3 INT4 INT5 INT6 INT7 INT8 INT9 INT10 INT11 INT12 INT13 INT14 EXT1 EXT2 EXT3 EXT4 EXT5 EXT6
/c/l2/stg 1/off
/c/l2/stg 1/port EXT1/off
/c/l2/stg 1/port EXT2/off
/c/l2/stg 1/port EXT3/off
/c/l2/stg 1/port EXT4/off
/c/l2/stg 10/clear
/c/l2/stg 10/brg/prior 65535
/c/l2/stg 10/add 10
/c/l2/stg 11/clear
/c/l2/stg 11/brg/prior 65535
/c/l2/stg 11/add 11
/c/l2/stg 12/clear
/c/l2/stg 12/brg/prior 65535
/c/l2/stg 12/add 12
/c/l2/lacp/port EXT1
        mode active
/c/l2/lacp/port EXT2
        mode active
        adminkey 17
/c/l2/lacp/port EXT3
        mode active
        adminkey 17
/c/l2/lacp/port EXT4
        mode active
        adminkey 17
/c/l3/frwd/off
>> bc-io1 - Configuration#

Cisco :

Current configuration : 13689 bytes
!
! Last configuration change at 15:11:18 UTC Wed Mar 9 2016 by manager
! NVRAM config last updated at 14:16:34 UTC Mon Mar 7 2016 by manager
!
version 15.0
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
service sequence-numbers
!
hostname sw-ipu-stk
!
boot-start-marker
boot-end-marker
!
enable secret 5 $$$$
!
username manager secret 5 $$$$$
aaa new-model
!
!
aaa authentication login default local
!
!
!
!
!
!
aaa session-id common
switch 1 provision ws-c2960s-24ts-l
switch 2 provision ws-c2960s-24ts-l
!
!
ip domain-name srs.net
udld aggressive

!
mls qos map cos-dscp 0 8 16 24 32 46 46 56
mls qos srr-queue output cos-map queue 1 threshold 3 4 5
mls qos srr-queue output cos-map queue 2 threshold 1 2
mls qos srr-queue output cos-map queue 2 threshold 2 3
mls qos srr-queue output cos-map queue 2 threshold 3 6 7
mls qos srr-queue output cos-map queue 3 threshold 3 0
mls qos srr-queue output cos-map queue 4 threshold 3 1
mls qos srr-queue output dscp-map queue 1 threshold 3 32 33 40 41 42 43 44 45
mls qos srr-queue output dscp-map queue 1 threshold 3 46 47
mls qos srr-queue output dscp-map queue 2 threshold 1 16 17 18 19 20 21 22 23
mls qos srr-queue output dscp-map queue 2 threshold 1 26 27 28 29 30 31 34 35
mls qos srr-queue output dscp-map queue 2 threshold 1 36 37 38 39
mls qos srr-queue output dscp-map queue 2 threshold 2 24
mls qos srr-queue output dscp-map queue 2 threshold 3 48 49 50 51 52 53 54 55
mls qos srr-queue output dscp-map queue 2 threshold 3 56 57 58 59 60 61 62 63
mls qos srr-queue output dscp-map queue 3 threshold 3 0 1 2 3 4 5 6 7
mls qos srr-queue output dscp-map queue 4 threshold 1 8 9 11 13 15
mls qos srr-queue output dscp-map queue 4 threshold 2 10 12 14
mls qos queue-set output 1 threshold 1 100 100 50 200
mls qos queue-set output 1 threshold 2 125 125 100 400
mls qos queue-set output 1 threshold 3 100 100 100 400
mls qos queue-set output 1 threshold 4 60 150 50 200
mls qos queue-set output 1 buffers 15 25 40 20
mls qos
!
crypto pki trustpoint HTTPS_SS_CERT_KEYPAIR
 enrollment selfsigned
 serial-number
 revocation-check none
 rsakeypair HTTPS_SS_CERT_KEYPAIR
!
!
crypto pki certificate chain HTTPS_SS_CERT_KEYPAIR
 ...
 !
spanning-tree mode pvst
spanning-tree loopguard default
spanning-tree portfast bpduguard default
spanning-tree portfast bpdufilter default
spanning-tree extend system-id
spanning-tree vlan 10-12 priority 24576
!
!
!
!
!
errdisable recovery cause link-flap
errdisable recovery interval 60
port-channel load-balance dst-ip
!
!
!
!
vlan internal allocation policy ascending
!
ip ssh logging events
ip ssh version 2
lldp run
!
!
!
!
!
!
!
!
!
macro global description cisco-global
!
interface Port-channel1
 description UP_BC1-IO1
 switchport trunk native vlan 11
 switchport trunk allowed vlan 10-12
 switchport mode trunk
 switchport nonegotiate
 storm-control broadcast level 5.00
 spanning-tree bpdufilter enable
!
interface Port-channel2
 description UP_BC2-IO2
 switchport trunk native vlan 11
 switchport trunk allowed vlan 10-12
 switchport mode trunk
 switchport nonegotiate
 storm-control broadcast level 5.00
 spanning-tree bpdufilter enable
!
!
interface Port-channel5
 description UP_sw-core
 switchport trunk native vlan 11
 switchport trunk allowed vlan 1,11
 switchport mode trunk
 storm-control broadcast level 5.00
 spanning-tree bpdufilter enable
!
interface Port-channel6
 description UP_sw-admin
 switchport trunk native vlan 10
 switchport trunk allowed vlan 10
 switchport mode trunk
 storm-control broadcast level 5.00
!
interface FastEthernet0
 ip address 10.0.0.1 255.255.255.0
!
interface GigabitEthernet1/0/1
 description BC1-IO1-1
 switchport trunk native vlan 11
 switchport trunk allowed vlan 10-12
 switchport mode trunk
 switchport nonegotiate
 storm-control broadcast level 5.00
 channel-group 1 mode active
!
interface GigabitEthernet1/0/2
 description BC1-IO1-2
 switchport trunk native vlan 11
 switchport trunk allowed vlan 10-12
 switchport mode trunk
 switchport nonegotiate
 storm-control broadcast level 5.00
 channel-group 1 mode active
!
interface GigabitEthernet1/0/3
 description BC1-IO2-1
 switchport trunk native vlan 11
 switchport trunk allowed vlan 10-12
 switchport mode trunk
 switchport nonegotiate
 shutdown
 storm-control broadcast level 5.00
 channel-protocol lacp
 channel-group 2 mode active
!
interface GigabitEthernet1/0/4
 description BC1-IO2-2
 switchport trunk native vlan 11
 switchport trunk allowed vlan 10-12
 switchport mode trunk
 switchport nonegotiate
 shutdown
 storm-control broadcast level 5.00
 channel-protocol lacp
 channel-group 2 mode active
!
interface GigabitEthernet1/0/5
 shutdown
!
interface GigabitEthernet1/0/6
 shutdown
!
interface GigabitEthernet1/0/7
 shutdown
!
interface GigabitEthernet1/0/8
 shutdown
!
interface GigabitEthernet1/0/9
 shutdown
!
interface GigabitEthernet1/0/10
 shutdown
!
interface GigabitEthernet1/0/11
 shutdown
!
interface GigabitEthernet1/0/12
 shutdown
!
interface GigabitEthernet1/0/13
 description UP_sw-core
 switchport trunk native vlan 11
 switchport trunk allowed vlan 1,11
 switchport mode trunk
 storm-control broadcast level 5.00
 channel-protocol lacp
 channel-group 5 mode active
!
interface GigabitEthernet1/0/14
 description UP_sw-core
 switchport trunk native vlan 11
 switchport trunk allowed vlan 1,11
 switchport mode trunk
 storm-control broadcast level 5.00
 channel-protocol lacp
 channel-group 5 mode active
!
interface GigabitEthernet1/0/15
 shutdown
!
interface GigabitEthernet1/0/16
 shutdown
!
interface GigabitEthernet1/0/17
 shutdown
!
interface GigabitEthernet1/0/18
 shutdown
!
interface GigabitEthernet1/0/19
 shutdown
!
interface GigabitEthernet1/0/20
 shutdown
!
interface GigabitEthernet1/0/21
 shutdown
!
interface GigabitEthernet1/0/22
 shutdown
!
interface GigabitEthernet1/0/23
 shutdown
!
interface GigabitEthernet1/0/24
 description sw-adm-23
 switchport trunk native vlan 10
 switchport trunk allowed vlan 10
 switchport mode trunk
 storm-control broadcast level 5.00
 channel-protocol lacp
 channel-group 6 mode active
!
interface GigabitEthernet1/0/25
 shutdown
!
interface GigabitEthernet1/0/26
 shutdown
!
interface GigabitEthernet1/0/27
!
interface GigabitEthernet1/0/28
 shutdown
!
interface GigabitEthernet2/0/1
 description BC1-IO2-1
 switchport trunk native vlan 11
 switchport trunk allowed vlan 10-12
 switchport mode trunk
 switchport nonegotiate
 storm-control broadcast level 5.00
 channel-group 1 mode active
!
interface GigabitEthernet2/0/2
 description BC1-IO2-2
 switchport trunk native vlan 11
 switchport trunk allowed vlan 10-12
 switchport mode trunk
 switchport nonegotiate
 storm-control broadcast level 5.00
 channel-group 1 mode active
!
interface GigabitEthernet2/0/3
 description BC1-IO2-3
 switchport trunk native vlan 11
 switchport trunk allowed vlan 10-12
 switchport mode trunk
 switchport nonegotiate
 shutdown
 storm-control broadcast level 5.00
 channel-protocol lacp
 channel-group 2 mode active
!
interface GigabitEthernet2/0/4
 description BC1-IO2-4
 switchport trunk native vlan 11
 switchport trunk allowed vlan 10-12
 switchport mode trunk
 switchport nonegotiate
 shutdown
 storm-control broadcast level 5.00
 channel-protocol lacp
 channel-group 2 mode active
!
interface GigabitEthernet2/0/5
 shutdown
!
interface GigabitEthernet2/0/6
 shutdown
!
interface GigabitEthernet2/0/7
 shutdown
!
interface GigabitEthernet2/0/8
 shutdown
!
interface GigabitEthernet2/0/9
 shutdown
!
interface GigabitEthernet2/0/10
 shutdown
!
interface GigabitEthernet2/0/11
 shutdown
!
interface GigabitEthernet2/0/12
 shutdown
!
interface GigabitEthernet2/0/13
 description UP_sw-core
 switchport trunk native vlan 11
 switchport trunk allowed vlan 1,11
 switchport mode trunk
 storm-control broadcast level 5.00
 channel-protocol lacp
 channel-group 5 mode active
!
interface GigabitEthernet2/0/14
 description UP_sw-core
 switchport trunk native vlan 11
 switchport trunk allowed vlan 1,11
 switchport mode trunk
 storm-control broadcast level 5.00
 channel-protocol lacp
 channel-group 5 mode active
!
interface GigabitEthernet2/0/15
 shutdown
!
interface GigabitEthernet2/0/16
 shutdown
!
interface GigabitEthernet2/0/17
 shutdown
!
interface GigabitEthernet2/0/18
 shutdown
!
interface GigabitEthernet2/0/19
 shutdown
!
interface GigabitEthernet2/0/20
 shutdown
!
interface GigabitEthernet2/0/21
 shutdown
!
interface GigabitEthernet2/0/22
!
interface GigabitEthernet2/0/23
 shutdown
!
interface GigabitEthernet2/0/24
 description sw-adm-24
 switchport trunk native vlan 10
 switchport trunk allowed vlan 10
 switchport mode trunk
 storm-control broadcast level 5.00
 channel-protocol lacp
 channel-group 6 mode active
!
interface GigabitEthernet2/0/25
 shutdown
!
interface GigabitEthernet2/0/26
 shutdown
!
interface GigabitEthernet2/0/27
 shutdown
!
interface GigabitEthernet2/0/28
 shutdown
!
interface Vlan1
 no ip address
 shutdown
!
interface Vlan10
 ip address 172.1.2.250 255.255.255.0
!
ip default-gateway 172.1.2.254
ip http server
ip http secure-server
!
!
snmp-server community public RO
!
!
!
!
line con 0
line vty 0 4
 length 0
 transport preferred none
 transport input ssh
 transport output ssh
line vty 5 15
 length 0
 transport preferred none
 transport input ssh
 transport output ssh
!
ntp server 172.1.2.1
end

If you could help me with this configuration I would be graceful.

Philip D'Ath · ‎03-10-2016

Of course they are not reachable. Builds 1 and 2 only have connectivity only through gbesm 1 and 2, so if they are both powered off nothing will work.

eric.philippon · ‎03-11-2016

My problem is that bld1 and bld2 aren’t reachable if both Gbesm are powered ON, but if only one is up, the configuration is working.

After some tests, I succeeded pinging a blade if only 1 of its eth interfaces is up, even if both inner switch are up.

So, the pb seems to be concentrated inside the bladecenter between inner switches and blades.Yet, in the previous working configuration under redhat, this interfaces were bound with mode 6 (adaptive load balancing). It seems that I should change for mode 1 active-backup...

Could it be a Broadcom driver pb (but I can't change it, I am stuck with this old redhat version) ?