02-08-2010 09:43 AM - edited 03-06-2019 09:37 AM
What is the CLI trying to tell me when it talks about EtherChannels with an 'A' or 'B' designator following them? i.e. I understand 'Po41' ... but what are these 'Po41A' and 'Po42B' thingies? It seems to be a way of representing multiple ports in a bundle ... but I don't see why it is useful. Nor why the alpha designator is sometimes 'A' and sometimes 'B'. Is this conveying information which I want to know?
cat6513#show interface status | include Po
Port Name Status Vlan Duplex Speed
Po40 To tungsten-a e1b connected 42 a-full 10G
Po41 To tungsten-a e0b connected 42 a-full a-1000
Po41A connected 42 a-full a-1000
Po42 To tungsten-b e0a connected 42 a-full a-1000
Po42B connected 42 a-full a-1000
Po43 To tungsten-b e4c connected 42 a-full a-1000
--sk
cat6513# show etherchannel summary
Group Port-channel Protocol Ports
[...]
40 Po40(SU) LACP Te5/4(P)
41 Po41(SU) LACP Gi7/42(P)
41 Po41A(SU) LACP Gi7/43(P) Gi7/44(P)
42 Po42(SU) LACP Gi7/40(P)
42 Po42B(SU) LACP Gi7/41(P)
43 Po43(SU) LACP Gi7/39(P)
[...]
cat6513# show lacp internal
[...]
Channel group 40
LACP port Admin Oper Port Port
Port Flags State Priority Key Key Number State
Te5/4 SA bndl 32768 0x28 0x28 0x505 0x3D
Channel group 41
LACP port Admin Oper Port Port
Port Flags State Priority Key Key Number State
Gi7/42 SA bndl 32768 0x29 0x29 0x72B 0x3D
Gi7/43 SA bndl 32768 0x29 0x1029 0x72C 0x3D
Gi7/44 SA bndl 32768 0x29 0x1029 0x72D 0x3D
Channel group 42
LACP port Admin Oper Port Port
Port Flags State Priority Key Key Number State
Gi7/40 SA bndl 32768 0x2A 0x2A 0x729 0x3D
Gi7/41 SA bndl 32768 0x2A 0x102A 0x72A 0x3D
Channel group 43
LACP port Admin Oper Port Port
Port Flags State Priority Key Key Number State
Gi7/39 SA bndl 32768 0x2B 0x2B 0x728 0x3D
[...]
interface Port-channel40
description To tungsten-a e1b
switchport
switchport access vlan 42
switchport mode access
logging event bundle-status
spanning-tree portfast edge trunk
spanning-tree guard none
!
interface Port-channel41
description To tungsten-a e0b e4c e4d
switchport
switchport access vlan 42
switchport mode access
logging event bundle-status
spanning-tree portfast edge trunk
spanning-tree guard none
!
interface Port-channel42
description To tungsten-b e0a e4a
switchport
switchport access vlan 42
switchport mode access
logging event bundle-status
spanning-tree portfast edge trunk
spanning-tree guard none
!
interface Port-channel43
description To tungsten-b e4c
switchport
switchport access vlan 42
switchport mode access
logging event bundle-status
spanning-tree portfast edge trunk
spanning-tree guard none
!
[...]
interface TenGigabitEthernet5/4
description To tungsten-a e1b
switchport
switchport access vlan 42
switchport mode access
spanning-tree portfast edge
spanning-tree guard none
channel-protocol lacp
channel-group 40 mode active
!
[...]
interface GigabitEthernet7/39
description To tungsten-b e4c
switchport
switchport access vlan 42
switchport mode access
spanning-tree portfast edge
spanning-tree guard none
channel-protocol lacp
channel-group 43 mode active
!
interface GigabitEthernet7/40
description To tungsten-b e0a
switchport
switchport access vlan 42
switchport mode access
spanning-tree portfast edge
spanning-tree guard none
channel-protocol lacp
channel-group 42 mode active
!
interface GigabitEthernet7/41
description To tungsten-b e4a
switchport
switchport access vlan 42
switchport mode access
spanning-tree portfast edge
spanning-tree guard none
channel-protocol lacp
channel-group 42 mode active
!
interface GigabitEthernet7/42
description To tungsten-a e0b
switchport
switchport access vlan 42
switchport mode access
spanning-tree portfast edge
spanning-tree guard none
channel-protocol lacp
channel-group 41 mode active
!
interface GigabitEthernet7/43
description To tungsten-a e4c
switchport
switchport access vlan 42
switchport mode access
spanning-tree portfast edge
spanning-tree guard none
channel-protocol lacp
channel-group 41 mode active
!
interface GigabitEthernet7/44
description To tungsten-a e4d
switchport
switchport access vlan 42
switchport mode access
spanning-tree portfast edge
spanning-tree guard none
channel-protocol lacp
channel-group 41 mode active
02-17-2010 08:47 AM
I didn't know about this, but here is what I found out:
Aparently this means that within your bundles you have ports that are mutually incompatibly configured. It creates a series of bundles each grouping ports tghat are multually compatible. These are known as the secondary aggregators. It then aggegates the bundles into a super-bundle, known as the primary aggregator.
This reference looks pretty useful:
Also, from the command reference:
For LACP, multiple aggregators are supported. For example, if two different bundles are created, Po1 indicates the primary aggregator, and Po1A and Po1B indicates the secondary aggregators.
At first glance I don't see how your port configs are incompatible, but there might be some restrictions on your card, regarding which ports you can bundle.
Hope this helps
Kevin Dorrell
Luxembourg
04-01-2010 09:09 AM
OK, thanx for the links. The 'A' and 'B" designators indicate 'secondary aggregators', which IOS creates as part of an automagic effort to keep the LACP channel functioning, despite incompatible interface parameters.
Peering at my logs:
Mar 22 11:03:03 j4sr-b-esx 23230: 023225: Mar 22 11:03:02.645 pdt:
%EC-SP-5-CANNOT_BUNDLE_LACP: Gi7/42 is not compatible with aggregators
in channel 41 and cannot attach to them (flow control send of Gi7/42 is off, Gi7/44 is on)
And in fact, we can see this in 'sh int' output:
switch-a#sh int flowcontrol module 7
Port Send FlowControl Receive FlowControl RxPause TxPause
admin oper admin oper
----------- -------- -------- -------- -------- ------- -------
[...]
Gi7/42 desired off off off 0 0
Gi7/43 desired on off off 0 0
Gi7/44 desired on off off 0 0
[...]
Looking at the host side of things (NetApp filer/ONTAP), I can see that ONTAP is configured to auto-negotiate *and* has autonegotiated to 'flowcontrol full', which means 'send and receive' (http://ecserv1.uwaterloo.ca/netapp/man/man1/na_ifconfig.1.html) While the IOS is configured to autonegotiate (but asymmetrically, i.e. to prefer to send PAUSE frames but not to receive them).
toaster-a> ifconfig -a
[...]
e0b: flags=948043
ether 02:a0:98:12:db:1e (auto-1000t-fd-up) flowcontrol full
trunked dmmvif3
[...]
e4c: flags=948043
ether 02:a0:98:12:db:1e (auto-1000t-fd-up) flowcontrol full
trunked dmmvif3
e4d: flags=948043
ether 02:a0:98:12:db:1e (auto-1000t-fd-up) flowcontrol full
trunked dmmvif3
[...]
I'm guessing that auto-negotiation isn't behaving the way I want it to, namely, I want ONTAP to honor the switch's request not to receive PAUSE frames.
Nevertheless. I don't think this matters -- i.e. what does the switch care if the host sends PAUSE frames? It isn't listening.
The part which kicks off the LACP oddness relates to how Gi7/42 sees the host wanting to *disable* its receipt of PAUSE frames ... and then of course the switch dutifully turns 'off' its sending of PAUSE frames, Gi7/43 and Gi7/44 stay as they were, and now the LACP channel contains three interfaces which disagree on flowcontrol settings.
I think the question becomes: is IOS disabling the transmission of PAUSE frames on a whim? Or is the host actually encoding its distaste for receiving PAUSE frames in the FLPs it is sending?
I don't even know what kind of tool I would need to analyze this ... I have in-line sniffers (Finisar THG and Cace TurboCap) ... but these boxes don't see FLPs.
Suggestions on how I could dig deeper into this?
--sk
05-10-2010 08:18 AM
OK, so I've chewed a little deeper into this. The folks who write the driver for the NIC in the host believe they have a flow-control bug, which results in the mismatch between flow control settings. LACP wants all members to be configured identically, so it doesn't want to bundle the 'odd' link. Thus far, I think my story is fairly solid. Where I stumble is on what happens next -- looks like the IOS, rather than yanking these members, creates these 'secondary' aggregates, which it then bundles into a primary aggregate ... effectively hiding the problem from us. But here I'm speculating.
--sk
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: