Solved: Hi Matthew, - Page 2

Matthew Millman · ‎05-10-2016

I've just encountered some behavior with dynamic link aggregation between switches which I wasn't expecting -

I have this scenario, I'm expecting 2.0gbps aggregate bandwidth from the server to the PCs, and I get 2.0gbps, as expected

     +---------+
     |  Server | <--- Using NIC Teaming
     +---------+
        |   | GbE x2 (802.3ad)
     +---------+
     | 2960G 1 |
     +---------+
        |   |
    +--+   +--+
    |         | 
+-------+ +-------+ 
|  PC   | |  PC   |
+-------+ +-------+

But when I change the topology to this:

     +---------+
     |  Server | <--- Using NIC Teaming
     +---------+
        |   | GbE x2 (802.3ad)
     +---------+
     | 2960G 1 |
     +---------+
        |   | GbE x2 (802.3ad)
     +---------+
     | 2960G 2 |
     +---------+
        |   |
    +--+    +--+
    |          | 
+-------+ +-------+ 
|  PC   | |  PC   |
+-------+ +-------+

Now I'm down to 1.0gbps, and I don't get it. Everything appears to be OK.

On switch 1:

Switch#show lacp nei

Partner's information:

(Link to server)

LACP port Admin Oper Port Port
Port Flags Priority Dev ID Age key Key Number State
Gi0/1 FA 0 0030.xxxx.xxxx 25s 0x0 0x0 0x2 0x3F
Gi0/2 FA 0 0030.xxxx.xxxx 26s 0x0 0x0 0x1 0x3F

Channel group 2 neighbors

Partner's information:

(Switch to switch link)

LACP port Admin Oper Port Port
Port Flags Priority Dev ID Age key Key Number State
Gi0/3 SA 32768 0025.xxxx.xxxx 24s 0x0 0x1 0x102 0x3D
Gi0/4 SA 32768 0025.xxxx.xxxx 10s 0x0 0x1 0x103 0x3D

Switch 2:

Channel group 1 neighbors

Partner's information:

LACP port Admin Oper Port Port
Port Flags Priority Dev ID Age key Key Number State
Gi0/1 SA 32768 04c5.xxxx.xxxx 13s 0x0 0x2 0x104 0x3D
Gi0/2 SA 32768 04c5.xxxx.xxxx 12s 0x0 0x2 0x105 0x3D

Grateful for any thoughts as to how to solve this!

Matthew Millman · ‎09-02-2016

> traffic goes through on all ethernet ports of the etherchannel
What evidence do you have that it's going through all ports equally ?

Have you checked the frame counters? Do they all increment equally when passing your test traffic?

juanruzafa · ‎09-02-2016

Yes, we did clear counters during tests and we saw the traffic goes trhough all ports (in this case 2 ports) as packet counters are incremented, about percent is mostly the same and near 50%. In the production environment (4 ports) we also see traffic going through all ports, obviously not all of them having the same percent of traffic... but ever obtaining a maximum near 1 Gbps.

Matthew Millman · ‎09-02-2016

So in your production environment, you have confirmed that when you transmit 10000 frames, the result was an increment of 2500 on each port?

If that turns out to be the case, I don't think I can be of any more assistance!

juanruzafa · ‎09-13-2016

Hi Matthew,

Thanks a lot for your support. Finally the problem wasn't as I described. The thechnicians of my customer reported incorrect data observations to me. They were seeing the traffic on other ports than the correct ones.

So the problem is related to the hashing algorithm not applied in an intermediate cisco switch that only sends traffic through a single port and using src-mac hashing method.

We changed the hashing method to src-dst-ip but we could'nt make a succesful test yet.

My apologies to the forum, as I couldn't connect to that switch remotely and I've posted in trust that the info I've received from my customer was correct, obviously after explaining them what I wanted to know in detail... but the personnel communication failed too.

Regards

Matthew Millman · ‎09-02-2016

Another thing I'd mention - using "src-dst-ip" as the hashing mechanism when you are using both IPv4 and IPv6 sounds like a bit of confusing/unpredictable setup.

As I mentioned earlier in this thread, I also use both protocols and chose not to use this mechanism because frames can end up with different hashes (meaning I could get either 1.0Gbps or 2.0Gbps) depending on whether the application used IPv4 or IPv6 (Typically you end up seeing both used at once, and end up with something in the middle).

Instead I used src-dst-mac and changed the MAC addresses until I got unique hashes. I know changing MAC addresses isn't straight forward but it meant I could guarantee maximum bandwidth across the LAG link regardless of the protocol in use.

Matthew Millman · ‎09-01-2016

Interestingly, on here: http://www.cisco.com/c/en/us/support/docs/lan-switching/etherchannel/12023-4.html

In some cases it's possible to evaluate the frame distribution policy out-of-band!

That would have been a real time saver

 Console> (enable) show channel hash 865 10.10.10.1 10.10.10.2
?Selected channel port: 1/1

 Console> (enable) show channel hash 865 00-02-fc-26-24-94 
 00-d0-c0-d7-2d-d4

802.3ad link aggregation switch to switch bandwidth: expecting 2Gb, getting 1Gb