cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1037
Views
0
Helpful
3
Replies
Stanislav Zaikin
Beginner

Load balancing on sub-interfaces (3 links)

Hello.

I am trying to load balance between the three links of a bundle. Traffic comes and goes with the same bundle interface.

Launched 5 threads TCP\UDP with different SRC DST IP addresses and see the following balances:

IOS-XR               Monitor Time: 00:00:30          SysUptime: 106:39:28

                          Last Clear:   00:00:22

Protocol:General

Interface             In(pps)      Out(pps)      InPkts/Delta   OutPkts/Delta

Te0/1/0/0             11381           628        102062/25512       256/64

Te0/1/0/1             33849         55965        303244/75700    505364/126230

Te0/1/0/2             11363             0        100800/25200         0/0

Quit='q',     Clear='c',    Freeze='f', Thaw='t',

Next set='n', Prev set='p', Bytes='y',  Packets='k'

(General='g', IPv4 Uni='4u', IPv4 Multi='4m', IPv6 Uni='6u', IPv6 Multi='6m')

We have 10G switch connected to asr9010 three ports and the following configuration:

interface TenGigE0/1/0/0

bundle id 1 mode active

bundle port-priority 2

interface TenGigE0/1/0/1

bundle id 1 mode active

bundle port-priority 2

interface TenGigE0/1/0/2

bundle id 1 mode active

interface Bundle-Ether1.75

ipv4 address 25.0.0.1 255.255.255.252

encapsulation dot1q 75

interface Bundle-Ether1.76

ipv4 address 26.0.0.1 255.255.255.252

encapsulation dot1q 76

RP/0/RSP0/CPU0: ios # sh bundle load-balancing bundle-e1 detail location 0/1/CPU0

Tue Jun 4 07:03:07.605 UTC

Bundle-Ether1

  Type: Ether (L3)

  Members <current/max>: 3/3

  Total Weighting: 3

  Load balance: Default

  Locality threshold: 65

  Avoid rebalancing? False

  Sub-interfaces: 3

  Member Information:

    Port: LON ULID BW

    -----------------------------

    Te0/1/0/0 0 0 1

    Te0/1/0/1 1 1 1

    Te0/1/0/2 2 2 1

  Sub-interface Information:

    Sub-interface Type Load Balance Locality

                                        Hash Threshold

    -------------------------------------------------- ---

    Bundle-Ether1.76 L3 Default 65

    Bundle-Ether1.75 L3 Default 65

    Bundle-Ether1.100 L3 Default 65

  Platform Information:

  =====================

                  * Bundle Summary Information *

                    --------------------------

Interface: Bundle-Ether1 Ifhandle: 0x08000160

Lag ID: 1 Virtual Port: 255

Number of Members: 3 Local to LC: Yes

Hash Modulo Index: 3

Member Information:

LON Interface ifhandle SFP port slot remote / rack_id

-------------------------------------------------- -----

Te0/1/0/0 0x02000140 0 12 0 1 0/0

Te0/1/0/1 0x02000180 1 13 0 1 0/0

Te0/1/0/2 0x020001c0 11 2 0 1 0/0

                   * Bundle Table Information *

                     ------------------------

[NP 0]:

----------------------------------

   Unicast (Global) LAG table

----------------------------------

idx local LON VQI port

----------------------------------

   1 0 0 12 0

   2 0 1 13 0

   3 0 2 11 0

[NP 1]

----------------------------------

   Unicast (Global) LAG table

----------------------------------

idx local LON VQI port

----------------------------------

   1 0 0 12 0

   2 0 1 13 0

   3 0 2 11 0

[NP 2]:

----------------------------------

   Unicast (Global) LAG table

----------------------------------

idx local LON VQI port

----------------------------------

   1 0 0 12 0

   2 0 1 13 0

   3 0 2 11 0

[NP 3]

-------------------------------------------------- -----------------

   Unicast (Global) LAG table | Multicast (Local) LAG table

-------------------------------------------------- -----------------

idx local LON VQI port | idx local LON VQI port

-------------------------------------------------- -----------------

   1 0 0 12 0 1 1 2 11 0

   2 0 1 13 0 2 0 0 0 0

   3 1 2 11 0 3 0 0 0 0

[NP 4]:

-------------------------------------------------- -----------------

   Unicast (Global) LAG table | Multicast (Local) LAG table

-------------------------------------------------- -----------------

idx local LON VQI port | idx local LON VQI port

-------------------------------------------------- -----------------

   1 1 0 12 0 1 1 0 12 0

   2 0 1 13 0 2 0 0 0 0

   3 0 2 11 0 3 0 0 0 0

[NP 5]

-------------------------------------------------- -----------------

   Unicast (Global) LAG table | Multicast (Local) LAG table

-------------------------------------------------- -----------------

idx local LON VQI port | idx local LON VQI port

-------------------------------------------------- -----------------

   1 0 0 12 0 1 1 1 13 0

   2 1 1 13 0 2 0 0 0 0

   3 0 2 11 0 3 0 0 0 0

[NP 6]

----------------------------------

   Unicast (Global) LAG table

----------------------------------

idx local LON VQI port

----------------------------------

   1 0 0 12 0

   2 0 1 13 0

   3 0 2 11 0

[NP 7]

----------------------------------

   Unicast (Global) LAG table

----------------------------------

idx local LON VQI port

----------------------------------

   1 0 0 12 0

   2 0 1 13 0

   3 0 2 11 0

================================================== =============================

3 REPLIES 3
xthuijs
Cisco Employee

Yeah 5 flows is not enough to create enough spread.

You could use the show cef exact route and or the bundle bundle-hash commands to findout which path or member a flow owuld take. Note however that there was a bug in these show commands if you are not running 423 with pack2.

Also have you maybe found the asr9000 loadbalancing architecture?

https://supportforums.cisco.com/docs/DOC-26687

‎it holds a lot of detail on the LB architecture.

xander

20 flows and a bit better result:

IOS-XR               Monitor Time: 00:00:08          SysUptime: 133:33:44

                     Last Clear:   00:00:06

Protocol:General

Interface             In(pps)      Out(pps)      InPkts/Delta   OutPkts/Delta

Te0/1/0/0             11794         14977             0/44696         0/44484

Te0/1/0/1             10682          8786             0/37924         0/25456

Te0/1/0/2             18243         16958             0/44596         0/57579

Quit='q',     Clear='c',    Freeze='f', Thaw='t',

Next set='n', Prev set='p', Bytes='y',  Packets='k'

(General='g', IPv4 Uni='4u', IPv4 Multi='4m', IPv6 Uni='6u', IPv6 Multi='6m')

Can the ASR9K more or less normal balance on uneven number of links?

Yup, that confirms the suspicion that the entropie in your flows was too little.

You'll also notice that instead of using, sequential addresses for your test flows using more randomized L3 and L4 combo's (especially random high source ports to simulate clients), the spread will get better too.

We use a hash result of 256 buckets, these buckets are shared over the members you have.

So the spread, when entropie is good enough, will be a near equal spread. 20 flows is likely not good enough to produce that spread. In non facto 2 number of members with a facto 2 number of buckets, the spread will never be a perfect x/members.

In older releases we only had 8 buckets which inherently means that a non facto 2 number of members will

be problematic in terms of member loading.

Make sure you verify this with either XR 4.2.3 and smu pack2 or XR 4.3.1

regards

xander

Content for Community-Ad