Performance Routing Basic Load-balancing

neil grant · ‎03-11-2013

Hello,

I have a requirement to set up PFR as a load balancing mechanism for i/eBGP (LAB environment for the moment)

Topology:

Iperf Client192.168.1.254

I

Primary 20Mb --------iBGP--------Secondary 5Mb

I I

eBGP (PE) eBGP (PE)

I I

Primary 20Mb --------iBGP--------Secondary 5Mb

I

Iperf Server192.168.3.254

Now when generating TCP traffic with Iperf i only seem to be getting the 20Mb despite the prefix 192.168.3.0/24 being OOP:

Mar 11 16:51:25.675: %OER_MC-5-NOTICE: Range OOP BR 192.168.1.1, i/f Gi0/0.100, percent 28. Other BR 192.168.1.2, i/f Gi0/0, percent 0

Mar 11 16:51:25.675: %OER_MC-5-NOTICE: Exit 192.168.1.1 intf Gi0/0.100 OOP, Tx BW 5730, Rx BW 246, Tx Load 28, Rx Load 1

Mar 11 16:51:45.907: %OER_MC-5-NOTICE: Range OOP BR 192.168.1.1, i/f Gi0/0.100, percent 80. Other BR 192.168.1.2, i/f Gi0/0, percent 0

Mar 11 16:51:45.907: %OER_MC-5-NOTICE: Load OOP BR 192.168.1.1, i/f Gi0/0.100, load 16104 policy 12000

Mar 11 16:51:45.907: %OER_MC-5-NOTICE: Exit 192.168.1.1 intf Gi0/0.100 OOP, Tx BW 16104, Rx BW 685, Tx Load 80, Rx Load 3

Mar 11 16:52:05.739: %OER_MC-5-NOTICE: Range OOP BR 192.168.1.1, i/f Gi0/0.100, percent 70. Other BR 192.168.1.2, i/f Gi0/0, percent 0

Mar 11 16:52:05.739: %OER_MC-5-NOTICE: Load OOP BR 192.168.1.1, i/f Gi0/0.100, load 14072 policy 12000

Mar 11 16:52:05.739: %OER_MC-5-NOTICE: Exit 192.168.1.1 intf Gi0/0.100 OOP, Tx BW 14072, Rx BW 594, Tx Load 70, Rx Load 2

If someone could point out where i am going wrong it would be appreciated:

Border Status UP/DOWN AuthFail Version

192.168.1.1 ACTIVE UP 00:08:04 0 3.0

Gi0/1 INTERNAL UP

Gi0/0.100 EXTERNAL UP

External Capacity Max BW BW Used Load Status Exit Id

Interface (kbps) (kbps) (kbps) (%)

--------- -------- ------ ------- ------- ------ ------

Gi0/0.100 Tx 20000 12000 18473 92 UP 2

Rx 20000 800 4

-------------------------------------------------------------------------------------------

Border Status UP/DOWN AuthFail Version

192.168.1.2 ACTIVE UP 00:08:04 0 3.0

Gi0/0 EXTERNAL UP

Gi0/1 INTERNAL UP

External Capacity Max BW BW Used Load Status Exit Id

Interface (kbps) (kbps) (kbps) (%)

--------- -------- ------ ------- ------- ------ ------

Gi0/0 Tx 5000 4500 0 0 UP 1

Rx 5000 0 0

Customer_PE#

key chain pfr

key 0

key-string cisco

pfr master

max-range-utilization percent 10

logging

!

border 192.168.1.1 key-chain pfr

interface GigabitEthernet0/1 internal

interface GigabitEthernet0/0.100 external

max-xmit-utilization percentage 60

!

border 192.168.1.2 key-chain pfr

interface GigabitEthernet0/0 external

max-xmit-utilization percentage 90

interface GigabitEthernet0/1 internal

!

learn

throughput

delay

periodic-interval 0

monitor-period 1

holddown 90

mode route control

periodic 180

resolve range priority 1

resolve utilization priority 5 variance 20

no resolve delay

!

pfr border

local GigabitEthernet0/1

master 192.168.1.1 key-chain pfr

Regards Neil

Regards Neil http://uk.linkedin.com/pub/neil-grant/20/5b0/267

neil grant · ‎03-11-2013

BGP Table:

Customer_PE#show ip bgp

BGP table version is 11, local router ID is 10.173.0.1

Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,

r RIB-failure, S Stale, m multipath, b backup-path, x best-external, f RT-Filter

Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Weight Path

*> 10.10.10.0/30 0.0.0.0 0 32768 i

* 10.10.10.1 0 0 4445 i

*> 10.10.10.4/30 10.10.10.1 0 0 4445 i

*>i172.31.0.8/30 192.168.1.2 0 100 0 i

*>i172.31.0.12/30 192.168.1.2 0 100 0 4445 i

* 10.10.10.1 0 4445 64513 i

*> 192.168.1.0 0.0.0.0 0 32768 i

* i 192.168.1.2 0 100 0 i

* i192.168.3.0 192.168.1.2 0 100 0 4445 64513 i

*> 10.10.10.1 0 4445 64513 i

Regards Neil http://uk.linkedin.com/pub/neil-grant/20/5b0/267

Joseph W. Doherty · ‎03-11-2013

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

Now when generating TCP traffic with Iperf i only seem to be getting the 20Mb despite the prefix 192.168.3.0/24 being OOP:

Your fastest path only seems to have 20 Mbps, correct?

PfR, I believe, sends all flows for one of its prefixes per one path. If true, an OOP prefix wouldn't be able to obtain more bandwidth. If the prefix wasn't being actively managed, then ECMP could exceed one path's bandwidth for multiple flows under the same prefix.

neil grant · ‎03-12-2013

Hi Joseph,

In this lab 20Mb is the fastest link (My eBGP neighbor) the iBGP neighbor actually hasnt got any shapping configured at the minute so is 100Mb

Ok since following your suggestion that load balancing is per prefix i have configured the following:

Topology:

Iperf Client192.168.1.254

I

Primary 20Mb --------iBGP--------Secondary 5Mb

I I

eBGP (PE) eBGP (PE)

I I

Primary 20Mb --------iBGP--------Secondary 5Mb

I

Iperf Server192.168.3.254 / 192.168.4.254

* 10.10.10.0/30 10.10.10.1 0 0 4445 i

*> 0.0.0.0 0 32768 i

*> 10.10.10.4/30 10.10.10.1 0 0 4445 i

*>i172.31.0.8/30 192.168.1.2 0 100 0 i

*>i172.31.0.12/30 192.168.1.2 0 100 0 4445 i

* 10.10.10.1 0 4445 64513 i

* i192.168.1.0 192.168.1.2 0 100 0 i

*> 0.0.0.0 0 32768 i

* i192.168.3.0 192.168.1.2 0 100 0 4445 64513 i

*> 10.10.10.1 0 4445 64513 i

* i192.168.4.0 192.168.1.2 0 100 0 4445 64513 i

*> 10.10.10.1 0 4445 64513 i

So when generating traffic at 20Mb to 192.168.3.254 at 20Mb the PFR master acknowledges the interface is OOP:

Mar 12 10:08:58.699: %OER_MC-5-NOTICE: Range OOP BR 192.168.1.1, i/f Gi0/0.100, percent 99. Other BR 192.168.1.2, i/f Gi0/0, percent 0

But still does not utilise the other link when i start a Iperf session to the 192.168.4.0 prefix

Customer_PE#show pfr master border detail

Border Status UP/DOWN AuthFail Version

192.168.1.2 ACTIVE UP 00:01:51 0 3.0

Gi0/1 INTERNAL UP

Gi0/0 EXTERNAL UP

External Capacity Max BW BW Used Load Status Exit Id

Interface (kbps) (kbps) (kbps) (%)

--------- -------- ------ ------- ------- ------ ---- --

Gi0/0 Tx 5000 4500 0 0 UP 2

Rx 5000 0 0

--------------------------------------------------------------------------------- ----------

Border Status UP/DOWN AuthFail Version

192.168.1.1 ACTIVE UP 00:01:51 0 3.0

Gi0/0.100 EXTERNAL UP

Gi0/1 INTERNAL UP

External Capacity Max BW BW Used Load Status Exit Id

Interface (kbps) (kbps) (kbps) (%)

--------- -------- ------ ------- ------- ------ ---- --

Gi0/0.100 Tx 20000 12000 19957 99 UP 1

Rx 20000 8 0

Configuration as it stands at the minute:

Customer_PE#show run | sec pfr

key chain pfr

key 0

key-string cisco

pfr master

max-range-utilization percent 10

logging

!

border 192.168.1.2 key-chain pfr

interface GigabitEthernet0/1 internal

interface GigabitEthernet0/0 external

max-xmit-utilization percentage 90

!

border 192.168.1.1 key-chain pfr

interface GigabitEthernet0/0.100 external

max-xmit-utilization percentage 60

interface GigabitEthernet0/1 internal

!

learn

throughput

delay

periodic-interval 0

monitor-period 1

aggregation-type prefix-length 32

holddown 90

mode route protocol pbr

mode route control

periodic 180

resolve range priority 1

resolve utilization priority 5 variance 20

no resolve delay

!

pfr border

local GigabitEthernet0/1

master 192.168.1.1 key-chain pfr

Customer_PE#

Regards Neil

Regards Neil http://uk.linkedin.com/pub/neil-grant/20/5b0/267

neil grant · ‎03-12-2013

I think PFR isnt aware of the routes via iBGP: (G0/1)

Customer_PE#show pfr master prefix

OER Prefix Statistics:

Pas - Passive, Act - Active, S - Short term, L - Long term, Dly - Delay (ms),

P - Percentage below threshold, Jit - Jitter (ms),

MOS - Mean Opinion Score

Los - Packet Loss (packets-per-million), Un - Unreachable (flows-per-million),

E - Egress, I - Ingress, Bw - Bandwidth (kbps), N - Not applicable

U - unknown, * - uncontrolled, + - control more specific, @ - active probe all

# - Prefix monitor mode is Special, & - Blackholed Prefix

% - Force Next-Hop, ^ - Prefix is denied

Prefix State Time Curr BR CurrI/F Protocol

PasSDly PasLDly PasSUn PasLUn PasSLos PasLLos

ActSDly ActLDly ActSUn ActLUn EBw IBw

ActSJit ActPMOS ActSLos ActLLos

-------------------------------------------------------------------------------------------

192.168.3.0/24 DEFAULT* 95 192.168.1.1 Gi0/0.100 U

U

Mar 12 10:43:48.791: OER BR: Timer event, 0

Mar 12 10:43:50.231: OER BR: Timer event, 2

Mar 12 10:43:50.231: OER BR: Timer event, 2 U 0 0 0 0

U 10 0 500000 20384 0

N N

192.168.4.0/24 HOLDDOWN 53 192.168.1.1 Gi0/0.100 BGP

U U 0 0 0 0

12 9 500000 500000 20444 0

N N

Customer_PE#

Regards Neil http://uk.linkedin.com/pub/neil-grant/20/5b0/267