GLBP with PBR

johnnylingo · ‎08-31-2015

So I've configured GLBP on a pair of Cisco 6509 switches. It should be balancing 50/50, but instead I see 1/99. Any ideas what the problem could be? There's a pre-existing Policy-Based routing configuration, and I'm wondering if that would interfere with GLBP operation.

R1 is the STP root, yet R2 is taking 99% of the input traffic.

R1

interface Vlan1
ip address 192.168.1.2 255.255.255.0
no ip redirects
no ip proxy-arp
ip route-cache policy
ip policy route-map PBR_ROUTE-MAP
load-interval 30
glbp 1 ip 192.168.1.1
glbp 1 priority 255
glbp 1 preempt
glbp 1 weighting 254
glbp 1 load-balancing weighted

R2

interface Vlan1
ip address 192.168.1.3 255.255.255.0
no ip redirects
no ip proxy-arp
ip route-cache policy
ip policy route-map PBR_ROUTE-MAP
load-interval 30
glbp 1 ip 192.168.1.1
glbp 1 priority 254
glbp 1 preempt
glbp 1 weighting 254
glbp 1 load-balancing weighted

Jon Marshall · ‎08-31-2015

How are you measuring the amount of traffic ?

When you say input do you mean from clients ?

PBR should only affect traffic once it has arrived at the interface ie. it should not affect which 6500 the client uses to send traffic to.

Jon

johnnylingo · ‎08-31-2015

I'm just looking at input rate on the SVIs. R1 is doing 114 Mbps while R2 is doing 3.8 Gbps.

R1>sh int vl1
Vlan1 is up, line protocol is up
MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
reliability 255/255, txload 3/255, rxload 29/255
30 second input rate 114621000 bits/sec, 10089 packets/sec
30 second output rate 14325000 bits/sec, 23138 packets/sec

R2>sh int vl1
Vlan1 is up, line protocol is up
MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 130/255
30 second input rate 3807281000 bits/sec, 337836 packets/sec
30 second output rate 5921000 bits/sec, 10625 packets/sec

Jon Marshall · ‎08-31-2015

Couple of things to bear in mind -

1) traffic from the other "side" ie. return traffic to the client vlan can use either switch

2) the rates do not tell you how much traffic individual clients are sending or receiving ie. you may have some very busy clients all tied to the same switch

Not saying you are not load balancing but a better way to check would be to look at the clients and see what mac address they are using for their default gateway.

You don't need to look at all clients, after all if all clients are tied to the same switch you should be able to see that pretty quickly.

Jon

johnnylingo · ‎08-31-2015

Yes, the return traffic is controlled by Multipath BGP, so that's a different beast. I'm strictly trying to figure out why R2 is taking so much input traffic. Even if I set the GLBP weight on R1 to 254 and R2 to 1, it has no effect. And yes, I did wait several hours for ARP to be refreshed.

There are about 200 clients on the VLAN, 50 of which are doing the heavy traffic. So while I don't expect the traffic to be a perfect 50/50, I would expect to see a better result than 1/99. The "show glbp" command shows that both switches are sending about the same number of ARP replies.

Jon Marshall · ‎08-31-2015

It's difficult to say from just the SVI stats.

If 50 clients are doing heavy traffic and the majority of these happened to end up with the same switch as their gateway then you would expect to see this.

You really would have to look at the end clients and see what they think is their default gateway in terms of mac address because each switch will be handing out a different one.

Jon

johnnylingo · ‎08-31-2015

This looks like it's an issue with the clients. When I logged in to them, all were resolving 192.168.1.1 to 00:07:B4:02:5C:01. I deleted the ARP entry manually, and at that point they refreshed to 00:07:B4:02:5C:01 half the time and 00:07:B4:02:5C:02 the other half.

The clients are Linux servers, so they should be refreshing every 60 seconds. Apparently there's behavioral a difference between an ARP entry timing out and it being manually deleted.