PBR problem on 6509

mreevesh · ‎10-20-2010

Hi,

I'm having a problem with Policy Based Routing on a 6509 with a FWSM installed.

The network is configured as follows.

I have 2 x 6509's SW1 and SW2

I have a server DC 1 connected to a vlan 10 connected to the 6509's behind the FWSM subnet 10.2.0.0/24

I also have 2 x ASA 5510's FW1 and FW2.

SW1 and SW2 are operating as a HSRP pair on all their VLANS

The internal side of the FWSM is connected on vlan 20

SW1 and SW2 are connected to FW1 on VLAN 30

SW1 and SW2 are connected to FW2 on VLAN 40

FW1 is connected to an external network 10.1.0.0 /24

FW2 is also connected to an external network with the same subnet, but this is not the same as the one connected to FW1.

There is no NAT running.

The internal routing within my network routes 10.1.0.0/24 to FW1.

Now comes the PBR bit,the subnet 10.2.0.0/24 needs to talk to the 10.1.0.0/24 connected to FW2 not FW1.

Both of the 6509's have the following config for PBR:

ip access-list extended 10-2-to-10-1

permit ip 10.2.0.0 0.0.0.255 10.1.0.0 0.0.0.255

ip access-list extended 10-1-to-10-2

permit ip 10.1.0.0 0.0.0.255 10.2.0.0 0.0.0.255

route-map 10-2-out permit 10

match ip address 10-2-to-10-1

set ip next hop (IP address of FW2)

route-map 10-2-in permit 10

match ip address 10-1-to-10-2

set ip next hop (IP address of FWSM vlan 20)

int vlan 40

ip policy route-map 10-2-in

int vlan 10 (Note if also tried this on int vlan 20)

ip policy route-map 10-2-out

When I send traffic from my DC on VLAN 10 to 10.2.0.1 in goes to FW1 not FW2.

Any ideas I've tried all sort of things with this and from what I've seen the above should be working.

Thanks

Matthew

Jon Marshall · ‎10-20-2010

Matthew

When you say DCI is behind the FWSM what do you actually mean ? It's not clear where the server sits in relation to the FWSM ? Could you knock up a very quick schematic ?

Jon

mreevesh · ‎10-21-2010

Hi,

I've uploaded a diagram showing the connections in Visio format.

Thanks

Matthew

trevorpeters · ‎10-21-2010

Hi Matthew

a couple of things do not look right to me.

1. You have VLAN 40 both as you live office VLAN and the FW2 VLAN in your drawing, This would by pass the FWSM

2. VLAN 20 looks like the right place to apply the PBR, I ssume that this is an SVI with an IP address.

NOTE By default the FWSM will only allow one SVI to be attached, you have to enable multiple SVI's with the firewall command on the 6509 IOS. (not FWSM).

DANGER, With Multiple SVI's you can route around the FWSM's be careful !!!!

3. You might have to turn off CEF/fast switching on the VLAN interface so that the 6500 can process switch.

4. You might be getting the traffic at both FW1 and FW2, if both 6500's have dual routes to the 10.1.0.0/24 networks, due to routing load balancing.

good luck

Trev

mreevesh · ‎10-21-2010

Hi Trevor,

Thanks for that, you are indeed correct the vlan 40 on the FWSM was incorrect and was a type, it should be vlan 50.

I've tried the PBR for traffic from 10.2 to 10.1 on both vlan 10 and vlan 20 and get the same results.

Regards

Matthew

Jon Marshall · ‎10-21-2010

Sorry, Matthew, can't read visio's. If Trevor has not solved your question can you repost as a .jpg

Jon

mreevesh · ‎10-21-2010

Hi Jon,

Attached is a JPG file with the vlan corrected which Trevor pointed out.

I'ts still not working so any pointers would be great.

Thanks

Matthew

Jon Marshall · ‎10-21-2010

Matthew

Couple of things -

1) From the looks of the diagram vlan 50 is routed on the FWSM so

i) there shouldn't be an SVI for it which i believe is what Trevor said. And you shouldn't therefore need to apply PBR to that vlan interface as it shouldn't exist.

By the away, Trevor is also spot on about not having an SVI for vlan 50 on the 6500 as you will simply route around the FWSM.

2) So looking at the diagram all you should need to do is apply PBR to vlan 20 on the 6500 vlan 20 SVIs.

It should also be noted that PBR is hardware switched in the 6500 so relying on the acl counters is not a good guide as to whether the acl is getting hits or not. Your best bet is traceroute which shows the path being taken although you would obviously have to temporarily open that up through the FWSM.

Jon

davegibelli · ‎10-21-2010

Matthew

Am I missing something here...

You have a PBR on vlan 10 that sets the next-hop to an IP address on vlan 40. Surely this would bypass the FWSM because you have set the next-hop to be the ASA5510 FW2.

To my simple mind you are telling the SVI on vlan 10 to forward the packet directly to a device on vlan 40...

Dave

trevorpeters · ‎10-22-2010

Hi Matthew,

To me the place to apply the PBR is VLAN 20.

When you have the PBR in place do you get any HITS on the access-lists ? I never trust Route-map counters

Have you tried to debug IP routing/Ip policy (i think you can). If on a live network make sure you use an access -list with debug ip packet.

Trev

trevorpeters · ‎10-22-2010

Hi Matthew,

Found this on another site

"We've run into problems with route-maps with the SXH train.

PBR behavior seemed to have changed when jumping to SXH; I thought this was supposed to have been fixed since SXH3a though...
If I remember correctly, we were using a route-map to do simple PBR (acl match then set next-hop to another IP) and the IOS stopped doing the required ARP lookup for the next-hop.
Our workaround was to configure a static arp entry for that next-hop ip address.
Edit: Here's the bug id: CSCsm08087

Not sure if it was the same issue, but we also faced a bug with route-map application on SXH where if you have a FWSM in that 6500, not having the "monitor session service" command would break the pbr (TCAM issues). Checkout CSCsl39710.

Edit: There's been quite a few issues with route-maps in SXH if you search the bug toolkit... I wouldn't even be surprised if you stumbled upon a new one..."

regards

Trev