Split Traffic between 2 Firewall from my Core Switch C3850

[Herald Sison]

The goal i want to achieve here based on from the diagram below:

i want user1 - 192.168.20.x to pass FG-FW01 192.168.1.1

i want user 2 -192.168.50.x to pass FG-FW02 192.168.1.111

currently the routing is going to FG-FW01 with static ip route 0.0.0.0 0.0.0.0 192.168.1.1 

Switching, Catalyst 3000routing, Catalyst 2000routing, Catalyst 2000

- tried adding a static route ip route 192.168.50.0 255.255.255.0 192.168.1.111 - nothing is happening 

- tried removing the 0.0.0.0 0.0.0.0 192.168.1.1 and retain the 192.168.50.0 255.255.255.0 192.168.1.111 - all traffic down

-tried adding 192.168.20.0 255.255.255.0 192.168.1.1 and 192.168.50.0 255.255.255.0 192.168.1.111 - all traffic still down

- tried removing all ip route and added 0.0.0.0 0.0.0.0 192.168.1.111 - all traffic went to FG-FW02

is there another approach to split these 2 vlans to each FW?

[Herald Sison]

ive done some checking when the vlan 30 is routed to 172.21.1.111, here are my observations

1) vlan 30 users cannot access the internet but can ping 8.8.8.8, and cannot contact our internal dns servers or any other vlans like vlan 50 and or vlan 1

2) when tried to add 8.8.8.8 to the DNS on the network adapter setting of the computer of a user using vlan 30, they can access the internet but cannot contact our internal dns servers or any other vlans like vlan 50 and or vlan 1

3) tried to ping or tracert other (from and to) vlans like vlan 1 and 50, it got RTO's.

my observation is, when ip policy route-map PBR is added to vlan30 it seems that vlan 30 will bypass everything and will go straight to the firewall without checking other vlans and gateway and also withouth contacting our internal dns servers, it seems it was isolated to everyone. IDK anymore.

[Richard Burts]

You make an interesting observation "my observation is, when ip policy route-map PBR is added to vlan30 it seems that vlan 30 will bypass everything". And you are quite right. This is due to the acl used in PBR. You have access-list 100 permit ip 172.20.30.0 0.0.0.255 any. This says that any packet sourced from vlan 30 is forwarded to the firewall. And apparently the firewall does not forward that traffic to inside destinations back inside. I suggest that the solution is to modify the acl. Before that line insert lines with deny traffic originating from vlan 30 to the subnets of the other vlans. Have the line I reference as the last line in the acl.

[Herald Sison]

Thank you sir, What do you mean by "have the line as i reference the last line in the acl."

You mean i will do this config below?

access-list 100 permit ip 172.20.100.0 0.0.0.255 any
access-list 100 permit ip 172.21.1.0
0.0.0.255 any
access-list 100 permit ip 172.20.50.0 0.0.0.255 any
access-list 100 permit ip 172.20.30.0 0.0.0.255 any

route-map PBR permit 10
match ip address 100
set ip next-hop 172.21.1.111

interface vlan 30
ip helper-address 172.20.1.100
ip helper-address 172.20.1.102
ip policy route-map PBR

And then add all vlans to the access list 100 and place vlan 30 to the last line the only apply the ip policy route-map to vlan 30?

[Richard Burts · Accepted solution]

You misunderstood an important detail in my suggestion:  insert lines with deny traffic originating from vlan 30 to the subnets of the other vlans. So it would be like

access-list 100 deny 172.20.30.0 0.0.0.255 172.20.192.0 0.0.0.255

access-list 100 deny 172.20.30.0 0.0.0.255 172.20.1.0 0.0.0.255

access-list 100 permit 172.20.30.0 0.0.0.255 any

[Richard Burts]

Let me explain a bit about my suggestion. Some people get worried about implementing an acl with deny statements. They associated acl deny statements with packets being dropped. That is not the case here. To explain it let us start with an understanding that acl can be used in different ways and has different results. An acl applied to an interface using ip access-group does result in dropped packets when the acl specifies deny. But an acl applied in a route map for PBR does not.

To understand this better let us start from the point that PBR provides an over ride to normal routing logic. Normal routing logic builds a routing table based on locally connected subnets, configured static routes, and dynamic learned routes. PBR provides an alternative way to forward traffic. PBR uses a route map which provides alternative routing logic. In the PBR acl if a packet is permitted it is forwarded using this alternative logic. In the PBR if a packet is denied it is not dropped but is forwarded using normal routing table logic.

[Cristian Matei]

Hi,

   Default router logic is route for destination based on most specific / longer prefix-length match in RIB. 

   PBR has, by design, more than on logic applicable. Most commonly implemented, as long as traffic matches conditions and set actions are valid, ignore RIB and route traffic as PBR states. However, less known, PBR can also be implemented using the following logic: as long as traffic matches conditions and set actions are valid, look into RIB for a match on non-default route, if match found route as RIB states, if match not found, route as PBR states; this alternative logic of PBR is used when you want to have the benefit of a default route for traffic coming ingress only on a specific interface, so without having a default route in the RIB.

Best,

Cristian.

[Herald Sison]

This works now sir. Thank you so much. 

One last thing. if ever i need to add more VLANS, woulde these config below is fine also?

access-list 100 deny 172.20.30.0 0.0.0.255 172.20.192.0 0.0.0.255

access-list 100 deny 172.20.30.0 0.0.0.255 172.20.1.0 0.0.0.255

access-list 100 permit 172.20.30.0 0.0.0.255 any

access-list 200 deny 172.20.10.0 0.0.0.255 172.20.192.0 0.0.0.255

access-list 200 deny 172.20.10.0 0.0.0.255 172.20.1.0 0.0.0.255

access-list 200 permit 172.20.10.0 0.0.0.255 any

access-list 300 deny 172.20.20.0 0.0.0.255 172.20.192.0 0.0.0.255

access-list 300 deny 172.20.20.0 0.0.0.255 172.20.1.0 0.0.0.255

access-list 300 permit 172.20.20.0 0.0.0.255 any

route-map PBR30 permit10

match ip address 100

set ip next-hop 172.21.1.111

route-map PBR10 permit11

match ip address 200

set ip next-hop 172.21.1.111

route-map PBR20 permit12

match ip address 300

set ip next-hop 172.21.1.111

interface vlan 30
ip helper-address 172.20.1.100
ip helper-address 172.20.1.102
ip policy route-map PBR30

interface vlan 10
ip helper-address 172.20.1.100
ip helper-address 172.20.1.102
ip policy route-map PBR10

interface vlan 20
ip helper-address 172.20.1.100
ip helper-address 172.20.1.102
ip policy route-map PBR20

[Richard Burts]

Glad to hear that it is working now. To answer your question about adding more vlans we need a bit more information. It would help if we knew what IP addressing/subnetting you will use for the new vlans. And in particular we need to know how you want the new vlans to work. Do you want some/all of the new vlans to use the default route and forward to first ISP? No PBR needed for them. If you want some/all of new vlans to forward to second ISP then they need PBR.

Conceptually the logic you suggest for the new vlans is appropriate. But some details are not. In particular there need to be changes to the access lists used. Remember that the basic principle was that the ACL would deny any traffic whose source address was the local vlan subnet and whose destination was any other subnet in your network. So any ACL used for PBR will need to deny all other vlan subnets.

[Herald Sison]

The VLANs that may be added soon will be this below, these 2 new vlans will be rerouted to the other ISP so we may need a PBR for this one,

interface vlan 10
ip address 172.20.10.1 255.55.255.0

interface vlan 20
ip address 172.20.20.1 255.255.255.0

i am wondering if these access list below would not cause any conflicts.

access-list 100 deny 172.20.30.0 0.0.0.255 172.20.192.0 0.0.0.255

access-list 100 deny 172.20.30.0 0.0.0.255 172.20.1.0 0.0.0.255

access-list 100 permit 172.20.30.0 0.0.0.255 any

access-list 200 deny 172.20.10.0 0.0.0.255 172.20.192.0 0.0.0.255

access-list 200 deny 172.20.10.0 0.0.0.255 172.20.1.0 0.0.0.255

access-list 200 permit 172.20.10.0 0.0.0.255 any

access-list 300 deny 172.20.20.0 0.0.0.255 172.20.192.0 0.0.0.255

access-list 300 deny 172.20.20.0 0.0.0.255 172.20.1.0 0.0.0.255

access-list 300 permit 172.20.20.0 0.0.0.255 any

[Richard Burts]

These access lists are a beginning. But they are incomplete. The basic principle here is that the ACL for PBR needs to deny traffic from its source subnet to EACH of the other local subnets. So in the earlier part of the discussion when there were 3 subnets the ACL needed 2 deny statements. If you add 2 additional subnets the the ACL for PBR will need 4 deny statements. And this impacts not only the new interfaces bot also the original interfaces.

[Herald Sison]

thank you sir. i ran some test and it seems to working well for now. thank you so much

[Richard Burts]

You are quite welcome. I am glad to know that your test shows that it is working well for now. This community is an excellent place to learn about networking. I hope to see you continue to be active in the community.

[Cristian Matei]

Hi,

   If you will end up having many more subnets (now you only have two), but option from simplicity and scale point of view, would be to move to VRF-lite on Cisco 3850, with one VRF per VLAN, and thus control routing per VRF/VLAN towards a single FW, or towards both FW's in Primary/Secondary fashion if you want failover in case of primary FW failure.

  As for only two VLAN's you'll do PBR, where you'll leave traffic between 192.168.20.0/24 and 192.168.50.0/24 to be performed by Cisco 3850 (I somehow assumed this is what you want, if not, let me know) and all other traffic to go through firewalls. First configuration examples does not allow for failover (so if FW at 192.168.1.1 is down, traffic from 192.168.20.0 will no go through FW at 192.168.1.111), while second configuration example allows for failover; ensure FW's (I guess it's Fortigate) allows ICMP packets on 192.168.1.x interface, otherwise SLA/track/PBR on Cisco 3850 will fail.

First Config Example:

ip access-list extended VLAN20
 10 deny ip 192.168.20.0 0.0.0.255 192.168.50.0 0.0.0.255
 20 permit ip 192.168.20.0 0.0.0.255 any
!
route-map PBR_VLAN20 permit 10 
 match ip address VLAN20
 set ip next-hop 192.168.1.1
!
interface Vlan20
 ip policy route-map PBR_VLAN20

SecondConfig Example:

ip access-list extended VLAN20
 10 deny ip 192.168.20.0 0.0.0.255 192.168.50.0 0.0.0.255
 20 permit ip 192.168.20.0 0.0.0.255 any
!
route-map PBR_VLAN20 permit 10 
 match ip address VLAN20
 set ip next-hop verify-availability 192.168.1.1 10 track 100
 set ip next-hop 192.168.1.111
!
track 100 ip sla 100 state
ip sla 100
 icmp-echo 192.168.1.1 source-ip 192.168.1.2
ip sla schedule 100 life forever start-time now

Both configuration examples are given for VLAN20, you can easily replicate it for VLAN50 as well.

Best,

Cristian.