897VA Multiple VLAN to Dual WAN Routing

[Abe_00]

I have configured a Cisco 897VA with two WAN interfaces, and 6 VLANs.  What I would like to achieve is that hosts in one particular VLAN go out to the Internet via one WAN interface, and everything else goes out via the other one, while all VLANs are able to communicate with each other (later on I'll look to lock down access to the rest of the network from the GuestWiFi VLAN). 

Please could someone offer some advice on how I could achieve this?

Here's the summary, followed by the full config below.

1. VDSL connecting to BT Infinity (tested ok)

2. GE WAN connecting to a Huawei B535 and out to the Internet on the Three 4G network (tested ok)

I have also configured 6 VLANs (all tested ok, with inter-connectivity between them and out to the Internet, currently all going via the GE WAN on GigabitEthernet 8):

1. Cameras (192.168.2.0/24)

2. Users (192.168.3.0/24)

3. Secure Users (192.168.4.0/24)

4. Servers (192.168.5.0/24)

5. Guest WiFi (192.168.6.0/24)

6. Management (192.168.7.0/24)

I've experimented a bit with Policy-Based Routing, but I don't understand it well enough and haven't been able to get it working.

!
! Last configuration change at 09:11:08 UTC Mon Apr 27 2020
!
version 15.5
service timestamps debug datetime msec
service timestamps log datetime msec
service password-encryption
!
hostname Router
!
boot-start-marker
boot-end-marker
!
!
enable secret 5 xxxx
enable password 7 xxxx
!
no aaa new-model
ethernet lmi ce
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!

!
ip dhcp excluded-address 192.168.2.1 192.168.2.99
ip dhcp excluded-address 192.168.2.240 192.168.2.254
ip dhcp excluded-address 192.168.3.1 192.168.3.99
ip dhcp excluded-address 192.168.3.240 192.168.3.254
ip dhcp excluded-address 192.168.4.1 192.168.4.99
ip dhcp excluded-address 192.168.4.240 192.168.4.254
ip dhcp excluded-address 192.168.5.1 192.168.5.99
ip dhcp excluded-address 192.168.5.200 192.168.5.254
ip dhcp excluded-address 192.168.6.1 192.168.6.39
ip dhcp excluded-address 192.168.6.240 192.168.6.254
ip dhcp excluded-address 192.168.7.1 192.168.7.99
ip dhcp excluded-address 192.168.7.240 192.168.7.254
!
ip dhcp pool Cameras
import all
network 192.168.2.0 255.255.255.0
default-router 192.168.2.254
dns-server 192.168.2.254
lease 7
!
ip dhcp pool Users
import all
network 192.168.3.0 255.255.255.0
default-router 192.168.3.254
dns-server 192.168.3.254
lease 7
!
ip dhcp pool Secure
import all
network 192.168.4.0 255.255.255.0
default-router 192.168.4.254
dns-server 192.168.4.254
lease 7
!
ip dhcp pool Servers
import all
network 192.168.5.0 255.255.255.0
default-router 192.168.5.254
dns-server 192.168.5.254
lease 7
!
ip dhcp pool GuestWiFi
import all
network 192.168.6.0 255.255.255.0
default-router 192.168.6.254
dns-server 192.168.6.254
!
ip dhcp pool Management
import all
network 192.168.7.0 255.255.255.0
default-router 192.168.7.254
dns-server 192.168.7.254
lease 7
!
!
!
ip name-server 8.8.8.8
ip name-server 8.8.4.4
ip cef
no ipv6 cef
!
!
!
!
!
multilink bundle-name authenticated
!
!
!
!
!
!
!
license udi pid C897VA-M-K9 sn xxxxx
!
!
vtp mode transparent
!
!
!
!
!
controller VDSL 0
!
vlan 2
name Cameras
!
vlan 3
name Users
!
vlan 4
name Secure
!
vlan 5
name Servers
!
vlan 6
name GuestWiFi
!
vlan 7
!
vlan 10
name LAN-VRF01666
!
vlan 30
name LAN-VRF01667
!
!
!
!
!
!
!
!
!
!
!
!
interface ATM0
no ip address
shutdown
no atm ilmi-keepalive
!
interface Ethernet0
no ip address
!
interface Ethernet0.101
encapsulation dot1Q 101
no ip redirects
no ip proxy-arp
ip virtual-reassembly in
pppoe enable group global
pppoe-client dial-pool-number 1
!
interface GigabitEthernet0
no ip address
!
interface GigabitEthernet1
no ip address
!
interface GigabitEthernet2
switchport access vlan 2
no ip address
!
interface GigabitEthernet3
switchport access vlan 3
no ip address
!
interface GigabitEthernet4
switchport access vlan 4
no ip address
!
interface GigabitEthernet5
switchport access vlan 5
no ip address
!
interface GigabitEthernet6
switchport access vlan 6
no ip address
!
interface GigabitEthernet7
switchport access vlan 7
no ip address
!
interface GigabitEthernet8
ip address 192.168.8.254 255.255.255.0
ip nat outside
ip virtual-reassembly in
no ip route-cache
duplex auto
speed auto
!
interface Vlan1
ip address 192.168.1.254 255.255.255.0
ip nat inside
ip virtual-reassembly in
!
interface Vlan2
ip address 192.168.2.254 255.255.255.0
ip nat inside
ip virtual-reassembly in
!
interface Vlan3
ip address 192.168.3.254 255.255.255.0
ip nat inside
ip virtual-reassembly in
!
interface Vlan4
ip address 192.168.4.254 255.255.255.0
ip nat inside
ip virtual-reassembly in
!
interface Vlan5
ip address 192.168.5.254 255.255.255.0
ip nat inside
ip virtual-reassembly in
!
interface Vlan6
ip address 192.168.6.254 255.255.255.0
ip nat inside
ip virtual-reassembly in
!
interface Vlan7
ip address 192.168.7.254 255.255.255.0
ip nat inside
ip virtual-reassembly in
!
interface Dialer1
description Dialer interface for VDSL
mtu 1492
ip address negotiated
no ip redirects
no ip unreachables
no ip proxy-arp
ip nbar protocol-discovery
ip flow ingress
ip nat outside
ip virtual-reassembly in
encapsulation ppp
ip tcp adjust-mss 1452
dialer pool 1
dialer-group 1
ppp authentication pap chap ms-chap callin
ppp chap hostname bthomehub@btbroadband.com
ppp chap password 7 xxxxx
ppp ipcp address accept
no cdp enable
!
ip forward-protocol nd
ip http server
no ip http secure-server
!
!
ip dns server
ip nat inside source list 1 interface GigabitEthernet8 overload
ip nat inside source list NAT interface Dialer1 overload
ip route 0.0.0.0 0.0.0.0 192.168.8.1
!
ip access-list extended NAT
permit ip 192.168.0.0 0.0.255.255 any
remark Access list for NAT
!
!
access-list 1 permit 192.168.0.0 0.0.255.255
!
control-plane
!
!
!
mgcp behavior rsip-range tgcp-only
mgcp behavior comedia-role none
mgcp behavior comedia-check-media-src disable
mgcp behavior comedia-sdp-force disable
!
mgcp profile default
!
!
!
!
!
!
!
line con 0
password 7 xxxxx
login
no modem enable
line aux 0
line vty 0
exec-timeout 40 0
privilege level 15
password 7 xxxxx
logging synchronous
login
transport input telnet
line vty 1 4
privilege level 15
password 7 xxxxx
logging synchronous
login
transport input telnet
!
scheduler allocate 20000 1000
!
end

[Abzal]

Hi @Abe_00 

This PBR config is pretty straightforward just make sure you match your NAT ACL with PBR ACL. 

Let's say a subnet 192.168.2.0/24 access Internet via GE WAN:

access-list 101 permit ip 192.168.2.0 0.0.0.255 192.168.1.0 0.0.0.255

access-list 101 permit ip 192.168.2.0 0.0.0.255 192.168.2.0 0.0.0.255

access-list 101 permit ip 192.168.2.0 0.0.0.255 192.168.3.0 0.0.0.255

access-list 101 permit ip 192.168.2.0 0.0.0.255 192.168.4.0 0.0.0.255

access-list 101 permit ip 192.168.2.0 0.0.0.255 192.168.5.0 0.0.0.255

access-list 101 permit ip 192.168.2.0 0.0.0.255 192.168.6.0 0.0.0.255

access-list 101 permit ip 192.168.2.0 0.0.0.255 192.168.7.0 0.0.0.255

! ACL #102 to ensure InterVlan routing isn't affected by pbr.

!

access-list 102 permit ip 192.168.2.0 0.0.0.255 any

access-list 10 permit 192.168.2.0 0.0.0.255

ip nat inside source list 10 interface GigabitEthernet8 overload

!

route-map GE-WAN deny 10
 match ip address 101

!

route-map GE-WAN permit 20
 match ip address 102
set ip next-hop 192.168.8.1

!

interface Vlan2
 ip policy route-map GE-WAN

!

A guide for PBR.

https://community.cisco.com/t5/networking-documents/dual-internet-links-nating-with-pbr-and-ip-sla/ta-p/3126275

Kindly mark helpful answers.

[Abe_00]

Hey @Abzal , thanks for the reply.  This helps a lot, I've proven this config and now have PBR working so that hosts in one subnet can get to the Internet via the GE WAN interface.

I should mention that due to an error on my part, I actually posted this question twice.  Here's the link to the other one:

https://community.cisco.com/t5/routing/897va-multiple-vlan-to-dual-wan-routing/m-p/4075070#M333437

However, I have an unwanted side-effect, which is now hosts in the other VLANs can talk to each other, but they cannot get to the Internet via the Dialer1 interface.

If I try to ping www.google.com from a host in the 192.168.3.0/24 subnet, I can see that it resolves the name ok, but it doesn't ping.

If I try to ping directly from the router (via a telnet session) then it pings no problem.

Any ideas?

[Abzal]

Hi,

Probably because config still has a default route over GE. Remove it and put a new default route over Dialer interface.

no ip route 0.0.0.0 0.0.0.0 192.168.8.1

ip route 0.0.0.0 0.0.0.0 dialer1

[Abe_00]

Hi @Abzal ,

Thank you for your help with this, I've created a problem for myself by posting this question twice (the reason was the first post I put up was this one, but it was immediately marked as spam for some reason, so I thought no-one could see it, and I posted it a second time).

However, I have made some further changes, latest config is below.  These are the results of the latest tests I did:

1. Internet access from VLAN 4 through GE WAN - tested successfully

2. Inter-VLAN connectivity between VLAN 4 and other VLANs - tested successfully

3. Internet access from VLAN 3 through Dialer1 - tested UNsuccessfully

4. Internet access direct from router - tested successfully (ping + traceroute showing traffic going out through Dialer1)

Latest config:

!
! Last configuration change at 09:51:59 UTC Tue Apr 28 2020
!
version 15.5
service timestamps debug datetime msec
service timestamps log datetime msec
service password-encryption
!
hostname Router
!
boot-start-marker
boot-end-marker
!
!
enable secret 5 xxxxx
enable password 7 xxxxx
!
no aaa new-model
ethernet lmi ce
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!


!
ip dhcp excluded-address 192.168.2.1 192.168.2.99
ip dhcp excluded-address 192.168.2.240 192.168.2.254
ip dhcp excluded-address 192.168.3.1 192.168.3.99
ip dhcp excluded-address 192.168.3.240 192.168.3.254
ip dhcp excluded-address 192.168.4.1 192.168.4.99
ip dhcp excluded-address 192.168.4.240 192.168.4.254
ip dhcp excluded-address 192.168.5.1 192.168.5.99
ip dhcp excluded-address 192.168.5.200 192.168.5.254
ip dhcp excluded-address 192.168.6.1 192.168.6.39
ip dhcp excluded-address 192.168.6.240 192.168.6.254
ip dhcp excluded-address 192.168.7.1 192.168.7.99
ip dhcp excluded-address 192.168.7.240 192.168.7.254
!
ip dhcp pool Cameras
 import all
 network 192.168.2.0 255.255.255.0
 default-router 192.168.2.254 
 dns-server 192.168.2.254 
 lease 7
!
ip dhcp pool Users
 import all
 network 192.168.3.0 255.255.255.0
 default-router 192.168.3.254 
 dns-server 192.168.3.254 
 lease 7
!
ip dhcp pool Secure
 import all
 network 192.168.4.0 255.255.255.0
 default-router 192.168.4.254 
 dns-server 8.8.8.8 8.8.4.4 
 lease 7
!
ip dhcp pool Servers
 import all
 network 192.168.5.0 255.255.255.0
 default-router 192.168.5.254 
 dns-server 192.168.5.254 
 lease 7
!
ip dhcp pool GuestWiFi
 import all
 network 192.168.6.0 255.255.255.0
 default-router 192.168.6.254 
 dns-server 192.168.6.254 
!
ip dhcp pool Management
 import all
 network 192.168.7.0 255.255.255.0
 default-router 192.168.7.254 
 dns-server 192.168.7.254 
 lease 7
!
!
!
ip name-server 8.8.8.8
ip name-server 8.8.4.4
ip cef
no ipv6 cef
!
!
!
!
!
multilink bundle-name authenticated
!
!
!
!
!
!
!
license udi pid C897VA-M-K9 sn xxxxx
!
!
vtp mode transparent
!
!
!
!
!
controller VDSL 0
!
vlan 2
 name Cameras
!
vlan 3
 name Users
!
vlan 4
 name Secure
!
vlan 5
 name Servers
!
vlan 6
 name GuestWiFi
!
vlan 7 
!
vlan 10
 name LAN-VRF01666
!
vlan 30
 name LAN-VRF01667
!
! 
!
!
!
!
!
!
!
!
!
!
interface ATM0
 no ip address
 shutdown
 no atm ilmi-keepalive
!
interface Ethernet0
 no ip address
!
interface Ethernet0.101
 encapsulation dot1Q 101
 no ip redirects
 no ip proxy-arp
 ip virtual-reassembly in
 pppoe enable group global
 pppoe-client dial-pool-number 1
!
interface GigabitEthernet0
 no ip address
!
interface GigabitEthernet1
 no ip address
!
interface GigabitEthernet2
 switchport access vlan 2
 no ip address
!
interface GigabitEthernet3
 switchport access vlan 3
 no ip address
!
interface GigabitEthernet4
 switchport access vlan 4
 no ip address
!
interface GigabitEthernet5
 switchport access vlan 5
 no ip address
!
interface GigabitEthernet6
 switchport access vlan 6
 no ip address
!
interface GigabitEthernet7
 switchport access vlan 7
 no ip address
!
interface GigabitEthernet8
 ip address 192.168.8.254 255.255.255.0
 ip nat outside
 ip virtual-reassembly in
 no ip route-cache
 duplex auto
 speed auto
!
interface Vlan1
 ip address 192.168.1.254 255.255.255.0
 ip nat inside
 ip virtual-reassembly in
!
interface Vlan2
 ip address 192.168.2.254 255.255.255.0
 ip nat inside
 ip virtual-reassembly in
!
interface Vlan3
 ip address 192.168.3.254 255.255.255.0
 ip nat inside
 ip virtual-reassembly in
!
interface Vlan4
 ip address 192.168.4.254 255.255.255.0
 ip nat inside
 ip virtual-reassembly in
 ip policy route-map GE-WAN
!
interface Vlan5
 ip address 192.168.5.254 255.255.255.0
 ip nat inside
 ip virtual-reassembly in
!
interface Vlan6
 ip address 192.168.6.254 255.255.255.0
 ip access-group 102 in
 ip nat inside
 ip virtual-reassembly in
!
interface Vlan7
 ip address 192.168.7.254 255.255.255.0
 ip nat inside
 ip virtual-reassembly in
!
interface Dialer1
 description Dialer interface for VDSL
 mtu 1492
 ip address negotiated
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 ip nbar protocol-discovery
 ip flow ingress
 ip nat outside
 ip virtual-reassembly in
 encapsulation ppp
 ip tcp adjust-mss 1452
 dialer pool 1
 dialer-group 1
 ppp authentication pap chap ms-chap callin
 ppp chap hostname bthomehub@btbroadband.com
 ppp chap password 7 xxxxx
 ppp ipcp address accept
 no cdp enable
!
ip forward-protocol nd
ip http server
no ip http secure-server
!
!
ip dns server
ip nat inside source list 1 interface GigabitEthernet8 overload
ip nat inside source list 2 interface Dialer1 overload
ip route 0.0.0.0 0.0.0.0 Dialer1
!
ip access-list extended GuestWiFi
 deny   ip any 192.168.1.0 0.0.0.255
 deny   ip any 192.168.2.0 0.0.0.255
 deny   ip any 192.168.3.0 0.0.0.255
 deny   ip any 192.168.4.0 0.0.0.255
 deny   ip any 192.168.5.0 0.0.0.255
 deny   ip any 192.168.7.0 0.0.0.255
 permit ip any any
!
!
route-map GE-WAN permit 10
 match ip address 100
 set ip next-hop 192.168.8.1
!
access-list 1 permit 192.168.0.0 0.0.255.255
access-list 2 permit 192.168.0.0 0.0.255.255
access-list 100 deny   ip 192.168.4.0 0.0.0.255 192.168.1.0 0.0.0.255
access-list 100 deny   ip 192.168.4.0 0.0.0.255 192.168.2.0 0.0.0.255
access-list 100 deny   ip 192.168.4.0 0.0.0.255 192.168.3.0 0.0.0.255
access-list 100 deny   ip 192.168.4.0 0.0.0.255 192.168.5.0 0.0.0.255
access-list 100 deny   ip 192.168.4.0 0.0.0.255 192.168.6.0 0.0.0.255
access-list 100 deny   ip 192.168.4.0 0.0.0.255 192.168.7.0 0.0.0.255
access-list 100 permit ip 192.168.4.0 0.0.0.255 any
access-list 102 deny   tcp any any eq telnet
access-list 102 permit ip any any
!
control-plane
!
!
!
mgcp behavior rsip-range tgcp-only
mgcp behavior comedia-role none
mgcp behavior comedia-check-media-src disable
mgcp behavior comedia-sdp-force disable
!
mgcp profile default
!
!
!
!
!
!
!
line con 0
 password 7 xxxxx
 login
 no modem enable
line aux 0
line vty 0
 exec-timeout 40 0
 privilege level 15
 password 7 xxxxx
 logging synchronous
 login
 transport input telnet
line vty 1 4
 privilege level 15
 password 7 xxxxx
 logging synchronous
 login
 transport input telnet
!
scheduler allocate 20000 1000
!
end

[paul driver]

Hello
When you have dual wan connections like you have shown you will need to accomodate for primary link failure not only for PBR but for global routing and NAT because if that goes down then traffic could be blackholed due to the default static route you have applied, So having conditional static routing and nat would be applicable.

Example:

no ip nat inside source list 1 interface GigabitEthernet8 overload
no ip nat inside source list NAT interface Dialer1 overload
no ip route 0.0.0.0 0.0.0.0 192.168.8.1

route-map 4G_NAT
match ip address 1
match interface GigabitEthernet 8

route-map VDSL_NAT
match ip address 1
match interface Dialer1

ip nat inside source list route-map VDSL_NAT interface Dialer1
ip nat inside source list route-map 4G_NAT interface GigabitEthernet8 

ip sla 1
icmp-echo 8.8.8.8 source-interface GigabitEthernet8
timeout 2000
frequency 5
ip sla schedule 1 life forever start-time now
track 10 rtr 1 reachability

ip route 0.0.0.0 0.0.0.0 interface GigabitEthernet8 192.168.8.1 track 10
ip route 8.8.8.8 255.255.255.255 Null0 2
ip route 0.0.0.0 0.0.0.0 Dialer1 2

Lastly for the PBR would suggest if you cannot specify an ip next-hop and not able to verify its reachability then try appending the set interface (dialer1,GigabitEthernet8) instead so if the dialer interface goes down then the PBR will try to choose the BT interface.

[Abe_00]

Thanks for this, Paul.  I've read your message a number of times and am working hard to understand it fully.

The comment about fail-over definitely resonated and got me thinking about a slightly different scenario.

I suppose the preferred behavior would be:

1. All 'normal' internet-bound traffic, i.e. from VLANs 1, 2, 3, 5, 6 and 7 should go out via the 'BT' link (VDSL), if that link should fail then these VLANs should NOT fail over to the 4G link.

2. Internet-bound traffic from VLAN 4 should go out via the '4G' link (GE WAN), if that link should fail, then it would be useful for VLAN 4 to fail-over to the 'BT' link.

The example you outlined is very useful, my understanding is:

Section 1 is undoing some of the existing config (actually I don't have these lines any more, but they are in my original post)

Section 2 is defining two new route maps, one each for the outbound interfaces

Section 3 is redefining the ip nat statements to use the route maps

Section 4 appears to be setting up a way that the router can test using an external ip address whether the link is up or not, and return a value if it is?

Section 5 appears to define route statements, but I'm not sure what the resultant behavior would be.

I also don't understand how to append the set interface statement to achieve the actual fail-over.

Apologies for all the follow-up questions.

Kind Regards,

Abe.

[Abe_00]

I'm going to attempt to close-down this discussion thread, as it is effectively a duplicate (a problem that I created mysef) of the following discussion:

https://community.cisco.com/t5/routing/897va-multiple-vlan-to-dual-wan-routing/m-p/4075070

Apologies for the confusion.

[paul driver]

Hello

You are correct in your assumption witch section 1-4, let me clarify the others for you is I may.

---

@Abe_00 wrote:

Section 5 appears to define route statements, but I'm not sure what the resultant behavior would be.

---

Static routes by default have no way of knowing about failed links so when you append one to a rtr especially a default static then you are saying to the rtr always forward via this path which isn’t a very deterministic way of routing when you have dual wan links because as I stated previously if any failure to the primary wan connection occurs the static default will still be forwarding to a path that isn’t valid thus black holing it, However you can add a condition to the default route with some tracking (interface or ip) so if/when the  tracked object fails the conditional default route is removed based on the failed tracked object and is re-added if/when its reachable again.

What you can see is that ip sla monitoring will be polling 8.8.8.8 through the primary wan interface sourced from the BT connected interface designated by the primary default static route

Now this primary default static route has a preferred administrative distance of 1 and has tracking 10 appended which correlates to the ip sla monitoring.

ip sla 1
icmp-echo 8.8.8.8 source-interface GigabitEthernet8
timeout 2000
frequency 5
ip sla schedule 1 life forever start-time now
track 10 rtr 1 reachability

ip route 0.0.0.0 0.0.0.0 interface GigabitEthernet8 192.168.8.1 track 10

If/when that track object fails the primary default static will be removed, at this point you can rely on the ADSL rtr to advertise to you a default route or you can control that yourself with adding another default static route pointing towards the ADSL interface but with a higher administrative distance value then the original primary default static
ip route 0.0.0.0 0.0.0.0 Dialer1 2

Now when this ADSL interface becomes the primary path, what you don’t want is for the tracking to become active again as no doubt that ip sla monitored ip address would become reachable via the ADSL connection which you don’t want to happen so what you can do here is to add a null route for that monitored ip with a administrative distance higher than the original primary default static route
ip route 8.8.8.8 255.255.255.255 Null0 2

Lastly for the PBR, what I mean is here is policy routing can also be made conditional usually by verifying the reachability of the next hop ip address you specify within the policy again with ip sla tracking, However has you don’t know the ip next hop address you could try and specify a policy route to use multiple interfaces so then if one interface fails the rtr will attempt to use another interface, Now if you don’t have any condition whatsoever then if the ADSL were to fail PBR by default won’t preempt the failure to the an alternate default path it will  keep trying to PBR your traffic to a failed interface.

Example
route-map PBR_ip
match ip address <ACL>
set ip next-hop verify-availability x.x.x.x 1 track 20

or
route-map PBR_interface
match ip address <ACL>
set interface dialer 1 GigabitEthernet8
set ip next-hop verify-availability