For the Benefit of anyone reading this thread when its Necro this is what the fix was.
My router had a static default route that was pushing ALL traffic to the VPN. In order to effectivly split the traffic I had to create static routes for each network segment that was on the VPN and use a Default route to send any remaining traffic out the ISP. I had to Nat outside both Interface. That required a rule for each interface as you see can see in the Config. I mad it simple and used the same access list for each Nat rule. This is not ideal for everyone as it doesn't allow for granular control to each interface. (I'll fix it later It was just to get it working.) With that done and the static routes in place it worked. Here is my final config with Notes added *Notes:
Interfaces:
interface GigabitEthernet0/0 *Not in use
ip address 192.168.161.1 255.255.255.0
duplex auto
speed auto
!
interface GigabitEthernet0/1 *Not in use
ip address 192.168.162.1 255.255.255.0
duplex auto
speed auto
!
interface GigabitEthernet0/2 *This goes to my primary switch connecting the local network to the Router
ip address 192.168.163.1 255.255.255.0
ip nat inside *This was a key Configuration item
ip virtual-reassembly
duplex auto
speed auto
!
interface GigabitEthernet0/0/0 * Not in use
ip address 192.168.164.1 255.255.255.0
negotiation auto
!
interface GigabitEthernet0/1/0 *This is my VPN uplink. Initially a default route was forwarding all traffic to it.
description PRN_VPN
ip address 172.16.248.113 255.255.255.248
negotiation auto
!
interface FastEthernet0/2/0 *This is can be ignored as its just a bridge to another network segment and not relevant to the problem
ip address 192.168.60.100 255.255.255.0
ip nat outside
ip virtual-reassembly
duplex auto
speed auto
!
interface FastEthernet0/2/1 * Not in use
ip address 192.168.101.1 255.255.255.0
duplex auto
speed auto
!
interface FastEthernet0/3/0 *This is My ISP connection I set the Interface IP to 50.2 as the gateway on the other side was 50.1
description ISP Wan
ip address 192.168.50.2 255.255.255.0
ip nat outside *This was key to getting this interface to work. I couldn't properly route traffic out this interface without nat.
ip virtual-reassembly
duplex auto
speed auto
!
interface FastEthernet0/3/1 * Not In use
description PRN_VPN
no ip address
shutdown
duplex auto
speed auto
!
interface GigabitEthernet4/0 *Not In use
ip address 192.168.160.1 255.255.255.0
Rules:
ip nat inside source list 1 interface FastEthernet0/3/0 overload * This is the Nat Rule that uses an access list to allow traffic going to specific subnets.
ip route 0.0.0.0 0.0.0.0 192.168.50.1 Global Catch all rule to send any traffic that doesn't match a static route out the ISP
ip route 10.255.254.0 255.255.255.0 172.16.248.114
ip route 141.221.0.0 255.255.0.0 172.16.248.114
ip route 172.16.0.0 255.255.0.0 172.16.248.114
ip route 172.16.134.0 255.255.255.0 172.16.248.114
ip route 192.168.98.0 255.255.255.0 192.168.101.2
ip route 192.168.99.0 255.255.255.0 192.168.101.2
Note* All of these routes are Just to forward traffic to the VPN. All of the subnets are behind the VPN Interface. The route forces any traffic with a matching network destination to go through the VPN Wan Interface. It was a challenge to track down all the subnets.
*Note. This access list allows each subnet to access the Nated interface gi0/3/0
access-list 1 permit 192.168.163.0 0.0.0.255
access-list 1 permit 192.168.160.0 0.0.0.255
access-list 1 permit 172.16.0.0 0.0.255.255
access-list 1 permit 172.16.134.0 0.0.0.255
access-list 1 permit 192.168.101.0 0.0.0.255
access-list 1 permit 192.168.161.0 0.0.0.255
access-list 1 permit 192.168.162.0 0.0.0.255
access-list 1 permit 192.168.164.0 0.0.0.255
access-list 1 permit 172.16.248.0 0.0.0.255
access-list 1 permit 141.221.0.0 0.0.255.255
access-list 1 permit 10.255.254.0 0.0.0.255
access-list 1 permit 10.10.48.0 0.0.0.255
Huge Thanks to George Pauwen for helping me set this straight.
