Issues with NAT on a brand new ISR4321

[hayden.whizzit]

HI All,

This is my first post here, after much lurking, and I am hoping this is the right place to come for some assistance with this painful NAT issue I am having. I would consider myself to be fairly experienced with these sorts of network, and have done many like it before without issue. Sadly not the case this time!! This is my first experience with the ISR4321 and I am not sure what I am missing here.

Quick rundown of the network:

5 VLAN's, which are routed between each other by the core switch, which is a SG300-28P. 

VLAN IP's are as follows:

VLAN1   = 10.0.10.0/24 - Used for native/servers/management
VLAN20 = 10.0.20.0/23 - Used for PC's
VLAN30 = 10.0.30.0/23 - Used for wireless clients
VLAN40 = 10.0.40.0/24 - Used for VoIP
VLAN50 = 10.0.50.0/24 - Used for CCTV

- The core switch, the SG300, is on IP 10.0.10.254 and is the default gateway for all clients on all VLAN's.
- The default gateway for the core switch is the router in question, a brand new ISR4321 with a NIM-VAB-A for ADSL 2+ Internet connection (ISP is Telstra).This router is on IP 10.0.10.253
- There is a  second gateway, and older Cisco 2811, is on IP 10.0.10.252, and has a HWIC-1ADSL internet connection (ISP is iinet).

There is a policy based route configured on the ISR4321 to push traffic from ALL VLAN's out via the second gateway, the 2811, EXCEPT VLAN40. This is done with an ACL and route-map, as you will see in the below config. The reason for this is the ADSL connection on the ISR4321 is reserved for VoIP data only, and is not used by other devices at all.

Now, when I first went to configure and deploy these two new routers into this network, which replaced an old Linsys X2000 SOHO router, I encountered some issues getting NAT to work properly on the ISR4321. I configured NAT on the ISR4321 just as I would on any other cisco router, but for some reason dynamic NAT translations were not being populated. I went in circles with this for a few hours then at some point, it just started working!! I was not sure what was the issue but I was happy it was working so the router was deployed.

Afterward, I deployed the second 2811 router which had no issues at all with the NAT configuration.

About 1 month down the track now, NAT has again stopped working on the ISR4321 which means our VoIP is no longer working. I have looked at the config again and again but I must be missing something here. For the life of me I dont know what it is. No debugging has helped, but I am also not an expert in NAT on Cisco, as in the past it has usually worked just fine with my configurations.

It is worth noting the following things before looking at this config:

- This config works perfectly on the 2811 on this ADSL connection (telstra), as well as the iinet ADSL.
- The ISR4321 has perfect connectivity back to Telstra, I can also SSH in to the router from its outside IP address.
- The clients are all able to communicate with the router no problem, and all other VLAN's except VLAN40 still have perfect internet connectivity due to passing through the ISR4321 to the 2811 via the PBR that is in place.
- I have tried changing ACL types that is used for my dynamic NAT source list, as well as commands on different interfaces etc.
- As mentioned before, this config was working for some time without changing a thing, but now dynamic nat is failing again!

This almost seems like an IOS bug to me, but I cant find anything to support that claim.
I can only assume something changed on the ISR4321, but even after reading all of the Cisco guides pertaining to NAT config on these routers, I cannot see what.

Please help!

And my apologies for the long winded post.

Thanks!

ISR4321 Config:

version 15.5
service timestamps debug datetime msec
service timestamps log datetime msec
no platform punt-keepalive disable-kernel-core
!
hostname XXX_Router_ISR4321
!
boot-start-marker
boot-end-marker
!
!
vrf definition Mgmt-intf
 !
 address-family ipv4
 exit-address-family
 !
 address-family ipv6
 exit-address-family
!
!
aaa new-model
!
!
aaa authentication login default local
aaa authorization exec default local
!
!
!
!
!
aaa session-id common
!
!
!
!
!
!
!
!
!


ip domain name XXXXX
!
subscriber templating
!
multilink bundle-name authenticated
!
!
!
!
license udi pid ISR4321/K9 sn XXXXX
spanning-tree extend system-id
!
username XXXX privilege 15 secret 5 XXXX
!
redundancy
 mode none
!
!
!
!
controller VDSL 0/1/0
 operating mode adsl2+ annex A
!
!
vlan internal allocation policy ascending
!
ip tftp source-interface GigabitEthernet0
ip ssh rsa keypair-name ssh_key
!
interface GigabitEthernet0/0/0
 no ip address
 negotiation auto
!
interface GigabitEthernet0/0/1
 ip address 10.0.10.253 255.255.255.0
 ip nat inside
 ip policy route-map voip_data_out
 negotiation auto
 ip virtual-reassembly
!
interface ATM0/1/0
 no ip address
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 load-interval 30
 no atm oversubscribe
 no atm ilmi-keepalive
 no atm enable-ilmi-trap
!
interface ATM0/1/0.1 point-to-point
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 no atm enable-ilmi-trap
 pvc 8/35
 vbr-rt 969 969 1
 pppoe-client dial-pool-number 1
 !
!
interface Ethernet0/1/0
 no ip address
 no negotiation auto
!
interface GigabitEthernet0
 vrf forwarding Mgmt-intf
 no ip address
 shutdown
 negotiation auto
!
interface Vlan1
 no ip address
!
interface Dialer1l
 ip address negotiated
 ip mtu 1452
 ip nat outside
 encapsulation ppp
 dialer pool 1
 dialer-group 1
 ppp authentication pap callin
 ppp chap refuse
 ppp pap sent-username XXXXXX password 0 XXXXX
 no cdp enable
 ip virtual-reassembly
!
ip nat inside source list nat_source_list interface Dialer1 overload
ip nat inside source static tcp 10.0.XX.XX XXXX interface Dialer1 XXXX
ip nat inside source static tcp 10.0.XX.XX XXXX interface Dialer1 XXXX
ip nat inside source static tcp 10.0.XX.XX XXXX interface Dialer1 XXXX
ip nat inside source static tcp 10.0.XX.XX XXXX interface Dialer1 XXXX
ip nat inside source static tcp 10.0.XX.XX XXXX interface Dialer1 XXXX
ip forward-protocol nd
no ip http server
no ip http secure-server
ip route 0.0.0.0 0.0.0.0 Dialer1
ip route 10.0.20.0 255.255.254.0 10.0.10.254
ip route 10.0.30.0 255.255.254.0 10.0.10.254
ip route 10.0.40.0 255.255.255.0 10.0.10.254
ip route 10.0.50.0 255.255.255.0 10.0.10.254
!
!
ip access-list extended nat_source_list
 permit ip 10.0.10.0 0.0.0.255 any
 permit ip 10.0.20.0 0.0.1.255 any
 permit ip 10.0.30.0 0.0.1.255 any
 permit ip 10.0.40.0 0.0.0.255 any
 permit ip 10.0.50.0 0.0.0.255 any
ip access-list extended voip_vlan
 permit ip 10.0.10.0 0.0.0.255 any
 permit ip 10.0.20.0 0.0.1.255 any
 permit ip 10.0.30.0 0.0.1.255 any
 permit ip 10.0.50.0 0.0.0.255 any
 deny ip 10.0.40.0 0.0.0.255 any
!
!
route-map voip_data_out permit 10
 match ip address voip_vlan
 set ip next-hop 10.0.10.252
!
!
!
!
!
control-plane
!
!
line con 0
 stopbits 1
line aux 0
 stopbits 1
line vty 0 4
 logging synchronous
 transport input ssh
!
!
end

[hayden.whizzit]

It is also worth mentioning the command "show ip nat translations" ONLY shows the static NAT translations, nothing more. Not a single dynamic entry. 

Sad face.

Thanks,

Hayden

[nandau1082]

May be it helps.. I think you are doing NAT overload on interface Dialer1.  In looking at the interface Dialer1 configuration, IP address is set to negotiate.  The IP may be dynamically changing and breaking the NAT translation.  

What happens when you clear nat translation with issues occur?

[hayden.whizzit]

Thanks for your response.

The ip address is being set by the ISP but is "static" in the sense it does not change and is always the same address.

Clearing translations does not have any effect, even a reboot seems  to do nothing.... i am stumped with this one!

I am actually considering replacing the ISR4321 with another spare 2811 that i know will work!!

[hayden.whizzit]

This router has been temporarily replaced with a Cisco 2811, and is going to be returned for replacement. We simply cannot find the cause of the issue, which is a first for us with a brand new Cisco router...

[Elliott Willink]

Call Cisco and lodge a TAC case for an RMA. Config looks fine, so (excluding the problem being somewhere else on your network) I wouldn't put it past being a bug.

From past experience with faulty UCS gear, the TAC team will almost certainly help you to find the cause (and work out if it is a bug), even without a service contract, if you are actually about to return it because it doesn't work properly. 

[hayden.whizzit]

Thanks Elliott, I did already call Cisco today but TAC would not help without a service contract. Because I feel also that this is a bug I do not want to spend the $300 odd dollars for a contact only for TAC then to turn around and tell me its a bug and to return it!

I was pretty disappointed that I was only able to go that far. I have already lodged an RMA with the wholesaler in Australia and they seem happy to replace it.

I am glad to hear others do not see a config issue and its not just me!

[prasad_m897]

Instead of this route

ip route 10.0.30.0 255.255.254.0 10.0.10.254
Give 252 and try. but routing shows 254 and route map next hop is 252.

[hayden.whizzit]

Thanks for the response, the reason the route is set as 254 is because that is the IP Address of the VLAN Interface on the core switch, 10.0.10.254 Cisco SG300-28P  which does all the inter-VLAN routing.

Changing it to 252 would send it to the 2811 second gateway router, and that router would just send the traffic back to 254.

The issue is really only affecting the 10.0.40.0/24 network as this is the only one now going out directly from the ISR4321, all the other VLAN's are forwarded with the route map to .252 (Cisco 2811).

I hope that makes sense! Sorry I realise I should probably provide a network diagram.

[hayden.whizzit]

For those following this topic:

The issue was finally resolved by doing an RMA on the ISR4321 & NIM-VAB-A, and was replaced after a lengthy wait with a new unit. The new ISR4321 is running IOS-XE 3.16.2S and with the exact same config on it has been working flawlessly so far.

I can really only chalk this up to being some odd bug in the previous version of IOS-XE.

[alisathik]

Hi,

May be old thread, 

I have one suggestion if anyone having same iaaue which is not resolved.

When we do PAT, no need to configure static route for dialer interface.

*IP route 0 0 dialer 1*  (not required)

Just 

*IP Nat inside source list ....overload* 

is enough.

Might help...

Correct me if mistaken.

[Bernhard Roth]

Just a note:

According to Cisco, NAT on Dialer is not documented and thus unsupported

My experience is that is works well on IOS-XE 3.xx series but once you upgrade to 16.x.x (Denali etc..) it has severe problems.

All TCP, UDP or ICMP traffic going though Dialer-NAT might stop working without notice

That whole situation is very bad.

All cutomers coming from older ISR G2 series with xDSL are required to use NAT on Dialer. There is no other option.

So what are your options:

- Cisco denies support for NAT on Dialer on IOS-XE (ISR 4k platforms)

- There is no alternative

- Going back to older ISR G2?

If someone has a Cisco Account Manager, please let them know about that issue.

NAT on Dialer is essential in many situations and in particular for sales when replacing old ISR G2 units.

[bryan.watson237]

I have found out it seems to be the ARP request that is not playing well with NAT Pools. I have been able to create a loopback and apply an address that is within the NAT Translation Pool as an Outside Global Address to resolve the issue with NAT not working and allowing private addresses to be translated to public addresses. Not sure if it will work with a Dialer Interface. When using a Static NAT you will have to use an additional command to allow outside ip addresses to be able to connect to internal addresses, but im not sure that is you current issue as well. The command is ip nat allow-static-host which is applied on the internal interface connecting to possibly a DMZ Server.

[paul driver]

Hello
Can you try the following:

Access-list 100 permit ip 10.0.40.0 0.0.0.255 any
Route-map voip_data_out deny 5
match ip address 100

Also remove the relating ace entry from the voip_vlan acl