IOS-EX Static NAT on Secondary IP address

[AllertGen]

Hi everyone.

Not much time ago we bought a ISR 4451-X with IOS-EX OS on it. And now we have some problems with static nat rules. Here is a confog for NAT Rules:

interface Port-channel1.100
 description ---To_Internal_Network---
 bandwidth 1048576
 encapsulation dot1Q 100
 ip address 10.X.X.X 255.255.255.248
 ip nat inside
 delay 1000
exit

interface GigabitEthernet0/0/2
 description ---- ISP Beeline (inet) -----
 bandwidth 50000
 ip address X.X.X.226 255.255.255.248 secondary
 ip address X.X.X.227 255.255.255.248 secondary
 ip address X.X.X.228 255.255.255.248 secondary
 ip address X.X.X.230 255.255.255.248 secondary
 ip address X.X.X.229 255.255.255.248 secondary
 ip address Y.Y.Y.230 255.255.255.252
 ip nat outside
 media-type rj45
 negotiation auto
 no cdp enable
exit

ip route 0.0.0.0 0.0.0.0 Y.Y.Y.229

ip nat inside source list 110 interface GigabitEthernet0/0/2 overload
ip nat inside source static 10.X.X.253 X.X.X.227 extendable

ACL 110 has a permiteble rules for internal devices to internet. The host 10.X.X.253 presents at this rules.

So my problem is when I'm adding a line with static NAT rule (ip nat inside source static 10.X.X.2 X.X.X.229 extendable) a device 10.X.X.253 can't communicate by TCP or UDP protocols. But ICMP works well in both directions. I have response from X.X.X.229 at the internet and my device 10.X.X.253 can ping anything at the internet. But I can't connect to the device from the internet by TCP or UDP traffic (for example via Web traffic or DNS) and I can't connect to the internet from 10.X.X.253 device. But as soon as I delete line ip nat inside source static 10.X.X.253 X.X.X.227 extendable my device can connect to the internet freely.

I had a 3925 router before and all this rules did work well on it.

Any Ideas?

Here is some additional information:

#sh ver
Cisco IOS XE Software, Version 03.15.01c.S - Standard Support Release
Cisco IOS Software, ISR Software (X86_64_LINUX_IOSD-UNIVERSALK9-M), Version 15.5(2)S1c, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2015 by Cisco Systems, Inc.
Compiled Sat 15-Aug-15 21:40 by mcpre

When I'm trying to get access to the internet from 10.X.X.253:

sh ip nat tran | i 10.X.X.253                       
---  X.X.X.227        10.X.X.253           ---                   ---
tcp  X.X.X.227:50141  10.X.X.253:50141     X.X.233.56:80      X.X.233.56:80
tcp  X.X.X.227:50143  10.X.X.253:50143     X.X.233.56:80      X.X.233.56:80
tcp  X.X.X.227:50142  10.X.X.253:50142     X.X.233.56:80      X.X.233.56:80

I also tryed to use ACLs at the interfaces with log command to find out how the traffic goes. It went at this way (at the time when I tryed get access to the internet): 10.X.X.253 -> router (hit appears), X.X.X.227 -> X.X.233.56 (hit appears), [Response] X.X.233.56 -> X.X.X.227 (hit appears), router -> 10.X.X.253 (no mathes at the OUT ACL on Port-Channel 1.100 interface).

Problem solved. Solution at the end of the topic!

Best Regards.

[Peter Paluch · Accepted solution]

Hello,

It appears that the address X.X.X.227 used in your ip nat inside source static command is also assigned to your Gi0/0/2 interface as a secondary address. Considering the fact that you have used this entire IP address to represent your internal host 10.X.X.253, there is no reason for this IP address to be also assigned on the Gi0/0/2 interface. There may be some unfortunate interactions between this address as used in the NAT and as used as a secondary address on your Gi0/0/2.

Can you therefore try removing the ip address X.X.X.227 255.255.255.248 secondary line from your Gi0/0/2 line, then clear ip nat translation force and try again?

Best regards,
Peter

[AllertGen]

Hi, Peter Paluch.

Thanks a lot for your reply. I need to use static line to give access from the Internet to a 10.X.X.253 device. And i need to use X.X.X.227 for it. I'm also planning to use all another secondary (and only secondary) addresses to give access from the Internet to internal services. This is what for I need a static NAT.

I can try remove  line. But I thik it will breake NAT, becouse there is no other interfaces with X.X.X.227 address.

BTW I did try to delete the first NAT rule (ip nat source static list 110 interface Gi0/0/2 overload) but resault was the same. Just other hosts lost access to the internet.

Best Regards.

[Peter Paluch]

Hello,

But I thik it will breake NAT, becouse there is no other interfaces with X.X.X.227 address.

I do not think it will. On Cisco routers, interface addresses and addresses used in NAT are entirely decoupled and independent. You can NAT your internal server to any public address you want, without really having that public address configured on any of your interfaces. Doing NAT is one thing while delivering packets for that address back to you is another thing that is concerned with routing and plain basic IP connectivity.

Because your Gi0/0/2 is configured with a set of public addresses from the same subnet X.X.X.224/29 into which the X.X.X.227 also belongs, your router is smart enough to respond to ARP requests for X.X.X.227 even if that address is only used in NAT and is not assigned to your Gi0/0/2 interface as a secondary address.

So my suggestion still holds - please try removing that secondary address from the Gi0/0/2, clear the NAT translation table and test the connectivity again. Please be advised that clearing the NAT table will cause existing connections to the internet to be dropped. They will need to be reestablished by clients.

Best regards,
Peter

[AllertGen]

Hi, Peter Paluch.

Wow, you're really right. It started to work. Looks like my ISP has a stastic ARP records (I thought they didn't) for my secondary IP addresses.

Thanks a lot for the solution. But it's a little strange way to work...

Best Regards.

[Peter Paluch]

Hi,

I am glad to hear that it started working!

I do not think your ISP has static ARP records set up for you. I vaguely recall that with secondary IP addresses, there has been some peculiar issue specifically with regards to ARP handling, and I suspected this to be at the core of your problem. As NAT creates its proxy ARP entries independently, I wanted to make sure that the secondary address does not stand in the way.

I will try to look up more info on this - if I find anything, I will let you know.

Best regards,
Peter

[AllertGen]

Hi, Peter Paluch.

So, if I undestood you right it creates a proxy ARP at every interface that has a "ip nat outside" line? So if there is a 2 NAT outside ISP interfaces it will answer to the ARP request from any of this ISP?

It looks not well from a securety point of view... It's a strange move from a Cisco. Or maybe there is a way to turn off the proxy ARP and use a secondary IP addresses. I'll search at the Cisco guides too for this matter.

Once again, thank you vey much for your help.

Best Regards.

[Jon Marshall]

I think how it works is more to do with the ISP end than your router.

So it can work one of two ways -

1) your ISP interface connecting to you has a primary IP and a secondary one.

If you add a static NAT rule from the same IP subnet as the secondary IP the ISP is using on their router interface your router will respond for the NAT IP because of proxy arp.

Key thing here is it doesn't actually matter whether you have a secondary IP configured or not on your router because that won't affect proxy arp.

2) your ISP is using just the primary IP and then has a route for the other IP subnet pointing to the outside interface IP of your router.

In this case again you do not need any secondary IPs but this time the ISP will not be sending any arp requests for the NAT IPs for the other subnet, they will instead be routed to the outside interface of your router.

So whichever the ISP is doing you shouldn't need to use any secondary IPs on your router.

The only time you would need to use a secondary IP on your router would be if there were devices between your router and the ISP router using IPs from that subnet and you needed to connect to them from your router but in my experience this is not that common a setup.

Edit - if the ISP is using secondary IPs you need proxy arp but if they are routing the other subnet to your router you can safely turn off proxy arp on the router interface.

Jon

[AllertGen]

Hi, Jon Marshall.

Thank for your reply. It's a very interesting information. Need to test :)

Well, now i want to ask my ISP directly :) But maybe it somehow routed to me because  my secondary IP addresses started to work (by command ping ##### source X.X.X.227) only when i copied MAC address from the old router to a new one. But still it can be just some type of ARP protection...

Best Regards.

[Jon Marshall]

Not sure what you did with the ping test ?

If the ISP is relying on proxy arp ie. they have a secondary IP and you moved the NAT IP to a new router then their arp cache has the wrong entry because the router will respond with the mac address of it's outside interface and until that entry times out the ISP will continue to send it to the wrong mac address.

Perhaps that is what you mean copying the mac address.

If the ISP is routing the traffic to the router's outside interface for that IP subnet then their arp cache again has the wrong entry because the new router has a different outside interface mac address but because you are overloading all inside clients to that IP for general internet access it should get updated automatically as soon as any internal clients access the internet.

Jon

[Peter Paluch]

Hi,

In addition to Jon's answers, this is my twopence:

if I undestood you right it creates a proxy ARP at every interface that has a "ip nat outside" line?

Not entirely. What it does is merely make sure that if someone ARPs for X.X.X.227 in particular, the router will respond. This so-called ARP alias should only be created for the X.X.X.227 address because it falls into the directly connected network X.X.X.224/29, and so it is clear that if we're NATting into X.X.X.227, the router absolutely needs to respond to ARP queries for that address.

See the following document for more information:

http://www.cisco.com/c/en/us/support/docs/ip/network-address-translation-nat/26704-nat-faq-00.html#q10

Best regards,
Peter

[AllertGen]

Hello, Peter Paluch.

Funny but after some time it stoped to work. I think it because my ISP cleared ARP cashe table. And because I didn't have secondary address at my outside interface anymore my ISP can't get ARP response from me.

BTW my secondary IP address and default gateway in the different subnets. But my primary IP address and default gateway in the same subnet. And all IP address uses the same outside interface. Looks like my router is not smart anought to answer to an ARP requests... So I'm puzzeled again how to make it works.

I'll try ask my ISP to make a static ARP table for my interface, but I prefer a solution from my side.

Best Regards

[Jon Marshall]

Hope Peter doesn't mind me answering.

Removing the secondary IP from our router makes no difference to arp requests from the ISP.

Can you make sure you have proxy arp enabled on the router's interface ?

The common solution to this is to ask your ISP to simply route the other IP subnet to the outside IP of your router and then you have no need for proxy arp and the ISP does not need to use a secondary IP.

Jon

[AllertGen]

Hi, Jon Marshall.

Yes, I have proxy arp enabled at the external interface. And i'm planning to ask my ISP to check routing (as a brief answer they told me that there is no static ARP records so ISP tryes to get ARP responses from me).

Also i want to try turn of a proxy ARP and place secondary IP addresses to the interface. Maybe it will work.

But what I can't undenstand is why it's work fine at the 3925 with usual IOS OS and doesn't works at the 4451-x with IOS-EX OS?...

Thank you for your help.

Best Regards,

[Jon Marshall]

Perhaps a change in behaviour between different versions of software.

As I said I would simply ask the ISP to route that IP subnet to you then there is no need for the ISP to arp for anything other than the public IP assigned to the physical interface on your router and they shouldn't really have to arp for that as your internal users accessing the internet will constantly be refreshing their arp cache.

Jon