Nat destination with rotary pool not work in Cisco ISR 4331

[madmongoose]

Hello!

We have finally replaced the old Cisco 2851 on the more recent Cisco 4331. The config from the old 2851 was successful moved to 4331 except for one moment. Rotary nat was used to load balance external connections to internal mail servers on 2851, but it didn't work on the new 4331.

Ambiguous command: "ip nat inside destination list 100 pool pool-mail"

This is nat related config from Cisco 2851:
ip nat pool pool-mail 10.10.10.11 10.10.10.12 netmask 255.255.255.0 type rotary
ip nat inside source list acl-nat interface GigabitEthernet0/0 overload
ip nat inside destination list 100 pool-mail
access-list 100 permit tcp any host 100.100.100.100 eq www
access-list 100 permit tcp any host 100.100.100.100 eq 443
access-list 100 permit tcp any host 100.100.100.100 eq smtp

This from Cisco 4331:
ip nat pool pool-mail 10.10.10.11 10.10.10.12 netmask 255.255.255.0 type rotary
ip nat inside source list acl-nat interface GigabitEthernet0/0/0 overload
ip access-list extended 100
  permit tcp any host 100.100.100.100 eq www
  permit tcp any host 100.100.100.100 eq 443
  permit tcp any host 100.100.100.100 eq smtp
When I try setup nat destination, I see this "Ambiguous command: "ip nat inside destination list 100 pool pool-mail"

I read docs

But example did't work:

ip nat pool real-hosts 192.168.15.2 192.168.15.15 prefix-length 28 type rotary
access-list 2 permit 192.168.15.1
ip nat inside destination list 2 pool real-hosts
interface gigabitethernet 0/0/0
ip address 192.168.15.129 255.255.255.240
ip nat inside
interface serial 0
ip address 192.168.15.17 255.255.255.240
ip nat outside

rt-01(config)#$s 192.168.15.2 192.168.15.15 prefix-length 28 type rotary
rt-01(config)#access-list 2 permit 192.168.15.1
rt-01(config)#ip nat inside destination list 2 pool real-hosts
% Ambiguous command: "ip nat inside destination list 2 pool real-hosts"

I have Cisco ISR 4331 HSECK9 Version 16.9.7 Fuji

Community, please help.

[kubn2]

Hi, 

Based on the output that you've pasted you are using the command:

ip nat inside destination list 2 pool real-hosts

However in the documentation the command looks like this:

ip nat inside destination-list 2 pool real-hosts

So try with "destination-list" instead of "destination list"

[madmongoose]

Thank you for answer. This is example from Cisco Configuring NAT for isr 4300 Fuji firmware. My config above. But I try this...

Enter configuration commands, one per line. End with CNTL/Z.
rt-01(config)# ip nat pool real-hosts 192.168.15.2 192.168.15.15 prefix-length 28 type rotary
rt-01(config)#access-list 2 permit 192.168.15.1
rt-01(config)#ip nat inside destination-list 2 pool real-hosts

                                                               ^
% Invalid input detected at '^' marker.

rt-01(config)#ip nat inside destination list 2 ?
redundancy NAT redundancy operation
<cr> <cr>

[balaji.bandi]

try this post :

https://community.cisco.com/t5/routing/forward-range-ports-for-few-hosts-in-isr4331/td-p/3316899

still an issue please post-show version 

[madmongoose]

Thank you for answer!

Yes, named ACL passed.

Is the work config:

ip nat pool ais-pool-mail 10.131.1.11 10.131.1.12 netmask 255.255.255.0 type rotary

ip nat inside source list ais-nat interface GigabitEthernet0/0/0 overload
ip nat inside destination list ais-acl-mail pool ais-pool-mail

ip access-list extended ais-acl-mail
  permit tcp any any eq www

  permit tcp any any eq 443

  permit tcp any any eq smtp  

  permit tcp any any eq 465

And then an amazing poltergeist begins. The telnet test passes from external devices on Windows, http page opens, but it does not work from macos, iphone, linux devices.

From linux and macOS: 

telnet x.x.x.x 465

telnet: Unable to connect to remote host: Connection refused

telnet: can't connect to remote host (x.x.x.x Connection refused

From Windows:

220 mail-01.xxx.ru Microsoft ESMTP MAIL Service ready at Mon, 5 Apr 2021 12:28:10 +0300

quit

220 mail-02.xxx.ru Microsoft ESMTP MAIL Service ready at Mon, 5 Apr 2021 12:28:52 +0300

Load balancing is worked, but not for all.

rt-01#sh ver
Cisco IOS XE Software, Version 16.09.07
Cisco IOS Software [Fuji], ISR Software (X86_64_LINUX_IOSD-UNIVERSALK9-M), Version 16.9.7, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2021 by Cisco Systems, Inc.
Compiled Wed 10-Feb-21 09:23 by mcpre

ROM: IOS-XE ROMMON

ais-rt-01 uptime is 16 hours, 45 minutes
Uptime for this control processor is 16 hours, 47 minutes
System returned to ROM by Reload Command at 19:43:53 MSK Sun Apr 4 2021
System restarted at 19:48:39 MSK Sun Apr 4 2021
System image file is "bootflash:isr4300-universalk9.16.09.07.SPA.bin"
Last reload reason: Reload Command

Suite License Information for Module:'esg'

--------------------------------------------------------------------------------
Suite Suite Current Type Suite Next reboot
--------------------------------------------------------------------------------
FoundationSuiteK9 None None None
securityk9
appxk9

AdvUCSuiteK9 None None None
uck9
cme-srst
cube

Technology Package License Information:

-----------------------------------------------------------------
Technology Technology-package Technology-package
Current Type Next reboot
------------------------------------------------------------------
appxk9 appxk9 RightToUse appxk9
uck9 None None None
securityk9 securityk9 RightToUse securityk9
ipbase ipbasek9 Permanent ipbasek9

The current throughput level is 100000 kbps

Smart Licensing Status: Smart Licensing is DISABLED

cisco ISR4331/K9 (1RU) processor with 1784185K/6147K bytes of memory.
Processor board ID FDO2219A08H
3 Gigabit Ethernet interfaces
32768K bytes of non-volatile configuration memory.
4194304K bytes of physical memory.
3223551K bytes of flash memory at bootflash:.
0K bytes of WebUI ODM Files at webui:.

Configuration register is 0x102

Firmware upgrade to last version. It all looks like a bug.

[madmongoose]

Thank you for answer. I've written so much, but I don't see it here. I'll start again.

Named ACL is accept!

ip nat pool ais-pool-mail 10.131.1.11 10.131.1.12 netmask 255.255.255.0 type rotary
ip nat inside source list ais-nat interface GigabitEthernet0/0/0 overload
ip nat inside destination list ais-acl-mail pool ais-pool-mail

ip access-list extended ais-acl-mail
  permit tcp any host x.x.x.x eq www
  permit tcp any host x.x.x.x eq 443
  permit tcp any host x.x.x.x eq smtp
  permit tcp any host x.x.x.x eq 465

x.x.x.x - external ip

Now my config looks like this. And it worked! But not for everyone. This is some incredible poltergest. And it looks like a bug. Load balancing only works for external Windows clients. I can open the http page, connect telnet on port 465. But when I try to do the same with macos, iphone or linux the connection is refused.

--------------------------------

On Windows clients:

220 xxx-01.xxx Microsoft ESMTP MAIL Service ready at Mon, 5 Apr 2021 12:28:10 +0300

220 xxx-02.xxx Microsoft ESMTP MAIL Service ready at Mon, 5 Apr 2021 12:28:52 +0300

I try connect from other City - worked!

On *nix clients:

telnet x.x.x.x 465

telnet: can't connect to remote host (x.x.x.x Connection refused

telnet x.x.x.x 465
Trying x.x.x.x...
telnet: Unable to connect to remote host: Connection refused

--------------------------------

I have no idea how to diagnose it... 

#sh ver
Cisco IOS XE Software, Version 16.09.07
Cisco IOS Software [Fuji], ISR Software (X86_64_LINUX_IOSD-UNIVERSALK9-M), Version 16.9.7, RELEASE SOFTWARE (fc1)
ROM: IOS-XE ROMMON

ais-rt-01 uptime is 18 hours, 13 minutes
Uptime for this control processor is 18 hours, 15 minutes
System returned to ROM by Reload Command at 19:43:53 MSK Sun Apr 4 2021
System restarted at 19:48:39 MSK Sun Apr 4 2021
System image file is "bootflash:isr4300-universalk9.16.09.07.SPA.bin"
Last reload reason: Reload Command

Suite License Information for Module:'esg'

--------------------------------------------------------------------------------
Suite Suite Current Type Suite Next reboot
--------------------------------------------------------------------------------
FoundationSuiteK9 None None None
securityk9
appxk9

AdvUCSuiteK9 None None None
uck9
cme-srst
cube

Technology Package License Information:

-----------------------------------------------------------------
Technology Technology-package Technology-package
Current Type Next reboot
------------------------------------------------------------------
appxk9 appxk9 RightToUse appxk9
uck9 None None None
securityk9 securityk9 RightToUse securityk9
ipbase ipbasek9 Permanent ipbasek9

The current throughput level is 100000 kbps

Smart Licensing Status: Smart Licensing is DISABLED

cisco ISR4331/K9 (1RU) processor with 1784185K/6147K bytes of memory.
Processor board ID FDO2219A08H
3 Gigabit Ethernet interfaces
32768K bytes of non-volatile configuration memory.
4194304K bytes of physical memory.
3223551K bytes of flash memory at bootflash:.
0K bytes of WebUI ODM Files at webui:.

Configuration register is 0x102

Thanks for your help and your time.

[paul driver]

Hello
It seems you are not running the correct software or license to support the load balancing, have you tried upgrading the router?
Also regards your configuration  you are currently including the broadcast address for the subnet in the nat pool..

192.168.15.2 192.168.15.15

it should be

192.168.15.2 192.168.15.14

[balaji.bandi]

@paul - Good catch..

[madmongoose]

Please note this is taken from the official Cisco documentation. And there are claims to their writer. But no matter, my config is a different, I wrote about it above. In my config with addresses and mask everything is in order.

https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/ipaddr_nat/configuration/xe-16-8/nat-xe-16-8-book/iadnat-addr-consv.html?bookSearch=true

ip nat pool ais-pool-mail 10.10.10.11 10.10.10.12 netmask 255.255.255.0 type rotary
ip nat inside source list ais-nat interface GigabitEthernet0/0/0 overload
ip nat inside destination list ais-acl-mail pool ais-pool-mail
ip access-list extended ais-acl-mail
  permit tcp any host x.x.x.x eq www
  permit tcp any host x.x.x.x eq 443
  permit tcp any host x.x.x.x eq smtp
  permit tcp any host x.x.x.x eq 465

[paul driver]

Hello

---

@madmongoose wrote:

rt-01(config)#ip nat inside destination list 2 ?
redundancy NAT redundancy operation

---

Does your rtr except the pool the above suggests otherwise?

Also the access-list relatng to the public ip address for the serverpool, Is this separate from your public wan interface ip address address?

[madmongoose]

Thank you for answer.

I didn't quite understand the question.

Yes, pool only for two mail servers. ACL for mail different from nat acl. 

You are asking questions following an example taken from the documentation, which, as we found out, is not written correctly. And it no longer makes sense to disassemble it.

I would be grateful if you can figure out the main config. As I wrote above, balancing worked for Windows clients, but does not work for iphone, mac, linux. This is my actual config.

ip nat pool ais-pool-mail 10.131.1.11 10.131.1.12 netmask 255.255.255.0 type rotary
ip nat inside source list ais-nat interface GigabitEthernet0/0/0 overload
ip nat inside destination list ais-acl-mail pool ais-pool-mail

ip access-list extended ais-acl-mail
  permit tcp any host x.x.x.x eq www
  permit tcp any host x.x.x.x eq 443
  permit tcp any host x.x.x.x eq smtp
  permit tcp any host x.x.x.x eq 465

x.x.x.x - external ip

Now my config looks like this. And it worked! But not for everyone. This is some incredible poltergest. And it looks like a bug. Load balancing only works for external Windows clients. I can open the http page, connect telnet on port 465. But when I try to do the same with macos, iphone or linux the connection is refused.

--------------------------------

On Windows clients:

220 xxx-01.xxx Microsoft ESMTP MAIL Service ready at Mon, 5 Apr 2021 12:28:10 +0300

220 xxx-02.xxx Microsoft ESMTP MAIL Service ready at Mon, 5 Apr 2021 12:28:52 +0300

I try connect from other City - worked!

On *nix clients:

telnet x.x.x.x 465

telnet: can't connect to remote host (x.x.x.x Connection refused

telnet x.x.x.x 465
Trying x.x.x.x...
telnet: Unable to connect to remote host: Connection refused

--------------------------------

I have no idea how to diagnose it... 

#sh ver
Cisco IOS XE Software, Version 16.09.07
Cisco IOS Software [Fuji], ISR Software (X86_64_LINUX_IOSD-UNIVERSALK9-M), Version 16.9.7, RELEASE SOFTWARE (fc1)
ROM: IOS-XE ROMMON

[paul driver]

Hello

---

@madmongoose wrote:

ip nat pool ais-pool-mail 10.131.1.11 10.131.1.12 netmask 255.255.255.0 type rotary
ip nat inside source list ais-nat interface GigabitEthernet0/0/0 overload
ip nat inside destination list ais-acl-mail pool ais-pool-mail

ip access-list extended ais-acl-mail
permit tcp any host x.x.x.x eq www
permit tcp any host x.x.x.x eq 443
permit tcp any host x.x.x.x eq smtp
permit tcp any host x.x.x.x eq 465

---

Okay I think we got lost somewhere but now i believe we are on the same lines.

Now regards your nat configuration, I see two acls for nat, one performing PAT for the whole lan and one for NAT load balancing.

As a test can you deny the hosts that are being stated in acl ais-nat-mail from acl ais-nat making sure the deny ace's are above the permit aces statement

Example:
ip access-list extended ais-nat
1 deny tcp host x.x.x.x any
2 deny tcp host x.x.x.y any
etc

[madmongoose]

Good day!

My config with the ais-nat  acl look like this:

ip nat pool ais-pool-mail 10.131.1.11 10.131.1.12 prefix-length 29 type rotary
ip nat inside source static tcp 10.131.1.40 x.x.x.x extendable
ip nat inside source static tcp 10.131.1.40 x.x.x.x extendable
ip nat inside source static udp 10.131.1.40 x.x.x.x extendable
ip nat inside source list ais-nat interface GigabitEthernet0/0/0 overload

ip nat inside destination list ais-acl-mail pool ais-pool-mail
ip route 0.0.0.0 0.0.0.0 x.x.x.x
ip route 10.131.1.0 255.255.255.0 GigabitEthernet0/0/1

ip access-list standard ais-nat
10 permit 10.131.1.0 0.0.0.255
20 permit 10.131.3.0 0.0.0.255
30 permit 192.168.7.0 0.0.0.255
40 permit 10.131.10.0 0.0.0.255

ip access-list extended ais-acl-mail
10 permit tcp any host x.x.x.x eq www
20 permit tcp any host x.x.x.x eq 443
30 permit tcp any host x.x.x.x eq smtp
40 permit tcp any host x.x.x.x eq 465
50 permit tcp any host x.x.x.x eq 587

I didn't understand a bit why I need to make a deny rule?

Don't you find it interesting that Windows clients work in this configuration, but Linux, MacOS, iPhone, Android does not?

By the way, only telnet 587 passes for Unix.

Yesterday I updated the firmware to the latest possible Bengaluru 17.04.01b. I deleted all the config associated with NAT, made the settings again, but nothing changed.

Thank you for help!

[paul driver]

Hello
It does seem strange ,Are those other devcies accessing the internal server the way as the window machines?

As for the ammendment, What i mean is deny the hosts that are being stated in the ais-acl-mail acl in the ais-nat acl:
ip access-list standard ais-nat
4 deny host x.x.x.w
5 deny host x.x.x.x
6 deny host x.x.x.y
7 deny host x.x.x.z
10 permit 10.131.1.0 0.0.0.255
20 permit 10.131.3.0 0.0.0.255
30 permit 192.168.7.0 0.0.0.255
40 permit 10.131.10.0 0.0.0.255

[madmongoose · Accepted solution]

Good day!

Unfortunately it didn't help...(

But I solved the problem in a completely random way and it worked!

I added the ip address 10.131.2.10, which is not in my subnet and made a forwarding to it from external ip.

This is worked config:

x- external ips, w - another PAT

ip nat pool ais-pool-mail 10.131.1.11 10.131.1.12 prefix-length 29 type rotary
ip nat inside source static tcp 10.131.2.10 25 x.x.x.x 25 extendable
ip nat inside source static tcp 10.131.2.10 80 x.x.x.x 80 extendable
ip nat inside source static tcp 10.131.2.10 443 x.x.x.x 443 extendable
ip nat inside source static tcp 10.131.2.10 465 x.x.x.x 465 extendable
ip nat inside source static tcp 10.131.2.10 587 x.x.x.x 587 extendable
wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww

wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww

wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww

ip nat inside source list ais-nat interface GigabitEthernet0/0/0 overload
ip nat inside destination list ais-acl-mail pool ais-pool-mail
ip route 0.0.0.0 0.0.0.0 x.x.x.x
ip route 10.131.1.0 255.255.255.0 GigabitEthernet0/0/1
ip ssh version 2
!
!
ip access-list standard ais-nat
10 permit 10.131.1.0 0.0.0.255
!
ip access-list extended ais-acl-mail
10 permit tcp any host x.x.x.x eq www
20 permit tcp any host x.x.x.x eq 443
30 permit tcp any host x.x.x.x eq smtp
40 permit tcp any host x.x.x.x eq 465
50 permit tcp any host x.x.x.x eq 587

Thanks everyone for the help!