ASR1000 - Cisco IOS XE - %NAT-6-ADDR_ALLOC_FAILURE: Address allocation failed

[kasunrajapakse]

Hi Guys, 

I have a ASR1000 running Cisco IOS XE Software, Version 16.12.04.
I get the following on my logs. 
But I know the NAT pool has plenty of capacity left. 

--
Feb 4 07:47:53.913 GMT: %IOSXE-6-PLATFORM: R0/0: cpp_cp: QFP:0.0 Thread:036 TS:00001691789441877406 %NAT-6-ADDR_ALLOC_FAILURE: Address allocation failed; pool 3 may be exhausted [2]
Feb 4 07:47:58.941 GMT: %IOSXE-6-PLATFORM: R0/0: cpp_cp: QFP:0.0 Thread:033 TS:00001691794469654984 %NAT-6-ADDR_ALLOC_FAILURE: Address allocation failed; pool 3 may be exhausted [2]
Feb 4 07:48:05.933 GMT: %IOSXE-6-PLATFORM: R0/0: cpp_cp: QFP:0.0 Thread:055 TS:00001691801461317926 %NAT-6-ADDR_ALLOC_FAILURE: Address allocation failed; pool 3 may be exhausted [2]
Feb 4 08:03:04.502 GMT: %IOSXE-6-PLATFORM: R0/0: cpp_cp: QFP:0.0 Thread:078 TS:00001692700019933256 %NAT-6-ADDR_ALLOC_FAILURE: Address allocation failed; pool 3 may be exhausted [2]
Feb 4 08:03:09.898 GMT: %IOSXE-6-PLATFORM: R0/0: cpp_cp: QFP:0.0 Thread:023 TS:00001692705416760704 %NAT-6-ADDR_ALLOC_FAILURE: Address allocation failed; pool 3 may be exhausted [2]
Feb 4 08:03:14.995 GMT: %IOSXE-6-PLATFORM: R0/0: cpp_cp: QFP:0.0 Thread:007 TS:00001692710513092322 %NAT-6-ADDR_ALLOC_FAILURE: Address allocation failed; pool 3 may be exhausted [2]
Feb 4 08:03:21.939 GMT: %IOSXE-6-PLATFORM: R0/0: cpp_cp: QFP:0.0 Thread:077 TS:00001692717457580306 %NAT-6-ADDR_ALLOC_FAILURE: Address allocation failed; pool 3 may be exhausted [2]
Feb 4 08:18:17.515 GMT: %IOSXE-6-PLATFORM: R0/0: cpp_cp: QFP:0.0 Thread:098 TS:00001693613023495736 %NAT-6-ADDR_ALLOC_FAILURE: Address allocation failed; pool 3 may be exhausted [2]
Feb 4 08:18:22.928 GMT: %IOSXE-6-PLATFORM: R0/0: cpp_cp: QFP:0.0 Thread:111 TS:00001693618435755122 %NAT-6-ADDR_ALLOC_FAILURE: Address allocation failed; pool 3 may be exhausted [2]
Feb 4 08:18:29.028 GMT: %IOSXE-6-PLATFORM: R0/0: cpp_cp: QFP:0.0 Thread:039 TS:00001693624535922760 %NAT-6-ADDR_ALLOC_FAILURE: Address allocation failed; pool 3 may be exhausted [2]
Feb 4 08:18:35.921 GMT: %IOSXE-6-PLATFORM: R0/0: cpp_cp: QFP:0.0 Thread:083 TS:00001693631429068824 %NAT-6-ADDR_ALLOC_FAILURE: Address allocation failed; pool 3 may be exhausted [2]
Feb 4 08:33:29.507 GMT: %IOSXE-6-PLATFORM: R0/0: cpp_cp: QFP:0.0 Thread:034 TS:00001694525005423058 %NAT-6-ADDR_ALLOC_FAILURE: Address allocation failed; pool 3 may be exhausted [2]
Feb 4 08:33:34.508 GMT: %IOSXE-6-PLATFORM: R0/0: cpp_cp: QFP:0.0 Thread:110 TS:00001694530006544240 %NAT-6-ADDR_ALLOC_FAILURE: Address allocation failed; pool 3 may be exhausted [2]
Feb 4 08:33:41.439 GMT: %IOSXE-6-PLATFORM: R0/0: cpp_cp: QFP:0.0 Thread:099 TS:00001694536936983678 %NAT-6-ADDR_ALLOC_FAILURE: Address allocation failed; pool 3 may be exhausted [2]
Feb 4 08:33:47.507 GMT: %IOSXE-6-PLATFORM: R0/0: cpp_cp: QFP:0.0 Thread:087 TS:00001694543004627620 %NAT-6-ADDR_ALLOC_FAILURE: Address allocation failed; pool 3 may be exhausted [2]

-------------------

ASR1000#show ip nat translations total
Total number of translations: 101507

------------

ASR1000#show ip nat pool name mobile_data_out

NAT Pool Statistics

Pool name mobile_data_out, id 3
Assigned Available
Addresses 4 0
UDP Low Ports 169 1879
TCP Low Ports 0 2048
UDP High Ports 6563 251485
TCP High Ports 93350 164698

(Low ports are less than 1024. High ports are greater than or equal to 1024.)

ASR1000#

------------

ip nat translation timeout 600
ip nat translation tcp-timeout 600
ip nat translation udp-timeout 120
ip nat translation icmp-timeout 120
ip nat translation max-entries 2000000

I have the above configured as well. 
Any help will be highly appreciated. 

[MHM Cisco World]

I think it is Bug

[marce1000 · Accepted solution]

                FYI : https://bst.cloudapps.cisco.com/bugsearch/bug/CSCut24320

 M.

[kasunrajapakse · Accepted solution]

Hi Team, 

The issue was that non PATtable traffic was utilizing my NAT pool. Cisco TAC suggested to do the following changes and the issue resolved. 

sh ip nat translations filter entry-type half all                <<<<<< This will show if the NAT pool is being  overutilized by NONPATtable traffic. 

ip nat settings nonpatdrop
clear ip nat translation *