topic Hi in Network Security

ASA NAT weirdness, connection reports wrong interface, return traffic not forwarded.

Garry Cross — Tue, 12 Mar 2019 07:45:24 GMT

Running 9.2(2)4

We have some static nats, they're object nats.

Just static nating ip's from inside to outside, and dmz to outside. It was working fine. Now it's not.

We have some identity nat's before the static nats and some dynamic nats after.

Here is a list of the interfaces, the outside is a fake address.

GigabitEthernet0/0       Outside                2.2.2.10 255.255.255.248 CONFIG
GigabitEthernet0/1       inside                 10.20.1.1       255.255.240.0   manual
GigabitEthernet0/2       dmz                    10.15.1.1       255.255.255.0   manual
GigabitEthernet0/3       Meraki                 10.40.9.1       255.255.255.0   CONFIG
GigabitEthernet0/3.8     WirelessGuest          10.40.8.1       255.255.255.0   CONFIG

Here is an example

object network MISHQ01CRM001
nat (inside,Outside) static 2.2.2.11 dns

object network MISHQ01CRM001
host 10.20.1.66

type telnet 2.2.2.11 443 on a windows machine on the Internet.

show conn | inc 10.20.1.66
TCP Meraki 70.27.173.10:52502 inside 10.20.1.66:443, idle 0:00:01, bytes 0, flags aB

Should say Outside not Meraki

Packet capture show the traffic and responses from server on inside.

Packet capture shows nothing on interface Outside

Packet capture on interface Meraki show the inbound packet from 70.27.173.10

show capture - the capture filters are picking up the 70.27.173.10 address and denying others.

The cap4 acl is denying the real networks on interface Meraki and accepting all else, other sources than below for the nat being discussed also show up in cap4.

ASA5525X(config)# show capture
capture cap1 type raw-data access-list cap1 interface inside headers-only [Capturing - 5750 bytes]
capture cap2 type raw-data access-list cap1 interface Meraki [Capturing - 2420 bytes]
capture cap4 type raw-data access-list cap4 interface Meraki [Capturing - 5940 bytes]

Sample from cap4

   1: 15:01:53.908430       70.27.173.10.52011 > 10.20.1.66.443: S 855525282:855525282(0) win 8192 <mss 1380,nop,wscale 2,nop,nop,sackOK>
   2: 15:01:56.926679       70.27.173.10.52011 > 10.20.1.66.443: S 855525282:855525282(0) win 8192 <mss 1380,nop,wscale 2,nop,nop,sackOK>
   3: 15:02:02.933606       70.27.173.10.52011 > 10.20.1.66.443: S 855525282:855525282(0) win 8192 <mss 1380,nop,nop,sackOK>

Sample from cap1

60: 15:01:53.908491 70.27.173.10.52011 > 10.20.1.66.443: S 2218087414:2218087414(0) win 8192 <mss 1380,nop,wscale 2,nop,nop,sackOK>
61: 15:01:53.908812 10.20.1.66.443 > 70.27.173.10.52011: S 2734516970:2734516970(0) ack 2218087415 win 8192 <mss 1460,nop,wscale 8,nop,nop,sackOK>
62: 15:01:56.908690 10.20.1.66.443 > 70.27.173.10.52011: S 2734516970:2734516970(0) ack 2218087415 win 8192 <mss 1460,nop,wscale 8,nop,nop,sackOK>
63: 15:01:56.926679 70.27.173.10.52011 > 10.20.1.66.443: S 2218087414:2218087414(0) win 8192 <mss 1380,nop,wscale 2,nop,nop,sackOK>
64: 15:02:02.909178 10.20.1.66.443 > 70.27.173.10.52011: S 2734516970:2734516970(0) ack 2218087415 win 65535 <mss 1460,nop,nop,sackOK>
65: 15:02:02.933621 70.27.173.10.52011 > 10.20.1.66.443: S 2218087414:2218087414(0) win 8192 <mss 1380,nop,nop,sackOK>
66: 15:02:14.909117 10.20.1.66.443 > 70.27.173.10.52011: R 2734516971:2734516971(0) win 0

There is no chance that Internet traffic is coming into the Meraki interface.

All other nat's dynamic and identity are working fine.

That is a very old version of

Philip D'Ath — Mon, 16 May 2016 20:02:01 GMT

That is a very old version of code. I would upgrade to a known gold star release like asa942-11-smp-k8.bin fist.

https://software.cisco.com/download/release.html?mdfid=284143129&softwareid=280775065&release=9.4.2%20Interim&relind=AVAILABLE&rellifecycle=&reltype=latest

Hi

Henrik Grankvist — Tue, 17 May 2016 20:37:17 GMT

It would be nice to see the output from this command:

packet-tracer input inside tcp 10.20.1.10 4564 2.2.2.11 443

You will be able to see what NAT statement you are actually hitting.

If you don't realise what the problem is after you have run the packet-tracer command could you also post the output of "show nat".

And as an advice, use manual NAT statements when doing static NAT so they dont get tangled up with other dynamic NAT statements that also are using object NAT.

That will hit...

Garry Cross — Tue, 17 May 2016 21:44:07 GMT

That will hit...

object network MeadowPineLan
nat (inside,Outside) dynamic interface

Dynamic translate 10.20.1.10/4564 to 2.2.2.10/4564

PS The problem has disappeared. Everything is working again.

Hello Garry, are you doing

Sergio Ceron Ramirez — Wed, 18 May 2016 19:41:43 GMT

Hello Garry, are you doing the Telnet test from the inside network? I don't think it will give you the output you are expecting...

Could you bring up the output of 'show route' and run the following:

packet-tracer input outside tcp 8.8.8.8 1025 2.2.2.11 443 detailed

The packet-tracer above is only if you are allowing traffic from any source. If you are delimiting traffic from specific sources on the outside ACL, please change the source.

Yes from outside.

Garry Cross — Wed, 18 May 2016 19:58:35 GMT

Yes from outside.

When I did the packet-tracer using the port numbers of the telnet I was doing at the same time it said the three-way handshake was incomplete.

When I did it with random source port numbers dest 443 it allowed it and translated it to 10.20.1.66 which we knew already by the packet capture and show conn. I expect somehow the state table was messed up and the return packet did not have a state on interface Outside and so the packet was dropped, since it was a syn/ack and so there was no established session on Outside but rather on Meraki.

Phase: 2
Type: UN-NAT
Subtype: static
Result: ALLOW
Config:
object network MISHQ01CRM001
nat (inside,Outside) static 2.2.2.11 dns
Additional Information:
NAT divert to egress interface inside
Untranslate 2.2.2.11/443 to 10.20.1.66/443

Not sure what benefit this is but here you go. Keep in mind the failure is gone but default was correct at the time of the failure.

S*    0.0.0.0 0.0.0.0 [1/0] via 2.2.2.9, Outside
C        10.15.1.0 255.255.255.0 is directly connected, dmz
L        10.15.1.1 255.255.255.255 is directly connected, dmz
C        10.20.0.0 255.255.240.0 is directly connected, inside
L        10.20.1.1 255.255.255.255 is directly connected, inside
O        10.20.20.0 255.255.252.0 [110/11] via 10.40.9.253, 5d05h, Meraki
S        10.30.1.0 255.255.255.0 [1/0] via 10.20.1.1, inside
S        10.30.1.97 255.255.255.255 [1/0] via 216.183.93.9, Outside
S        10.30.1.99 255.255.255.255 [1/0] via 216.183.93.9, Outside
S        10.30.1.100 255.255.255.255 [1/0] via 216.183.93.9, Outside
S        10.30.1.101 255.255.255.255 [1/0] via 216.183.93.9, Outside
S        10.30.1.102 255.255.255.255 [1/0] via 216.183.93.9, Outside
O        10.40.0.0 255.255.252.0 [110/11] via 10.40.9.254, 5d05h, Meraki
O        10.40.4.0 255.255.255.0 [110/11] via 10.40.9.254, 5d05h, Meraki
O        10.40.5.0 255.255.255.0 [110/11] via 10.40.9.254, 5d05h, Meraki
C        10.40.8.0 255.255.255.0 is directly connected, WirelessGuest
L        10.40.8.1 255.255.255.255 is directly connected, WirelessGuest
C        10.40.9.0 255.255.255.0 is directly connected, Meraki
L        10.40.9.1 255.255.255.255 is directly connected, Meraki
C        192.168.255.252 255.255.255.252 is directly connected, FO_Link
L        192.168.255.253 255.255.255.255 is directly connected, FO_Link
C        2.2.2.8 255.255.255.248 is directly connected, Outside
L        2.2.2.10 255.255.255.255 is directly connected, Outside

Going back to your initial

Sergio Ceron Ramirez — Wed, 18 May 2016 20:34:07 GMT

Going back to your initial post, what source ports did you use on the packet-tracer? I see that connection table shows port 52502 and the capture is showing port 52011, meaning they are two different connections.

Is there any IP SLA configured there?

No ip SLA configured.

Garry Cross — Thu, 19 May 2016 15:00:51 GMT

No ip SLA configured.

Port numbers were different from multiple examples of attempts to connect. Copied different outputs to show the results. Always the same except for source port.

Thanks for your interest.