Solved: Phase8 NAT drops

WStoffel1 · ‎10-16-2013

I'm having trouble getting traffic from one sub interface to another. Logically I know what the issue is, I'm just having issues working through it based on this packet tracer output.

I sit behind the K_Inc interface and need to get to the FOS interface. There is no ACL to allow the traffic.

You'll have to pardon the santized config, i realize that's a pain, but hopefully it's enough to get me pointing in the right direction

There are routes to my core switch that send traffic for 10/10 to my core from the 192.168.10.0/24 network.

interface GigabitEthernet0/0

nameif outside

security-level 0

ip address 7.x.x.21 255.255.255.0

interface GigabitEthernet0/3.10

vlan 10

nameif K_Inc

security-level 100

ip address 192.168.10.254 255.255.255.0

interface GigabitEthernet0/3.177

vlan 177

nameif FOS

security-level 100

ip address 192.168.177.254 255.255.255.0

same-security-traffic permit inter-interface

same-security-traffic permit intra-interface

global (outside) 1 interface

nat (K_Inc) 1 10.0.0.0 255.0.0.0

nat (FOS) 1 192.168.177.0 255.255.255.0

static (FOS,outside) 7.x.x190 192.168.177.10 netmask 255.255.255.255

static (FOS,outside) 7.x.x.191 192.168.177.5 netmask 255.255.255.255

static (FOS,outside) 7.x.x.192 192.168.177.9 netmask 255.255.255.255

route outside 0.0.0.0 0.0.0.0 7.x.x.1 1

route K_Inc 10.0.0.0 255.192.0.0 192.168.10.252 1

packet-tracer input K_Inc tcp 10.10.80.49 1065 192.168.177.10 3389 det

yields:

Phase: 1

Type: ACCESS-LIST

Subtype:

Result: ALLOW

Config:

Implicit Rule

Additional Information:

Forward Flow based lookup yields rule:

in id=0xb88abc98, priority=1, domain=permit, deny=false

hits=253726471, user_data=0x0, cs_id=0x0, l3_type=0x8

src mac=0000.0000.0000, mask=0000.0000.0000

dst mac=0000.0000.0000, mask=0000.0000.0000

Phase: 2

Type: FLOW-LOOKUP

Subtype:

Result: ALLOW

Config:

Additional Information:

Found no matching flow, creating a new flow

Phase: 3

Type: ROUTE-LOOKUP

Subtype: input

Result: ALLOW

Config:

Additional Information:

in 192.168.177.0 255.255.255.0 FOS

Phase: 4

Type: ACCESS-LIST

Subtype:

Result: ALLOW

Config:

Implicit Rule

Additional Information:

Forward Flow based lookup yields rule:

in id=0xb88ac280, priority=2, domain=permit, deny=false

hits=667120, user_data=0x0, cs_id=0x0, flags=0x3000, protocol=0

src ip=0.0.0.0, mask=0.0.0.0, port=0

dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

Phase: 5

Type: IP-OPTIONS

Subtype:

Result: ALLOW

Config:

Additional Information:

Forward Flow based lookup yields rule:

in id=0xb88adb60, priority=0, domain=permit-ip-option, deny=true

hits=13897343, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0

src ip=0.0.0.0, mask=0.0.0.0, port=0

dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

Phase: 6

Type: FOVER

Subtype: standby-update

Result: ALLOW

Config:

Additional Information:

Forward Flow based lookup yields rule:

in id=0xb88ab3b8, priority=20, domain=lu, deny=false

hits=7400090, user_data=0x0, cs_id=0x0, flags=0x0, protocol=6

src ip=0.0.0.0, mask=0.0.0.0, port=0

dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

Phase: 7

Type: NAT

Subtype: host-limits

Result: ALLOW

Config:

static (K_Inc,dmz) 10.10.0.0 10.10.0.0 netmask 255.255.0.0

match ip K_Inc 10.10.0.0 255.255.0.0 dmz any

static translation to 10.10.0.0

translate_hits = 48365, untranslate_hits = 38813

Additional Information:

Forward Flow based lookup yields rule:

in id=0xacf39b88, priority=5, domain=host, deny=false

hits=8592687, user_data=0xacc23618, cs_id=0x0, reverse, flags=0x0, protocol=0

src ip=10.10.0.0, mask=255.255.0.0, port=0

dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

Phase: 8

Type: NAT

Subtype:

Result: DROP

Config:

nat (K_Inc) 1 10.0.0.0 255.0.0.0

match ip K_Inc 10.0.0.0 255.0.0.0 FOS any

dynamic translation to pool 1 (No matching global)

translate_hits = 44, untranslate_hits = 0

Additional Information:

Forward Flow based lookup yields rule:

in id=0xacf4c000, priority=1, domain=nat, deny=false

hits=43, user_data=0xacf4bf60, cs_id=0x0, flags=0x0, protocol=0

src ip=10.0.0.0, mask=255.0.0.0, port=0

dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

Result:

input-interface: K_Inc

input-status: up

input-line-status: up

output-interface: FOS

output-status: up

output-line-status: up

Action: drop

Drop-reason: (acl-drop) Flow is denied by configured rule

What I thought I needed only seems to break my access to the internet:

access-list FOS_In extended permit tcp 10.10.0.0 255.255.0.0 192.168.177.0 255.255.255.0

access-group FOS_In in interface K_Inc

static (FOS,K_Inc) 192.168.177.0 192.168.177.0 netmask 255.255.255.0

static (K_Inc,FOS) 10.10.0.0 10.10.0.0 netmask 255.255.0.0

Thanks in advance!

Jouni Forss · ‎10-16-2013

Hi,

But the original traffic wasnt blocked by ACL to begin with. It was dropped because of missing/incomplete NAT configuration.

If you removed the ACL with the "no access-group in interface K_Inc" command this should still work because of

same-security-traffic permit inter-interface

And because of the fact that the interfaces are both "security-level 100"

- Jouni

View solution in original post

Jouni Forss · ‎10-16-2013

Hi,

I imagine that the "FOS_In" ACL contains more rules than this single one? If it doesnt then it naturally doesnt allow the traffic to the Internet as the only allowed destination network is a private network.

By the way, why is the ACL for the interface "K_Inc" named "FOS_In" ?

I would have personally just tried to configure

static (FOS,K_Inc) 192.168.177.0 192.168.177.0 netmask 255.255.255.0

And tried the "packet-tracer" again.

Seems to me that the traffic originally matched the "nat" statement and had no matching "global" configuration for the destination interface and failed because of that.

So to my understanding the only thing needed is the above "static" and you can leave the out the "access-group" that you added.

- Jouni

WStoffel1 · ‎10-16-2013

First thing, I thought of the static NAT first, and it didn't work and I just tried it again, and I get the same result. It's the same Phase8 nat error.

But I had a very similar issue once before that you helped me with, i was hoping you'd be answering this one.

https://supportforums.cisco.com/message/3999166#3999166

Which is what made me think that traffic isn't allowed. So I did in fact just fix it, you're first comment above was the key, i did have only the one allow statement to the private network. Here's what fixed it:

access-list FOS_In extended permit tcp 10.10.0.0 255.255.0.0 192.168.177.0 255.255.255.0

access-list FOS_In extended permit ip 10.10.0.0 255.255.0.0 any

access-group FOS_In in interface K_Inc

static (FOS,K_Inc) 192.168.177.0 192.168.177.0 netmask 255.255.255.0

static (K_Inc,FOS) 10.10.0.0 10.10.0.0 netmask 255.255.0.0

It's only temporary while they migrate some data from one network to the other.

In answer to your question:

By the way, why is the ACL for the interface "K_Inc" named "FOS_In"?

I'm allowing traffic IN to the FOS network, isnt' that appropriate? Hahaha just kidding. It was just for me because I'll be ripping it out before the end of the day today anyway.

Thank you.

Jouni Forss · ‎10-16-2013

Hi,

Are you saying that the problems with the ACL was sorted but the problems with the "packet-tracer" and NAT still remains?

What seems wrong to me in the "packet-tracer" output is the fact that we should see an UN-NAT Phase at the very start if you had this configured while you took another "packet-tracer" output.

static (FOS,K_Inc) 192.168.177.0 192.168.177.0 netmask 255.255.255.0

It would also seem to me that the original traffic was allowed as there was no ACL configure and the "security-level" values permitted this traffic (same-security-traffic commands + equal security-level)

As I said before, the original output would just seem to point to a situation where you have a "nat" that matches the source address but there is no matching "global" for the destination IP address. To my understanding the "static" command should avoid that problem.

- Jouni

WStoffel1 · ‎10-16-2013

Oh man, now I'm confused.

I added this:

access-list FOS_In extended permit tcp 10.10.0.0 255.255.0.0 192.168.177.0 255.255.255.0

access-list FOS_In extended permit ip 10.10.0.0 255.255.0.0 any

access-group FOS_In in interface K_Inc

static (FOS,K_Inc) 192.168.177.0 192.168.177.0 netmask 255.255.255.0

static (K_Inc,FOS) 10.10.0.0 10.10.0.0 netmask 255.255.0.0

and now everything is working.

here's the packet tracer output which shows the packet's allowed, and i can in fact RDP to the server I needed to.

this also now has the UN NAT you were looking for.

Phase: 1

Type: ACCESS-LIST

Subtype:

Result: ALLOW

Config:

Implicit Rule

Additional Information:

MAC Access list

Phase: 2

Type: FLOW-LOOKUP

Subtype:

Result: ALLOW

Config:

Additional Information:

Found no matching flow, creating a new flow

Phase: 3

Type: UN-NAT

Subtype: static

Result: ALLOW

Config:

static (FOS,K_Inc) 192.168.177.0 192.168.177.0 netmask 255.255.255.0

match ip FOS 192.168.177.0 255.255.255.0 K_Inc any

static translation to 192.168.177.0

translate_hits = 0, untranslate_hits = 1857

Additional Information:

NAT divert to egress interface FOS

Untranslate 192.168.177.0/0 to 192.168.177.0/0 using netmask 255.255.255.0

Phase: 4

Type: ACCESS-LIST

Subtype: log

Result: ALLOW

Config:

access-group FOS_In in interface K_Inc

access-list FOS_In extended permit tcp 10.10.0.0 255.255.0.0 192.168.177.0 255.255.255.0

Additional Information:

Phase: 5

Type: IP-OPTIONS

Subtype:

Result: ALLOW

Config:

Additional Information:

Phase: 6

Type: FOVER

Subtype: standby-update

Result: ALLOW

Config:

Additional Information:

Phase: 7

Type: NAT

Subtype:

Result: ALLOW

Config:

static (K_Inc,FOS) 10.10.0.0 10.10.0.0 netmask 255.255.0.0

match ip K_Inc 10.10.0.0 255.255.0.0 FOS any

static translation to 10.10.0.0

translate_hits = 1856, untranslate_hits = 0

Additional Information:

Static translate 10.10.0.0/0 to 10.10.0.0/0 using netmask 255.255.0.0

Phase: 8

Type: NAT

Subtype: host-limits

Result: ALLOW

Config:

static (K_Inc,dmz) 10.10.0.0 10.10.0.0 netmask 255.255.0.0

match ip K_Inc 10.10.0.0 255.255.0.0 dmz any

static translation to 10.10.0.0

translate_hits = 51213, untranslate_hits = 40162

Additional Information:

Phase: 9

Type: QOS

Subtype:

Result: ALLOW

Config:

Additional Information:

Phase: 10

Type: NAT

Subtype: rpf-check

Result: ALLOW

Config:

static (FOS,K_Inc) 192.168.177.0 192.168.177.0 netmask 255.255.255.0

match ip FOS 192.168.177.0 255.255.255.0 K_Inc any

static translation to 192.168.177.0

translate_hits = 0, untranslate_hits = 1857

Additional Information:

Phase: 11

Type: QOS

Subtype:

Result: ALLOW

Config:

Additional Information:

Phase: 12

Type: NAT

Subtype: host-limits

Result: ALLOW

Config:

static (FOS,dmz) 192.168.177.0 192.168.177.0 netmask 255.255.255.0

match ip FOS 192.168.177.0 255.255.255.0 dmz any

static translation to 192.168.177.0

translate_hits = 0, untranslate_hits = 0

Additional Information:

Phase: 13

Type: IP-OPTIONS

Subtype:

Result: ALLOW

Config:

Additional Information:

Phase: 14

Type: FLOW-CREATION

Subtype:

Result: ALLOW

Config:

Additional Information:

New flow created with id 74528300, packet dispatched to next module

Phase: 15

Type: ROUTE-LOOKUP

Subtype: output and adjacency

Result: ALLOW

Config:

Additional Information:

found next-hop 192.168.177.10 using egress ifc FOS

adjacency Active

next-hop mac address 0025.9027.a0ae hits 244

Result:

input-interface: K_Inc

input-status: up

input-line-status: up

output-interface: FOS

output-status: up

output-line-status: up

Action: allow

As i said it is working, but I'm a little bit stumped as to why i had to create this acl to allow this traffic considering the same security traffic commands and sec levels.

Jouni Forss · ‎10-16-2013

Hi,

But the original traffic wasnt blocked by ACL to begin with. It was dropped because of missing/incomplete NAT configuration.

If you removed the ACL with the "no access-group in interface K_Inc" command this should still work because of

same-security-traffic permit inter-interface

And because of the fact that the interfaces are both "security-level 100"

- Jouni

WStoffel1 · ‎10-16-2013

Told you i was confused. It was just the NATs.

Thank you.

Removed the acl and it all continues to function.

So all i needed was:

static (FOS,K_Inc) 192.168.177.0 192.168.177.0 netmask 255.255.255.0

static (K_Inc,FOS) 10.10.0.0 10.10.0.0 netmask 255.255.0.0

You're the best. Thanks again.