Re: Cisco ASA 5525 - Connection Loops observed on Outside interface

ManadarDesai2895 · ‎05-22-2023

Dear Community,

We have HQ Site & 2 remote offices Site-A & Site-B. All 3 site are connected within MPLS managed cloud. We are seeing the Connection loops of the internal LAN subnets of Site-A & B on the outside interface of HQ firewall(Cisco 5525-X). We have verified the routing on BGP & found on issues. Internal LAN subnets are configured behind the L3 switch with SVIs for both Site - A & Site -B.

Site-A : 10.28.0.0/16

Site-B : 10.41.0.0/16

Find logs, pls help us to understand this behavior , if it would have any adverse effect on the performance of the perimeter firewall?

pri/Firewall/act# show conn all | in OUTSIDE.*OUTSIDE
UDP OUTSIDE 10.28.17.58:5060 OUTSIDE 10.28.12.92:0, idle 0:00:16, bytes 0, flags ti
UDP OUTSIDE 10.28.17.58:5060 OUTSIDE 10.28.12.92:0, idle 0:00:16, bytes 0, flags ti
UDP OUTSIDE 10.28.17.58:9363 OUTSIDE 10.28.12.92:0, idle 0:00:16, bytes 0, flags mi
UDP OUTSIDE 10.28.17.58:9362 OUTSIDE 10.28.12.92:0, idle 0:00:16, bytes 0, flags mi
UDP OUTSIDE 10.41.18.144:5060 OUTSIDE 10.41.12.12:0, idle 0:00:48, bytes 0, flags ti
UDP OUTSIDE 10.41.18.144:5060 OUTSIDE 10.41.12.12:0, idle 0:00:48, bytes 0, flags ti
UDP OUTSIDE 10.41.18.144:8093 OUTSIDE 10.41.12.12:0, idle 0:00:48, bytes 0, flags mi
UDP OUTSIDE 10.41.18.144:8092 OUTSIDE 10.41.12.12:0, idle 0:00:48, bytes 0, flags mi

Rob Ingram · ‎05-22-2023

@ManadarDesai2895 traffic is spoofed or more likely a routing issue.

We'd need more information to determine the cause, please provide the output of "show route" and also run packet-tracer from the CLI between 10.41.18.144 and 10.41.12.12.

ManadarDesai2895 · ‎05-22-2023

Hi Rob,

Find below output.

SH ROUTE:

sec/FIREWALL/act# sh route

Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2, V - VPN
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, + - replicated route
SI - Static InterVRF
Gateway of last resort is 10.82.111.13 to network 0.0.0.0

S* 0.0.0.0 0.0.0.0 [1/0] via 10.82.111.13, OUTSIDE
S 10.0.222.212 255.255.255.255 [1/0] via 10.82.111.8, OUTSIDE
S 10.0.222.213 255.255.255.255 [1/0] via 10.82.111.8, OUTSIDE
D EX 10.0.236.204 255.255.255.255
[170/136960] via 10.82.111.8, 1d06h, OUTSIDE
C 10.1.1.0 255.255.255.0 is directly connected, LANFAIL
L 10.1.1.2 255.255.255.255 is directly connected, LANFAIL
D EX 10.1.101.97 255.255.255.255
[170/136960] via 10.82.111.8, 1d06h, OUTSIDE
D EX 10.1.103.0 255.255.255.0 [170/136960] via 10.82.111.8, 1d06h, OUTSIDE
C 10.2.1.0 255.255.255.0 is directly connected, LINKFAIL
L 10.2.1.2 255.255.255.255 is directly connected, LINKFAIL
D EX 10.4.139.0 255.255.255.0 [170/136960] via 10.82.111.8, 1d06h, OUTSIDE
D EX 10.4.140.22 255.255.255.255
[170/136960] via 10.82.111.8, 1d06h, OUTSIDE
D EX 10.4.238.220 255.255.255.255
[170/136960] via 10.82.111.8, 1d06h, OUTSIDE
D EX 10.10.0.0 255.255.0.0 [170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.10.111.0 255.255.255.224
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.10.135.9 255.255.255.255
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.10.135.10 255.255.255.255
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.10.135.11 255.255.255.255
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.28.0.0 255.255.0.0 [170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.28.52.0 255.255.254.0 [170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.28.111.0 255.255.255.240
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.57.0.0 255.255.0.0 [170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.57.111.0 255.255.255.240
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.57.111.224 255.255.255.252
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.59.119.66 255.255.255.255
[170/136960] via 10.82.111.8, 1d06h, OUTSIDE
D EX 10.59.120.52 255.255.255.255
[170/136960] via 10.82.111.8, 1d06h, OUTSIDE
D EX 10.41.0.0 255.255.0.0 [170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.41.111.0 255.255.255.240
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D 10.82.0.0 255.255.0.0 is a summary, 2d19h, Null0
D 10.82.2.0 255.255.254.0 [90/3072] via 10.82.4.3, 2d19h, SBI
[90/3072] via 10.82.4.2, 2d19h, SBI
C 10.82.4.0 255.255.254.0 is directly connected, SBI
L 10.82.4.210 255.255.255.255 is directly connected, SBI
C 10.82.6.0 255.255.254.0 is directly connected, BKOFFICE
L 10.82.6.1 255.255.255.255 is directly connected, BKOFFICE
D 10.82.8.0 255.255.255.0 [90/3072] via 10.82.4.3, 2d19h, SBI
[90/3072] via 10.82.4.2, 2d19h, SBI
D 10.82.9.0 255.255.255.128 [90/3072] via 10.82.4.3, 2d19h, SBI
[90/3072] via 10.82.4.2, 2d19h, SBI
D 10.82.9.128 255.255.255.192 [90/3072] via 10.82.4.3, 2d19h, SBI
[90/3072] via 10.82.4.2, 2d19h, SBI
D 10.82.10.0 255.255.255.240 [90/3072] via 10.82.4.3, 2d19h, SBI
[90/3072] via 10.82.4.2, 2d19h, SBI
S 10.82.11.0 255.255.255.0 [1/0] via 10.82.4.1, SBI
D 10.82.12.0 255.255.255.128 [90/3072] via 10.82.4.2, 2d19h, SBI
D 10.82.12.128 255.255.255.128 [90/3072] via 10.82.4.2, 2d19h, SBI
S 10.82.16.0 255.255.254.0 [1/0] via 10.82.4.1, SBI
D EX 10.82.18.0 255.255.254.0
[170/2585856] via 10.82.111.13, 2d19h, OUTSIDE
S 10.82.20.0 255.255.255.0 [1/0] via 10.82.4.1, SBI
D 10.82.30.16 255.255.255.240 [90/3072] via 10.82.4.3, 2d19h, SBI
[90/3072] via 10.82.4.2, 2d19h, SBI
D 10.82.30.32 255.255.255.240 [90/3072] via 10.82.4.3, 2d19h, SBI
[90/3072] via 10.82.4.2, 2d19h, SBI
D 10.82.36.0 255.255.255.224 [90/3072] via 10.82.4.3, 2d19h, SBI
[90/3072] via 10.82.4.2, 2d19h, SBI
D 10.82.37.0 255.255.255.224 [90/3072] via 10.82.4.3, 2d19h, SBI
[90/3072] via 10.82.4.2, 2d19h, SBI
D 10.82.40.0 255.255.255.224 [90/3072] via 10.82.4.3, 2d19h, SBI
[90/3072] via 10.82.4.2, 2d19h, SBI
C 10.82.50.0 255.255.255.192 is directly connected, HCC
L 10.82.50.1 255.255.255.255 is directly connected, HCC
D 10.82.52.0 255.255.254.0 [90/3072] via 10.82.111.13, 2d19h, OUTSIDE
C 10.82.54.0 255.255.255.0 is directly connected, MPLS-DMZ
L 10.82.54.1 255.255.255.255 is directly connected, MPLS-DMZ
C 10.82.111.0 255.255.255.240 is directly connected, OUTSIDE
L 10.82.111.6 255.255.255.255 is directly connected, OUTSIDE
D 10.82.111.16 255.255.255.252
[90/3072] via 10.82.111.13, 2d19h, OUTSIDE
D 10.82.253.0 255.255.255.240 [90/3072] via 10.82.4.3, 2d19h, SBI
[90/3072] via 10.82.4.2, 2d19h, SBI
D EX 10.85.0.0 255.255.0.0 [170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.85.111.0 255.255.255.240
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.92.32.16 255.255.255.252
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.92.32.56 255.255.255.252
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.92.32.136 255.255.255.252
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.92.33.132 255.255.255.252
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.92.33.160 255.255.255.252
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.92.35.72 255.255.255.252
[170/239104] via 10.82.111.3, 2d19h, OUTSIDE
D EX 10.92.35.248 255.255.255.252
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.92.35.252 255.255.255.252
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.92.42.40 255.255.255.252
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.92.42.224 255.255.255.252
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.92.42.240 255.255.255.252
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.92.43.12 255.255.255.252
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.92.43.240 255.255.255.252
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.92.43.244 255.255.255.252
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.176.0.0 255.255.0.0
[170/136960] via 10.82.111.8, 00:10:41, OUTSIDE
D EX 10.177.128.57 255.255.255.255
[170/136960] via 10.82.111.8, 00:10:41, OUTSIDE
D EX 10.186.49.146 255.255.255.255
[170/136960] via 10.82.111.8, 01:24:06, OUTSIDE
D EX 10.187.52.146 255.255.255.255
[170/136960] via 10.82.111.8, 00:10:41, OUTSIDE
D EX 10.189.0.0 255.255.0.0
[170/136960] via 10.82.111.8, 01:24:06, OUTSIDE
D EX 10.191.0.0 255.255.0.0
[170/136960] via 10.82.111.8, 01:24:06, OUTSIDE
S 10.191.148.156 255.255.255.255 [1/0] via 10.82.111.8, OUTSIDE
S 10.191.148.157 255.255.255.255 [1/0] via 10.82.111.8, OUTSIDE
D EX 10.200.0.0 255.255.128.0 [170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.200.49.0 255.255.255.0
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.200.111.0 255.255.255.224
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.200.128.0 255.255.128.0
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.200.152.0 255.255.254.0
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.200.222.0 255.255.255.224
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.200.222.80 255.255.255.240
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.200.240.0 255.255.255.192
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.200.240.64 255.255.255.192
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
D EX 10.200.240.96 255.255.255.240
[170/208640] via 10.82.111.2, 2d19h, OUTSIDE
S 100.81.5.137 255.255.255.255 [1/0] via 10.82.111.7, OUTSIDE
S 100.81.5.185 255.255.255.255 [1/0] via 10.82.111.7, OUTSIDE
D EX 192.168.242.25 255.255.255.255
[170/136960] via 10.82.111.8, 01:24:06, OUTSIDE
D EX 192.168.243.31 255.255.255.255
[170/136960] via 10.82.111.8, 00:10:41, OUTSIDE
D EX 192.168.243.32 255.255.255.255
[170/136960] via 10.82.111.8, 00:10:41, OUTSIDE
D EX 192.168.247.110 255.255.255.255
[170/136960] via 10.82.111.8, 1d06h, OUTSIDE
D EX 192.168.247.122 255.255.255.255
[170/136960] via 10.82.111.8, 1d06h, OUTSIDE
D EX 192.168.253.0 255.255.255.0
[170/136960] via 10.82.111.8, 1d06h, OUTSIDE
S 198.148.79.58 255.255.255.255 [1/0] via 10.82.111.13, OUTSIDE

Packet Tracer Output:

sec/FIREWALL/act# packet-tracer input outside udp 10.41.18.144 5060 10.41.12.12 5060
Phase: 1
Type: INPUT-ROUTE-LOOKUP
Subtype: Resolve Egress Interface
Result: ALLOW
Config:
Additional Information:
Found next-hop 10.82.111.2 using egress ifc OUTSIDE

Phase: 2
Type: ECMP load balancing
Subtype:
Result: ALLOW
Config:
Additional Information:
ECMP load balancing
Found next-hop 10.82.111.2 using egress ifc OUTSIDE

Phase: 3
Type: ACCESS-LIST
Subtype: log
Result: DROP
Config:
access-group OUTSIDE in interface OUTSIDE
access-list OUTSIDE extended deny ip any4 any4
Additional Information:

Result:
input-interface: OUTSIDE
input-status: up
input-line-status: up
output-interface: OUTSIDE
output-status: up
output-line-status: up
Action: drop
Drop-reason: (acl-drop) Flow is denied by configured rule, Drop-location: frame 0x000055fe77d4d73c flow (NA)/NA

Rob Ingram · ‎05-22-2023

@ManadarDesai2895 output interface from packet-tracer is the OUTSIDE interface.

Result:
input-interface: OUTSIDE
input-status: up
input-line-status: up
output-interface: OUTSIDE
output-status: up
output-line-status: up
Action: drop
Drop-reason: (acl-drop) Flow is denied by configured rule, Drop-location: frame 0x000055fe77d4d73c flow (NA)/NA

This route is routing out the outside interface. - D EX 10.41.0.0 255.255.0.0 [170/208640] via 10.82.111.2, 2d19h, OUTSIDE

If this network (10.41.12.X ) is on the inside interface, add a static or resolve your routing issues.

ManadarDesai2895 · ‎05-22-2023

HI Rob,

Both 10.41.12.XX & 10.41.18.XX are belongs to remote SITE-A. These both would learn from the OUTSIDE interface only.

Rob Ingram · ‎05-22-2023

@ManadarDesai2895 so the upstream device routing is incorrect if that traffic is routed to the ASA.

ManadarDesai2895 · ‎05-22-2023

No Rob Have verified it on upstream router. Its forwarding towards PE router to the service provider end.

MHM Cisco World · ‎05-22-2023

Wait sec

Both for site A then sure asa point to outside interface'

Why the source of packet is from ASA?

MHM Cisco World · ‎05-22-2023

Make double check subnet

I Think asa is run as hub interconnect both site

I.e.

Site a to connect to site b the traffic must pass through asa.

ManadarDesai2895 · ‎05-22-2023

No , Site-A would directly communicate with Site-B & vice versa. HQ is not a HUB for Site-A & Site-B. Its a mesh infra. Just wanted to know if this behavior would have any impact on the performance of the ASA?

MHM Cisco World · ‎05-22-2023

sec/FIREWALL/act# packet-tracer input outside udp 10.41.18.144 5060 10.41.12.12 5060 <<- both for site-A so the packet tracer is OK no issue

UDP OUTSIDE 10.28.17.58:5060 OUTSIDE 10.28.12.92:0, idle 0:00:16, bytes 0, flags ti
UDP OUTSIDE 10.28.17.58:5060 OUTSIDE 10.28.12.92:0, idle 0:00:16, bytes 0, flags ti
UDP OUTSIDE 10.28.17.58:9363 OUTSIDE 10.28.12.92:0, idle 0:00:16, bytes 0, flags mi
UDP OUTSIDE 10.28.17.58:9362 OUTSIDE 10.28.12.92:0, idle 0:00:16, bytes 0, flags mi
ARE THE 10.28.17.x is for Site-B ??
UDP OUTSIDE 10.41.18.144:5060 OUTSIDE 10.41.12.12:0, idle 0:00:48, bytes 0, flags ti
UDP OUTSIDE 10.41.18.144:5060 OUTSIDE 10.41.12.12:0, idle 0:00:48, bytes 0, flags ti
UDP OUTSIDE 10.41.18.144:8093 OUTSIDE 10.41.12.12:0, idle 0:00:48, bytes 0, flags mi
UDP OUTSIDE 10.41.18.144:8092 OUTSIDE 10.41.12.12:0, idle 0:00:48, bytes 0, flags mi <<- this need to investigate. the site-A send packet to ASA and ASA reforward it to Site-A
why Site-A assume that this IP is not direct connect ?
can I see show route of Site-A

MHM Cisco World · ‎05-22-2023

you meaning the overlapping ?
If Yes
then the SVI use /16 and what you receive from the MPLS is must be /24 not /16

ManadarDesai2895 · ‎05-22-2023

Hi MHM,

SVIs are created with CIDR /23 or /24 & from BGP we have advertised /16 network.

MHM Cisco World · ‎05-22-2023

Asa always prefer longest match'

SVI is correct/23 or /24

The bgp use redistrubte try' disable autosummary.

Or make bgp advertise/24 not /16

ManadarDesai2895 · ‎05-22-2023

HI MHM,

IS there any impact of this doing to the production?