Redundant WAN Design – Asymmetric Routing Issue

[duahimanshu816]

Hello Expert,

I'm working on a somewhat complex network design and could use your expertise. I have two sites, each with a similar setup: a firewall and a Layer 3 switch.

At Site 1, the firewall is deployed in a "router-on-a-stick" configuration with two interfaces – Port 1 and Port 2. Port 2 is a trunk port with multiple subinterfaces for VLANs 10, 20, and 30. This trunk is connected to the Layer 3 switch, where static VLAN interfaces (SVIs) are configured for VLANs 10, 20, and 30. VLAN 30 is used as the WAN link, and an IPsec tunnel is established between the two sites over this link (firewall to firewall). The same architecture is mirrored at Site 2. This setup works well, and the IPsec tunnel is functioning as expected.

Now, we have added a secondary WAN link at both sites to provide redundancy. This second link is terminated directly on the Layer 3 switch/router at each site, with public IPs assigned. Over this new link, we’ve successfully established GRE over IPsec and OSPF peering directly between the Layer 3 switches. All of this configuration (GRE/IPsec/OSPF) is done on the Layer 3 switch.

Additionally, we enabled OSPF on the firewalls. On each firewall, the VLAN 10 interface (e.g., 192.168.10.1/24) faces the Layer 3 switch (e.g., 192.168.10.100/24), and OSPF neighbor relationships are correctly formed between them.

Here’s where the issue arises:

When I ping from the Layer 3 switch at Site 1 to the Layer 3 switch interface at Site 2 (e.g., from 192.168.10.100 to 192.168.30.100), the ping succeeds.
However, when I try to ping from Site 1 to the firewall interface at Site 2 (e.g., 192.168.30.1 – the firewall’s VLAN 10 IP), the packet reaches the firewall but the reply doesn’t come back.

After troubleshooting, it seems the firewall at Site 2 does not know the return path for the packet same with site 1

I’ve done my best to explain the situation, though I understand it may be a bit confusing. I’ve also attached a rough network diagram for your reference. Based on this, do you think the issue could be with the switch routing, or possibly asymmetric routing due to dual WAN paths? I'm struggling to pinpoint the exact cause.

Any guidance or help to resolve this issue would be greatly appreciated.

i have also shared the one of the layer 3 switch ospf configuration similar type of will be on site 1

 show running-config routing ospf | tab
routing ospf
enabled
no auto-cost-reference-bandwidth
no compatible-rfc1583
no default-information-originate
no default-metric
no distance
no distance-ospf external
no distance-ospf inter-area
no distance-ospf intra-area
no opaque-lsa
passive-default
no refresh-timer
router-id 1.1.1.1
area
AREA NETWORK SHORTCUT
------------------------------------
0.0.0.0 10.10.10.0/24 default
0.0.0.0 192.168.10.0/24 default
0.0.0.0 192.168.20.0/24 default
0.0.0.0 192.168.60.0/24 default

interface
MINIMAL
DEAD HELLO HELLO
IFNAME AUTHENTICATION COST INTERVAL MULTIPLIER INTERVAL PRIORITY PAS
--------------------------------------------------------------------------------
cel-6-1 - - 40 - 10 1 fal
fe-cm-1 - - 40 - 10 1 tru
ge-4-1 - - 40 - 10 1 tru
gre-test - - 40 - 10 1 fal
switch.0001 - - 40 - 10 1 tru
switch.0010 - - 40 - 10 1 fal
switch.0020 - - 40 - 10 1 tru
switch.0200 - - 40 - 10 1 tru

[balaji.bandi]

how is your OSPF peering - May be try increasing one of the Link Cost over other and see if that works.

Do you need to Load-Balance or Failover the second Link ? these kind of setup i always prefer to use BGP for traffic engineering made easy.

example :

ip ospf cost XXX (XXX is increasing the cost over other)

[duahimanshu816]

At the moment, I’m not concerned about link failover. My primary concern is that traffic is reaching the firewall site 1 to site2 vice versa via  Gre tunnel, but the firewall is not responding it back

Do you think this could be related to a switching issue, a firewall configuration problem, or something else? I’d appreciate any suggestions or possible solutions.

[balaji.bandi]

Same process, choose the prefer link with cost manipulation on OSPF.

[MHM Cisco World]

This can Happened if cost is mismatch in link

One side use low cost other side use high cost (cost or link BW)

So you need to match that

The FW is drop traffic becuase of it come from vlanx and go from vlanz 

MHM

[duahimanshu816]

Hi, thanks. I've currently brought down the firewall-to-firewall IPsec tunnel as I'm testing the GRE tunnel connectivity.

You're right — the firewall is dropping traffic because it's entering through VLAN X and exiting through VLAN Z. upon troubleshooting i found from firewall logs "RPF (Reverse Path Forwarding) check failure: The return path doesn’t match the expected route/interface."

Do you know why this is happening? And is there any solution to prevent the firewall from dropping this kind of traffic?

[MHM Cisco World]

What IGP you run between L3SW and FW? 

MHM

[duahimanshu816]

Firewall to Layer 3 switch ospf,

[MHM Cisco World]

I think you run OSPF for both VLAN SVI between FW and L3SW 
this make L3SW have two path and do load balance which is not accpet from FW 
check this point in L3SW

and sure that only one VLAN SVI is use for ospf other use as ospf passive only 

MHM

[MHM Cisco World]

Just to confirm both VPN is run between L3SW not bypass L3SW

MHM