>>>RV340 ( 192.168.15.1 )192.168.15.0/24 LAN 1- Gateway ( 192.168.15.2 / 172.16.1.1 ) 172.16.1.0/24 LAN 2
I strongly suggest that you should change/update the existing config on the RV340 as below for everything to work correctly
Note: Iam assuming that you have presently connected the "Gateway" (the Internal-router) directly to one of the lan-ports on the RV340. If not i suggest that you do so, if possible.
1. Keep the existing vlan1 configs (192.168.15.x/24) as it is on the RV340
2. Add another vlan on RV340, say vlan3001
a) configure a ipaddress for vlan3001 in a subnet you are NOT using in your lan for any hosts, such as "10.30.30.1/24"
- apply and do a permanent save
b) Assuming that you have connected one of the interfaces of "Gateway" to lan-port-3 (for example), in the vlan-settings config on RV340, set for LAN-3 port as "untagged" for vlan3001. Leave all other settings as it is.
3. Next in Routing/Static-route page, add a static route (to the lan2-network via 10.30.30.2 ip of the Gateway, using vlan3001 interface on RV340)
172.16.1.0 ; 255.255.255.0; 10.30.30.2; vlan3001
- apply and do a permanent save
4. Now on Gateway router, instead of the existing ipaddress of 192.168.15.2,
- change it to the ipaddress 10.30.30.2/24
- and then configure the default-gateway (for the default-route) as 10.30.30.1
Note: Since we have set the LAN3-port on RV340 as untagged, you can connect the Gateway interface (with the ipaddress 10.30.30.2/24) directly to this LAN3-port without configuring a tagged vlan3001...
So in the lan-side network perspective your lan-network on RV340 will look like below:
Followup explanation/observation of your issue (as understood by me) is
>>>>For example, PC A 172.16.1.10 tries to ping PC B 192.168.15.20. The ICMP request reaches PC B and the response is sent. But it >>>never reaches the gateway.
>>>I'm not sure if I can monitor packets on the RV340 but it must be going there. What happens after that?
So what's happening here is:
i) the ping-request (icmp-ping-request) from PC-A 172.16.1.10 to 192.168.15.20 is forwarded to its Gateway (172.16.1.1)
ii) Here on the Gateway, it is also connected to the vlan1 network with a ip-addr 192.168.15.2
iii) now on Gateway, when it sees that the destination-ipaddr of the ping-request is 192.168.15.20, AND this network is directly-connected on it, and it has a ipaddr in the same-subnet 192.168.15.x/24
iv) and therefore, instead of forwarding/routing this icmp-ping-request packet to the RV340, it correctly (as per standard network behavior and design) does a ARP-request for finding out the "mac-address of 192.168.15.20"....and in this case the PC-B being in the same subnet does a ARP-reply with its mac-address as its configured with the ipaddr 192.168.15.20
v) So now once gateway gets to know the mac-addr of PC-B, it simply forwards/sends the icmp-ping-request packet (in a ethernet-frame) directly to PC-B
Note: Here please note, that till this point, the RV340 does not know anything about this icmp-ping sesssion...AND please note that on RV340, there is a full-fledged statefull-firewall running on it...and if you look closely at the icmp-ping-packets, it too also has a session-id for every ping-request-reply session between 2 hosts..
vi) So now, as expected, PC-B will process the icmp-ping-request packet recieved AND as expected it will prepare to send a icmp-ping-reply....BUT since the destination-ipaddr of this icmp-ping-reply is 172.16.1.10, and since this is NOT a local-network (which is 192.168.15.x), therefore PC-B will refer to its default-gateway/default-route which is configured as 192.168.15.1 (the RV340 vlan1 interface) and therefore correctly forwards this icmp-ping-reply to the RV340...
BUT on RV340, the firewall does not have any info of any "existing" icmp-ping "session" entry....and this is NOT a icmp-ping-request, therefore it is simply dropping it....which is correct behavior for the statefull-firewall
>>>Meanwhile, if I ping from PC B to PC A, it completes without issue.
So what's happening in this case/direction is:
i) PC-B is sending a icmp-ping-request to 172.16.1.10 and since the destination is not a local-network, therefore its sending this to the default-gateway 192.168.15.1
ii) Now on RV340, there is a static-route that says "route traffic to 172.16.1.0 network to 192.168.15.2 on vlan1 interface"....and since it has to route to 192.168.15.2...on vlan1(192.168.15.1) interface, it would have done a ARP-request and got to know the mac-address of 192.168.15.2...
lets-say for example the mac-address of 192.168.15.2 is say "zzzz:zzzzz:zzzz"
iii) So as RV340 is a router too, and since the icmp-ping-request is from a host in the local-subnet 192.168.15.x, and since the gateway to reach the destination network (172.16.1.x) is also in the same local-subnet...
- for the purpose of efficient routing, RV340 router is simply sending a "IP-Redirect" message to PC-B, such as "IP-Redirect for 172.16.1.x to 192.168.15.2"...
- and i dont recall exactly now whether the macaddr of 192.168.15.2 is also included in this IP-Redirect message....but anyways the summary processing done is that RV340 has sent a IP-Redirect message to PC-B saying you should send your icmp-ping-request for 172.16.1.10 directly to the next-hop address 192.168.15.2....
iv) and that is what PC-B has done....it has simply "redirected" the icmp-ping-request for the destination 172.16.1.10 to 192.168.15.2
v) Now when the icmp-ping-reply is sent from PC-A to 192.168.15.20....and arrives at the Gateway, instead of routing it to RV340, a ARP-resolution is done for getting the mac-add of 192.168.15.20 and thus the icmp-ping-reply is sent directly via L2-forwarding in a Ethernet-frame to PC-B.....and thus in this case, the RV340 is no longer involved further in the ongoing icmp-ping-session (except for sending IP-Redirects to PC-B)
>>>>Additionally, if I add a static route on PC B - the same static route on the RV340 - then both ping scenarios work.
Correct. Thats becos, now you are completely avoiding the statefull-firewall on the RV340 (and therefore the PC-A to PC-B works now)
Due to all that is happening (as described above) presently in your network-deployment, hence my recommendation to re-configure your network as suggested in my previous post