I am moving a PPPOE-based internet connection from an FTDv running on ESXi to an FTDv running on an ENCS5412.
I have both the inside and outside interface of the FTDv connected via SRIOV ports on the ENCS. The FTDv virtual ports have MTU set to 1492.
On the old connection (FTDv on ESXi), Path MTU Discovery works. I can't ping out to the internet with a full 1500-byte packet with the DF bit set, but I have no connectivity issues (standard web browsing) because PMTUD works and lowers my PC's transmission units to stay under what the pppoe connection supports.
When I cut over to the new connection (FTDv on ENCS) there are many websites that don't work. Looking at the ASP drops on the FTDv I see it dropping packets with errors stating fragmentation required but df-bit set.
To confirm the issue I hard-coded my PC MTU to 1450 and I have no problem browsing websites. As soon as I set it to 1500 I have problems. It sounds to me like PMTUD is broken when traffic flows through the new connection.
I don't know if this is due to the FTDv, or the ENCS. I see the physical ports of the ENCS have an MTU of 9216, but I don't see any way to change that. Any thoughts?
Thanks!
Update on some troubleshooting I did:
On the new connection I did a packet capture on my PC. I can see packets leaving my PC @ 1514 bytes. I see ICMP Destination unreachable (Fragmentation needed) packets from the firewall.
As soon as I move the connection to the old firewall, no change to the PC, I run a capture and never see a single ICMP Dest Unreachable and the max size I ever see leave the PC is 1434 bytes.
I don't get what's happening
