topic Re: HTTPS web page not loading in Application Centric Infrastructure

HTTPS web page not loading

LewisD1 — Thu, 09 Dec 2021 00:54:20 GMT

Hello All,

I am experiencing a weird issue in my Multipod ACI Deployment.

We will have 2 machines part of the same EPG/BD.

Machine 1 and Machine 2

Machine 1 is in Pod 1

Machine 2 is in Pod 2

From machine 1 we can ping, ssh, telnet and telnet on port 443 to machine 2 without issue. When we try to load machine 2 in a web browser on port 443 we just get this page cannot be displayed. If machine 1 and machine 2 are in the same pod it all works perfectly.

We are running version 5.2(3f)

We are not restricting Intra-EPG traffic

I have tried disabling VRF enforcement to rule that out but still the same.

I am drawing up a blank.

Thanks

Re: HTTPS web page not loading

balaji.bandi — Thu, 09 Dec 2021 03:14:26 GMT

Looks like a browser issue, clear the cache and test it or use a different browser?

Re: HTTPS web page not loading

LewisD1 — Thu, 09 Dec 2021 09:05:50 GMT

I have tried this. Multiple browsers and cleared the cache.

I am more leaning down the route it is an ACI issue. In the example above I used 2 machines. There are more machines on this BD that are having this issue. The only machines affected are in another POD.

Re: HTTPS web page not loading

Sergiu.Daniluk — Thu, 09 Dec 2021 09:10:56 GMT

Have you tried to do a tcpdump/wireshark capture on machine 2, while connecting via browser from machine 1?

This way you confirm if the traffic arrives or not on Machine2 and see if there is any response.

If you see a response, perform tcpdump on machine 1 and confirm if return traffic is there.

The results will help you in which direction to focus your tshoot.

Cheers,

Sergiu

Re: HTTPS web page not loading

balaji.bandi — Thu, 09 Dec 2021 09:16:12 GMT

May be i have missed here - telnet on port 443 (the one having issue worked ?)

telnet using IP or FQDN or hostname ? are you able to resolve IP to DNS ?

are you trying https://IP address or https://domain.com

if the the One not working above and in different POD, then sure some EPG access issue.

Re: HTTPS web page not loading

LewisD1 — Thu, 09 Dec 2021 09:41:13 GMT

So in command prompt i telnet to 10.255.1.112 on port 443 get a response. I can even open and maintain an ssh session on 22.

When i go to a browser and do https://10.255.1.112 i get nothing.

In wireshark i can see the below.

I can see bidirectional traffic. which adds to the confusion.

But like i say its happening to all machines in the other POD and so far it is only impacting web browsing.

Re: HTTPS web page not loading

LewisD1 — Thu, 09 Dec 2021 09:42:05 GMT

I have messaged above with the wireshark output. I had ran this already and see bidirectional traffic which just adds to the confusion.

Re: HTTPS web page not loading

Robert Burns — Thu, 09 Dec 2021 12:43:34 GMT

Can you migrate/clone the endpoint from Pod2 to Pod1 and re-test this? If this works, it could point to a multicast config issue in the IPN. If its still failing within the same pod between the same VMs we'll have to continue debugging.

Robert

Re: HTTPS web page not loading

balaji.bandi — Thu, 09 Dec 2021 12:48:30 GMT

what is the outcome, the device working in other pod and connect here ? is that works ? ( before we conclude anything here)

Re: HTTPS web page not loading

LewisD1 — Thu, 09 Dec 2021 13:35:15 GMT

Hi Robert,

These are bare metal servers so moving them is not an option.

I have been thinking the IPN network. I have checked and mapped out the IGMP Joins and the PIM and nothing sticks out. The PIMs are not getting sent to the spines. I can see the route that the BD is taking and all seems above board.

Thanks

Lewis

Re: HTTPS web page not loading

Robert Burns — Thu, 09 Dec 2021 14:16:23 GMT

Can you also test the MTU from Pod1 > Pod2. You might have a mis-match MTU somewhere in the path. Ensure you can send at least 1550 across without fragmentation.

Robert

Re: HTTPS web page not loading

LewisD1 — Thu, 09 Dec 2021 14:49:49 GMT

What would be the best way to do this in an ACI fabric?

Re: HTTPS web page not loading

Robert Burns — Thu, 09 Dec 2021 15:32:49 GMT

Just ping from Src endpoint to destination endpoint. Depending on the OS of your hosts, there's appropriate flags to set on the command. This will validate your Dataplane MTU. The Control Plane MTU can be separately configured, but typically should be set as high as the IPN will allow.

See: https://www.pcwdld.com/ping-mtu

Robert

Re: HTTPS web page not loading

LewisD1 — Thu, 09 Dec 2021 16:18:21 GMT

These server are running ESXI i have enable MTU of 9000 on the DVS but cannot ping via jumbo frames. I get a message saying message too long.

Re: HTTPS web page not loading

Robert Burns — Thu, 09 Dec 2021 17:13:04 GMT

Keep adjusting it until you find out what your max MTU is. Just keep halving the value up/down until you identify the largest packet you can successfully send w/o fragmentation.

Ex.
9000? -No (split 1/2 down)
4500 - Yes (split 1/2 up)

6750 - Yes (split 1/2 up)

7875 - Yes (split 1/2 up)

etc...

Robert

Re: HTTPS web page not loading

Sergiu.Daniluk — Thu, 09 Dec 2021 17:24:19 GMT

Since we are talking about unicast traffic (https), not sure why focus should be on mcast config in IPN.

@LewisD1 I would continue with the packet captures on ACI Leaf/Spines (use ELAM Assistant) and confirm first where the traffic is seen/not seen.

Since you already confirmed that the EP from Pod2 receives and responds, then continue with a captures in Pod2 (direction EP->Leaf->Spine, Source EP-Pod2 Destination EP-Pod1). Then make the same captures on Spines/Leafs in Pod1.

Cheers,

Sergiu

Re: HTTPS web page not loading

LewisD1 — Thu, 09 Dec 2021 21:57:49 GMT

@Robert Burns

I kept digging i could not get anything past over 1450MTU. It turns out that the Subinterfaces of the IPN devices had the default MTU of 1500. All other interfaces has 9126. I have changes the sub interfaces which then brought everything into life.

You hit the nail on the head.

Thank you for your help.

Lewis

Re: HTTPS web page not loading

Robert Burns — Fri, 10 Dec 2021 00:44:09 GMT

Great to hear. Since telnet worked fine (lower payload) figured a full web connection would exhibit this symptom if there was fragmentation.

Cheers,

Robert