Web Proxy Packet Flow

[Ex-Engineer1968]

I am looking for a detailed packet trace that shows hop by hop exactly what happens when a web proxy is being used and an endpoint does a DNS resolution. 

Does the DNS server return the IP address of the actual website on the internet? Or does it return its own IP address so that the traffic gets redirected to it? 

How exactly is the traffic rerouted to the proxy? 

[Ken Stieers]

Are you using PAC file/Explicit redirection? Or transparent redirection?

[Ex-Engineer1968]

Hi, Im not using anything, per se. Im just trying to understand how traffic is forwarded in an environment in which a proxy is used... I dont know the difference between any of those you mentioned... lets say explicit redirection... 

[Ken Stieers]

If using explicit redirection, either the endpoint got a PAC file or the browser was configured to send web requests to the proxy. It's actually independent of the DNS lookups. Those happen as normal, but the browser sends the request to the proxy anyway.

If you're using transparent redirection via WCCP or Policy routing, the DNS request happens as normal, the packet is built as normal and sent. Then a router or firewall where WCCP or PBR sends it to the proxy instead. The proxy will then do the web request and send back the results to the endpoint, essentially spoofing the web site.

[balaji.bandi]

Depends on the model of proxy deployment transparent or explicit

All the DNS resolution done by Web proxy depends on the site user requested to access via Web Proxy.

check below guide :

https://www.cisco.com/c/en/us/products/collateral/security/web-security-appliance/guide-c07-742373.html

[Ex-Engineer1968]

Sorry, but this did not help at all. Doesn't even address what Im asking. 

[M02@rt37]

Hello @Ex-Engineer1968 

Let's walk through a detailed packet trace for a scenario where a web proxy is used, and an endpoint performs a DNS resolution.

The key steps involve DNS resolution, request redirection to the proxy, and the subsequent data flow.

- DNS Resolution
The endpoint sends a DNS query to its configured DNS server, typically located on the local network or provided by the ISP.
The DNS server performs a recursive lookup and returns the IP address associated with the requested domain (e.g., www.example.com).

- Proxy Configuration on the Endpoint
The endpoint is configured to use a web proxy for internet access. This configuration is often set either manually or through a network policy.

- DNS Response
The DNS server returns the actual IP address of the website (www.example.com) to the endpoint.

- Endpoint Sends HTTP Request
The endpoint initiates an HTTP request to the resolved IP address (or domain) of the website. However, since it's configured to use a proxy, it won't connect directly to the web server.

- Proxy Detection
The endpoint checks its proxy configuration and recognizes that web traffic should be directed through the proxy server.

- Request Sent to Proxy
The endpoint sends the HTTP request to the IP address of the proxy server, specifying the original destination (www.example.com) in the request headers.

- Proxy Receives Request
The proxy server receives the request and, based on the destination specified in the headers, performs its own DNS resolution to determine the actual IP address of www.example.com.

- Proxy Establishes Connection
The proxy establishes a connection to the web server on behalf of the endpoint. From the web server's perspective, it appears that the request is coming from the proxy, not the original endpoint.

- Web Server Responds
The web server responds to the proxy with the requested web content.

- Proxy Forwards Response to Endpoint
Proxy forwards the web server's response back to the endpoint that initiated the request.

This process ensures that the proxy is an intermediary between the endpoint and the web server. The DNS server provides the actual IP address of the web server, and the proxy redirects and handles the traffic on behalf of the endpoint.

To sum up, DNS server returns the IP add. of the actual website, and the traffic is rerouted to the proxy based on the endpoint's proxy configuration. The proxy acts as an intermediary, forwarding requests to the web server and returning responses to the endpoint.

[Ex-Engineer1968]

Thank you. Can you clarify this step?

"Request Sent to Proxy
The endpoint sends the HTTP request to the IP address of the proxy server, specifying the original destination (www.example.com) in the request headers."? 

[M02@rt37 · Accepted solution]

Certainly @Ex-Engineer1968 

The user, on their endpoint (computer or device), initiates a request to access a website. For example, they type "www.example.com" in their web browser.

The endpoint performs a DNS resolution to determine the IP address associated with "www.example.com." This involves querying a DNS server to translate the domain name to an IP address.

The user's system is configured to use a proxy server for internet access. This configuration may be set manually or through network policies.

Instead of directly connecting to the destination web server (www.example.com), the endpoint sends the HTTP request to the IP address of the configured proxy server. This is done by forming an HTTP request with the original destination (www.example.com) specified in the request headers.

Example HTTP Request:
```http
GET / HTTP/1.1
Host: www.example.com
...
```

In this request, the "Host" header indicates the original destination server.

The proxy server receives the HTTP request from the endpoint. The proxy is now aware of the original destination specified in the "Host" header.

The proxy server, based on its own configuration and policies, initiates a separate request to the destination server (www.example.com) on behalf of the endpoint.

The destination web server (www.example.com) responds to the proxy's request as if the proxy itself initiated the connection.

The proxy server forwards the response received from the destination server back to the original endpoint that initiated the request. From the endpoint's perspective, it appears as if the response is coming directly from www.example.com.

To sum up, the "Request Sent to Proxy" step refers to the endpoint directing its HTTP request to the proxy server, specifying the original destination in the request headers. The proxy then handles the communication with the destination server on behalf of the endpoint. This process is part of how proxy servers are used to control and optimize internet access in a network.

[Ex-Engineer1968 · Accepted solution]

Good show! Yes, I researched the answer myself by Googling "how does a proxy server know where to forward the web traffic?" In other words, if the www.example.com destination IP address in the packet header (resolved by the DNS server initially as x.y.z.w) gets replaced with the proxy server's IP address (because the web browser has a proxy configured), how then does the proxy server know where the client originally wanted to go? And the answer is exactly as you stated - that the HTTP Request packet formed by the browser includes the original Internet website name that the client was initially trying to reach. 

[amojarra]

Generally speaking 

In Explicit mode ( Browser proxy configured or PAC file ) the Name resolution is happening on the Proxy.

In transparent mode the ( WCCP , PBR - Policy Based Routing ,... ) the Name resolution happens on the Client side and WSA respect the name resolution from client, and we have some configuration to force WSA use its resolved IP: 

https://www.cisco.com/c/en/us/support/docs/security/secure-web-appliance-virtual/220558-troubleshoot-secure-web-appliance-dns-se.html

Also, you can use T1/T2 interface to listen to Name resolutions in transparent deployment, and if there are any misbehavior, WSA will block the connection. 

Regards,

Amirhossein Mojarrad

+++++++++++++++++++++++++++++++++++++++++++++++++++

++++        If you find this answer helpful, please rate it as such      ++++

+++++++++++++++++++++++++++++++++++++++++++++++++++