I spent an entire day working on this issue that I decided to discuss with everyone today.
(click image to enlarge)
As you can see in the above topology, clients hanging off the inside interface of Local ASA are trying to acquire an IP address from the DHCP server that is located on the other side of the tunnel behind the Remote ASA.
Let us quickly run through how DHCP relay works.
1. Client starts the DHCP process by sending a DHCPDISCOVER message to destination address 255.255.255.255 - UDP port 67
2. ASA sees the broadcast and based on the dhcprelay server config it forwards the DHCPDISCOVER message as a unicast packet to the server's IP sourcing from the interface IP close to the server. In this case the outside IP address.
3. Server sends back DHCPOFFER as a unicast packet to the ASA - UDP port 67
Server will send it to the destination IP address of the inside interface (giaddr) which is the dhcprelay enabled interface.
4. This packet will arrive on the outside interface and will be broadcast out the inside interface - UDP port 68.
5. Client receives the DHCPOFFER, and sends a DHCPREQUEST message to the server, that it accepts the offer.
6. The server will send back a DHCPACK message to the client.
7. Client upon receiving the DHCPACK, it will start communicating on the network.
Here is the partial config on the Local ASA:
dhcprelay server 10.48.65.13 outside dhcprelay enable inside
DHCP relay packets will be sent to the server with the source address of the outside interface IP address and we need to encrypt that.
crypto acl includes:
access-list L-VPN extended permit ip host 10.252.17.6 host 10.48.65.13
access-list L-VPN extended permit ip host 10.150.1.1 host 10.48.65.13
Here is the partial config on the Remote ASA:
static (inside,outside) 10.150.1.1 10.252.17.6 netmask 255.255.255.255 (this is the problematic static)
The first static 1-1 is a requirement because the DHCP server will only talk to IP addresses in the 10.150.1.0/24 network.
crypto acl includes:
access-list R-VPN extended permit ip host 10.48.65.13 host 10.252.17.6
access-list R-VPN extended permit ip host 10.48.65.13 host 10.150.1.1
Technically the offer packets from the server will be sent to the interface IP address that has dhcprelay enabled which is the inside interface 10.150.1.1 but, since this Remote ASA has static 1-1 NAT configured it changes the destination IP address to 10.252.17.6 upon receiving
these packets destined to 10.150.1.1
Everyone with me so far? Now comes the interesting part, the problem.
The clients off the Local ASA never get an IP address.
According to debugs on the Local ASA the DHCP server seems to be sending offers but, these packets do not egress the inside interface of the Local ASA.
dhcpd_forward_request: request from 0024.7e03.e3bf forwarded to 10.48.65.13. DHCPRA: Received a BOOTREPLY from interface 1 DHCPRA: relay binding found for client 0024.7e03.e3bf. DHCPRA: Adding rule to allow client to respond using offered address 10.150.1.10 DHCPRA: forwarding reply to client 0024.7e03.e3bf. Now, where could they be going? On the outside we cannot capture because the traffic is encrypted. Debugs and syslogs don't indicate any problem. I started to wonder if the ASA was silently dropping these packets. Well, that should show in the asp drop captures shouldn't it? Nope. They weren't there either.
It was a mystery. I decided to throw a capture on the outside interface for udp 67 and udp 68 packets and wasn't I surprised to see the ASA broadcasting the offer packets off the outside interface! Oh boy!! Why would it do that? The clients are on the inside so, why would this ASA send these packets off the outside interafce and also create arp entries on the outside?
sh arp | i outside arp outside 10.151.1.19 0024.7e03.d45d arp outside 10.151.1.10 0024.7e03.e3bf
This was extremly strange. MAC address belongs to clients on the inside. DHCP offered IP address is mapped to the MAC address properly but the interface is wrong.No wonder the inside clients don't get the IP address!
The 'giaddr' field can be used to identify the logical interface from which the reply must be sent (i.e., the host or router interface connected to the same network as the BOOTP client). If the content of the 'giaddr' field does not match one of the relay agent's directly-connected logical interfaces, the BOOTREPLY messsage MUST be silently discarded ....
The ASA is supposed to receive these offer packets destined to the inside interface IP on the outside interface but, due to the static 1-1 on the Remote ASA this is not the case. The offer packet's destinated IP address gets changed to the outside IP address. Upon receiving these packets, without looking at the "giaddr" in the packet the ASA chooses to broadcast it out on the outside interface thinking the clients are off of the outside interface which is clearly not enabled for dhcprelay.
I know what everyone is thinking. Time to file a bug. Right. But, we need to fix the probelm at hand. What can we do resolve the problem right now?Using static IP address on the client PCs is not an option. Somehow we need to make sure that the destination IP address in the offer packets sent by the server does not get changed. How can we accomplish this? If we can do that then, things will start to work and we can file a bug later.
We decided to change the static 1-1 on the Remote ASA as follows: static (inside,outside) 10.150.1.6 10.252.17.6 netmask 255.255.255.255 We just picked this address 10.150.1.6that was not used anywhere. Since these are just udp packets, when going to the server these will have the source address of 10.150.1.6 but, when the server responds back it will respond back to the "giaddr" which is 10.150.1.1, will take the identity static and reach the Local ASA.
Once we did the above, DHCP relay started to work as expected. I have filed this defect CSCtj68732.(NOTE: you will need to login with your Cisco Connection Online "CCO"credentials to access)
Hi everyone,Following up a SOHO network that I am designing, I have an ASA 5505 firewall behind a router, and I need some end devices like the NAS and FTP server to be configured with static IP addresses so that I can access them remotely via DDNS. Howeve...
"Choose one of the topics below to help you on your journey with NGFW/ASA"
Getting Started with Next-Generation ...
Hello, I got a reported alarm from the customer, so it looks like if phase 2 VPN (IPsec) has some issue, however when I validate the status of the IPsec it is active. Alarm:Último interno &nbs...
Do Cisco IronPorts start with an MID of 0 (zero) or 1 (one)? I have only seen MID 0 for "Message size exceeds limit". Is 0 reserved for this notice or is it ALSO a valid MID when a system restarts/rolls over. Thank you