encapsulation failed

[chuck007]

I have a 2600 router on a stick setup with ip helper turned on for DHCP relays. But it's not working. I got encapsulation failed on the debug logs. However I can ping back and forth just fine. Here are the logs:

Dec 26 18:18:38.081 PST: IP: s=0.0.0.0 (FastEthernet0/0), d=255.255.255.255, len 328, rcvd 2

Dec 26 18:18:38.081 PST: UDP src=68, dst=67

Dec 26 18:18:38.085 PST: IP: s=10.209.2.254 (local), d=10.69.96.30 (FastEthernet0/0), len 328, sending

Dec 26 18:18:38.085 PST: UDP src=67, dst=67

Dec 26 18:18:38.085 PST: IP: s=10.209.2.254 (local), d=10.69.96.30 (FastEthernet0/0), len 328, encapsulation failed

Dec 26 18:18:38.085 PST: UDP src=67, dst=67

Dec 26 18:18:38.089 PST: IP: s=10.209.2.254 (local), d=10.69.96.40 (FastEthernet0/0), len 328, sending

Dec 26 18:18:38.089 PST: UDP src=67, dst=67

Dec 26 18:18:38.089 PST: IP: s=10.209.2.254 (local), d=10.69.96.40 (FastEthernet0/0), len 328, encapsulation failed

Dec 26 18:18:38.089 PST: UDP src=67, dst=67

I have a very basic setup:

ip subnet-zero

!

interface FastEthernet0/0

ip address 10.209.2.254 255.255.255.0

ip helper-address 10.69.96.30

ip helper-address 10.69.96.40

no ip redirects

no ip directed-broadcast

duplex auto

speed auto

!

interface Serial0/0

no ip address

no ip directed-broadcast

no ip mroute-cache

shutdown

!

ip classless

ip route 0.0.0.0 0.0.0.0 10.209.2.22

no ip http server

The WAN router is at 10.209.2.22 which is managed by our ISP. From my 2600 router, I can ping the WAN router and my DHCP servers. And my servers can ping the router also. It seems like my router is confused because the ARP debugs showed the following:

Dec 26 18:21:27.933 PST: IP ARP: creating incomplete entry for IP address: 10.209.2.22 interface FastEthernet0/0

Dec 26 18:21:27.933 PST: IP ARP: sent req src 10.209.2.254 0007.eb37.f5c0,

dst 10.209.2.22 0000.0000.0000 FastEthernet0/0

Dec 26 18:21:27.933 PST: IP ARP: rcvd rep src 10.209.2.22 0019.556a.8660, dst 10.209.2.254 FastEthernet0/0

Dec 26 18:22:26.313 PST: IP ARP: creating incomplete entry for IP address: 10.69.96.30 interface FastEthernet0/0

Dec 26 18:22:26.317 PST: IP ARP: sent req src 10.209.2.254 0007.eb37.f5c0,

dst 10.69.96.30 0000.0000.0000 FastEthernet0/0

Dec 26 18:22:26.317 PST: IP ARP: creating incomplete entry for IP address: 10.69.96.40 interface FastEthernet0/0

Dec 26 18:22:26.317 PST: IP ARP: sent req src 10.209.2.254 0007.eb37.f5c0,

dst 10.69.96.40 0000.0000.0000 FastEthernet0/0

Dec 26 18:22:30.313 PST: IP ARP: sent req src 10.209.2.254 0007.eb37.f5c0,

dst 10.69.96.30 0000.0000.0000 FastEthernet0/0

Dec 26 18:22:30.317 PST: IP ARP: sent req src 10.209.2.254 0007.eb37.f5c0,

dst 10.69.96.40 0000.0000.0000 FastEthernet0/0

What is going on? Does "ip helper-address" not support router on a stick setup? Thanks.

[chuck007]

I have turned on IP CEF but didn't work. I wonder if my IOS version is messing up. But in theory, it should work. Any advise would help. Thanks.

Here is the show ip cef:

Prefix Next Hop Interface

0.0.0.0/0 10.209.2.22 FastEthernet0/0

0.0.0.0/32 receive

10.69.96.0/19 10.209.2.22 FastEthernet0/0

10.209.2.0/24 attached FastEthernet0/0

10.209.2.0/32 receive

10.209.2.1/32 10.209.2.1 FastEthernet0/0

10.209.2.13/32 10.209.2.13 FastEthernet0/0

I have turned on debug for ARP and looking for DHCP packets:

ARP:

ARP packet debugging is on

Generic IP:

IP packet debugging is on (detailed) for access list 102

access-list 102 permit udp any any eq bootps

access-list 102 permit udp any eq bootpc any

Here is my debug logs:

.Dec 29 09:30:23.656 PST: IP: s=0.0.0.0 (FastEthernet0/0), d=255.255.255.255, len 328, rcvd 2

.Dec 29 09:30:23.656 PST: UDP src=68, dst=67

.Dec 29 09:30:23.656 PST: IP: s=10.209.2.254 (local), d=10.69.96.30 (FastEthernet0/0), len 328, sending

.Dec 29 09:30:23.656 PST: UDP src=67, dst=67

.Dec 29 09:30:23.660 PST: IP ARP: creating incomplete entry for IP address: 10.69.96.30 interface FastEthernet0/0

.Dec 29 09:30:23.660 PST: IP ARP: sent req src 10.209.2.254 0007.eb37.f5c0,

dst 10.69.96.30 0000.0000.0000 FastEthernet0/0

.Dec 29 09:30:23.660 PST: IP: s=10.209.2.254 (local), d=10.69.96.30 (FastEthernet0/0), len 328, encapsulation failed

.Dec 29 09:30:23.660 PST: UDP src=67, dst=67

.Dec 29 09:30:23.660 PST: IP: s=10.209.2.254 (local), d=10.69.96.40 (FastEthernet0/0), len 328, sending

.Dec 29 09:30:23.664 PST: UDP src=67, dst=67

.Dec 29 09:30:23.664 PST: IP ARP: creating incomplete entry for IP address: 10.69.96.40 interface FastEthernet0/0

.Dec 29 09:30:23.664 PST: IP ARP: sent req src 10.209.2.254 0007.eb37.f5c0,

dst 10.69.96.40 0000.0000.0000 FastEthernet0/0

.Dec 29 09:30:23.664 PST: IP: s=10.209.2.254 (local), d=10.69.96.40 (FastEthernet0/0), len 328, encapsulation failed

.Dec 29 09:30:23.668 PST: UDP src=67, dst=67

.Dec 29 09:30:26.652 PST: IP: s=0.0.0.0 (FastEthernet0/0), d=255.255.255.255, len 328, rcvd 2

.Dec 29 09:30:26.652 PST: UDP src=68, dst=67

.Dec 29 09:30:26.656 PST: IP: s=10.209.2.254 (local), d=10.69.96.30 (FastEthernet0/0), len 328, sending

.Dec 29 09:30:26.656 PST: UDP src=67, dst=67

.Dec 29 09:30:26.656 PST: IP ARP: sent req src 10.209.2.254 0007.eb37.f5c0,

dst 10.69.96.30 0000.0000.0000 FastEthernet0/0

.Dec 29 09:30:26.656 PST: IP: s=10.209.2.254 (local), d=10.69.96.30 (FastEthernet0/0), len 328, encapsulation failed

.Dec 29 09:30:26.660 PST: UDP src=67, dst=67

.Dec 29 09:30:26.660 PST: IP: s=10.209.2.254 (local), d=10.69.96.40 (FastEthernet0/0), len 328, sending

.Dec 29 09:30:26.660 PST: UDP src=67, dst=67

.Dec 29 09:30:26.660 PST: IP ARP: sent req src 10.209.2.254 0007.eb37.f5c0,

[Richard Burts]

Chuck

The additional information that you have posted pretty much confirms what I had said in an earlier post. Having both debug arp and debug ip packet looking for DHCP is helpful. It clearly demonstrates that you receive a DHCP packet, that your router is attempting to send to both configured helper address destinations, that it is sending ARP request for the helper address destinations, it receives no response, and having no destination MAC address it can not send the packet and it fails with encapsulation failed.

The basic problem is that the IOS should send the helper address packets via the default route. It should not ARP for the helper address destination. Harold says that this should work. So obviously there is some bug in the version of IOS that you are using. As Harold has pointed out the version of IOS that you are running is somewhat old. If you upgrade to a newer version of IOS it is quite possible that the buggy behavior will not be present. (Or if upgrading the code is not feasible I would suggest looking at possibilities of redesigning your network so that it is not such an odd version of router on a stick.)

HTH

Rick

[Richard Burts]

Chuck

It occurs to me that the behavior that you are having is the behavior that would result if the default route just pointed at the FastEthernet interface rather than at the address of the next hop router. Is it possible that at one point you had the default route just specifying the outbound interface. If you reboot the router do you get the same behavior?

HTH

Rick

[george.fouad]

This is definitely an IOS bug

You may try the following as a workaround

1/add these two static routes

ip route 10.69.96.30 255.255.255.255 10.209.2.22

ip route 10.69.96.40 255.255.255.255 10.209.2.22

it will solve your problem, if the bug is limited to the traffic routed by the default route

2/if possible, ask the admin for 10.209.2.22 to enable proxy arp on the LAN interface facing your router

Hope this helps

George

[george.fouad]

A third good workaround would be configuring static arp entries for 10.69.96.30 and 10.69.96.40 as follows

conf t

arp 10.69.96.30 [mac_address_for_10.209.2.22] arpa

arp 10.69.96.40 [mac_address_for_10.209.2.22] arpa

George

[george.fouad · Accepted solution]

I made some more researches... you're hitting this BUG : CSCdp36754

----QUOTE----

CSCdp36754

Forwarding of bootp/dhcp address request UDP packets fail because of encapsulation failure.

Workaround:

1) Use Cisco IOS Release 12.0(5)T

2) For DDR, define IP DHCP server on the local router.

UniverCD document "Configuring DHCP" at:

http://www.cisco.com/univercd/cc/td/doc/product/software/ios121/121cgcr/ip_c/ipcprt1/1cddhcp.htm#35657

3) Configure a static arp entry for the "next hop" adresse's

mac-address.

Here is an example workaround with DHCP server on the other

side of a FDDI ring:

interface ethernet 0

ip helper-address 192.82.247.98

arp 192.82.247.98 4000.7507.0301 SNAP

----UNQUOTE----

So I think you may try the suggestion I made in the previous post.. or consider an IOS upgrade to 12.1T or 12.2.

or even downgrade to 12.0(5)T as it doesn't hit this bug

I hope it works by the next year ;)

Happy new year to all

George

[chuck007]

George,

Thanks for helping me get to the bottom of this. I'm testing the work-arounds and will post the results.

Have a wonderfull new year to all.

--cd

[chuck007]

For the record, the static ARP entries worked!!! It solved the router's ARP confusion. Thanks.

--chuck