Re: Routing Mystery - where is it going?

bleucube · ‎12-18-2002

I have a mystery and before I call Leonard Nimoy in to do a "In Search Of...." episode thought I post here.

PROBLEM:

I'm converting a remote site from ISDN (At the moment using Ascend Pipelines) and converting the site to Point-to-point using a 1760 Cisco on the remote end and a 7204 vxr on the other. After conversion, everything works except DHCPOFFER doesn't hit the workstation. EVERYTHING works except that. Ping/Telent/Http..blahblah. I can see DHCPDISCOVER packet hit the DHCP server, I can see the DHCP server sending a DHCPOFFER but workstation doesn't get it. If I put a different IP subnet for this location the offer goes through. Routing problem right?

SETUP:

[workstation] - IP 172.22.36.7

|

[1706 Cisco] - E0=172.22.36.1 S0=172.22.136.1 (ip helper on E0)

|

[7204 VXR Cisco] - S0=172.22.136.2 E0=172.22.1.8 (Ip helper on E0)

|

[2948G-L3 Cisco] - 172.22.1.1

|

[DHCP Server] - 172.22.1.50

TROUBLESHOOTING THUS FAR:

1. Check routes on all routers/servers/workstations

2. Clear routers on everything. We are using EIGRP.

3. Enable Show Debug on all routers

with this I see the DHCPDISCOVER hit everything,

I don't see an DHCPOFFER hit the 172.22.1.1 on the way back

4. Put a Sniffer on DHCP server and on the Workstation ends

- Sniffer shows its heading to the right place

5. Enable DHCP debug on the server, shows everything is fine except it can't send DHCPOFFER.

6. If I go to a workstation after converting the site, and it still has an IP, the dhcpserver can ping the workstation no problem. Once I release the IP it can't get another one.

7. I have 7 total sites, 4 sites are working - three are not, using the exact same config, except different ips.

8. I removed all sites from the 7200VXR and cleared all routes and then attempt switch over but still had problem. I was thinking something was advertising the wrong route.

SUGGESTIONS?

I know I can get it to work if I change the IP subnet. Really don't want to do this, need to figure out issue, who knows when and where it will rear its ugly head.

Any help will be great!

prafuljaded · ‎12-18-2002

Hi,

I dont think U need a ip helper-address command on e0 of the DHCP Server subnet. Well if telnet,http applications are running fine, then there shouldnt be a routing problem.

Chk out: http://www.cisco.com/warp/public/779/smbiz/service/knowledge/tcpip/dhcp.htm

prafuljaded · ‎12-18-2002

Is your workstation directly connected to e0 of the router or there is some L3 device between the two. If yes,configure ip helper-address on the port facing the workstation. Also your Router/L3 should have path to DHCP Server subnet.

bleucube · ‎12-18-2002

No Workstation is connected from the Router on the remote end, goes directly into a stupid hub.

At one point in my troubleshooting i had Helper address on every interface.

Also, I made sure the DHCP is setup properly with option 3 (router) to the correct IP. Like I said, I have four sites working and the setup is the same except IP's. I had three other set of eyes check the configs and to compare and yep - all the same.

a.balian · ‎12-18-2002

My friend :

You have a very interesting problem. I suggest you look into the scope of your DHCP server. Use netstat -a to see if the port 67 68 are listening. Your IP variation and DHCP scope have something to do with your problem that you anly can solve it.

Good luck

lgijssel · ‎12-19-2002

There is one sneaky issue with cisco routers that might cause this type of problem. Most important questions:

- To what range did you change the IP's to get it to work?

- How do you route from your remote sites?

My assumption is that you are using supernet routes from the remote locations, and you have configured no ip classless on the 1760 router.

Changing to an ip-network outside 172.22 will result in a working situation as the supernet route is now the best match. Without ip classless, the packet will be discarded by the 1760.

Effectively all you should do, if this is the problem, is to enable ip classless.

bleucube · ‎12-19-2002

- I changed the range to our test subnet which is 172.22.15.0

- I use EIGRP from my remote sites, however the existing sites that I need to transfer over to T1 are using RIP (not cisco routers) and I have RIP setup on core router (172.22.1.1). During troubleshooting I did remove the sites using RIP from the core router and clear ip routes.

- Actually have IP CLASSESS on ... here's the config. Very short and simple. Good guess tho.

version 12.2

no service pad

service timestamps debug datetime msec

service timestamps log datetime msec

service password-encryption

!

hostname #######

!

enable password 7 #######

!

no ip subnet-zero

no ip domain lookup

!

interface FastEthernet0/0

ip address 172.22.36.1 255.255.255.0

ip helper-address 172.22.1.50

speed auto

!

interface Serial0/0

ip address 172.22.136.1 255.255.255.0

no fair-queue

!

router eigrp 200

network 172.22.0.0

no auto-summary

no eigrp log-neighbor-changes

!

ip classless

ip route 0.0.0.0 0.0.0.0 172.22.136.2

no ip http server

!

line con 0

password 7 #######

login

line aux 0

password 7 #######

login

line vty 0 4

password 7 #############

login

!

end

lgijssel · ‎12-19-2002

That is a simple config indeed. There should be no problem.

Rereading the posting I noticed that DHCP is sent to the server and arrives there. Therefore, th sole thing that I still can think of is related to another posting. C800 routers with IOS 12.0(x)T are having an identical issue with dhcp. Are all 1760's using the same IOS-version?

To check this you can exchange a working 1760 with a non-working one.

Regards and good hunting,

Leo

bleucube · ‎12-19-2002

All the 1760's and the 7200 VXR are using 12.2

This indeed is weird. I don't understand how a DHCPDISCOVER packet can tranverse the route back to the DHCP server but the DHCPOFFER disappears out of the DHCP server.

So if you looked at the ascii map above, this is what it looks like:

[DHCP] (a)

|

| ---|hub| (b) --------------- Sniffer

|

[172.22.1.1] (c)

A) The DHCP sends the offer out heading in the right direction

B) The Sniffer see's it going the right way

C) 172.22.1.1 show ip debug UDP never see's the packet

It's hard to imagine its getting lost on the physical line... especially since everything else works all DNS request go to this server too.

uniemeyer · ‎12-19-2002

Hi,

you explained that you did a Sniffer trace on the server site:

Is the IP address of the ethernet interface of the router 1706 included in the

DHCP DISCOVER message?

Is the DHCP server answering with a DHCP OFFER to the MAC-address

of the 7204 router or is the offer also a broadcast?

Ulrich Marzoli

bleucube · ‎12-19-2002

1) No, just the address of the DHCP server

2) Answer is a broadcast I believe, Need to look up the sniff log. Will respond in a few... need to fire up laptop

msingla · ‎12-19-2002

Hi,

Strange problem u have .Please do send the other side configuration so that problem can be analyzed in detail.....

Regards,

Munit

bleucube · ‎12-19-2002

Here's the 7200 VXR Config. Pretty simple too.

version 12.2

no service pad

service timestamps debug datetime msec

service timestamps log datetime msec

service password-encryption

!

hostname #############

!

enable secret 5 #############

enable password 7 #############

!

no ip subnet-zero

ip cef

!

no ip domain lookup

!

controller T1 1/0

framing esf

clock source internal

linecode b8zs

channel-group 1 timeslots 1-24

description "#############"

!

controller T1 1/1

framing esf

clock source internal

linecode b8zs

channel-group 1 timeslots 1-24

description "#############"

!

controller T1 1/2

framing esf

clock source internal

linecode b8zs

channel-group 1 timeslots 1-24

description "#############"

!

controller T1 1/3

framing esf

clock source internal

linecode b8zs

channel-group 1 timeslots 1-24

description "#############"

!

controller T1 1/4

framing esf

clock source internal

linecode b8zs

channel-group 1 timeslots 1-24

description "#############"

!

controller T1 1/5

framing esf

clock source internal

linecode b8zs

channel-group 1 timeslots 1-24

description "#############"

!

controller T1 1/6

framing esf

clock source internal

linecode b8zs

channel-group 1 timeslots 1-24

description "#############"

!

controller T1 1/7

framing esf

linecode b8zs

!

speed auto

!

interface FastEthernet0/1

no ip address

shutdown

duplex auto

speed auto

!

interface Serial1/0:1

description "#############"

ip address 172.22.141.2 255.255.255.0

!

interface Serial1/1:1

description "#############"

ip address 172.22.161.2 255.255.255.0

!

interface Serial1/2:1

description "#############"

ip address 172.22.136.2 255.255.255.0

!

interface Serial1/3:1

description "#############"

ip address 172.22.156.2 255.255.255.0

!

interface Serial1/4:1

description "#############"

ip address 172.22.146.2 255.255.255.0

!

interface Serial1/5:1

description "#############"

ip address 172.22.131.2 255.255.255.0

!

interface Serial1/6:1

description "#############"

ip address 172.22.151.2 255.255.255.0

!

router eigrp 200

network 172.22.0.0

no auto-summary

no eigrp log-neighbor-changes

!

ip classless

ip route 0.0.0.0 0.0.0.0 172.22.1.1

no ip http server

!

gatekeeper

shutdown

!

line con 0

password 7 #############

login

line aux 0

password 7 #############

login

line vty 0 4

password 7 #############

login

!

end

msingla · ‎12-19-2002

Hi,

Thanx for sending the configuration.I analyzed everthing nothing is faulty.Can you tell which dhcp server u are using most probably i think windows and please do write the source address u are getting of DHCP discover messages in sniffer and moreover the other any traces from sniffer.

Lets try all together where the issue is.

The other way to test is implement the dhcp on cisco IOS temporarily on l3 switch and check now....most probably no routing issues as everything works fine and also as everybody suggested please remove the ip helper command on the dhcp server side as it is no need.

Regards,

Munit

bleucube · ‎12-19-2002

Thanks for your help!

Our DNS/DHCP runs on Netware 5.1 service pack5 with all the updates thus far, our DHCP server version is 3.1.2c with TCP/IP 5.81o

Here is a copy of the DHCP server debug, its from 172.22.1.50 trying to send the DHCPOFFER to one of the remote sites we are having an issue with - this is after the DHCPDISCOVER hits the DHCP server. It never receives the DHCPACK from the workstation so it keeps sending out the OFFER - finally the workstation saids I GIVE UP and gets the Microsoft IP

2002/12/04 13:48:13 packet received from client <0:50:4:C2:68:AF>.

2002/12/04 13:48:13 Sending BOOTP/DHCP reply to <0:50:4:C2:68:AF> as <172.22.56.1>

85 : Get type:4, IPAddr: 172.22.56.11,

LeaseTime:0,MacIndx:3476,pIP=AC16380B

Using existing IP with subnet AC163800

DetermineLeaseTime: proposed=0, return=86400, pSubnet-

>leaseTime=86400

AMAGet() exit type=4, err=0, addr=172.22.56.11

2002/12/04 13:48:17 packet received from client <0:50:4:C2:68:AF>.

2002/12/04 13:48:17 Sending BOOTP/DHCP reply to <0:50:4:C2:68:AF> as <172.22.56.1>.

92 : Get type:4, IPAddr: 172.22.56.11,

LeaseTime:0,MacIndx:3476,pIP=AC16380B

Using existing IP with subnet AC163800

DetermineLeaseTime: proposed=0, return=86400, pSubnet-

>leaseTime=86400

AMAGet() exit type=4, err=0, addr=172.22.56.11

2002/12/04 13:48:24 packet received from client <0:50:4:C2:68:AF>.

2002/12/04 13:48:24 Sending BOOTP/DHCP reply to <0:50:4:C2:68:AF> as <172.22.56.1>.

105 : Get type:2, IPAddr: 169.254.190.37,

LeaseTime:0,MacIndx:3476,pIP=AC16380B

AMAGet() exit type=2, err=30, addr=169.254.190.37

2002/12/04 13:48:37 packet received from client <0:50:4:C2:68:AF>, IP Address <169.254.190.37>.

107 : Get type:2, IPAddr: 169.254.190.37,

LeaseTime:0,MacIndx:3476,pIP=AC16380B

AMAGet() exit type=2, err=30, addr=169.254.190.37

2002/12/04 13:48:39 packet received from client <0:50:4:C2:68:AF>, IP Address <169.254.190.37>.

129 : Get type:4, IPAddr: 172.22.56.11,

LeaseTime:0,MacIndx:3476,pIP=AC16380B

Using existing IP with subnet AC163800

DetermineLeaseTime: proposed=0, return=86400, pSubnet-

>leaseTime=86400

AMAGet() exit type=4, err=0, addr=172.22.56.11

2002/12/04 13:49:02 packet received from client <0:50:4:C2:68:AF>.

2002/12/04 13:49:02 Sending BOOTP/DHCP reply to <0:50:4:C2:68:AF> as <172.22.56.1>.

133 : Get type:4, IPAddr: 172.22.56.11,

LeaseTime:0,MacIndx:3476,pIP=AC16380B

Using existing IP with subnet AC163800

DetermineLeaseTime: proposed=0, return=86400, pSubnet-

>leaseTime=86400

AMAGet() exit type=4, err=0, addr=172.22.56.11

This is what the SHOW DEBUG IP UDP on the 1706 at the remote end *172.22.51.1 sending to the DCHP Server *172.22.1.50

UDP: rcvd src=0.0.0.0(68), dst=255.255.255.255(67), length=308

UDP: sent src=172.22.51.1(67), dst=172.22.1.50(67), length=328

UDP: rcvd src=0.0.0.0(68), dst=255.255.255.255(67), length=308

UDP: sent src=172.22.51.1(67), dst=172.22.1.50(67), length=328

UDP: rcvd src=0.0.0.0(68), dst=255.255.255.255(67), length=308

UDP: sent src=172.22.51.1(67), dst=172.22.1.50(67), length=328

In one that works it shows a src of 172.22.50, dst:172.22.51.1

I will need to upload sniffer log tomorrow.