How to use Vlans and DHCP Pools Configured on Another, Directly Connected Switch?

[Matthew Martin]

Hello All,

Core Switch: WS-C4510R+E
Secondary Attached Switch: WS-C3560-48PS-S

We have our Core switch, i.e. the 4510, which has all the Vlans already configured on it. Both the 4510 Core Switch and the 3560 switch are both directly connected to one another via Trunk ports, like so:

********** 4510 Trunk Port **********
4510R-HQ# show run int Gi9/22
Building configuration...

Current configuration : 230 bytes
!
interface GigabitEthernet9/22
 description IT Room Switch 
 switchport mode trunk
 auto qos voip cisco-phone 
 service-policy input AutoQos-4.0-Cisco-Phone-Input-Policy
 service-policy output AutoQos-4.0-Output-Policy
end

********** 3560 Trunk Port **********
3560sw1-IT# show run int Fa0/1
Building configuration...

Current configuration : 149 bytes
!
interface FastEthernet0/1
 description uplink to 4510R+E
 switchport trunk encapsulation dot1q
 switchport mode trunk
 ip dhcp snooping trust
end

*FYI: On the 4510's Trunk port, which now has a switch on it, was previously configured as a regular User workstation port which is why the auto-qos stuff is on there, and I figured since we will probably have cisco ip phones connected to the 3560 so they can be configured, it wouldn't hurt to leave it on there...

Also, the 3560 is now configured so that connected devices can be authenticated via our Cisco ISE server, which is why the 3560's trunk port has the dhcp snooping command configured on it, which is for device Profiling purposes.

On the 4510, the Vlans I need access to are the User Access Vlan, vlan114. As well as the Voice Vlan, vlan124. From one of the guides I found on cisco.com for configuring InterVLAN Routing seemed like you only really needed to include:

3560sw1-SP(config)#vlan 114
3560sw1-SP(config-vlan)#name Access
3560sw1-SP(config-vlan)#exit
3560sw1-SP(config)#
3560sw1-SP(config)#vlan 124   
3560sw1-SP(config-vlan)#name Voice 
3560sw1-SP(config-vlan)#exit

However, PCs and IP Phones connected to the Switchports on the 3560 aren't getting IP Addresses.

Below is an example of how I configured the Switchports on the 3560 that will be used as User Workstation ports (*i.e. for IP Phones and/or PCs). And like I had mentioned earlier, the 3560 switch is already configured to Authenticate devices through the Cisco ISE server, and I can see on the ISE server and on the switch (*via "show auth sess...") that the connected devices are authenticating properly, they just aren't getting IP Addresses.

3560sw1-SP#show run int Fa0/3
Building configuration...

Current configuration : 671 bytes
!
interface FastEthernet0/3
 switchport access vlan 114
 switchport mode access
 switchport voice vlan 124
 authentication event fail action next-method
 authentication event server dead action authorize vlan 114
 authentication event server dead action authorize voice
 authentication event server alive action reinitialize 
 authentication host-mode multi-auth
 authentication order dot1x mab
 authentication priority dot1x mab
 authentication port-control auto
 authentication violation restrict
 mab
 snmp trap mac-notification change added
 snmp trap mac-notification change removed
 dot1x pae authenticator
 dot1x timeout tx-period 10
 spanning-tree portfast
end

I'm sure there is something I am missing to allow devices connected via the 3560 switch to use the Access and Voice Vlans that are already configured on the 4510... So if anyone has ANY thoughts or suggestions, it would be greatly appreciated..! Or if you need anymore info from me just let me know and I'll reply back with that info, Thanks!

Thanks in Advance,
Matt

[Matthew Martin]

Hey Jon,

Yea I agree... After reading over that guide that I mentioned in the comment I just posted, it was pretty much the same steps that everyone who replied had said to do, which was only a couple of very simple steps... So I'm stumped as to why it isn't working when configured that way. Do you think having "ip routing" command on the 3560 is causing the problem?

Yes, the 4510 has ACLs configured on it but nothing is applied to the interface connecting to the 3560. The ACLs configured on the 4510 are only applied when the Cisco ISE server sends the Authorization profile to the Authenticated device, and then again that only happens when the port is configured to authenticate via Radius/AAA server.

I'm almost thinking of just resetting the 3560 back to factory default config and starting from scratch, because most of what's on the 3560 was already there when my boss gave it to me...

UPDATE:
Just saw your most recent post... Will comment back shortly.

-Matt

[Matthew Martin · Accepted solution]

Sorry for the consecutive posts, but I have some new info...

Ok, so I tried following the guide at the link below, which seems pretty much what I want to achieve. Where you have an Access switch (*the 3560 in my case) connected to a Layer-3 switch (*the 4510). The Layer-3 switch has all of the Vlan interfaces configured on it, and then devices connecting to the Access switch should get IP Addresses via those Vlans on the Layer-3 switch. The guide pretty much states to do exactly what you guys have suggested to do, and which I've done already...

InterVLAN Routing - Layer-3 and Layer-2 Switch Configuration

Basically, the steps were to create trunk ports on both switches for the ports connecting them together, and then enable dot1q encapsulation on those trunk ports.
Then, on the 3560 run these commands to create the matching Vlans on the 3560 by entering the following:

# conf t
(config)# vlan 114
(config-if)# exit
(config)#
(config)# vlan 124
(config-if)# end

Then, assign the "switchport access vlan <vlan-#>" to the switchports for the PCs. After they do that in the guide, they create the actual Vlan Interfaces (*SVIs) on the Layer-3 switch, which for me is the 4510. So, in my case, these SVIs were already created... I'm not positive, but I didn't think this should cause an issue by the order in which these things are created, I assume it doesn't, but just want to make sure...

######################################################

After I did those steps above again and nothing changed, I decided to try a different approach...

On the 3560 I decided to try and create the SVI "interface Vlan114", which I did. Then, I assigned it an IP Address in that subnet that is configured as an excluded address in the DHCP Pool. And the moment I clicked enter after issuing the "ip address 10.20.114.14 255.255.255.0" the DHCP debugging commands that I had enabled started spitting out lots of lines in the terminal monitor, like these below:

Mar  1 13:31:12.438 EST: DHCPD: Reload workspace interface Vlan114 tableid 0.
Mar  1 13:31:12.438 EST: DHCPD: tableid for 10.60.114.14 on Vlan114 is 0
Mar  1 13:31:12.438 EST: DHCPD: client's VPN is .
Mar  1 13:31:12.438 EST: DHCPD: using received relay info.
Mar  1 13:31:14.493 EST: DHCPD: Reload workspace interface Vlan114 tableid 0.
Mar  1 13:31:14.493 EST: DHCPD: tableid for 10.60.114.14 on Vlan114 is 0
Mar  1 13:31:14.493 EST: DHCPD: client's VPN is .
Mar  1 13:31:14.493 EST: DHCPD: Finding a relay for client 0100.249b.1008.53 on interface Vlan114.
Mar  1 13:31:18.100 EST: DHCPD: Reload workspace interface Vlan114 tableid 0.
Mar  1 13:31:18.100 EST: DHCPD: tableid for 10.60.114.14 on Vlan114 is 0
Mar  1 13:31:18.100 EST: DHCPD: client's VPN is .
Mar  1 13:31:18.100 EST: DHCPD: Finding a relay for client 0100.1d09.1382.6e on interface Vlan114.
Mar  1 13:31:22.949 EST: DHCPD: Reload workspace interface Vlan114 tableid 0.
Mar  1 13:31:22.949 EST: DHCPD: tableid for 10.60.114.14 on Vlan114 is 0
Mar  1 13:31:22.949 EST: DHCPD: client's VPN is .
Mar  1 13:31:22.949 EST: DHCPD: using received relay info.

These lines above continued printing over and over for all different mac addresses, which I'm not sure where exactly those mac addresses are coming from...

But, even after that my connected laptop was still NOT getting an IP Address. So I added this line below, which is the same line configured in Vlan114 on the 4510:

ip helper-address [dhcp-server-IP]

to the SVI for Vlan114 on the 3560 and now my laptop can successfully get an IP Address... However, since it sounds like from that guide above and from what you guys stated, this should be working without me needing to create the SVI on the 3560, so is this just basically a hack/workaround for the problem I'm having?

Not sure what else to try since it seems like this "should" be working the way I had it configured before I added the SVI for Vlan114 to the 3560...?

Thanks Again,
Matt

[Jon Marshall]

Oops, accidentally endorsed your post :-)

Right, this just gets stranger.

Can you post the full configuration of the 3560, add it as an attachment to keep the thread space down.

And as a further test. I understand you said the 3560 must have routing enabled for ISE but could you temporarily remove the SVI you created, disable routing and then retest.

This is a weird one :)

Jon

[Matthew Martin]

Hey Jon, sorry for the delay, had a bit of a computer crash and just got it back up again...

Ok, so I disabled IP Routing on the switch and then tried to disconnect and reconnect the test laptop I have connected to the 3560, and it seemed to keep the same IP Address that it got via DHCP after I had added the helper-address to the DHCP server on the SVI for Vlan114. Even after I did a ipconfig /release & /renew, so I had thought it was working... But, I just got back to the laptop after it had gone to sleep and upon coming back up, its not getting an IP Address now... Darn, thought removing ip routing fixed it, guess not...!

Question regarding the ip routing command. I thought you needed to have ip routing enabled in order to assign IP Addresses to any interfaces, including Vlans, is that correct? Because I am still able to telnet to the 3560's Vlan1 IP Address, even after disabling ip routing. The only difference I am seeing is that after removing ip routing (*with "no ip routing"), any interface that had an IP Address assigned to it got the command "no ip route-cache" added to it. What does that command do exactly?

I have attached the Running-config of the 3560. I wasn't sure if you wanted the config from before or after removing ip routing, so I just attached the config from Before removing it. If you want the other too just let me know and I'll attach it.

*Config FYI: In the config I have my test laptop connected to Fa0/31 and the test IP Phone connected to Fa0/29

***EDIT*** Do you think I need to have some kind of DHCP relay configured..? I believe DHCP relay is for any switch that forwards dhcp packets to an actual DHCP server...? Never used the realy commands before so I wasn't sure if this was needed.

Thanks Again,
Matt

[Jon Marshall]

Matt

If the switch is L2 only ie.routing is not enabled then you usually have just one SVI with an IP address and this is for management of the switch ie. so you can connect to it. You also usually set a default gateway as well.

Only if the switch is L3 do you have multiple SVIs with IP addresses.

The  "no ip route-cache" command tells the switch to process switch the packets ie. send them to the main CPU not sure why that was added but it should not matter as the SVIs won't be in use.

I notice from the attached configuration you are using a L3 port to connect to an ISR in which case you would need routing enabled.

You should not need to configure any type of DHCP relay, this is simply a DHCP broadcast from the client which should be seen by the 4500 and forwarded to the DHCP server.

I think it may be an idea to simply start from scratch in terms of configuration on the 3560, you have a copy of it so you can always copy and paste bits back in if we can get it working.

Jon

[Matthew Martin]

Thanks for the reply Jon, and the explanations. Much appreciated!

Yea, I'm thinking the same thing, about starting over from scratch, which I'll get started on in the morning.... It's been a long day today!

*One last thing before I start this config over...

Do you think it could have anything to do with the IP Routes configured on the 4510, at least the ones configured to route to the 3560? Or, maybe I'm MISSING a route that is needed to perhaps receive the reply from the 4510 or something along those lines?
Also, are there any debugging commands that could help diagnose where the trouble is? It seems like the ones I have configured, which I showed in a previous comment, don't really show much except for the line, "DHCPD: no option 125" which keeps showing up in the log, as well as all those lines that printed once I created the SVI for Vlan114 and added the helper-address (*also show in a previous post).

The IP Routes related to the 3560 that are configured on the 4510 include:

!!!These Routes are for the Test ISR4321 (Connected to Fa0/2)!!!
ip route 10.19.0.0 255.255.0.0 192.168.3.3
ip route 10.241.241.0 255.255.255.0 192.168.3.3

And to explain the need for the ip routing, which like you said, because of the ISR on Fa0/2. This is because I connect new Network devices (*switches, routers, etc...) to Fa0/2, which are usually for one of our remote branch offices. So this was sort of to try and mimic this setup as if it were a remote office location, which would require ip routing.

But either way, it sounds like your saying; Ip routing or NO ip routing, the DHCP stuff should be working whether that's configured or not. RIght?

Thanks Again,
Matt

[Jon Marshall]

Matt

Should have nothing to do with the routes. Basically this all happens at L2 from the 3560 to the 4500 and it is only when it gets to the L3 SVI on the 4500 that it is then routed to the DHCP server. 

So you can have the switch acting as L3 to connect to the ISR and still have clients getting DHCP from the 4500 although it's not clear exactly how the routing is going to work ie. if the 4500 handles all the local vlans then how are you going to advertise those routes to the 3560 so it can advertise them to the ISR because the 3560 is connected via a trunk not a L3 connection to the 4500.

I am perhaps not understanding your topology correctly.

In terms of debugging it is basically DHCP debugging you need, does the DHCP server allow any debugging because it would help to know whether the DHCP request is actually getting to the server or not.

I'll have another read of this thread in case I have missed anything and let you know if I spot anything.

Jon

[Matthew Martin]

Hey Jon, thanks for the info! Sorry, still getting familiar with Layer-2/Layer-3 interface differences, Vlans, SVIs, etc...

So I went onto the Linux DHCP server and checked out the log file for DHCPD and I was able to find the laptop's MAC Address. However, it looks like the only time my MAC shows up was when I had created the Vlan interface and had the helper-address in there. It shows up with the IP Address that I received when that Vlan interface was configured on the 3560. And just to be sure I also checked the backup DHCP server's log, but there wasn't any requests in there from the test laptop.

So I'm not seeing any DHCP messages from the Vlan interface of the 4510 for my test Laptop's MAC Address, only from when I had the Vlan configured on the 3560.

Also, I'm seeing DHCP messages from the Vlan IP Address I had configured on the 3560, but the requests are for a few different MAC Addresses that are NOT connected to the 3560, they're actually connected to the 4510. In the log it shows DHCP requests from BOTH the 4510 and the 3560's Vlan interfaces, see below:

4510 Vlan114 IPAddr = 10.20.114.1
3560 Vlan114 IPAddr = 10.20.114.14

*FYI, I checked on the 4510 and this MAC Address that shows in the log below is for a Printer that is directly connected to the 4510, which is configured with a static IP Address...

Mar  1 13:39:08 dhcp-hq dhcpd: DHCPDISCOVER from 8c:dc:d4:yy:xx:d3 via 10.20.114.1
Mar  1 13:39:08 dhcp-hq dhcpd: DHCPOFFER on 10.20.114.231 to 8c:dc:d4:yy:xx:d3 via 10.20.114.1
Mar  1 13:39:08 dhcp-hq dhcpd: DHCPDISCOVER from 8c:dc:d4:yy:xx:d3 via 10.20.114.14
Mar  1 13:39:08 dhcp-hq dhcpd: DHCPOFFER on 10.20.114.231 to 8c:dc:d4:yy:xx:d3 via 10.20.114.14
Mar  1 13:39:08 dhcp-hq dhcpd: DHCPREQUEST for 10.20.114.231 (192.168.2.25) from 8c:dc:d4:yy:xx:d3 via 10.20.114.14
Mar  1 13:39:08 dhcp-hq dhcpd: DHCPACK on 10.20.114.231 to 8c:dc:d4:yy:xx:d3 via 10.20.114.14
Mar  1 13:39:08 dhcp-hq dhcpd: DHCPDECLINE of 10.20.114.231 from 8c:dc:d4:yy:xx:d3 via 10.20.114.14: not found
Mar  1 13:39:08 dhcp-hq dhcpd: DHCPREQUEST for 10.20.114.231 (192.168.2.25) from 8c:dc:d4:yy:xx:d3 via 10.20.114.1
Mar  1 13:39:08 dhcp-hq dhcpd: DHCPACK on 10.20.114.231 to 8c:dc:d4:yy:xx:d3 via 10.20.114.1
Mar  1 13:39:08 dhcp-hq dhcpd: DHCPDECLINE of 10.20.114.231 from 8c:dc:d4:yy:xx:d3 via 10.20.114.1: not found

*Question... Is it a bad idea to have a matching SVI (*i.e. Vlan interface) configured on the 3560, which is also configured on the 4510? I'm just wondering because if that works, and it's not bad practice to do so, then maybe I could just go that route instead... I'm just worried maybe there could be issues seeing that there are other, non-connected devices which are somehow using Vlan114 on the 3560 for DHCP.

Thanks again for all your help, much appreciated!

Thanks,
Matt

[Matthew Martin]

Think I found the problem!!

I was just reading an article I found at the link below, which talks about Layer-2 port security. Below is a snippet from this URL. I've highlighted the important part in red:

http://www.cisco.com/c/en/us/support/docs/switches/catalyst-3750-series-switches/72846-layer2-secftrs-catl3fixed.html

DHCP Snooping

DHCP snooping acts like a firewall between untrusted hosts and DHCP servers. You use DHCP snooping to differentiate between untrusted interfaces connected to the end user and trusted interfaces connected to the DHCP server or another switch. When a switch receives a packet on an untrusted interface and the interface belongs to a VLAN that has DHCP snooping enabled, the switch compares the source MAC address and the DHCP client hardware address. If the addresses match (the default), the switch forwards the packet. If the addresses do not match, the switch drops the packet. The switch drops a DHCP packet when one of these situations occurs: .........

So after reading that I decided to remove any/all dhcp snooping commands from the 3560 and almost immediately the laptop was acquiring an IP Address. So I just double checked the log on the DHCP server and found the following, which has a source address of Vlan114 on the 4510. So it looks like the DHCP packets are now reaching the 4510:

Mar  2 12:42:44 dhcp-hq dhcpd: DHCPDISCOVER from 78:2b:cb:e6:xx:yy via 10.20.114.1
Mar  2 12:42:45 dhcp-hq dhcpd: DHCPOFFER on 10.20.114.33 to 78:2b:cb:e6:xx:yy (MATT-Test-PC) via 10.20.114.1
Mar  2 12:42:45 dhcp-hq dhcpd: DHCPREQUEST for 10.20.114.33 (192.168.2.25) from 78:2b:cb:e6:xx:yy (MATT-Test-PC) via 10.20.114.1
Mar  2 12:42:45 dhcp-hq dhcpd: DHCPACK on 10.20.114.33 to 78:2b:cb:e6:xx:yy (MATT-Test-PC) via 10.20.114.1
Mar  2 12:42:48 dhcp-hq dhcpd: DHCPINFORM from 10.20.114.33 via 10.20.114.1
Mar  2 12:42:48 dhcp-hq dhcpd: DHCPACK to 10.20.114.33 (78:2b:cb:e6:xx:yy) via eth1

So, I'm confused why these messages were being dropped because of the DHCP snooping since I had the "ip dhcp snooping trust" on the Trunk port. But, maybe I misunderstood that part and that command would need to be on any port where a DHCPREQUEST/DHCPDISCOVER packet would come from, *i.e. like a User workstation switchport.

I guess I'll try re-enabling dhcp snooping on the 3560 and then add the trust command to the test switchports and see if it works that way. I know we never setup dhcp snooping on the 4510, so this switch was sort of a guinea pig when it comes to that. If you're wondering why I was using DHCP snooping, it was because in the Cisco ISE Admin Guide, the chapter on configuring your Network Access Devices (*i.e. your Routers and Switches) says to enable dhcp snooping for the Vlans configured for ISE to help with device tracking and IP substitution in dynamic ACLs on switchports. However, the official ISE Admin guide doesn't show the "ip dhcp snooping trust" command. I believe I actually found that on another site that had its own switch configuration guide for Cisco ISE integration.

Here are the commands I removed from the switch just before it started working:

Global Commands:
ip dhcp snooping vlan 114,124
ip dhcp snooping

Command Removed from Trunk Port:
ip dhcp snooping trust

I'll post back if I find out why the dhcp snooping was dropping the packet... But, I'm assuming I must have configured it wrong, or maybe the trust command was NOT even needed in the first place since it's not in the "offical" guide... We'll see.

Thanks Again,
Matt

[Jon Marshall]

Matt

Sounds promising and good find.

By all means post back and if you have more queries etc. please feel free to ask.

Jon

[Jon Marshall]

Matt

Can you also post "sh vlan" from the 4500 as well.

Jon

[Matthew Martin]

I think we can mark this thread as solved...

After a bit more testing it looks like DHCP Snooping was the issue for sure.

According to this TrustSec Switch Configuration Guide located at this link, it states that you need to add the "ip dhcp snooping trust" command to the "uplink interface port" where the DHCP server is located. And, for me, the "Uplink" port would be Fa0/1, which goes to the 4510 where the DHCP server is connected. This is shown in the screenshot I inserted below, see the 2nd paragraph that starts with "Before configuring DHCP snooping..."

Wouldn't you consider Fa0/1 to be the Uplink port to the DHCP server..?

---

---

However, I have the trust command on the Trunk port that goes to the 4510. Then, I connected a new PC device to one of the FastEthernet ports and it did NOT get an IP Address. So I then added the trust command to that specific port where the new device was connected, detached and reattached the PC to the port in order reattempt to get an IP Address, and this time it successfully got a DHCP address.

So I'm not sure where exactly the issue is here, unless something has changed and it is required on every switchport, but if that's the case that seems like sort of a pain...

I think I'm going to create a new post under the Network Security section and see if I can find out what the deal is with this DHCP Snooping stuff and what actually needs to be considered as "trusted", i.e. which ports, and go from there...

Thanks again for your help and taking the time to assist, very much appreciated!

Thanks Again,
Matt