Endpoints getting IP from switchport VLAN before ISE changes VLAN

[Josh Morris]

I just migrated from ISE 2.2 to ISE 3.1 and I am having an issue now where a lot of my endpoints come online on the switch, and ISE puts them in the correct VLAN, but by that point, the endpoint has already received an IP address from the VLAN on the switchport. And these devices (mainly security cameras) are not smart enough to refresh their IP. I'm assuming this is just a timing issue because I'm running 'authentication open' on my switchports. I guess the device is coming up on the port and is getting an IP address on the VLAN before ISE can respond with the correct VLAN. 

The thing that is weird to me is that this was not an issue in my previous 2.2 deployment. Here is an example of a switchport. The issue in this case is that the endpoint would come up with an IP out of VLAN 58, but ISE then changes the port to VLAN 1001 and the endpoint can't change IP address. 

Also, these are not 802.1X endpoints, they are simply MAB in a static group in ISE. 

interface GigabitEthernet1/47
 switchport access vlan 58
 switchport mode access
 switchport voice vlan 90
 ip device tracking maximum 10
 logging event link-status
 authentication control-direction in
 authentication event fail action next-method
 authentication event server dead action authorize vlan 58
 authentication event server dead action authorize voice
 authentication event server alive action reinitialize 
 authentication host-mode multi-auth
 authentication open
 authentication order mab dot1x
 authentication priority dot1x mab
 authentication port-control auto
 authentication periodic
 authentication timer reauthenticate server
 authentication violation restrict
 mab
 dot1x pae authenticator
 dot1x timeout quiet-period 300
 dot1x timeout tx-period 7
 dot1x timeout ratelimit-period 300
 dot1x timeout held-period 300
 service-policy input QoS-Input-Policy
 service-policy output QoS-Host-Port-Output-Policy
end

If you think this is the issue, is there a way for me to fix this without removing authentication open? I'm not ready to go to closed mode yet. 

[ahollifield · Accepted solution]

It's best practice not to change VLANs for the exact issues you describe above.  What is your use-case?  Why not use another form of enforcement such as dACL or TrustSec?

If you MUST change VLANs then use a port-bounce CoA and/or really short DHCP leases.

[Josh Morris]

Use case is segmentation at a higher layer (ex: vlan in a vrf behind a firewall). I am using DACL/SGT for lateral segmentation however. 

[ahollifield]

 Then yeah short DHCP lease times or port-bounce CoA.

[Josh Morris]

Thanks, you mean port bounce globally or on profile? Unfortunately the majority of my cameras are in a static group and I dont know how to cause a port bounce based on an authorization profile. 

[Greg Gibbs · Accepted solution]

As @ahollifield was saying, this is a common problem with dynamic VLAN assignment. You can configure a port-bounce CoA based on a custom Profiling Policy, but I don't believe port-bounce CoA will resolve the issue as that will just start the authentication process over again and likely just end up in a never-ending loop.

The two main methods to resolve this DHCP issue are typically:

1. Use a super aggressive (like 10 seconds) DHCP lease timer on the starting VLAN. This is not possible with a Windows DHCP server, so I've only see this done at Universities that use Unix-based DHCP servers.

2. Block the initial DHCP requests. You could do this with a Firewall/ACL rule upstream, remove any dhcp helper entries, or configure a pre-auth ACL on the switchport (considered Low Impact Mode) that blocks DHCP but allows all other traffic. Some endpoints with old or basic network stacks, however, can be sensitive to DHCP timeouts and could stop requesting DHCP after a few timeouts, so you could run into issues.

[Josh Morris]

Thanks, this keeps getting better! I have seen guides on this, but considering the scenario of low-impact mode with a pre-auth ACL blocking DHCP vs Closed mode...I have profiles that rely on certain DHCP parameters to make decisions. What happens to those? I'm guessing that in the low-impact scenario, the DHCP request still makes it to the switchport and gets picked up by device sensor and sent to ISE? In closed mode, I wouldn't expect this to happen at all, so I don't know how profiling even takes place.

[ahollifield]

It doesn't your "base condition" for closed-mode when you're using profiling is typically a dACL that permits nothing but DHCP.  

[Josh Morris]

By allowing DHCP in closed mode, wouldn't I be back in the same predicament where endpoints would get an IP from the statically set VLAN then potentially have to change IPs based on the dynamic VLAN?

[ahollifield]

Yes, this is precisely why you shouldn’t change VLANs at all. If your dACL allows DHCP only for example, why does it matter what VLAN the device is on?

[Josh Morris]

Yeah, I hear you. I am going to reevaluate some things based on this issue. In the meantime, I did add the port bounce CoA action to this particular profile, and so far, it's working as expected. 

[Rob R.]

Hi Josh, 

When you "add the port bounce CoA action" was that a change within ISE? 

[Josh Morris]

In this case, I added it to the profile for the device type. As seen below.

[Rob R.]

I've been trying to get this to work in my environment for several days but the cameras will not pull a new DHCP from the second VLAN. I've updated the CoA Type to Port Bounce as shown and still no luck. Thanks for sharing though, this has been interesting to say the least.. 

[ianbirchall]

One way that I was able to overcome this in one environment was to have the access vlan be configured as a "blackhole/Road to nowhere" vlan. Where there was no DHCP request.
Every switchport was configured on access port 66, and wouldnt be able to eve get an IP address until ISE have dynamically changed the VLAN to the correct one.