Cisco 300 - VLAN DHCP packets not passing

[Tim Butters]

I am seeing a problem with our Cisco 300 switches. We use these switches as access switches, with a stacked 3750-G at the core, two 2960-S at the distribution layer, and about 10 300 Series switches at the access layer (10 port and 28 ports, all PoE).

We use Voice VLAN (VLAN 14) for our Mitel phones – there is a DHCP server on the Mitel system. Phones come up, get tagged VLAN 14 (LLDP), Traffic flows (including Broadcast for DHCP etc…). The system works, and has worked for months.

One day, suddenly, I find that all the Mitel phones on a particular access switch are not working. I look on the Mitel system and the lease on DHCP has expired, and the phone is stuck on renewing its DHCP IP address. I run port mirroring on the switch for VLAN 14 to see what is happening. The phones are stuck on DHCP discover, and I see the DHCP Discover broadcast packets on the switch but nothing else, no DHCP offer packets – hence the phone stuck at boot cycle.

I then do a port mirror from another access switch (that is currently working) – I can see the broadcast packets from the Mitel phones on the broken switch, but on this switch I can also see the DHCP offer packets from the Mitel system. I run two port mirrors simultaneous from the two switches (one working, one not) and I can see that the DHCP offer packets are not coming through to the broken switch. Panic ensues – I look at the distribution layer and there is no problem what so ever.

For some strange reason, the Cisco 300 28 port has stopped passing DHCP broadcast packets on a particular VLAN, even though they are being sent. I power cycle the switch – and hey presto, DHCP offer packets are coming through, and the phones get an IP address and boot properly.

I wipe the sweat off my brow, note the issue down and carry on my daily duties.

Forward a couple of weeks later, and to today. I have another phone that is showing the same symptoms, luckily it is the only phone on this particular Cisco 300 28 port. The same issue is occurring as described above. I gather as much diagnostic information I can then reboot the switch – but still no joy. I then remember that this switch is not directly attached to the distribution layer and instead gets trunked to another Cisco 300 28 port. I give that a reboot and 5 minutes later, DHCP broadcast offers are passing and the phone boots.

I am listing this problem as not just a ‘one off’ now, and is recurring. It has happened to two of my 300 28 port switches.

All Switches running 1.1.2.0.
No link to up time – first instance of the problem, switch was up for 14 days – second instance (another switch) uptime of 39 days
LLDP is working fine on the switches, as is Voice-VLAN (Port is tagged and broadcasts out DHCP Discover which is seen by other devices throughout network)
Nothing in the log file on the access switch
Nothing on the Dist/Core regarding STP – Spanning tree set up is fine throughout

Has anyone else experienced same? I’m hoping this is a bug that is getting fixed.

Many Thanks

Tim

[Tom Watts]

Hello Tim

Brayton Hackworth had a similar post as yours, found here;

https://supportforums.cisco.com/message/3684179#3684179

Brayton is using the Mitel 5330 phones where the LLDP no longer fed VLAN information to the phone network. But, he reverted to use a DHCP server to provide the VLAN information.

Unfortunately, I (personally) cannot test any Mitel resources (since I don't have them) so my labs usually only consist of either 7900 series or SPA500/900 series phones.

The best thing I can really recommend to you is to make a package of data consisting of;

Topology which consists of;

• Modem type
• Router including IP
• All devices including IP of switches
• Servers / relevant workstations
• # of attached devices and # of users on the LAN

Switch config file + show tech on a notepad

Syslog output from the switch

Working PCAP

PCAP showing failure

PCAP legend showing what IP address are who (unless topology contains all IP)

Where the PCAP is taken from and method

We then can create a service request for you and pass it for review.

-Tom

[Tim Butters]

Thomas,

Thanks for getting back to me.

Because this is a very intermittent issue I am going to find it hard to capture data in a timely manner. I managed to get a capture or two when it happened yesterday. I have:

A capture from the switch the phone was plugged into, showing the DHCP Discover packets but no DHCP offer broadcasts not being received whilst at the same time a capture from another 300 switch where the same DHCP offers are seen from the phone but the DHCP offers are also seen.

I then have packet captures after the switch (and the intermediate 300-28 port) had been restarted, showing normal behaviour with the DHCP offer packets being seen and the phone booting.

I have a network diagram detailing the above scenario and the config from the switch itself. I don't have a show tech or an exportable log file from the switch.

Because there is sensitive information in this case, can we create a Service Request and send this information through securely?

Thanks

Tim

[Tom Watts]

Send me a PM with your phone number. I will call you. Also if it is possible, can you please obtain the show tech and a show log ?

-Tom