04-12-2013 07:55 AM - edited 03-07-2019 12:47 PM
Hi
I have a fault where client pcs attached to HIPT phones (not cisco) cannot participate on the network if they are disconnected from the phone and then reconnected.
Workstations that are left in situ dont experience the problem, however laptops do: but not in every case....
Once the fault has been exhibited we have found that we can connect any device (workstation/laptop/printer) to the phone and the fault will remain until the phone is restarted (remotely by the telephony provider, physically by the user removing and reseating the network cable or by us shutting/no shutting the switchport or removing and reapplying ILP to the port).
The PC will have nic lights but will receive mesage "unidentified network detected". The PCs are locked down by GPO so users have limited ability to assist with remote troubleshooting.
I have seen the issue on c3560v2-48ps and c3570v2-48ps ipbasek9 12.2(53)SE
I can see the PC MAC address on the port (aswell as the phone), the client usually has a DHCP lease and can even have an arp entry on the core.
Captures (SPAN and from the affected PC) show the PC initially sending DHCP requests, failing over to an APIPA address and then sending DHCP Discovers onto the network. However no responses are seen from the server in the captures. The phone works fine in the voice vlan and can make and receive calls all ok. Its as if the client vlan is operating in a uni-directional manner. I have stripped the config on the switch interface off line by line and the fault will still be present with only;
switchport mode access
switchport access vlan x
switchport voice vlan y
speed 100
duplex full
I have tried changing the access vlan but the fault persists.
I have verified that the server does send DHCP ACKS and OFFERS back to the client in the correct vlan, DHCP SNOOPING debugs (we dont have IP DHCP SNOOPING enabled but I have enabled it for testing to gain more information) show that the switch knows that which vlan and egress interface the client PC is attached to and that it is forwarding the packet onwards. However they arent getting onto the wire between the phone and the switch. In fact I cant see any captured packets going towards the PC.
The interface counters dont show any output drops or errors, I cant seen any drops in mls qos statistics. The controller output doesnt show anything unusual. I can add port acls to affected interfaces to check for bootps and bootpc but they will only show the ingress packets.
Unless Im fundamentally misinterpreting the output (very possible!) Im at a complete loss to explain where, and why, this traffic is dissapearing.
I have no options to run captures in IOS on these devices. I cant see any commands to show fabric or check backplane. Im not sure where else to look. Next time I will clear all counters on the interface before I start work......
Can anyone assist?
SWITCH#sh mac- int f0/3
Mac Address Table
-------------------------------------------
Vlan Mac Address Type Ports
---- ----------- -------- -----
CLIENT XXXX.XXXX.XXXX STATIC Fa0/3
VOICE yyyy.yyyy.yyyy STATIC Fa0/3
Total Mac Addresses for this criterion: 2
SWITCH#sh int f0/3
FastEthernet0/3 is up, line protocol is up (connected)
Hardware is Fast Ethernet, address is f025.72b0.1e85 (bia f025.72b0.1e85)
MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 100Mb/s, media type is 10/100BaseTX
input flow-control is off, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:01:31, output 00:00:00, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 3000 bits/sec, 4 packets/sec
24182472 packets input, 5314908683 bytes, 0 no buffer
Received 819072 broadcasts (729095 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 729095 multicast, 0 pause input
0 input packets with dribble condition detected
67606534 packets output, 12571462460 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 PAUSE output
0 output buffer failures, 0 output buffers swapped out
Transmit FastEthernet0/3 Receive
3984993305 Bytes 1023196232 Bytes
26979925 Unicast frames 23402864 Unicast frames
26646033 Multicast frames 729128 Multicast frames
14021788 Broadcast frames 90007 Broadcast frames
0 Too old frames 913015998 Unicast bytes
0 Deferred frames 70381604 Multicast bytes
0 MTU exceeded frames 10113214 Broadcast bytes
0 1 collision frames 0 Alignment errors
0 2 collision frames 0 FCS errors
0 3 collision frames 0 Oversize frames
0 4 collision frames 0 Undersize frames
0 5 collision frames 0 Collision fragments
0 6 collision frames
0 7 collision frames 2986936 Minimum size frames
0 8 collision frames 8352962 65 to 127 byte frames
0 9 collision frames 7756773 128 to 255 byte frames
0 10 collision frames 3663553 256 to 511 byte frames
0 11 collision frames 385172 512 to 1023 byte frames
0 12 collision frames 1076603 1024 to 1518 byte frames
0 13 collision frames 0 Overrun frames
0 14 collision frames 0 Pause frames
0 15 collision frames
0 Excessive collisions 0 Symbol error frames
0 Late collisions 0 Invalid frames, too large
0 VLAN discard frames 0 Valid frames, too large
0 Excess defer frames 0 Invalid frames, too small
6732272 64 byte frames 0 Valid frames, too small
42863394 127 byte frames
11262180 255 byte frames 0 Too old frames
2641393 511 byte frames 0 Valid oversize frames
395025 1023 byte frames 0 System FCS error frames
3753482 1518 byte frames 0 RxPortFifoFull drop frame
0 Too large frames
0 Good (1 coll) frames
0 Good (>1 coll) frames
Name: Fa0/3
Switchport: Enabled
Administrative Mode: static access
Operational Mode: static access
Administrative Trunking Encapsulation: negotiate
Operational Trunking Encapsulation: native
Negotiation of Trunking: Off
Access Mode VLAN: X (Client)
Trunking Native Mode VLAN: 1 (default)
Administrative Native VLAN tagging: enabled
Voice VLAN: Y (Voice)
Administrative private-vlan host-association: none
Administrative private-vlan mapping: none
Administrative private-vlan trunk native VLAN: none
Administrative private-vlan trunk Native VLAN tagging: enabled
Administrative private-vlan trunk encapsulation: dot1q
Administrative private-vlan trunk normal VLANs: none
Administrative private-vlan trunk associations: none
Administrative private-vlan trunk mappings: none
Operational private-vlan: none
Trunking VLANs Enabled: ALL
Pruning VLANs Enabled: 2-1001
Capture Mode Disabled
Capture VLANs Allowed: ALL
Protected: false
Unknown unicast blocked: disabled
Unknown multicast blocked: disabled
Appliance trust: none
SWITCH#sh int fastEthernet 0/3 summary
*: interface is up
IHQ: pkts in input hold queue IQD: pkts dropped from input queue
OHQ: pkts in output hold queue OQD: pkts dropped from output queue
RXBS: rx rate (bits/sec) RXPS: rx rate (pkts/sec)
TXBS: tx rate (bits/sec) TXPS: tx rate (pkts/sec)
TRTL: throttle count
Interface IHQ IQD OHQ OQD RXBS RXPS TXBS TXPS TRTL
-------------------------------------------------------------------------
* FastEthernet0/3 0 0 0 0 33000 50 37000 56 0
FastEthernet0/3 (All statistics are in packets)
dscp: incoming
-------------------------------
0 - 4 : 16434978 0 0 0 0
5 - 9 : 0 0 0 0 0
10 - 14 : 0 0 0 0 0
15 - 19 : 0 0 0 2338 0
20 - 24 : 0 0 0 0 0
25 - 29 : 0 187967 0 0 0
30 - 34 : 0 0 0 0 0
35 - 39 : 0 0 0 0 0
40 - 44 : 0 0 0 0 0
45 - 49 : 0 7215587 0 503 0
50 - 54 : 0 0 0 0 2487
55 - 59 : 0 0 0 0 0
60 - 64 : 0 0 0 0
dscp: outgoing
-------------------------------
0 - 4 : 34190940 0 0 0 0
5 - 9 : 0 0 0 0 0
10 - 14 : 0 0 0 0 0
15 - 19 : 0 0 0 23327 0
20 - 24 : 0 0 0 0 0
25 - 29 : 0 176706 0 0 0
30 - 34 : 0 0 0 0 0
35 - 39 : 0 0 0 0 0
40 - 44 : 0 0 0 0 0
45 - 49 : 0 7293648 0 4087 0
50 - 54 : 0 0 0 0 612
55 - 59 : 0 0 0 0 0
60 - 64 : 0 0 0 0
cos: incoming
-------------------------------
0 - 4 : 17156425 0 37064 187967 0
5 - 7 : 7215587 0 0
cos: outgoing
-------------------------------
0 - 4 : 37308601 0 34038 176706 0
5 - 7 : 7293648 4699 58351
output queues enqueued:
queue: threshold1 threshold2 threshold3
-----------------------------------------------
queue 0: 2 0 7295184
queue 1: 0 3203 23715920
queue 2: 0 0 34076
queue 3: 0 0 37404993
output queues dropped:
queue: threshold1 threshold2 threshold3
-----------------------------------------------
queue 0: 0 0 0
queue 1: 0 0 0
queue 2: 0 0 0
queue 3: 0 0 0
Policer: Inprofile: 0 OutofProfile: 0
Mar 4 14:53:04: DHCP_SNOOPING: received new DHCP packet from input interface (FastEthernet0/3)
Mar 4 14:53:04: DHCP_SNOOPING: process new DHCP packet, message type: DHCPDISCOVER, input interface: Fa0/3, MAC da: ffff.ffff.ffff, MAC sa: XXXX.XXXX.XXXX, IP da: 255.255.255.255, IP sa: 0.0.0.0, DHCP ciaddr: 0.0.0.0, DHCP yiaddr: 0.0.0.0, DHCP siaddr: 0.0.0.0, DHCP giaddr: 0.0.0.0, DHCP chaddr: XXXX.XXXX.XXXX
Mar 4 14:53:04: DHCP_SNOOPING_SW: bridge packet get invalid mat entry: FFFF.FFFF.FFFF, packet is flooded to ingress VLAN: (X)
Mar 4 14:53:04: DHCP_SNOOPING_SW: bridge packet send packet to cpu port: VlanX.
Mar 4 14:53:04: DHCPSNOOP(hlfm_set_if_input): Setting if_input to Gi0/1 for pak. Was VlX
Mar 4 14:53:04: DHCPSNOOP(hlfm_set_if_input): Setting if_input to VlX for pak. Was Gi0/1
Mar 4 14:53:04: DHCPSNOOP(hlfm_set_if_input): Setting if_input to Gi0/1 for pak. Was VlX
Mar 4 14:53:04: DHCP_SNOOPING: received new DHCP packet from input interface (GigabitEthernet0/1)
Mar 4 14:53:04: DHCP_SNOOPING: process new DHCP packet, message type: DHCPOFFER, input interface: Gi0/1, MAC da: ffff.ffff.ffff, MAC sa: ZZZZ.ZZZZ.ZZZZ, IP da: 255.255.255.255, IP sa: 10.n.n.1, DHCP ciaddr: 0.0.0.0, DHCP yiaddr: 10.n.n.48, DHCP siaddr: 10.n.n.50, DHCP giaddr: 10.n.n.1, DHCP chaddr: XXXX.XXXX.XXXX
Mar 4 14:53:04: DHCP_SNOOPING: received new DHCP packet from input interface (FastEthernet0/3)
Mar 4 14:53:04: DHCP_SNOOPING: process new DHCP packet, message type: DHCPDISCOVER, input interface: Fa0/3, MAC da: ffff.ffff.ffff, MAC sa: XXXX.XXXX.XXXX, IP da: 255.255.255.255, IP sa: 0.0.0.0, DHCP ciaddr: 0.0.0.0, DHCP yiaddr: 0.0.0.0, DHCP siaddr: 0.0.0.0, DHCP giaddr: 0.0.0.0, DHCP chaddr: XXXX.XXXX.XXXX
Mar 4 14:53:04: DHCP_SNOOPING_SW: bridge packet get invalid mat entry: FFFF.FFFF.FFFF, packet is flooded to ingress VLAN: (X)
Mar 4 14:53:04: DHCP_SNOOPING_SW: bridge packet send packet to cpu port: VlanX.
Mar 4 14:53:04: DHCPSNOOP(hlfm_set_if_input): Setting if_input to Gi0/1 for pak. Was VlX
Mar 4 14:53:04: DHCPSNOOP(hlfm_set_if_input): Setting if_input to VlX for pak. Was Gi0/1
Mar 4 14:53:04: DHCPSNOOP(hlfm_set_if_input): Setting if_input to Gi0/1 for pak. Was VlX
Mar 4 14:53:04: DHCP_SNOOPING: received new DHCP packet from input interface (GigabitEthernet0/1)
Mar 4 14:53:04: DHCP_SNOOPING: process new DHCP packet, message type: DHCPOFFER, input interface: Gi0/1, MAC da: ffff.ffff.ffff, MAC sa: ZZZZ.ZZZZ.ZZZZ, IP da: 255.255.255.255, IP sa: 10.n.n.1, DHCP ciaddr: 0.0.0.0, DHCP yiaddr: 10.n.n.48, DHCP siaddr: 10.n.n.91, DHCP giaddr: 10.n.n.1, DHCP chaddr: XXXX.XXXX.XXXX
SWITCH#sh spanning-tree interface fastEthernet 0/3
Vlan Role Sts Cost Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
VLANX Desg FWD 19 128.128 P2p Edge
VLANY Desg FWD 19 128.128 P2p Edge
04-23-2013 01:41 PM
Bump!
Looking for another example in the field
Plan is to get phone provider to statically assign ip on phone. clear interface and interface Ethernet controller counters, set port ACL to permit ingress bootps only. Turn off stp, Cdp and keeps Alives. Span the port.
I want only dhcp packets coming in from client. If the captures show no egress packets, but dhcp snooping debuts suggest packets are sent out correct interface, I want to see if the interface controller transmit counters increment in response to client dhcp packets.
Anyone know of a better way of getting this visibility?
Still not sure what to make if it if I do see this however....
Thanks
Sent from Cisco Technical Support iPhone App
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide