Basic steps to troubleshoot layer 2 LAN issue:

ashirkar · ‎08-31-2013

Introduction
Finding degraded performance across link
No Connectivity between Switches
Lack of reachability to devices in same VLAN
Intermittent reachability to devices in same VLAN

Introduction

This document will explain you initial layer 2 troubleshooting steps with some helpful IOS command.

Finding degraded performance across link

Approaching Steps:

Check for physical interface problems like duplex mismatch. By default, each Cisco Switch port uses Ethernet auto-negotiation to determine the speed and duplex setting (whether it can be half or Full).These switches can set their duplex setting with "duplex" interface subcommand and their speed with the “Speed" interface subcommand.

A duplex mismatch usually does not bring link down; it just creates suboptimal performance.

Duplex mismatch might be caused due to hard-coding one side of the link to full duplex but leaving other side to auto negotiates. You would suspect a duplex mismatch if you saw collision on a full-duplex link because a full-duplex link should never have collisions. Half duplex on both sides will show some error.

IMP IOS Command: "Show interface"

Example

R1#sh int fa0/0

FastEthernet0/0 is up, line protocol is up

Hardware is Gt96k FE, address is c000.3710.0000 (bia c000.3710.0000)

MTU 1500 bytes, BW 10000 Kbit/sec, DLY 1000 usec,

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Half-duplex, 10Mb/s, 100BaseTX/FX

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:33, output 00:00:00, output hang never

Last clearing of "show interface" counters never

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue: 0/40 (size/max)

5 minute input rate 0 bits/sec, 0 packets/sec

5 minute output rate 0 bits/sec, 0 packets/sec

5 packets input, 1765 bytes

Received 5 broadcasts, 0 runts, 0 giants, 0 throttles

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

0 watchdog

0 input packets with dribble condition detected

22 packets output, 2785 bytes, 0 underruns

0 output errors, 0 collisions, 1 interface resets

0 unknown protocol drops

0 babbles, 0 late collision, 0 deferred

0 lost carrier, 0 no carrier

0 output buffer failures, 0 output buffers swapped out

Watch for some of the errors like

Runts: Runts are frames smaller than 64bytes

CRC error: This is CRC called cyclic redundancy checksum value does not match one calculated by switch or router etc.

Collisions: Look for collisions on a full-duplex interface or excessive collision on a half-duplex interface.

Late collision on a half-duplex interface: This is occurs after first 64 bytes of a frame.

Frames: frame error has a CRC error.

There is another helpful command display interface statistics is "show controllers fa0/0".This will have very give you very long output but you can find the no of frames with bad frame check, CRC error, Collision, late collision and its own interface auto negotiation status, speed duplex capability as well as its neighbor.

Also read "Configuring and Troubleshooting Ethernet 10/100/1000Mb Half/Full Duplex Auto-Negotiation" document for more information.

No Connectivity between Switches

1) Check for interface shut down using “show ip interface” command”.

Here is example below:

R1#show ip interface fa0/0

FastEthernet0/0 is up, line protocol is up

Internet protocol processing disabled

R1#

If an interface shows and UP/UP means physical and logical connection has been made. If it is showing Up/down, you have some l2 troubleshooting to do. An interface status of err-disable could be caused by many different problem .common problem can be security violation or detection of a unidirectional link.

When a port is error disabled, it is effectively shut down and no traffic is sent or received on that port. The port LED is set to the color orange. You can check using “show interface status err-disabled” command on your device.

This example shows how to display the error disabled state of interfaces:

switch# show interface status err-disabled

-----------------------------------------------------------------------

Port Name Status Reason

-----------------------------------------------------------------------

Eth114/1/27 -- down BPDUGuard errDisable

Eth114/1/28 -- down BPDUGuard errDisable

Eth114/1/29 -- down BPDUGuard errDisable

Eth114/1/30 -- down BPDUGuard errDisable

Eth114/1/31 -- down BPDUGuard errDisable

Eth114/1/32 -- down BPDUGuard errDisable

Eth114/1/33 -- down BPDUGuard errDisable

Eth114/1/34 -- down BPDUGuard errDisable

--More--

switch#

2) Verify your trunk links and ether channel if configure using following command:

Useful command:

“Show interface trunk”
“Show etherchannel summary”

Here is a document for "Troubleshooting Switch Port and Interface Problems"

Lack of reachability to devices in same VLAN

1) Eliminate Layer 1 issue using "show ip interface “command.

R1#show ip interface fa0/0

FastEthernet0/0 is up, line protocol is up

Internet protocol processing disabled

R1#

2) Verify VLAN exist on the Switch using “Show VLAN “command.

SW#sh vlan

VLAN Name Status Ports

---- -------------------------------- --------- -------------------------------

1 default active Fa1/0, Fa1/1, Fa1/2, Fa1/3

Fa1/4, Fa1/5, Fa1/6, Fa1/7

Fa1/8, Fa1/9, Fa1/10, Fa1/11

Fa1/12, Fa1/13, Fa1/14, Fa1/15

2 cisco_test active

1002 fddi-default act/unsup

1003 token-ring-default act/unsup

1004 fddinet-default act/unsup

1005 trnet-default act/unsup

VLAN Type SAID MTU Parent RingNo BridgeNo Stp BrdgMode Trans1 Trans2

---- ----- ---------- ----- ------ ------ -------- ---- -------- ------ ------

1 enet 100001 1500 - - - - - 1002 1003

2 enet 100002 1500 - - - - - 0 0

1002 fddi 101002 1500 - - - - - 1 1003

1003 tr 101003 1500 1005 0 - - srb 1 1002

1004 fdnet 101004 1500 - - 1 ibm - 0 0

1005 trnet 101005 1500 - - 1 ibm - 0 0

SW#

3) Verify that the interface is assigned to the correct VLAN using “show interface switchport” command.

sw#show interfaces switchport fa1/15

Switchport: Enabled

Administrative Mode: static access

Operational Mode: static access

Administrative Trunking Encapsulation: dot1q

Operational Trunking Encapsulation: native

Negotiation of Trunking: Disabled

Access Mode VLAN: 1 (default)

Trunking Native Mode VLAN: 1 (default)

Trunking VLANs Enabled: ALL

Trunking VLANs Active: 1

Protected: false

Priority for untagged frames: 0

Override vlan tag priority: FALSE

Voice VLAN: none

Appliance trust: none

If it is not in correct VLAN assign port into correct VLAN using following steps:

Conf t

Int fa1/15

Switchport access vlan 2

4) Verify that VLAN is allowed on trunk port using “show interface trunk” command.

sw#show interfaces trunk

Port Mode Encapsulation Status Native vlan

Fa1/15 on 802.1q trunking 1

Port Vlans allowed on trunk

Fa1/15 1-2,1002-1005

Port Vlans allowed and active in management domain

Fa1/15 1-2

Port Vlans in spanning tree forwarding state and not pruned

Fa1/15 none

sw#

5) You can also use the Layer 2 traceroute utility to identifies the Layer 2 path that a packet takes from a source device to a destination device using “traceroute mac [interface type interface_number] source_mac_address [interface type interface_number] destination_mac_address [vlan vlan_id] [detail]” command.

Intermittent reachability to devices in same VLAN

1) Check for spanning-tree problems such as BPDU floods or flapping mac address.
Spanning-tree issues are possible in a network that has not been properly configured. One common STP problem is a change in Root Bridge. If Root Bridge is not properly configured a change in root can cause a flood of BPDUs and affect network connectivity. Another Known symptom of loop is flapping of MAC address.A port configuraed with loop guard or root guard put in an inconsistence state if it receive superior BPDU can be verify using “ show spanning-tree inconsistent port”

Some IOS useful command:
‘Show spanning-tree”
‘Show spanning-tree detail:
‘Show spanning-tree root”
"Show mac-address-table”

Here is another document you would like to look into it "Troubleshooting LAN Switching Environments"

Some Spanning-tree related troubleshooting docs:

"Troubleshooting STP on Catalyst Switches Running Cisco IOS System Software"
"Spanning Tree Loop Troubleshooting and Safeguards"
"Spanning Tree Protection"

Finding IP address connected to a cisco switch port

If you don’t know IP address of devices present on specific VLAN and wanted to track end device IP address please try following steps:

Steps 1: ping to broadcast IP address of subnet from your L3 device(Gateway)

For example: I have following connectivity.R1 connected to Sw1 and Sw1 to Sw2.Host H1 and H2 are connected to SW2.

R1--SW1--SW2---H1
|--H2

R1 is default gateway having L3 address. Please find below IP address for each device

R1..1.1.1.1/24

H1..1.1.1.2/24

H2.1.1.1.3/24

So for subnet 1.1.1.0/24 broadcast IP is 1.1.1.255

Let’s ping to 1.1.1.255 from your router. All hosts present to that LAN segment will reply as you can see below and your ARP table will get flood with IP address and respective mac-address on L3 device.

ping 1.1.1.255

Sending 1000, 100-byte ICMP Echos to 1.1.1.255, timeout is 2 seconds:

........

Reply to request 8 from 1.1.1.2, 28 ms

Reply to request 9 from 1.1.1.3, 64 ms

Step2: then Check arp entries using "show arp” command on L3 device and it will show you mac-address associate with IP address.

R1#sh ip arp

Protocol Address Age (min) Hardware Addr Type Interface

Internet 1.1.1.1 - c000.2498.0000 ARPA Vlan2

Internet 1.1.1.2 0 c003.2498.0000 ARPA Vlan2

From above table you can see host 1.1.1.2 machine mac-address is c003.2498.0000

Step3: Now check mac learned from specific port as shown below:

R1#sh mac address c003.2498.0000

Destination Address Address Type VLAN Destination Port

------------------- ------------ ---- --------------------

c003.2498.0000 Dynamic 2 FastEthernet1/1

Step4: Then use CDP (Cisco discovery protocol) to check what device connected to port on which you learn mac –address.

In our scenario we have learned Mac-address from F1/1; we need to check CDP detail for fa1/1.

R1#sh cdp ne fa1/1 detail

-------------------------

Device ID: SW1.lab.local

Once you find connected device, login into it and again use “sh mac address c003.2498.0000” command and “sh cdp ne fa1/1 detail” command till you find your actual end port to which your host is connected.The above method is useful when you CDP enable on your all switches and your end host responds to broadcast message.

The above point has written through discussion .
Please feel free to add your expertise experience into this document to make it helpful for beginners