server connected to switch port but no traffic passing through

linqksupport · ‎01-08-2013

Hello,

this is really embarrassing and has been frustrating to say the least.

I have a linux (centos 5) machine which has eth1 connected to a port on a Cisco 3560G. The switch port belongs to a VLAN which is public. It appears that this linux machine is actually connected to this port (I am not physically there...this equipment is sitting on the other side of the world, I'm VPN'ing).

If I shutdown the interface Gi0/20 to which this machine is apparently connected and use "ip monitor" or "mii-tool -w eth1" on the linux machine, I see the tranisition happen with the link dropping. The same happens the other way, if I shutdown eth1, I see the interface going down on the switch console.

I have statically assigned an available IP address from our pool to eth1 which belongs to the VLAN on Gi0/20, but for the life of me I cannot understand why I can't get any traffic going anywhere.

The routing table in the linux machine has been done and re-done and regardless of that, I should see the MAC address of the switch show in the arp cache of the linux machine. Incidentally the mac addr of eth1 does show up on the switch "sh mac address-table interface Gi0/20" when I turn it up or down, but after a little while it disappears from the switch arp table.

I haven't touched the cisco stuff for a while and have forgotten everything. Can't figure out how to troubleshoot this:

have tried ping (obv), traceroute, traceroute mac, traceroute mac ip etc. but doesn't come up with anything. It's as if the switch doesn't even know about this machine.

Please help...I've wasted 2 whole days on this and it's really annoying esp. when the machines are so far away and the response time and interaction with the network takes so damn long

Thanks in advance

Karsten Iwen · ‎01-09-2013

Please paste the output of "show int gig 0/20 switchport". Is the VLAN really the one you want to use?

If yes, take another free IP and configure the IP with:

interface vlan VLAN

ip add ...

Can you ping that IP from the PC?

And paste the output of "sh run int gig 0/20".

--
Don't stop after you've improved your network! Improve the world by lending money to the working poor:
http://www.kiva.org/invitedby/karsteni

linqksupport · ‎01-10-2013

Hi. Please see below the output you requested:

Please paste the output of "show int gig 0/20 switchport".

Name: Gi0/20

Switchport: Enabled

Administrative Mode: dynamic auto

Operational Mode: static access

Administrative Trunking Encapsulation: negotiate

Operational Trunking Encapsulation: native

Negotiation of Trunking: On

Access Mode VLAN: 100 (EXTERNAL_VLAN)

Trunking Native Mode VLAN: 1 (default)

Administrative Native VLAN tagging: enabled

Voice VLAN: none

Administrative private-vlan host-association: none

Administrative private-vlan mapping: none

Administrative private-vlan trunk native VLAN: none

Administrative private-vlan trunk Native VLAN tagging: enabled

Administrative private-vlan trunk encapsulation: dot1q

Administrative private-vlan trunk normal VLANs: none

Administrative private-vlan trunk associations: none

Administrative private-vlan trunk mappings: none

Operational private-vlan: none

Trunking VLANs Enabled: ALL

Pruning VLANs Enabled: 2-1001

Capture Mode Disabled

Capture VLANs Allowed: ALL

Protected: false

Unknown unicast blocked: disabled

Unknown multicast blocked: disabled

Appliance trust: none

 Is the VLAN really the one you want to use?

Yes...although I tried changing VLANs on this port to other available VLANs on this switch

and changing eth1 correspondingly, but I get the same behaviour.

If yes, take another free IP and configure the IP with:

interface vlan VLAN

ip add ...

Can you ping that IP from the PC?

No I can't ping anything from the PC, either this or anything else on the same VLAN. Like I said, 
even the arp cache expunges the record for the mac of this interface after a short time, so no one

on the network has any clue about this Interface. The only signs I see are when you either shutdown

Gi0/20 and monitor the state change on the linux box or vice-versa. So we know that they are indeed

connected but why nothing else is going on, I have no clue

And paste the output of "sh run int gig 0/20".

interface GigabitEthernet0/20

description EMS00 - LNK A

switchport access vlan 100

priority-queue out

spanning-tree portfast

end

--
Don't stop after you've improved your network! Improve the world by lending money to the working poor:
http://www.kiva.org/invitedby/karsteni

davy.timmermans · ‎01-10-2013

You're connected via vpn on the linux device via a second port? I can't visualize your setup

linqksupport · ‎01-10-2013

davy.timmermans wrote:

You're connected via vpn on the linux device via a second port? I can't visualize your setup

The linux machine in question is part of a much larger network..I'm VPN'ed into that network ...

have access to the switch via a serial console switch

have access to the linux console via a remote supervisor adapter so I can turn networking on/off / add/remove ethernet ports etc.

jose.vieira525 · ‎01-10-2013

Hi there

Can you paste the output from the following commands?

Sh ip arp vlan 100
Sh run int vlan 100
Sh ip int brief

Sent from Cisco Technical Support iPad App

Karsten Iwen · ‎01-10-2013

The switchport looks ok. For best practice you should configure "switchport host" on the port.

Can you replace the server against anything else to see how a different device works on that port? Can the server work on a different switch?

--
Don't stop after you've improved your network! Improve the world by lending money to the working poor:
http://www.kiva.org/invitedby/karsteni

linqksupport · ‎01-10-2013

karsten.iwen wrote:

The switchport looks ok. For best practice you should configure "switchport host" on the port.

Can you replace the server against anything else to see how a different device works on that port? Can the server work on a different switch?

Hi. Ok thanks for the advice on "switchport host" ...I don't know what it does but I'm assuming it tells the switch that the port is connected to a physical machine as opposed to another switch?

No I can't replace it or do anything that requires a physical change. As explained earlier, these servers are over 8,000 miles away or switching out the server with something else on that switch port would've been the first thing I'd have tried

After having conducted further investigation on the Linux machine, I am sure something is definitely wrong with that NIC on that machine. This machine is equipped with 2 x Dual GigE NICs. The 2 NICs are NOT identifcal (not the same model). The 2nd port of this NIC is connected to a different switch in a different VLAN. I turned that up, configured it etc. and it has the exact same problem

The loopback test of the offline ethtool test fails on both ports of this NIC with code "13", turns out this indicates that the hardware MAY be failing. These NICs have really old NIC e1000 driver versions. I will download the latest drivers for it, and run the test again at a later time as after having spent countless hours on it I have put in some workarounds in place for me to use the machine for the purpose I turned it back up (for now). Just scared that messing around with drivers and the sort may not go well (knowing my luck).

Karsten Iwen · ‎01-11-2013

Ok thanks for the advice on "switchport host" ...I don't know what it does but I'm assuming it tells the switch that the port is connected to a physical machine as opposed to another switch?

From the reference:

To optimize the port for a host connection, the switchport host command sets switch port mode to access, enables spanning tree Port Fast, and disables channel grouping. Only an end station can accept this configuration.

http://www.cisco.com/en/US/partner/docs/switches/lan/catalyst3750/software/release/12.2_55_se/commmand/reference/cli3.html#wp1948119

--
Don't stop after you've improved your network! Improve the world by lending money to the working poor:
http://www.kiva.org/invitedby/karsteni