Re: sho arp doesn't show mac address

gflorescu · ‎10-09-2007

I have 3 Cisco 3750 switches that are connected via trunk links to each other. When I do:

ping 172.31.6.11

sho arp

on 1 of my switches it doesn't even show the ip and the coresponding mac-address. This particular host is physically plugged into port 10 on that switch.

on the 2nd or 3rd switch switches it shows the mac-address and it's the same mac-address for a lot of different ip addresses, meaning that these ips are located on a different switch. I follow it up the chain and then when I get to the 1st switch and I try to do a "sho mac-address-table", it shows that it's going out of interface 1/0/1 which goes out to another location across town.

Here's my config for that 1st switch where this host is located:

Current configuration : 8085 bytes

!

! Last configuration change at 15:07:48 PDT Tue Oct 9 2007 by admin

! NVRAM config last updated at 15:10:07 PDT Wed Sep 19 2007 by admin

!

version 12.2

no service pad

service timestamps debug datetime localtime

service timestamps log datetime localtime

service password-encryption

!

hostname 1st3750

!

enable password xxx

!

username damiens privilege 15 password xxx

username admin privilege 15 secret xxx

no aaa new-model

clock timezone PST -8

clock summer-time PDT recurring

switch 1 provision ws-c3750g-24ts

vtp domain TEST

vtp mode transparent

ip subnet-zero

ip cef load-sharing algorithm universal CB41AB75

ip domain-name dcipa.com

ip name-server 172.16.x.x

!

mls qos

!

crypto pki trustpoint TP-self-signed-3281851776

enrollment selfsigned

subject-name cn=IOS-Self-Signed-Certificate-3281851776

revocation-check none

rsakeypair TP-self-signed-3281851776

no file verify auto

spanning-tree mode pvst

no spanning-tree optimize bpdu transmission

spanning-tree extend system-id

!

vlan internal allocation policy ascending

!

vlan 99

name voice

!

vlan 100

name IT

!

vlan 120

name TEST

!

vlan 121

name ABC-Servers

!

vlan 300

name management

!

interface GigabitEthernet1/0/1

description to HMP

switchport trunk encapsulation dot1q

switchport mode trunk

!

interface GigabitEthernet1/0/2

switchport access vlan 100

switchport voice vlan 99

spanning-tree portfast

!

interface GigabitEthernet1/0/3

switchport access vlan 100

switchport voice vlan 99

spanning-tree portfast

!

interface GigabitEthernet1/0/4

switchport access vlan 100

switchport voice vlan 99

spanning-tree portfast

!

interface GigabitEthernet1/0/5

switchport access vlan 100

switchport voice vlan 99

spanning-tree portfast

!

etc, etc......

interface GigabitEthernet1/0/22

switchport access vlan 100

switchport voice vlan 99

spanning-tree portfast

!

interface GigabitEthernet1/0/23

switchport trunk encapsulation dot1q

switchport mode trunk

!

interface GigabitEthernet1/0/24

switchport trunk encapsulation dot1q

switchport mode trunk

!

interface GigabitEthernet1/0/25

!

interface GigabitEthernet1/0/26

!

interface Vlan1

no ip address

no ip route-cache

shutdown

!

interface Vlan300

ip address 172.30.x.x 255.255.255.0

no ip route-cache

!

ip default-gateway 172.30.x.x

ip classless

ip http server

no ip http secure-server

!

snmp-server community abct RW

snmp-server community public RO

!

control-plane

Richard Burts · ‎10-09-2007

Gilbert

Your post does not clearly describe where you originate the ping from. Am I correct in assuming that it is from the console of the switch or from a telnet to the switch - so that it is coming from the management interface? If so it is originating from 172.30.x.x/24.

In your attempt to protect the security of your network you have obscured much information that might allow us to more quickly understand the environment and determine the cause of the problem. But the limited information that you include shows that the destination is 172.31.6.11 which is clearly in a different subnet from the management interface. So the switch must go to its default gateway whi is:ip default-gateway 172.30.x.x.

The default gateway is almost certainly the reason that there is the same mac-address for a lot of different ip addresses. That mac address is almost certainly the mac address of the default gateway. And the fact that the destination is in a different subnet and VLAN is the reason that the IP address does not show up in the ARP table of that switch. The ARP table is just for addresses that the management interface accesses and not for all the mac addresses in the switch.

HTH

Rick

HTH

Rick

gflorescu · ‎10-09-2007

You are correct in your assumptions. I am accessing the switch via telnet and I'm pinging from the switch itself.

The management interface IP is in a different subnet than the ip address I'm pinging.

So how do I fix this??

Should I assign another IP address to the switch that's in the same subnet/VLAN as the IP I'm trying to ping?? Then access the switch via telnet by using the new IP??

Is there any way to see all the mac-addresses on the switch and all the ports they connect to when using a management VLAN IP?

paul.matthews · ‎10-10-2007

I don't think you need to fix it, merely understand better.

If you ping from one of the routing devices attached to the VLAN the user is in, a sh ip arp should reveal the mac address of the device. you can then look at that mac address on any device that is carrying the traffic for that VLAN.

You can alswats see al the mac addresses known to a switch by using show mac-address-table, but that will not tell you any IP to MAC mapping.

Paul.

gflorescu · ‎10-10-2007

When you say "ping from one of the routing devices attached to the VLAN the user is in", you mean give the switch an interface ip of 172.31.6.2, then connect to the switch via telnet with that IP, then ping 172.31.6.11 and I should be able to see it all?

That makes it kind of hard because if I want to see the ARP for vlan 100, I'll have to connect to the switch via an ip address in that VLAN. Then if I want to see the ARP for VLAN 99, I'll have to connect to the switch via an ip address in VLAN 99.

paul.matthews · ‎10-10-2007

Don't just add extra devices in VLANs without planning - you could mess up the routing quite easily.

By a device I mean anything already in the VLAN - either anotherPC/Server or better whatever device is doing the routing between the VLANs and ping/check arp from there, then use sh mac-a to track the mac address to the port.

If you have a switch with five VLANs and an IP address in each, it does not matter what address you use to telnet to the switch, when you ping, the switch will use the appropriate IP address anyway.

gflorescu · ‎10-10-2007

That worked. I went to the device that does the routing between VLANs (a cisco 6513) and I was able to ping, then get the ip to mac address mapping.

A follow up question:

On the Cisco 6513, I also pinged an ip address of 172.31.0.150 which is located in the DMZ but the host is physically connected to a switch port on the 6513. When I do a sho arp, I don't see that ip address listed.

paul.matthews · ‎10-10-2007

If it is on the DMZ, I presume a firewall will be handling the traffic? If so the firewall wuld be the device that does the ARP for it.

Paul.

gflorescu · ‎10-13-2007

Not to beat a dead horse but when I go to my Cisco 6513 and I try to ping a different address in a different vlan/subnet and then do a

sho arp | include 172.16.0.30

it doesn't show anything.

If I do:

sho arp | include 172.31.6.11

it shows the mac address and the vlan.

The IP 172.16.0.30 is physically connected to one of the ports on the 6513. The IP 172.31.6.11 is not physically connected to the 6513.

Kevin Dorrell · ‎10-13-2007

It doesn't make any difference whether or not they are physically connected to this switch. What matters is whether they are on the same VLAN and subnet as a VLAN interface on this switch.

If you ping an address in a VLAN (and subnet) in which the 6513 has a layer-3 VLAN interface, then you will get a response and you will see an entry in the ARP cache.

If you ping an address in a VLAN (and subnet) in which the 6513 has no layer-3 VLAN interface, then you can only get a response by going through a router (or another L3 switch). You will not see any entry in the ARP cache because you were not talking directly to their VLAN - you were going through a router.

Kevin Dorrell

Luxembourg

Richard Burts · ‎10-13-2007

Gilbert

It seems like this thread has changed focus several times (looking at different switches and at different addresses) and that makes it more difficult for us to get to the bottom of the issue. So lets stay focused on this part of the issue and hope that we can make some progress.

Please provide us with some information about this issue:

- please post the output of show ip interface brief so that we can get a clear understanding of what subnets (and vlans) are routed on this 6513.

- please post the output of show ip route 172.16.0.30 so that we can verify how the switch will get to that address.

- please post the output of show ip route 172.31.6.11 so that we can verify how the switch will get to that address.

It does seem a bit odd that the 6513 would not have an arp entry for a device connected on one of its physical ports. But there are scenarios in which that would happen and would be correct. For example if 172.16.0.30 were in some VLAN (say for example VLAN 30) and the 6513 was doing layer 2 switching for that VLAN (so it has layer 2 VLAN definitions) but did not have a layer 3 interface (interface vlan 30) with an IP address in that VLAN then there would be some other device that was doing inter vlan routing and the 6513 would forward to that device to get layer 3 access to vlan 30. In this case there would be no ARP entry on the 6513.

In a similar way if 172.31.6.11 is physically connected to a different switch and the other switch is connected at layer 2 to the 6513 and if the 6513 is providing inter vlan routing for that VLAN then it is correct that the 6513 should have an ARP entry for that address.

So provide the information that I requested and perhaps we can make some progress in understanding this issue.

{edit} I see that while I was working on my response that Kevin has posted and that he and I are on the same track. Thanks Kevin.

HTH

Rick

HTH

Rick

gflorescu · ‎10-13-2007

sho ip interface brief

Vlan100 172.31.6.1 YES NVRAM up up

Vlan101 172.31.5.129 YES NVRAM up up

Vlan102 172.31.6.129 YES NVRAM up up

Vlan120 unassigned YES NVRAM up up

Vlan121 172.30.2.1 YES NVRAM up up

Vlan199 172.31.12.129 YES NVRAM up up

#sho ip route 172.16.0.30

Routing entry for 172.16.0.0/22

Known via "static", distance 1, metric 0

Routing Descriptor Blocks:

* 172.31.0.6

Route metric is 0, traffic share count is 1

#sho ip route 172.31.6.11

Routing entry for 172.31.6.0/25

Known via "connected", distance 0, metric 0 (connected, via interface)

Redistributing via eigrp 1, ospf 1

Advertised by ospf 1 subnets

Routing Descriptor Blocks:

* directly connected, via Vlan100

Route metric is 0, traffic share count is 1

-----------------------------------------

I noticed there's no ip address on interface vlan 120 and that's where 172.16.0.30 is located (on vlan 120). I am not sure if to put an ip address on that interface because I don't know if the 6513 does the layer 3 routing for that vlan.

That interface use to have an ip address of 172.16.0.1 before but then another technician who is no longer around put in a Citrix netscaler appliance and assigned the 172.16.0.1 address to it.

Richard Burts · ‎10-13-2007

Gilbert

Thanks for posting the information that I requested. It confirms the explanation that I suggested. The address 172.16.0.30 is not in a VLAN/subnet for which the 6513 is routing. So it forwards to some other device (172.31.0.6) and therefore will NOT arp for this address. This pretty clearly explains why there is no entry in the ARP table after you ping to this address.

And it shows that 172.31.6.11 is in a subnet for which the 6513 is routing and therefore the 6513 will ARP for it (even if it is physically connected on a different switch) and will have an entry in the ARP table.

To supplement the explanation that Kevin provided: we need to differentiate the operation of the switch at layer 2 and at layer 3. If the destination is within a layer 3 subnet for which the switch is routing, then it will ARP no matter whether the device is local at layer 2 or remote at layer 2. And if the device is not within a layer 3 subnet for which the switch is routing, then it will not ARP no matter whether the device is local at layer 2 or not.

HTH

Rick

HTH

Rick

paul.matthews · ‎10-15-2007

I would say do not pt an address there. You will be altering the routing if you do - you have already meantioned a DMZ, and suggested that me b beyond a firewall. If this subnet is also firewalled, adding an IP address will probably break things (traffic to the sobnet bypassing the firewall, but return traffic that the firewall is not expecting hitting the firewall and being dropped. you would also have a major security breach by having a path available that bypasses the firewall.

Kevin Dorrell · ‎10-10-2007

Does that mean you have an isolated (layer-2 only) VLAN in your switch representing the DMZ? In that case you will not see the DMZ server in your router ARP cache because the (internal) router is not talking directly to the DMZ server. Instead, you probably have a route for the 172.31.0.150 pointing to the trusted side of a firewall device. The firewall device will then do the onward routing to the DMZ server.

The best you can hope for in this situation is to see the ARP cache entry for the firewall device. To see the ARP entry for the DMZ server, you would have to interrogate the firewall itself.

Kevin Dorrell

Luxembourg