can't communicate on L3 on specific interfaces

[3moloz123]

Hi,

I have a Cisco 2821 with 2x HWIC-4ESW. Two of the ports on the first module (range 0/0/1-2) are access ports, with one vlan each. The vlans has ips configured in /30 nets, where the other usable address are ebgp neighbours. The port Im trying to use for management is on module number 2 (0/1/3).

Seems to me it's identical hwics, see output of 'show inventory'.

So port 0/1/3 has vlan 999, and vlan 999 is configured with address 10.1.0.132/25. I configured 0/1/2 as vlan 999 too, connected a laptop and it can ping through the router to the switch connected on 01/3 - but not other device can reach the routers ip 10.1.0.132, and the router cant reach any of the devices on 10.1.0.128/25.

To sumarize the problem: I can ping the other hosts in vlan 100 and vlan 101, but not the other hosts in vlan 999.

It's connected  like this:

       <laptop>
     /
<router>
     \
       <switch_with_vlan999>
                    \
                       <other_devices_on_vlan999>

# show version

NAME: "2821 chassis", DESCR: "2821 chassis"

PID: CISCO2821         , VID: V05 , SN: FCZ123456

NAME: "4 Port FE Switch on Slot 0 SubSlot 0", DESCR: "4 Port FE Switch"

PID: HWIC-4ESW         , VID: V01 , SN: FOC123456

NAME: "4 Port FE Switch on Slot 0 SubSlot 1", DESCR: "4 Port FE Switch"

PID: HWIC-4ESW         , VID: V01 , SN: FOC123456

# show run

interface FastEthernet0/0/0

description a specific IX

switchport access vlan 101

no cdp enable

!

!

interface FastEthernet0/0/1

description specific ISP

switchport access vlan 100

no cdp enable

!

!

interface FastEthernet0/1/3

description network management

switchport access vlan 999

!

!

interface Vlan100

description specific isp

ip address 1.2.3.150 255.255.255.252

no ip proxy-arp

!

!

interface Vlan101

description some IX

ip address 2.3.4.70 255.255.255.192

no ip proxy-arp

!

!

interface Vlan999

ip address 10.1.0.132 255.255.255.128

no ip proxy-arp

!

!

# show ip interface brief

FastEthernet0/0/0          unassigned      YES unset  up                    up 

FastEthernet0/0/1          unassigned      YES unset  up                    up 

FastEthernet0/1/3          unassigned      YES unset  up                    up

Vlan1                      unassigned      YES NVRAM  up                    down

Vlan100                    1.2.3.150  YES NVRAM  up                    up 

Vlan101                    2.3.4.70  YES NVRAM  up                    up 

Vlan999                    10.1.0.132      YES manual up                    up    <- does manual mean "not yet saved to startup-config", or is it an indicator of a difference between hwic 1 and hwic 2?

[cadet alain]

<< should this be in your routing table? possible overlapped subnet..?

This is normal entry with IOS 15 which is applying to Pv4 the Ipv6 way of always having a host  interface for every connected subnet.

Vlan999                    10.1.0.132      YES manual up                    u

This is not the source of the problem, simply means it was configure with the ip address command and not taken from startup config.

Regards.

Alain.

[3moloz123]

Hi, bbb bbb.

The default gateway for that particular network is 10.1.0.129, which happens to be a Cisco ASA.

As the laptop, the router and the switches are all on the same network, I fail to see what relevance the gateway has.

Can you elaborate more on what you have in mind?

[cadet alain]

Have you tried what I suggested above?

Regards.

Alain.

[3moloz123]

Im trying to fully understand what I should do.

10.1.0.143 is the switch to which the router is directly connected. What I see there is the opposite.

sw3#ping 10.1.0.132

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 10.1.0.132, timeout is 2 seconds:

34w5d: IP ARP: creating incomplete entry for IP address: 10.1.0.132 interface Vlan999

34w5d: IP ARP: sent req src 10.1.0.143 0017.0e8d.3c00,

                 dst 10.1.0.132 0000.0000.0000 Vlan999

34w5d: IP ARP throttled out the ARP Request for 10.1.0.132.

34w5d: IP ARP: sent req src 10.1.0.143 0017.0e8d.3c00,

                 dst 10.1.0.132 0000.0000.0000 Vlan999

34w5d: IP ARP throttled out the ARP Request for 10.1.0.132

Success rate is 0 percent (0/5)

On the router, here's spotted mac addresses for the vlan and ports involved:

0017.0e8d.3c00 Dynamic     999     FastEthernet0/1/3

0017.0e8d.3c0d Dynamic     999     FastEthernet0/1/3

0026.994c.6560 Self     999     Vlan999

same on fa0/13 of "switch3" (.143):

999    001e.1309.50fc    DYNAMIC     Fa0/13

The two dynamic mac addresses on the router correspond to two of the addresses of the switch 3 device:

sw3# show mac-address-table | inc 0017.0e8d.3c0d

All    0017.0e8d.3c0d    STATIC      CPU

sw3# show mac-address-table | inc 0017.0e8d.3c00

All    0017.0e8d.3c00    STATIC      CPU

And now while investigating, I suddently found a third mac address on the router:

0017.0e8d.3c0d Dynamic     999     FastEthernet0/1/3

0023.4719.a920 Dynamic     999     FastEthernet0/1/3

f062.8195.f100 Dynamic     999     FastEthernet0/1/3

# show arp

Internet  10.1.0.143              0   0017.0e8d.3c00  ARPA   Vlan999

But not vice versa:

Internet  10.1.0.132              0   Incomplete      ARPA

I still cant ping 10.1.0.143 from the router, and not the router from the switch. I am reading some information about basic capture usage now.

[cadet alain · Accepted solution]

Hi,

And now while investigating, I suddently found a third mac address on the router:

0017.0e8d.3c0d Dynamic     999     FastEthernet0/1/3

0023.4719.a920 Dynamic     999     FastEthernet0/1/3

f062.8195.f100 Dynamic     999     FastEthernet0/1/3

# show arp

Internet  10.1.0.143              0   0017.0e8d.3c00  ARPA   Vlan999

But not vice versa:

Internet  10.1.0.132              0   Incomplete      ARPA

The new mac addresses correspond to a HP procurve gears ,  is it normal?

Can you do a sh cdp nei .

Regards.

Alain.

[3moloz123]

on router

sw3              Fas 0/1/3          133          S I      WS-C3550- Fas 0/13

On switch

rtr1-core.mydomain.com                    Fas 0/13              134           R S I     2821      Fas 0/1/3

The HPs are two SAN-switches (from switch 3):

Internet  10.1.0.152              0   f062.8195.f100  ARPA   Vlan999

Internet  10.1.0.151              0   0023.4719.a920  ARPA   Vlan999

[3moloz123]

If any other device tries to ping to router, the router gets the mac addr of the remote device but ping does not work.

Apr 26 14:15:35: IP ARP: rcvd req src 10.1.0.152 f062.8195.f100, dst 10.1.0.129 Vlan999

Apr 26 14:15:37: IP ARP: rcvd req src 10.1.0.151 0023.4719.a920, dst 10.1.0.132 Vlan999

Apr 26 14:15:37: IP ARP: creating entry for IP address: 10.1.0.151, hw: 0023.4719.a920

Apr 26 14:15:37: IP ARP: sent rep src 10.1.0.132 0026.994c.6560,

                 dst 10.1.0.151 0023.4719.a920 Vlan999

The opposite is unfortunately not true, neither other cisco switches nor the procurves (all within the same network segment) get the routers mac addr when it try to ping them.

[3moloz123]

A capture reveals nothing. Seems to me it's not possible to run the capture on layer 2 though?

rtr1-core#monitor capture point ip cef cap1 FastEthernet 0/1/3
rtr1-core#monitor capture point ip cef cap1 FastEthernet 0/1/3 both
rtr1-core#monitor capture point associate cap1 buf1
rtr1-core#monitor capture point start cap1
rtr1-core#ping 10.1.0.141

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.1.0.141, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
rtr1-core#show monitor cap buffer buf1 dump
rtr1-core#

[3moloz123]

The first hwic works, so I guess there's better places to discuss eventual hardware problems for.

Thanks for the help .

[bbb bbb]

hi  3moloz123 ,

test a traceroute to any IP that doesnt belong to current subnet from your laptop and PC.. the first hop should point you to a router.. then try to login to the first hop IP via telnet.. it should show what equipment you have log on.. (either router or HP)

if first hop is unreachable, try to save and reload switch.. then try traceroute again..

you said that your laptop/gateway points to cisco asa..

and has the following policy.. does this statement contradicting to each other..?

Standard IP access list 25

    10 permit 10.1.0.0, wildcard bits 0.0.0.7

Extended IP access list 101

    10 permit ip host 7.8.9.126 any (6 matches)

   20 permit ip 10.1.0.0 0.0.0.255 any

    30 permit ip host 7.7.7.40 any (18 matches)

regards..

[3moloz123]

This has nothing to do with routes, as the switches, the router and my laptop are all on the same network.

If you read all posts, you'll see that the router does not answer on arp requests, or atleast it never reaches the clients.

I also know perfectly well which devices are hp and which are not. The second access rule was added for debugging, while the first is for allowing connections from one of my network monitor servers that reside in 10.1.0.0/29 network.

[glen.grant · Accepted solution]

   If you are running 2 hwic 4 esw's you have to stack them together to work correctly,

You have to tie one port from one hwic to aniother port on the other hwic , otherwise

it won't work correctly. Has this been done? If not they will work as isolated switches.  Go here to see how to configure stacking. http://www.cisco.com/en/US/docs/ios/12_3t/12_3t8/feature/guide/esw_cfg.html#wp1049119

Stacking of Cisco 10/100BASE-T Ethernet Switch HWICs

Stacking is the connection of two Ethernet switch HWICs resident in the  same chassis so that they behave as a single switch. Stacking is  accomplished by daisy-chaining the two cards together with an external  RJ-45 crossover cable that is connected to the specified stacking port  on each switch.

When a chassis is populated with two Ethernet switch HWICs, the user must configure the cards to operate in stacked mode.

---

Note There  is no option to unstack two Ethernet switch HWICs. When two Ethernet  switch HWICs are in the same chassis, they can operate only in stacked  mode. If you configure the cards to operate unstacked, they will not  operate correctly.

---

You must designate one port on each switch to be the stacking port. On  the HWIC-4ESW card, this port is nominally the first port (port 0),  although any port can be chosen. On the HWIC-D-9ESW card, this port is  nominally the ninth port (port 8), although any port can be chosen. We  recommend the use of port 8 as the stacking port, because it has been  designed as an extra port on the HWIC-D-9ESW card and does not provide  inline power.

---

Note Only one port on an Ethernet switch HWIC can be configured as a stacking port.

---

All combinations of Ethernet switch HWICs may be stacked: two  HWIC-D-9ESW cards, an HWIC-D-9ESW card with an HWIC-4ESW card, or two  HWIC-4ESW cards.

See the Configuration Guidelines for HWIC-4ESW and HWIC-D-9ESW Interface Cards document for information on how to configure stacking ports.

[3moloz123]

Thanks, this explains it all.

I should have started with the documentation of the HWICs.