Acess-list issue on Cisco Router 3945

[Claudio Minnetti]

Hi All,

I reported a really strange issue on a Cisco Router 3945. Here below info about release software used:

IOS:C3900-UNIVERSALK9-M, Version 15.2(2)T, Release software (fc1)

ROM: System Bootstrap, Version 15.0(1r)M13, release software (fc1)

System image file: "flash0:c3900-universalk9-mz.SPA.152-2.T.bin"

Please look at a brief extract of router running configuration file:

"

ip access list extended 180

permit ip any any

interface fastethernet 0/0/1

ip access-group 180 in

"

It’s an easy configuration of Extended ACL and the application on an Ethernet interface. The expected result is:

- The interface works properly (because access list is permitting every kind of data traffic in input)

- Checking “show access-list 180”, the counter of matched packets increments for all the packets that are forwarded inside the fa0/0/1.

But actually the Fastethernet 0/0/1 drops all the packets as if all the packets don’t match with access list (And this behavior is really incredible). The interface couldn't be used anymore because any kind of data traffic is denied.

I hope I made myself clear with the explanation of that issue.

Thank you

[Richard Burts]

Interface fastethernet0/0/1 does not look to me like one of the built in interfaces for the 3945. Can you tell us about this interface and what it is? Perhaps it would also help if you would post the output of show version and of show ip interface brief.

I am wondering if there are some restrictions on this interface, such as its being a layer 2 switching interface rather than a layer 3 interface?

HTH

Rick

[Claudio Minnetti]

Dear Richard,

The fastethernet interface is a HWIC-2FE mounted into Cisco router 3945. Here below the "show ip int fa0/0/1":

Router_1#show ip int fa0/0/1

FastEthernet0/0/1 is up, line protocol is up

Internet address is 10.255.1.252/24

Broadcast address is 255.255.255.255

Address determined by non-volatile memory

MTU is 1500 bytes

Helper address is not set

Directed broadcast forwarding is disabled

Multicast reserved groups joined: 224.0.0.2 224.0.0.5

Outgoing access list is not set

Inbound access list is not set

Proxy ARP is enabled

Local Proxy ARP is disabled

Security level is default

Split horizon is enabled

ICMP redirects are always sent

ICMP unreachables are always sent

ICMP mask replies are never sent

IP fast switching is enabled

IP fast switching on the same interface is disabled

IP Flow switching is disabled

IP CEF switching is enabled

IP CEF switching turbo vector

IP multicast fast switching is enabled

IP multicast distributed fast switching is disabled

IP route-cache flags are Fast, CEF

Router Discovery is disabled

IP output packet accounting is disabled

IP access violation accounting is disabled

TCP/IP header compression is disabled

RTP/IP header compression is disabled

Policy routing is enabled, using route map Turksat

Network address translation is disabled

BGP Policy Mapping is disabled

Input features: Policy Routing, MCI Check

WCCP Redirect outbound is disabled

WCCP Redirect inbound is disabled

WCCP Redirect exclude is disabled

Here below "show version" and "show ip int brief":

Router_1#show version

Cisco IOS Software, C3900 Software (C3900-UNIVERSALK9-M), Version 15.2(2)T, RELEASE SOFTWARE (fc1)

Technical Support: http://www.cisco.com/techsupport

Copyright (c) 1986-2011 by Cisco Systems, Inc.

Compiled Wed 16-Nov-11 00:34 by prod_rel_team

ROM: System Bootstrap, Version 15.0(1r)M13, RELEASE SOFTWARE (fc1)

Router_1 uptime is 1 week, 1 day, 23 hours, 53 minutes

System returned to ROM by power-on

System restarted at 08:16:01 UTC Sat Mar 10 2012

System image file is "flash0:c3900-universalk9-mz.SPA.152-2.T.bin"

Last reload type: Normal Reload

Last reload reason: power-on

This product contains cryptographic features and is subject to United

States and local country laws governing import, export, transfer and

use. Delivery of Cisco cryptographic products does not imply

third-party authority to import, export, distribute or use encryption.

Importers, exporters, distributors and users are responsible for

compliance with U.S. and local country laws. By using this product you

agree to comply with applicable laws and regulations. If you are unable

to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:

http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to

export@cisco.com.

Cisco CISCO3945-CHASSIS (revision 1.0) with C3900-SPE150/K9 with 2029568K/67584K bytes of memory.

Processor board ID FCZ155220G9

6 FastEthernet interfaces

3 Gigabit Ethernet interfaces

1 terminal line

DRAM configuration is 72 bits wide with parity enabled.

255K bytes of non-volatile configuration memory.

497448K bytes of ATA System CompactFlash 0 (Read/Write)

License Info:

License UDI:

-----------------------------------------------------------------

Device#    PID                                        SN

------------------------------------------------------------------

*0             C3900-SPE150/K9       FOC15463NPQ    

Technology Package License Information for Module:'c3900'

-----------------------------------------------------------------

Technology   Technology-package           Technology-package

                      Current       Type                 Next reboot

------------------------------------------------------------------

ipbase           ipbasek9     Permanent       ipbasek9

security         None           None                None

uc                 None           None                None

data              None           None                None

Configuration register is 0x2102

Router_1#show ip int brief

Interface                                       IP-Address     OK? Method       Status                          Protocol

Embedded-Service-Engine0/0      unassigned     YES NVRAM      administratively down   down  

GigabitEthernet0/0                        unassigned     YES NVRAM      up                                up    

GigabitEthernet0/0.400                172.16.0.2     YES NVRAM        up                                up    

GigabitEthernet0/1                       100.1.7.252     YES NVRAM      up                                up    

GigabitEthernet0/2                       100.1.3.252     YES NVRAM      up                                up    

FastEthernet0/0/0                        100.1.8.252     YES NVRAM      up                                up    

FastEthernet0/0/1                        10.255.1.252   YES NVRAM      up                                up

down

Thank you in advance for your support

Claudio

[Marko Leopold]

Your interface is administratively down? Is this right? Or was it just at the time when you made the output of the commands.

[Richard Burts]

Claudio

Thank you for the additional information which I requested. It is helpful to know that the interface is part of HWIC-2FE. This confirms that it should operate as a normal layer 3 interface and should do normal processing of access lists. And that makes this issue more puzzling.

If you want to investigate this further I would suggest to re-configure the access list and apply it to the interface. After you have run some traffic and observe the problem is still happening then post the output of show run interface fast0/0/1, the output of show ip interface fast0/0/1, and of show access list.

An additional question would be whether you would experience the same problem on the interface fast0/0/0 and on any of the built in Gig interfaces? I am wondering if it might be something related to the HWIC-2FE?

HTH

Rick

[Vasileios Bouloukos MSc, ITIL, CCIE]

Does this interface forward the traffic properly without this access-list applied?

[Claudio Minnetti]

Dear Vasileios,

the interface is working properly without the access-list applied.

Thank you

Claudio

[Vasileios Bouloukos MSc, ITIL, CCIE]

Hi Claudio,

I have a few comments from your show ip interface outputs.

It is stated the next for the fa0/1/1 interface.

Outgoing access list is not set

Inbound access list is not set

Did you have apply the access-list before this show command?

I also noticed a policy routing have been applied to the show ip interface

Policy routing is enabled, using route map Turksat

Could possibly cause a confilict with the routing via this interface?

Finally, could you try to also log the packets with the access-list and check these captured packets.

permit ip any any log

Do you have any errors to this interface?

Attention, if you use this router to a production network and the interface has high traffic utilization.

BR

Vasilis

[Richard Burts]

Vasilis

This is a very good observation that Polic Based Routing is applied to the interface. We certainly need to know what is in the configuration of PBR.

HTH

Rick

[Claudio Minnetti]

Dear all,

I tried to apply the access-list also to other interfaces (on board as gi0/1 or in another HWIC-2FE as fa0/1/0) but the issue persist.

Then, I erased the PBR configuration and the problem persist again.

Here below I report "show ip int fa0/0/1", "show in fa0/0/1" and "show access-list 180" after applying the access-list in the fa0/0/1 interface:

Router_2#show ip int fa0/0/1

FastEthernet0/0/1 is up, line protocol is up

Internet address is 10.255.1.252/24

Broadcast address is 255.255.255.255

Address determined by non-volatile memory

MTU is 1500 bytes

Helper address is not set

Directed broadcast forwarding is disabled

Multicast reserved groups joined: 224.0.0.2 224.0.0.5 224.0.0.6

Outgoing access list is not set

Inbound access list is 180

Proxy ARP is enabled

Local Proxy ARP is disabled

Security level is default

Split horizon is enabled

ICMP redirects are always sent

ICMP unreachables are always sent

ICMP mask replies are never sent

IP fast switching is enabled

IP fast switching on the same interface is disabled

IP Flow switching is disabled

IP CEF switching is enabled

IP CEF switching turbo vector

IP multicast fast switching is enabled

ICMP redirects are always sent

ICMP unreachables are always sent

ICMP mask replies are never sent

IP fast switching is enabled

IP fast switching on the same interface is disabled

IP Flow switching is disabled

IP CEF switching is enabled

IP CEF switching turbo vector

IP multicast fast switching is enabled

IP multicast distributed fast switching is disabled

IP route-cache flags are Fast, CEF

Router Discovery is disabled

IP output packet accounting is disabled

IP access violation accounting is disabled

TCP/IP header compression is disabled

RTP/IP header compression is disabled

Policy routing is disabled

Network address translation is disabled

BGP Policy Mapping is disabled

Input features: Access List, MCI Check

WCCP Redirect outbound is disabled

WCCP Redirect inbound is disabled

WCCP Redirect exclude is disabled

Router_2#show int fa0/0/1

FastEthernet0/0/1 is up, line protocol is up

Hardware is FastEthernet, address is c471.fe1b.a712 (bia c471.fe1b.a712)

Description: Voice

Internet address is 10.255.1.252/24

MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100 usec,

     reliability 255/255, txload 255/255, rxload 1/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Full-duplex, 100Mb/s, 100BaseTX/FX

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:00, output 00:00:00, output hang never

Last clearing of "show interface" counters never

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue: 0/40 (size/max)

5 minute input rate 42000 bits/sec, 33 packets/sec

5 minute output rate 4000 bits/sec, 7 packets/sec

     591386 packets input, 47238160 bytes

     Received 97102 broadcasts (0 IP multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 watchdog

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Full-duplex, 100Mb/s, 100BaseTX/FX

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:00, output 00:00:00, output hang never

Last clearing of "show interface" counters never

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue: 0/40 (size/max)

5 minute input rate 42000 bits/sec, 33 packets/sec

5 minute output rate 4000 bits/sec, 7 packets/sec

     591386 packets input, 47238160 bytes

     Received 97102 broadcasts (0 IP multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 watchdog

     0 input packets with dribble condition detected

     501910 packets output, 40504970 bytes, 0 underruns

     0 output errors, 0 collisions, 1 interface resets

     0 unknown protocol drops

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier

     0 output buffer failures, 0 output buffers swapped out

Router_2# show access-lists 180

Extended IP access list 180

   10 permit ip any any

[Marko Leopold · Accepted solution]

You got a txload of 100%?

Router_2#show int fa0/0/1

FastEthernet0/0/1 is up, line protocol is up

Hardware is FastEthernet, address is c471.fe1b.a712 (bia c471.fe1b.a712)

Description: Voice

Internet address is 10.255.1.252/24

MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100 usec,

reliability 255/255, txload 255/255, rxload 1/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Full-duplex, 100Mb/s, 100BaseTX/FX

[Claudio Minnetti]

Dear Marko,

I confirm that value. Even if I disable the access-list applied in that interface and there is no data traffic through it the txload is 255/255. I found that value not only in fa0/0/1 interface but in all the interfaces of that router.

Claudio

[Vasileios Bouloukos MSc, ITIL, CCIE]

Hi Claudio,

I  aslo noticed that you have the next bitrates to this interface

5 minute input rate 42000 bits/sec, 33 packets/sec

5 minute output rate 4000 bits/sec, 7 packets/sec

I recommend you to check the process of the router and check the processes with high value

sh proc cpu sorted

sh proc cpu history

Provide the output.

Try to capture these packets with debug ip packet AL, access-list which log packets or even better with Wireshark.

It is needed to find out what are these packets and why are produced

BR

Vasilis

[Claudio Minnetti]

Dear all,

I solved that issue with a downgrade of the IOS. Now I'm using a:

c3900-universalk9-mz.SPA.151-3.T2

and all the access-list are working properly and also txload now is 1/255.

Probably the IOS C3900-UNIVERSALK9-M, Version 15.2(2)T has got a bug regarding that.

Thank you all for your support

Claudio

[Richard Burts]

Claudio

Thanks for posting back to the forum indicating that you had solved the problem by using a different version of IOS. I had been thinking that the symptoms sounded like a possible IOS bug and had been looking for some reference that would support that. Thanks for confirming that this was indeed the issue. Now that the problem is solved perhaps you would mark the issue as answered so that other readers would know that there is a solution?

HTH

Rick