Re: ZBPF IPV6 Problem

rafael_rung · ‎05-24-2011

Hello,

i have a issue with IP Version 6 and Zone-Based-Policy-Firewall.

this is the setup:

Router1 (Cisco 1802, IOS 15.1(3)T1, i also tried older IOS):
#######
Dialer 0 to IPV4 Internet
Dialer 6 to IPV6 Internet
Tunnel 0 configured as DMVPN to Router 2
VLAN 1 as internal Interface with IP 192.168.0.0/24 and 2001:xxxx:xxxx:2::/64

Router2 (Cisco 1802, IOS 15.1(3)T1, i also tried older IOS):
#######
Dialer 0 to IPV4 Internet
Tunnel 0 configured as DMVPN to Router 1
VLAN 1 as internal Interface with IP 10.0.0.0/24 and 2001:xxxx:xxxx:1::/64

Router1 connects to IPV6 Internet via Dialer 6
Router2 connects to IPV6 Internet via Tunnel0 (Dynamic-Multipoint-VPN)

Router1 and Router2 are configured with Zone-Based-Policy-Firewall.

When i remove the ZBPF Config from Router1 everything works!
#################
When i configure ZBPF on Router1 the IPV6 Connection between two Routers works in both directions,
the Internet Connection IPV6 from Router1 also works great
but the Internet Connection IPV6 from Router2 doesn't work anymore in the outgoing direction.

(Tunnel0 and VLAN1 are in the ZONE_BUERO_VLAN1, Dialer0 and Dialer6 are in ZONE_INTERNET)
zone security ZONE_INTERNET
zone security ZONE_BUERO_VLAN1

zone-pair security ZP_BUERO_VLAN1__INTERNET source ZONE_BUERO_VLAN1 destination ZONE_INTERNET
service-policy type inspect FW_TO_INTERNET
zone-pair security ZP_INTERNET__BUERO_VLAN1 source ZONE_INTERNET destination ZONE_BUERO_VLAN1
service-policy type inspect FW_INTERNET__BUERO_VLAN1

policy-map type inspect FW_TO_INTERNET
class type inspect FW_GRE
pass
class type inspect FW_ESP
pass
class type inspect FW_ALLES_ERLAUBT_INSPECT2
inspect
class class-default
drop log

policy-map type inspect FW_INTERNET__BUERO_VLAN1
class type inspect FW_GRE
pass
class type inspect FW_ESP
pass
class type inspect FW_IPV6_ALLES
drop log
class class-default
drop log

class-map type inspect match-any FW_GRE
match access-group name FW_GRE
class-map type inspect match-any FW_ESP
match access-group name FW_ESP
class-map type inspect match-any FW_ALLES_ERLAUBT_INSPECT2
match protocol http
match protocol https
match protocol ntp
match protocol dns

ip access-list extended FW_GRE
permit gre any any
ip access-list extended FW_ESP
permit esp any any
!
ipv6 access-list FW_IPV6_ALLES
permit ipv6 any any
permit icmp any any
permit tcp any any
permit udp any any

Debug Messages: (Seen on Router1)

Client in VLAN1 Router1 is browsing: (2001:xxxx:xxxx:2:C8FD:5EFE:523E:FB55)
- no debug - works -

Client in VLAN1 Router2 is browsing: (2001:xxxx:xxxx:1:C8FD:5EFE:5111:FB55)

*May 24 18:38:18.016: %FW-6-DROP_PKT: Dropping tcp session [2001:xxxx:xxxx:1:C8FD:5EFE:5111:FB55]:56687 [2A02:2E0:3FE:100::7]:80 on zone-pair ZP_BUERO_VLAN1__INTERNET class class-default due to DROP action found in policy-map with ip ident 0
Router1#
*May 24 18:38:48.465: %FW-6-DROP_PKT: Dropping tcp session [2001:xxxx:xxxx:1:C8FD:5EFE:5111:FB55]:56696 [2A02:2E0:3FE:100::7]:80 on zone-pair ZP_BUERO_VLAN1__INTERNET class class-default due to DROP action found in policy-map with ip ident 0
Router1#
*May 24 18:39:21.705: %FW-6-DROP_PKT: Dropping tcp session [2001:xxxx:xxxx:1:C8FD:5EFE:5111:FB55]:56747 [2A02:2E0:3FE:100::7]:80 on zone-pair ZP_BUERO_VLAN1__INTERNET class class-default due to DROP action found in policy-map with ip ident 0

so can you tell me how this it possible, even if i have the same zone membership of two interfaces it works for traffic from vlan 1 but it is blocked for traffic from tunnel 0???

Michael Sullenberger · ‎05-25-2011

I went over your setup with a TAC Firewall expert and we feel that what you have configured should work.

To continue with the troubleshooting you will need to turn on debugging, so we can see why it is not matching the FW_ALLES_ERLAUBT_INSPECT2

class. Probably the best debugs to start with are:

debug policy-firewall ?

 L2-transparent  Policy-Firewall transparent firewall debug
 control-plane   Policy-Firewall Control-plane debug
 detail          Policy-Firewall detailed debug
 events          Policy-Firewall events debug
 list            Policy-Firewall Conditional debug
 function-trace  Policy-Firewall function-trace debug
 obj-creation    Policy-Firewall object creations debug
 obj-deletion    Policy-Firewall object deletions debug
 packet-path     Policy-Firewall packet-path debug
 protocol        Policy-Firewall protocol debug
 timers          Policy-Firewall timers related events debug

detail and events. If that doesn't seem to give you enough information try also turning on packet-path and then function-trace.

Note, as you turn on more debugs it is going to get more verbose and more detailed.

If this doesn't seem to give you a reason that it is not matching you can try turning on the following debugs:

debug cce dp ?
all            Debugging CCE DP All
detailed       Debugging CCE Data-plane with more details
dynamic-class Debugging Dynamic Class functionality
function-trace Debugging CCE DP Timers
icmp           Debugging CCE ICMP classification
named-db       Debugging Named Database functionality
packet-path    Debugging CCE Packet Path
pi-pd-api      Debugging CCE DP Hardware API
policy-config Debugging CCE policy configuration
policy-trace   Debugging CCE Policy
stats          Debugging CCE DP Statistics
target         Debugging Target class group
timers         Debugging CCE DP Timers
udp            Debugging CCE UDP classification
vfr            Debugging VFR functionality
vtcp           Debugging VTCP functionality

Here you can try detailed, packet-path, policy-trace and even function-trace (in that order). function-trace is extermely detailed, so try that one last.

Hopefully this helps get you started in troubleshooting this.

Mike.

rafael_rung · ‎05-27-2011

Hello Mike,

thanks for the answer,

for my first test the

debug policy-firewall detail

command killed all my connections (ssh, vpn and so on) for some minutes so i will try it another time with a physical console cable attached.

Rafael

Phillip Remaker · ‎05-30-2011

What may have killed your conenction was the interrupt activity on the console port.

For any high volume debug, it is best to issue the configuration command

"no logging console"

prior to starting the debug. This suppresses messages to the physical console, but reduces the interrupt load in the device.

r.rung · ‎06-07-2011

Hello,

@Phillip: Thank you, you're right, that solved my "loosing connection" problems while debugging

so here are the debugs: (i made only a few of them, because if i enable to much of them i don't get the dropped packets in the debugs)

Router1#debug policy-firewall events

Policy-Firewall events debugging is on

Router1#debug policy-firewall detail

Policy-Firewall detailed debugging is on

*Jun 7 22:39:50.481: FIREWALL: FW CCE got packet 0x865A3430 in process path

*Jun 7 22:39:50.481: FIREWALL: NEW PAK 865A3430 [2001:xxxx:xxxx:1:8076:7A29:ABFB:708E]:52385 [2A02:2E0:3FE:100::7]:80 tcp

*Jun 7 22:39:50.481: FIREWALL: DROP feature object 0xAAAA000F found

*Jun 7 22:39:50.481: FIREWALL: FW CCE dropping pak 0x865A3430 in process path

Router1#debug policy-firewall packet-path

Policy-Firewall PAK_PATH debugging is on

packet from tunnel 0:

*Jun 7 22:43:45.781: CCE-FW :classify no match (srcaddr:port)-([2001:xxxx:xxxx:1:8076:7A29:ABFB:708E]:0) (dstaddr:port)-([2A02:2E0:3FE:100::7]:0)

*Jun 7 22:43:45.781: CCE-FW :ACCESS_GROUP_NAMED:CCE_DP_NAMED_DB_NOT_MATCHED: type_1_filter = 8764B300, acl = FW_ESP vers = 1

*Jun 7 22:43:45.781: CCE-FW :classify no match (srcaddr:port)-([2001:xxxx:xxxx:1:8076:7A29:ABFB:708E]:0) (dstaddr:port)-([2A02:2E0:3FE:100::7]:0)

*Jun 7 22:43:45.781: CCE-FW :L7 protocol match CCE_DP_NAMED_DB_NOT_MATCHED

*Jun 7 22:43:45.781: CCE-FW :classify no match (srcaddr:port)-([2001:xxxx:xxxx:1:8076:7A29:ABFB:708E]:0) (dstaddr:port)-([2A02:2E0:3FE:100::7]:0)

Router1#

*Jun 7 22:43:45.781: %FW-6-DROP_PKT: Dropping tcp session [2001:xxxx:xxxx:1:8076:7A29:ABFB:708E]:52403 [2A02:2E0:3FE:100::7]:80 on zone-pair ZP_BUERO_VLAN1__INTERNET class class-default due to DROP action found in policy-map with ip ident 0

*Jun 7 22:43:46.221: CCE-FW*:cce_dp_named_db_inspect_port_to_l7_protocol:L7 protocol is 0 L4 protocol is 0 address is [FE80::215:FAFF:FE0C:A70C] port is 0

*Jun 7 22:43:46.221: CCE-FW*:Packet L7 is 9 L4 prot is 0 granular is 0 (srcaddr:port)-([2001:xxxx:xxxx:2:80B6:E54A:A19:6435]:50900) (dstaddr:port)-([FE80::215:FAFF:FE0C:A70C]:22)

*Jun 7 22:43:46.221: CCE-FW*:L7 protocol match CCE_DP_NAMED_DB_NOT_MATCHED

*Jun 7 22:43:46.221: CCE-FW*:classify no match (srcaddr:port)-([2001:xxxx:xxxx:2:80B6:E54A:A19:6435]:50900) (dstaddr:port)-([FE80::215:FAFF:FE0C:A70C]:22)

*Jun 7 22:43:46.221: CCE-FW*:cce_dp_named_db_inspect_classify:packet 85977DC4 is not matched

packet from vlan1 (works)

*Jun 7 22:52:16.735: CCE-FW*:Packet L7 is 9 L4 prot is 0 granular is 0 (srcaddr:port)-([2A02:2E0:3FE:100::7]:50929) (dstaddr:port)-([2001:xxxx:xxxx:2:5800:62D5:A2AE:282C]:80)

*Jun 7 22:52:16.735: CCE-FW*:L7 protocol match CCE_DP_NAMED_DB_NOT_MATCHED

*Jun 7 22:52:16.735: CCE-FW*:classify no match (srcaddr:port)-([2A02:2E0:3FE:100::7]:50929) (dstaddr:port)-([2001:xxxx:xxxx:2:5800:62D5:A2AE:282C]:80)

*Jun 7 22:52:16.735: CCE-FW*:cce_dp_named_db_inspect_classify:packet 856B3D60 is not matched

*Jun 7 22:52:16.735: CCE-FW*:cce_dp_named_db_inspect_port_to_l7_protocol:L7 protocol is 0 L4 protocol is 0 address is [FE80::8C8F:658:1A14:661D] port is 0

*Jun 7 22:52:16.735: CCE-FW*:Packet L7 is 9 L4 prot is 0 granular is 0 (srcaddr:port)-([FE80::215:FAFF:FE0C:A70C]:50929) (dstaddr:port)-([FE80::8C8F:658:1A14:661D]:80)

*Jun 7 22:52:16.735: CCE-FW*:L7 protocol match CCE_DP_NAMED_DB_NOT_MATCHED

*Jun 7 22:52:16.735: CCE-FW*:classify no match (srcaddr:port)-([FE80::215:FAFF:FE0C:A70C]:50929) (dstaddr:port)-([FE80::8C8F:658:1A14:661D]:80)

*Jun 7 22:52:16.735: CCE-FW*:cce_dp_named_db_inspect_classify:packet 856B420C is not matched

*Jun 7 22:52:16.739: CCE-FW*:cce_dp_named_db_inspect_port_to_l7_protocol:L7 protocol is 0 L4 protocol is 0 address is [FE80::215:FAFF:FE0C:A70C] port is 0

*Jun 7 22:52:16.739: CCE-FW*:Packet L7 is 9 L4 prot is 0 granular is 0 (srcaddr:port)-([FE80::5800:62D5:A2AE:282C]:50929) (dstaddr:port)-([FE80::215:FAFF:FE0C:A70C]:80)

*Jun 7 22:52:16.739: CCE-FW*:L7 protocol match CCE_DP_NAMED_DB_NOT_MATCHED

*Jun 7 22:52:16.739: CCE-FW*:classify no match (srcaddr:port)-([FE80::5800:62D5:A2AE:282C]:50929) (dstaddr:port)-([FE80::215:FAFF:FE0C:A70C]:80)

*Jun 7 22:52:16.739: CCE-FW*:cce_dp_named_db_inspect_classify:packet 85929DEC is not matched

*Jun 7 22:52:16.739: CCE-FW*:cce_dp_named_db_inspect_port_to_l7_protocol:L7 protocol is 0 L4 protocol is 0 address is [FE80::8C8F:658:1A14:661D] port is 0

Router1#

*Jun 7 22:52:17.303: CCE-FW*:ACCESS_GROUP_NAMED:CCE_DP_NAMED_DB_MATCHED: type_1_filter = 8764B300, acl = FW_ESP vers = 0

*Jun 7 22:52:17.303: CCE-FW*:Matched acl, user group or insp protocols

*Jun 7 22:52:17.303: CCE-FW*:L7 is 9 l7 token prot is 0

*Jun 7 22:52:17.331: CCE-FW*:cce_dp_named_db_inspect_port_to_l7_protocol:L7 protocol is 0 L4 protocol is 0 address is 217.92.41.131 port is 0

debug policy-firewall function-trace

- no debug output -

my results are that with "debug policy-firewall packet-path" i can't really see where it matches, but i can see that if the packet comes from tunnel 0 the debug can't see the destination port and therefor cant't match the packet.

i also tried to implement a rule that allows all traffic, not only http: then it is not dropped any more, but the reverse packet is dropped because the stateful inspection doesn't work for that packet.

for me it seems to be a bug.....what do you think?

sylvain.munaut · ‎07-06-2011

Any news on this ?

Phillip Remaker · ‎07-08-2011

Looks like it is a regulart http (port 80) conection but is being dropped desipte allowing http.

May be a bug, or maybe I don't understand something.

I would go ahead an open a TAC case.

rafael_rung · ‎07-09-2011

Hello Phillip,

thank you for your answer. i also think that this is a bug, but until now i have this problem only in my test lab and i don't think that cisco is happy if i open a case without a customer that is affected.

so i downgraded the ios to a version where ZBPF doesn't support IPv6...

sylvain.munaut · ‎07-09-2011

I would think Cisco would be happy for any reported bug. Especially if it gives them a chance to fix the issue _before_ it affects someone at a production site.