Logging drop packets on the devices with zbf inspect

[dijix1990]

I'm interested.

how can I see drop packets on the device's buffer (show log) if I chose action inspect?

 1. I tried to enable action "Audit Trail" but it didn't show the drop packets. To be honest it didn't show sessions from outside to device - self-zone (I have FW for self-zone) I only saw session from self-zone to outside

2. I tried to use "Network Wide path insight" but it didn't show information about VPN 0

maybe it doesn't have function to see drop packets on the device and I need to configure external syslog server

[Kanan Huseynli]

Hi,

when you enable audit-trail it is applied only to start and stop of session. Dropped packets (due to inspection) is not logged.

Enable High-Speed Logging for Firewall Actions section from:

https://www.cisco.com/c/en/us/td/docs/routers/sdwan/configuration/security/ios-xe-17/security-book-xe/m-firewall-17.html

• Enable audit trail messages.
  
  You can enable audit-trail to a parameter map to record the start, stop, and duration of a connection or session, and the source and destination IP addresses
  
  Device(config-profile)# audit-trail on 
  
  

When audit-trial is enabled below policy-map is added to router configuration and also it is "called" in inspect policies:

parameter-map type inspect audit-trail-pmap_
 audit-trial on

class type inspect FW-POLICY-seq-11-cm_
inspect audit-trail-pmap_

With above expected configuration, you have logs when session is created or stopped as below:

*Jun 23 20:23:03.764: %IOSXE-6-PLATFORM: R0/0: cpp_cp: QFP:0.0 Thread:001 TS:00000003567570530310 %FW-6-SESS_AUDIT_TRAIL_START: (target:class)-(ZP_VPN1_VPN1_FW-POLICY:FW-POLICY-seq-11-cm_):Start session: initiator (10.1.1.253:56241) -- responder (10.2.1.253:23) from GigabitEthernet2 (srcvrf:dstvrf)-(1:1)
Site1-RTR1#
*Jun 23 20:23:18.627: %IOSXE-6-PLATFORM: R0/0: cpp_cp: QFP:0.0 Thread:000 TS:00000003582434894655 %FW-6-SESS_AUDIT_TRAIL: (target:class)-(ZP_VPN1_VPN1_FW-POLICY:FW-POLICY-seq-11-cm_):Stop session: initiator (10.1.1.253:56241) sent 77 bytes -- responder (10.2.1.253:23) sent 254 bytes, from GigabitEthernet2

Now, I'm doing reverse telnet (from 10.2.1.253 to 10.1.1.253), in this case Syn is allowed (from overlay to LAN), but Syn-Ack will not be allowed (I've intra-VPN inspect rule between these 2 different site IPs), but there is no log (inspect is not logged in this way and there is no log or not log checkbox in configuration).

What can be done, add configuration to audit-trail policy-map with CLI-template:

parameter-map type inspect audit-trail-pmap_
 log dropped-packets

*Jun 23 20:32:59.529: %IOSXE-6-PLATFORM: R0/0: cpp_cp: QFP:0.0 Thread:001 TS:00000004163336243357 %FW-6-DROP_PKT: Dropping tcp/unknown pkt from GigabitEthernet2 10.1.1.253:23 => 10.2.1.253:17121(target:class)-(ZP_VPN1_VPN1_FW-POLICY:FW-POLICY-seq-11-cm_) due to Invalid TCP initiator with ip ident 48839 tcp flag 0x12, seq 1925920251, ack 153385589 (srcvrf:dstvrf)-(1:1) (srcvpn:dstvpn)-(1:1)

As you see, it is now logged.

[dijix1990]

Thanks for sharing you investigate! I've already had cli template with (but it doesn't work) 

parameter-map type inspect global
    log dropped-packets

I use it for legacy network, but yours is the different than mine, I will check you example. Maybe it's because of I need to enable audit trail with log dropped-packets? 

parameter-map type inspect audit-trail-pmap_
 log dropped-packets

[Kanan Huseynli]

---

 Maybe it's because of I need to enable audit trail with log dropped-packets? 

parameter-map type inspect audit-trail-pmap_
 log dropped-packets

---

Yes, only after this it worked. log dropped-packets should be enabled for audit-trial policy-map.

[dijix1990]

don't know why, but doesn't work. I don't see any drops from internet to public ip on the vpn0

I added log dropped to the parameter-map type inspect audit-trail-pmap_ and enable audit

parameter-map type inspect-global
 log dropped-packets
 multi-tenancy
 vpn zone security
 alert on
parameter-map type inspect audit-trail-pmap_
 audit-trail on
 log dropped-packets

 I can see start and stop session and thats all

when I try to ping or ssh from 10.10.100.3 there aren't drops in the log

When I try to telnet to 443 from 10.10.100.2 there aren't drops in the log

[Kanan Huseynli]

What is inter-zone configuration in your case?

EDIT:

I see your zone-pair configuration, self-zone is configured for both source and destination.  See what is written in "Apply Policy to a Zone Pair" of above shared configuration guide:

To apply policy to a zone pair:

1. Create security policy using Cisco vManage. For information see, Start the Security Policy Configuration Wizard.

2. Click Apply Zone-Pairs.

3. In the Source Zone field, choose the zone that is the source of the data packets.

4. In the Destination Zone field, choose the zone that is the destination of the data packets.

   Note	You can choose self zone for either a source zone or a destination zone, not both.

It seems, this results misbehavior in your case.

Create separate zone-pair rule

[Kanan Huseynli]

Also, how does your topology look like? 10.10.100.2 is router VPN0 interface? 10.10.100.3 is any node in outside? Is 10.10.100.2 NAT-ed?

share "sh policy-firewall config" as well from router

[dijix1990]

10.10.100.1 it's cedge (branch) with config zbf

10.10.100.2 it's cedge (dc HQ) 

10.10.100.3 it's linux

[dijix1990]

I will share tomorrow (it's very big configuration. I have 10 service vpn's and for every vpn's about 70 rules)

[dijix1990]

You mean I need two rules? one for traffic from VPN0 to SELF and Second for SELF to VPN0?

About two years ago TAC adviced me config like this, one rule for in/out traffic 

Before I had 3 rules and so I used self-zone is configured for both source and destination

1. 10.10.100.2 to 10.10.100.1 ssh
2. 10.10.100.1 to 10.10.100.10 syslog
3. any to 10.10.100.1 icmp

[Kanan Huseynli]

Ignore what I mentioned before regarding this. Self to self is not allowed in configuration, but if you have self to another-zone and also another zone to self, config is accepted.

I played a bit, it seems when traffic matches to default rule (in your case it is default deny), session is not logged. Traffic initiated outside does not match any rule statement but matches default deny and dropped without log. You can create explicit deny with log checkbox enabled .

*Jun 24 17:10:52.032: %IOSXE-6-PLATFORM: R0/0: cpp_cp: QFP:0.0 Thread:001 TS:00000078435838974722 %FW-6-DROP_PKT: Dropping tcp pkt from GigabitEthernet1 172.20.2.2:5063 => 172.20.2.1:443(target:class)-(ZP_Transport_self_Self_Transport:Self_Transport-seq-61-cm_) due to Policy drop:classify result with ip ident 58721 tcp flag 0x2, seq 3423449996, ack 0 (srcvpn:dstvpn)-(0:65534)

[Kanan Huseynli]

The reason for this behavior is default parameter map.

Site1-RTR1#sh parameter-map type inspect default | inc log
log dropped-packet off
log flow off

Below is zone-pair configuration (only self to transport, to limit output transport to self is not shown, both have the same rules) of picture from previous comment:

Zone-pair : ZP_self_Transport_Self_Transport
Source Zone : self
Description: System defined zone
Destination Zone : Transport
Service-policy inspect : Self_Transport
Class-map : Self_Transport-seq-1-cm_ (match-all)
Match access-group name Self_Transport-seq-Inspect_SSH_Out-acl_
Action : inspect
Parameter-map : Default
Class-map : Self_Transport-seq-11-cm_ (match-all)
Match access-group name Self_Transport-seq-Rule_2-acl_
Action : pass log
Parameter-map : Default
Class-map : Self_Transport-seq-21-cm_ (match-all)
Match access-group name Self_Transport-seq-Rule_3-acl_
Action : pass log
Parameter-map : Default
Class-map : Self_Transport-seq-31-cm_ (match-all)
Match access-group name Self_Transport-seq-Rule_4-acl_
Action : pass log
Parameter-map : Default
Class-map : Self_Transport-seq-41-cm_ (match-all)
Match access-group name Self_Transport-seq-Rule_5-acl_
Action : pass log
Parameter-map : Default
Class-map : Self_Transport-seq-51-cm_ (match-all)
Match class-map Self_Transport-sRule_6-l4-cm_
Match access-group name Self_Transport-seq-Rule_6-acl_
Action : inspect
Parameter-map : Default
Class-map : class-default (match-any)
Match any
Action : drop log
Parameter-map : Default

 What is interesting, output shows with "log" keyword for any action. Seems, output issue, because actual config does not have log enabled:

policy-map type inspect Self_Transport
class type inspect Self_Transport-seq-1-cm_
inspect audit-trail-pmap_
class type inspect Self_Transport-seq-11-cm_
pass
class type inspect Self_Transport-seq-21-cm_
pass
class type inspect Self_Transport-seq-31-cm_
pass
class type inspect Self_Transport-seq-41-cm_
pass
class type inspect Self_Transport-seq-51-cm_
inspect audit-trail-pmap_
class class-default
drop

zone-pair security ZP_Transport_self_Self_Transport source Transport destination self
service-policy type inspect Self_Transport
zone-pair security ZP_self_Transport_Self_Transport source self destination Transport
service-policy type inspect Self_Transport 

Below is zone-pair configuration (only self to transport, to limit output transport to self is not shown, both have the same rules) of when audit-trial is enabled in policy summary:

Zone-pair : ZP_self_Transport_Self_Transport
Source Zone : self
Description: System defined zone
Destination Zone : Transport
Service-policy inspect : Self_Transport
Class-map : Self_Transport-seq-1-cm_ (match-all)
Match access-group name Self_Transport-seq-Inspect_SSH_Out-acl_
Action : inspect
Parameter-map : audit-trail-pmap_
Class-map : Self_Transport-seq-11-cm_ (match-all)
Match access-group name Self_Transport-seq-Rule_2-acl_
Action : pass log
Parameter-map : Default
Class-map : Self_Transport-seq-21-cm_ (match-all)
Match access-group name Self_Transport-seq-Rule_3-acl_
Action : pass log
Parameter-map : Default
Class-map : Self_Transport-seq-31-cm_ (match-all)
Match access-group name Self_Transport-seq-Rule_4-acl_
Action : pass log
Parameter-map : Default
Class-map : Self_Transport-seq-41-cm_ (match-all)
Match access-group name Self_Transport-seq-Rule_5-acl_
Action : pass log
Parameter-map : Default
Class-map : Self_Transport-seq-51-cm_ (match-all)
Match class-map Self_Transport-sRule_6-l4-cm_
Match access-group name Self_Transport-seq-Rule_6-acl_
Action : inspect
Parameter-map : audit-trail-pmap_
Class-map : class-default (match-any)
Match any
Action : drop log
Parameter-map : Default

As you see, parameter-map is changed and since I additionally added "logging dropped-packets", it now logs dropped packets for inspect rules.

[dijix1990]

Hm, I have vmanage 20.9.3 and by default default parameter map goes with log enable when I apply security policy

[Kanan Huseynli · Accepted solution]

Don't confuse default with global. Whatever you see in policy summary is for global parameter map and log is enabled there. However, it is not parameter map that is used by class-maps.

Also, class-maps can have its own log option, if you check it in GUI. But there is no checkbox for default action (seems missing feature). So, you need explicit rule set instead of default rule set (with class-default).

Note: maybe additional CLI-template would also help, but better to have one more line as explicit deny.

[dijix1990]

With explicit drop log it works, but I got Traceback error. It leaved when I delete explicit log

Jun 27 07:34:22.597: %CPPOSLIB-3-ERROR_NOTIFY: F0/0: cpp_cp_svr: cpp_cp encountered an error -Traceback= 1#a6cf5f9f42f7c50f651190d386f15edb  errmsg:7F3C2E8A0000+E9A cpp_common_os:7F3C34F0B000+1D5B0 cpp_common_os:7F3C34F0B000+13FA3 cpp_fw_svr_lib:7F3C4F493000+71402 cpp_common_os:7F3C34F0B000+24A59 evlib:7F3C31E56000+8FF6 evlib:7F3C31E56000+9D50 cpp_common_os:7F3C34F0B000+22BA4 cpp_common_os:7F3C34F0B000+22B24 :55EC2C3C9000+3D20C c:7F3C286F9000+27ED0 :55EC2C3C9000+2F34A