Switch C9300 Checking

[sangtx92]

Dear Every One,

Currently, according to the attached network diagram, I am experiencing a situation where network devices below the Firewall layer sometimes lose connection to the network (cannot ping the IP addresses of access switches).
I checked the following steps and found that Port Gi2/0/2 had packet drop status as shown in the attached picture.

- Using SNMP polling of that port at one-minute intervals
   + The attached PNG file shows
       +9:34~9:35, About 400 packets were discarded even though output traffic was only 60Mbps
       +9:58~9:59, Approximately 2,700 packets were discarded

Can anyone tell me if this is related to a bug and is there any way to investigate more specifically?

[marce1000]

                 >...is there any way to investigate more specifically?
  - Configure a central syslog server for all involved Cisco equipment , and follow up on logging arriving at it , also during normal production times , because that can bring insights too ,

 M.

[Torbjørn]

As @marce1000 said you should consider implementing a central logging server. In the meantime, can you post the logs from the 9300 stack?

[ahmedshoaib]

Dear;

The Packet drop is generally due to physical layer issue (i.e. Cable, SFP and some time grounding of devices). As per my understanding after view the topology your case might be cable need to replaced & verify, verify SFP on Firewall side (if you are using).

Are you face the packet drop issue if you failover the firewall?  

Just want to double check whether you are running Port-channel b/w Catalyst 9300 and Firewall. (Red circle create confusing to me)

Thanks

[sangtx92]

Dear Ahmedshoaib,

I'm sorry for not taking careful notes, but the red circle shows I am using the port channel on both the switch's and Firewall's.

Let me provide more information about Firewall devices, currently, 2 Fortigate 201F Firewalls are configured HA with transparent mode.

When a connection loss occurs, the C9300 Switch will not be able to ping the devices behind the Firewall layer (this is only occasionally, not often).

Thanks!

[ahmedshoaib]

Dear Sangtx;

Now as you mentioned you are running port-channel b/w Switches & Firewall. There will be 2 scenario:

1. Both ports on Switch (1/0/2 & 2/0/2) part of single port channel & Port 1 at Fortigate FW also configure port-channel.

If you configure above scenario is wrong configuration. Switches send the traffic on both port of switches, while Fortigate firewall configure as Active/Passive. All traffic toward Passive firewall will drop.

You need to remove the 2nd port toward the passive firewall from the port-channel

2. If you configure 2 port-channel on switches (1 port-channel for each firewall).

Design is correct but there is no sense to configure port-channel and assign 1 interface in port-channel.

Thanks

[MHM Cisco World]

PO to FW need to not cross' i.e. 

One FW connect to both SW via ONE PO

Other FE connect to both SW via Other PO

The SW see two different device in one PO and hence it suspend one link ypu can check that by 

Show etherchannel summary 

Or 

Show port-channel summary 

Also dow 

Show lacp neighbor <<- share this here 

MHM

[sangtx92]

Dear Ahmedshoaib,

I have just rechecked the configuration and want to update you with the information below.

• The 201F Firewall is operating in HA mode (Active/Passive).
• The C9300 Switch (Gi1/0/2 and Gi2/0/2) is not configured for Port-Channel but is in trunk mode.

According to the information you shared above:

• The packets passing through the Passive Firewall -> Port Gi1/0/2 will be dropped, and this port needs to be removed (I understand).
• However, why are the packets passing through the Active Firewall -> Port Gi2/0/2 on the C9300 Switch also being dropped? This part is confusing to me.

Based on your experience, what could be the possible cause for this?

Thanks!

[ahmedshoaib]

Dear Sangtx;

Thanks for clarification with reference to Port-channel.

Now the issue most-probably the patch-cord or port issue (either Switch or Firewall side). You can verify 1 by 1 as below:

1. Failover the Firewall & verify the packet drop. It will eliminated the Cable & Firewall & Switch port issue. If the problem fixed means issue identified problem b/w Sw 2/0/2 & Primary Firewall.

2. Replace the patch-cord b/w Switch G2/0/2 and Primary Firewall.

3.  Try to change the port 1 by 1 on switch side (G2/0/2 to G2/0/X) which is easily will be manageable. Then try to reconfigure the port on Firewall.

4. Can you double check the cable path there will be not magnetic field (cable not mess with power cord). Or there will be less chance but need to double check the devices/cabinet should be properly ground (ask your electrician).

Thanks & Best regards;