Packet drop between FWSM and 6500

saifuddin.miyaji · ‎09-04-2011

Hi,

I have packet drop issue between FWSM and 6500, I believe. Packets transiting through a particular vlan are intermittently being dropped. I have taken packet capture on FWSM and it shows that some of the echo requests to a servers does not get response back. At the same time packet capute was taked on the NIC of server and it shows no problems, all echo requests are being replied back properly. So this concludes that the unreplied echo request never reached server.

'sh np blocks' has all 0's in threshhold values

Here is the output of the etherchannel beween FWSM and 6500 (of course created automatically)

DR-KMPLF#sh int po 578 counters etherchannel

Port InOctets InUcastPkts InMcastPkts InBcastPkts

Po578 30043323428709 42076894290 257207347 4018763

Gi1/2/1 176 1 0 1

Gi1/2/2 176 1 0 1

Gi1/2/3 850300 7625 0 577

Gi1/2/4 176 1 0 1

Gi1/2/5 176 1 0 1

Gi1/2/6 597674383 3552695 0 67

Port OutOctets OutUcastPkts OutMcastPkts OutBcastPkts

Po578 29904465074209 41224244267 388975402 2251786

Gi1/2/1 7190383150 27868993 21660120 39163

Gi1/2/2 1062054141 5637578 876883 644905

Gi1/2/3 738425356 2511 2088658 5327

Gi1/2/4 408526405 5787 3371754 4085

Gi1/2/5 649356137 4973 4267284 14212

Gi1/2/6 351483674 182554 1413610 67163

This is a VSS setup with latest software on FWSM (4.1[6]) and SUP (12.2(33)SXI3).

Experts, please advise

Saif

Richard Michael · ‎09-04-2011

Hello saifuddin,

Can you please give me more details on your problem,

How is your switch MAC aging timer and ARP timers are configured?

Is this issue happening only for one server or with multiple servers/hosts?

no mac-add aging-type routed-mac - Do you see this command in your switch?

Do you have MAC-address table Sync command enabled in your 6500(If you are using DFC line cards)?

Can you please share the Show module output from the switch, Need to know if you have a DFC linecards.

Share the output of sh mac-add all detail

Thanks,

Ricky Micky

saifuddin.miyaji · ‎09-04-2011

Hi Richard,

All the timers for MAC ageing are at default, we didn't change any of those.

The issue is visible on a few particular servers in a vlan, We dint check if the issue exists with other vlan as well.

'no mac-add aging-type routed-mac' command does NOT exist in the configuration.

------------------ show module switch all ------------------

Switch Number: 1 Role: Virtual Switch Active

---------------------- -----------------------------

Mod Ports Card Type Model Serial No.

--- ----- -------------------------------------- ------------------ -----------

1 48 CEF720 48 port 10/100/1000mb Ethernet WS-X6748-GE-TX SAL142XXXXX

2 6 Firewall Module WS-SVC-FWM-1 SAL142XXXXX

3 16 CEF720 16 port 10GE WS-X6716-10GE SAL142XXXXX

5 5 Supervisor Engine 720 10GE (Active) VS-S720-10G SAL142XXXXX

Mod MAC addresses Hw Fw Sw Status

--- ---------------------------------- ------ ------------ ------------ -------

1 c84c.75b8.0210 to c84c.75b8.023f 3.4 12.2(18r)S1 12.2(33)SXI3 Ok

2 5475.d062.57a0 to 5475.d062.57a7 4.5 7.2(1) 4.1(6) Ok

3 8843.e1d1.64c0 to 8843.e1d1.64cf 1.1 12.2(18r)S1 12.2(33)SXI3 Ok

5 0025.84bf.e108 to 0025.84bf.e10f 3.2 8.5(4) 12.2(33)SXI3 Ok

Mod Sub-Module Model Serial Hw Status

---- --------------------------- ------------------ ----------- ------- -------

1 Centralized Forwarding Card WS-F6700-CFC SAL142XXXXX 4.1 Ok

3 Distributed Forwarding Card WS-F6700-DFC3C SAL142XXXXX 1.4 Ok

5 Policy Feature Card 3 VS-F6K-PFC3C SAL142XXXXX 1.1 Ok

5 MSFC3 Daughterboard VS-F6K-MSFC3 SAL142XXXXX 5.0 Ok

Mod Online Diag Status

---- -------------------

1 Pass

2 Pass

3 Pass

5 Pass

Switch Number: 2 Role: Virtual Switch Standby

---------------------- -----------------------------

Mod Ports Card Type Model Serial No.

--- ----- -------------------------------------- ------------------ -----------

1 48 CEF720 48 port 10/100/1000mb Ethernet WS-X6748-GE-TX SAL142XXXXX

2 6 Firewall Module WS-SVC-FWM-1 SAL142XXXXX

3 16 CEF720 16 port 10GE WS-X6716-10GE SAL142XXXXX

5 5 Supervisor Engine 720 10GE (Hot) VS-S720-10G SAL142XXXXX

Mod MAC addresses Hw Fw Sw Status

--- ---------------------------------- ------ ------------ ------------ -------

1 c84c.75b8.0c90 to c84c.75b8.0cbf 3.4 12.2(18r)S1 12.2(33)SXI3 Ok

2 5475.d062.55e0 to 5475.d062.55e7 4.5 7.2(1) 4.1(6) Ok

3 8843.e1d1.5a70 to 8843.e1d1.5a7f 1.1 12.2(18r)S1 12.2(33)SXI3 Ok

5 c47d.4ffd.e360 to c47d.4ffd.e367 3.2 8.5(4) 12.2(33)SXI3 Ok

Mod Sub-Module Model Serial Hw Status

---- --------------------------- ------------------ ----------- ------- -------

1 Centralized Forwarding Card WS-F6700-CFC SAL142XXXXX 4.1 Ok

3 Distributed Forwarding Card WS-F6700-DFC3C SAL142XXXXX 1.4 Ok

5 Policy Feature Card 3 VS-F6K-PFC3C SAL142XXXXX 1.1 Ok

5 MSFC3 Daughterboard VS-F6K-MSFC3 SAL142XXXXX 5.0 Ok

Mod Online Diag Status

---- -------------------

1 Pass

2 Pass

3 Pass

5 Pass

------------------ show mac-address-table ------------------

Legend: * - primary entry

age - seconds since last seen

n/a - not available

vlan mac address type learn age ports

------+----------------+--------+-----+----------+--------------------------

14 e41f.1365.0b40 dynamic Yes 0 Po104

* 101 5475.d0e5.4900 dynamic Yes 0 Po594

* 151 3333.0001.0002 static Yes -

* --- 0000.0000.0000 static No - Router

13 e41f.1364.d310 dynamic Yes 0 Po103

13 e41f.1365.1304 dynamic Yes 10 Po101

13 e41f.1343.c898 dynamic Yes 0 Po103

* 1 0100.0ccc.cccc static No - Switch

* 1 c84c.7581.8000 static No - Router

* 19 5475.d0e5.5b80 dynamic Yes 170 Po594

13 e41f.1365.13c8 dynamic Yes 20 Po101

* 16 5475.d0e5.5b80 dynamic Yes 135 Po594

* 8 c84c.7581.8000 static No - Router

* 14 5475.d0e5.5b80 dynamic Yes 250 Po594

* 13 5475.d0e5.5b80 dynamic Yes 130 Po594

* 101 5475.d0e5.5b80 dynamic Yes 0 Po578

13 e41f.1365.0d64 dynamic Yes 15 Po104

13 e41f.1364.cf10 dynamic Yes 10 Po102

12 e41f.1365.0c70 dynamic Yes 5 Po102

13 e41f.1365.0404 dynamic Yes 5 Po103

* --- 0000.0000.aaaa static No - Switch

* --- c84c.7581.8000 static No - Router

16 e41f.1365.035c dynamic Yes 0 Po101

13 e41f.1365.047c dynamic Yes 20 Po103

* 151 d485.64cc.2b88 dynamic Yes 160 Gi1/1/5

* 205 c84c.7581.8000 static No - Router

* 151 d485.646b.d3ca dynamic Yes 5 Gi1/1/2

* 151 d485.646b.d3c8 dynamic Yes 0 Gi1/1/4

14 e41f.1365.04a4 dynamic Yes 10 Po104

* 151 5475.d0e5.5b80 dynamic Yes 5 Po594

13 e41f.1363.a53c dynamic Yes 10 Po101

8 0080.e518.0c54 dynamic Yes 170 Po101

* 8 3333.0001.0002 static Yes -

14 e41f.1363.fecc dynamic Yes 20 Po104

13 e41f.1361.8c18 dynamic Yes 0 Po101

13 e41f.1364.e494 dynamic Yes 20 Po102

13 e41f.1364.ea50 dynamic Yes 10 Po102

205 0026.9964.2440 dynamic Yes 0 Po3

* 12 5475.d0e5.5b80 dynamic Yes 95 Po594

* 15 5475.d0e5.5b80 dynamic Yes 215 Po594

* 7 c84c.7581.8000 static No - Router

* 19 c84c.7581.8000 static No - Router

Richard Michael · ‎09-05-2011

Hello saifuddin,

This is too early to predict but i guess you are hitting the bug CSCth04998. If you can try upgrading it to SXI5 or SXJ and let me know if this problem re-appears.

Thanks,

Ricky Micky

*Pls rate useful posts