Couldn't Clear Bus

nanang · ‎09-10-2021

We have problem in Nexus 3064PQ 10GX,

2021 Sep 11 04:50:03 NX-MMR-LDP3 Sep 11 04:38:20 %KERN-2-SYSTEM_MSG: [701137.055200] clk_flush: Couldn't Clear Bus, Attempt 2 of 5 - kernel

2021 Sep 11 04:50:03 NX-MMR-LDP3 Sep 11 04:38:20 %KERN-2-SYSTEM_MSG: [701137.055220] clk_flush: Couldn't Clear Bus, Attempt 3 of 5 - kernel

2021 Sep 11 04:50:03 NX-MMR-LDP3 Sep 11 04:38:20 %KERN-2-SYSTEM_MSG: [701137.055241] clk_flush: Couldn't Clear Bus, Attempt 4 of 5 - kernel

2021 Sep 11 04:50:03 NX-MMR-LDP3 Sep 11 04:38:22 %KERN-2-SYSTEM_MSG: [701139.067808] clk_flush: Couldn't Clear Bus, Attempt 1 of 5 - kernel

2021 Sep 11 04:50:03 NX-MMR-LDP3 Sep 11 04:38:22 %KERN-2-SYSTEM_MSG: [701139.067958] clk_flush: Couldn't Clear Bus, Attempt 1 of 5 - kernel

2021 Sep 11 04:50:03 NX-MMR-LDP3 Sep 11 04:38:22 %KERN-3-SYSTEM_MSG: [701139.068055] neutron_usd/3469 - failed to set mux addr 0x73 ch 64 err I2C_

ERR_SMBUS_FAILED - kernel

LDP3# show system resources

Load average: 1 minute: 1.48 5 minutes: 1.42 15 minutes: 1.42

Processes : 331 total, 4 running

CPU states : 13.50% user, 43.00% kernel, 43.50% idle

CPU0 states : 12.00% user, 17.00% kernel, 71.00% idle

CPU1 states : 15.00% user, 69.00% kernel, 16.00% idle

Memory usage: 3665240K total, 2329168K used, 1336072K free

NX-MMR-LDP3# show processes cpu sort

Please note that only processes from the requested vdc are shown above

PID Runtime(ms) Invoked uSecs 1Sec Process

----- ----------- -------- ----- ------ -----------

3380 2631072 3410977 771 38.40% syslogd

3416 58778284 1446204879 40 7.15% bcm_usd

3524 5003137 36946856 135 7.15% snmpd

3654 372 1005 370 6.25% netstack

3382 1905831 8800817 216 3.15% pfma

3670 2034701 16774533 121 2.70% ethpm

3431 1307149 64584448 20 2.25% klogd

3600 171 46 3729 1.35% arp

LDP3# show processes cpu sort | ex 0.00

Please note that only processes from the requested vdc are shown above

PID Runtime(ms) Invoked uSecs 1Sec Process

----- ----------- -------- ----- ------ -----------

3380 2636268 3414954 771 38.75% syslogd

3416 58803869 1446558377 40 12.10% bcm_usd

3654 372 1005 370 5.34% netstack

3453 9850517 13575464 725 3.40% statsclient

3431 1317772 65120942 20 1.95% klogd

3600 171 46 3729 1.45% arp

3469 1877556 2646688 709 0.94% neutron_usd

4 221006 3082321 71 0.50% ksoftirqd/0

3477 358060 7032761 50 0.50% fwm

3747 4648932 78672341 59 0.50% stp

Any idea guys?

marce1000 · ‎09-10-2021

- Probably a bug , check current software version , upgrade and or use an advisory release , if applicable.

M.

-- Each morning when I wake up and look into the mirror I always say ' Why am I so brilliant ? '
When the mirror will then always repond to me with ' The only thing that exceeds your brilliance is your beauty! '

Georg Pauwen · ‎09-11-2021

Hello,

check if the bug below applies:

N3064: continue clk_flush: Couldn't Clear Bus - kernel and I2c access failed - neutron_usd errors
CSCvn34433
Description
Symptom:
A N3064 may report continuous clk_flush: Couldn't Clear Bus - kernel or neutron_usd I2c access failed errors when Eth1/49-52 QSFP ports are used as 40Gig or used as 10Gig breakout.

2018 Nov 16 20:12:16.042 N3064 16 20:12:03 %KERN-2-SYSTEM_MSG: [38998325.685862] clk_flush: Couldn't Clear Bus - kernel
2018 Nov 16 20:12:16.047 N3064 16 20:12:03 %KERN-3-SYSTEM_MSG: [38998325.685867] neutron_usd/3485 - failed to set mux addr 0x73 ch 16 err I2C_ERR_CLKFLUSH_FAILED - kernel

2018 Nov 13 23:38:57 N3064 %USER-2-SYSTEM_MSG: neutron_usd_i2c_prsnt_check:617: I2c access failed for port 50 - neutron_usd
2018 Nov 13 23:38:57 N3064 %USER-2-SYSTEM_MSG: neutron_usd_i2c_prsnt_check:617: I2c access failed for port 51 - neutron_usd
2018 Nov 13 23:38:57 N3064 %USER-2-SYSTEM_MSG: neutron_usd_i2c_prsnt_check:617: I2c access failed for port 52 - neutron_usd
2018 Nov 13 23:38:57 N3064 %USER-2-SYSTEM_MSG: neutron_usd_i2c_prsnt_check:617: I2c access failed for port 49 - neutron_usd

Conditions:
There are 2 conditions which can trigger these errors.
1. A faulty of wrongly seated QSFP/QSA used in ports (1/49-52) may generate continuous interrupts and cause the I2C bus to hang. - this is addresses via bug CSCuy12344 which can help to identify the port which caused the I2C errors instead of trying re-seat of one QSFP/QSA at a time to narrow down the faulty QSA/QSFP
2. Requesting for DOM data for the optics inserted in port 1/49-52 can hold the I2C bus for longer time and generate the I2C errors - the fix in this bug is addresses this issue.

Workaround:
Stop SNMP polling for DOM data or stop executing "show interface transceiver detail" command to avoid requesting DOM data.

Further Problem Description:
1. Due to busy I2C bug, the show version output may give an error message.
2. The PS and Fan Tray may report as missing/faulty, but they're working fine.

Once I2C errors are cleared, these issues also get cleared.