05-04-2017 12:59 AM - edited 03-01-2019 01:10 PM
Hi,
Recently upgrade our test Fabric Interconnects and 5108 Chassis to 3.1(2e) and since this the 5108 Chassis fan's have all been running at full speed (9372).
I can see there are bugs for other versions for fan speed but can't find one for this version, does anyone know the bug ID or a fix to reduce the fan speed?
Thanks,
Richard
05-04-2017 04:22 AM
Greetings.
Many of the issues resolved in firmware upgrades, still require a reseat of the PSUs and Fans in order for those devices to get the updated I2C timing programming.
If you haven't already, please do a reseat of the Fans and PSUs, one at a time, and confirm if the fan speeds go back down to normal ranges instead of the 'safemode' max speeds.
Thanks,
Kirk...
05-04-2017 05:42 AM
Hi Kirk,
Thanks for getting back so promptly, i will certainly test this tomorrow - the Chassis has been fully powered down since the update. I will let you know tomorrow how it goes.
Thanks,
Richard
05-04-2017 06:12 AM
Greetings.
The power down should have had the same result.
If you still have the fans maxed out after the reseats, you'll want to open a TAC case and let them look at your Chassis tech support logs to see if something still causing the FAN I2C segment to have a problem.
Thanks,
Kirk...
05-04-2017 08:26 AM
I'll see if i can get a case raised for this tomorrow, in the mean time is there any troubleshooting i can look into myself for this?
Thanks,
Richard
05-04-2017 08:39 AM
run from ucsm cli (connect to each FI node IP via ssh)
#connect iom 1 (or whatever chassis # is in question)
#show plat software cmcctrl fancontrol all
#show plat software cmcctrl showi2c
Thanks,
Kirk...
05-04-2017 12:14 PM
From FI-A
fex-1# show platform software cmcctrl fancontrol all
fex-1#
fex-1#
fex-1# show platform software cmcctrl showi2c
# I2C Bus Statistics Thu May 4 15:08:17 EDT 2017
# I2C Bus 1
busn=0 nseg=2
segment 0 local
segment 1 extended
wait_gt_deadline 27
error_pca9541_per_device:
# I2C Bus 2
busn=1 nseg=5
segment 0 local
segment 1 chassis
norxack 192
wait_gt_deadline 2
segment 2 blade
segment 3 fan
segment 4 psu
gilroy.counter.reserved 4366
gilroy.counter.released 4366
gilroy.counter.status_chassis_off 4518
error_pca9541_per_device:
# I2C Device Statistics
iom.fru={SUCCESS=12}
iom.rtc={SUCCESS=12}
NAME=iom.woodside
No such path /sys/devices/platform/fsl-i2c.1/i2c-0/0-0048
NAME=iom.dcdc0
detach_state driver iout iout_cal_offset iout_oc_fault_limit mfr_date mfr_id mfr_location mfr_model mfr_revision mfr_serial name ot_fault_limit ot_warn_limit temperature1 temperature2 vin vin_ov_fault_limit vin_uv_fault_limit vout vout_mode vout_n vout_ov_fault_limit vout_uv_fault_limit
NAME=iom.dcdc1
detach_state driver iout iout_cal_offset iout_oc_fault_limit mfr_date mfr_id mfr_location mfr_model mfr_revision mfr_serial name ot_fault_limit ot_warn_limit temperature1 temperature2 vin vin_ov_fault_limit vin_uv_fault_limit vout vout_mode vout_n vout_ov_fault_limit vout_uv_fault_limit
iom.gpio0={SUCCESS=1374}
iom.gpio1={SUCCESS=1374}
iom.gpio2={SUCCESS=1374}
iom.gpio3={SUCCESS=2814}
iom.temp.inlet1={SUCCESS=5251}
iom.temp.inlet2={SUCCESS=5251}
iom.temp.woodside={SUCCESS=4595}
c.fru={SUCCESS=1}
c.seeprom={SUCCESS=42}
f.fm0.fru={SUCCESS=1}
f.fm1.fru={SUCCESS=1}
f.fm2.fru={SUCCESS=1}
f.fm3.fru={SUCCESS=1}
f.fm4.fru={SUCCESS=1}
f.fm5.fru={SUCCESS=1}
f.fm6.fru={SUCCESS=1}
f.fm7.fru={SUCCESS=1}
p.psu3.psmi={SUCCESS=3}
p.psu0.fru={SUCCESS=1}
p.psu1.fru={SUCCESS=1}
p.psu1.psmi={SUCCESS=3}
p.psu2.fru={SUCCESS=1}
p.psu3.fru={SUCCESS=1}
c.gpio0={ENXIO=8}
c.gpio1={ENXIO=8}
c.gpio2={ENXIO=8}
c.gpio3={ENXIO=8}
# I2C Driver sysctl entries
sysctl: error: permission denied on key 'net.ipv4.route.flush'
sysctl: error: permission denied on key 'kernel.cad_pid'
sysctl: error: permission denied on key 'kernel.cap-bound'
dev.i2c.disconnect_retry = 3
dev.i2c.post_trigger = 64
dev.i2c.norxack_blink = 5
dev.i2c.norxack_blink = 5
dev.i2c.fixup_blink = 0
dev.i2c.pca9541-workaround = 16
dev.i2c.wait_deadline = 30
dev.i2c.chassis_reservation.demand = 0
dev.i2c.chassis_reservation.lock_state = 2
dev.i2c.chassis_reservation.auto_release = 1
dev.i2c.chassis_reservation.on_demand = 0
dev.i2c.chassis_reservation.pause_gilroy_thread = 0
dev.i2c.chassis_reservation.min_notheld_ms = 150
dev.i2c.chassis_reservation.wait_extra_ms = 3000
dev.i2c.chassis_reservation.grace_ms = 750
dev.i2c.chassis_reservation.hold_ms = 1500
dev.i2c.chassis_reservation.wait_ms = 4500
dev.i2c.gilroy-debug-level = 3
dev.i2c.debug-level = 1
dev.i2c.pca9541-businit = 1
dev.i2c.pca9541-delay = 250
dev.i2c.bus2.write-cdelay = 100
dev.i2c.bus2.write-delay = 100
dev.i2c.bus1.write-cdelay = 30
dev.i2c.bus1.write-delay = 30
From FI-B
fex-1# show platform software cmcctrl fancontrol all
Fan 0 Speed (FRONT): 9358
Fan 0 Speed (REAR): 9473
Fan 0 dutycycle: 100
Fan 1 Speed (FRONT): 9137
Fan 1 Speed (REAR): 9457
Fan 1 dutycycle: 100
Fan 2 Speed (FRONT): 9075
Fan 2 Speed (REAR): 9294
Fan 2 dutycycle: 100
Fan 3 Speed (FRONT): 8985
Fan 3 Speed (REAR): 9608
Fan 3 dutycycle: 100
Fan 4 Speed (FRONT): 9262
Fan 4 Speed (REAR): 9818
Fan 4 dutycycle: 100
Fan 5 Speed (FRONT): 9030
Fan 5 Speed (REAR): 9473
Fan 5 dutycycle: 100
Fan 6 Speed (FRONT): 8985
Fan 6 Speed (REAR): 9391
Fan 6 dutycycle: 100
Fan 7 Speed (FRONT): 9262
Fan 7 Speed (REAR): 9660
Fan 7 dutycycle: 100
fex-1#
fex-1#
fex-1# show platform software cmcctrl showi2c
# I2C Bus Statistics Thu May 4 15:13:37 EDT 2017
# I2C Bus 1
busn=0 nseg=2
segment 0 local
segment 1 extended
wait_gt_deadline 9
error_pca9541_per_device:
# I2C Bus 2
busn=1 nseg=5
segment 0 local
segment 1 chassis
norxack 199
wait_gt_deadline 2
segment 2 blade
segment 3 fan
norxack 7
wait_gt_deadline 205
segment 4 psu
wait_gt_deadline 114
gilroy.counter.reserved 4572
gilroy.counter.released 4572
gilroy.counter.status_chassis_off 4572
error_pca9541_per_device:
# I2C Device Statistics
iom.fru={SUCCESS=12}
iom.rtc={SUCCESS=12}
NAME=iom.woodside
No such path /sys/devices/platform/fsl-i2c.1/i2c-0/0-0048
NAME=iom.dcdc0
detach_state driver iout iout_cal_offset iout_oc_fault_limit mfr_date mfr_id mfr_location mfr_model mfr_revision mfr_serial name ot_fault_limit ot_warn_limit temperature1 temperature2 vin vin_ov_fault_limit vin_uv_fault_limit vout vout_mode vout_n vout_ov_fault_limit vout_uv_fault_limit
NAME=iom.dcdc1
detach_state driver iout iout_cal_offset iout_oc_fault_limit mfr_date mfr_id mfr_location mfr_model mfr_revision mfr_serial name ot_fault_limit ot_warn_limit temperature1 temperature2 vin vin_ov_fault_limit vin_uv_fault_limit vout vout_mode vout_n vout_ov_fault_limit vout_uv_fault_limit
iom.gpio0={SUCCESS=1454}
iom.gpio1={SUCCESS=1460}
iom.gpio2={SUCCESS=1454}
iom.gpio3={SUCCESS=2974}
iom.temp.inlet1={SUCCESS=5515}
iom.temp.inlet2={SUCCESS=5515}
iom.temp.woodside={SUCCESS=4826}
c.fru={SUCCESS=1}
c.seeprom={SUCCESS=76}
f.fm0.fc={SUCCESS=6040}
f.fm0.fru={SUCCESS=1}
f.fm1.fc={SUCCESS=6039}
f.fm1.fru={SUCCESS=1}
f.fm2.fc={SUCCESS=6039}
f.fm2.fru={SUCCESS=1}
f.fm3.fc={SUCCESS=6039}
f.fm3.fru={SUCCESS=1}
f.fm4.fc={SUCCESS=6041}
f.fm4.fru={SUCCESS=1}
f.fm5.fc={SUCCESS=6039}
f.fm5.fru={SUCCESS=1}
f.fm6.fc={SUCCESS=6039}
f.fm6.fru={SUCCESS=1}
f.fm7.fc={SUCCESS=6039}
f.fm7.fru={SUCCESS=1}
p.psu3.psmi={SUCCESS=18732}
p.psu0.fru={SUCCESS=4}
p.psu0.psmi={SUCCESS=18791}
p.psu1.fru={SUCCESS=4}
p.psu1.psmi={SUCCESS=18760}
p.psu2.fru={SUCCESS=4}
p.psu2.psmi={SUCCESS=18791}
p.psu3.fru={SUCCESS=4}
c.gpio0={ENXIO=8}
c.gpio1={ENXIO=8}
c.gpio2={ENXIO=8}
c.gpio3={ENXIO=8}
# I2C Driver sysctl entries
sysctl: error: permission denied on key 'net.ipv4.route.flush'
sysctl: error: permission denied on key 'kernel.cad_pid'
sysctl: error: permission denied on key 'kernel.cap-bound'
dev.i2c.disconnect_retry = 3
dev.i2c.post_trigger = 64
dev.i2c.norxack_blink = 5
dev.i2c.norxack_blink = 5
dev.i2c.fixup_blink = 0
dev.i2c.pca9541-workaround = 16
dev.i2c.wait_deadline = 30
dev.i2c.chassis_reservation.demand = 0
dev.i2c.chassis_reservation.lock_state = 1
dev.i2c.chassis_reservation.auto_release = 1
dev.i2c.chassis_reservation.on_demand = 0
dev.i2c.chassis_reservation.pause_gilroy_thread = 0
dev.i2c.chassis_reservation.min_notheld_ms = 150
dev.i2c.chassis_reservation.wait_extra_ms = 3000
dev.i2c.chassis_reservation.grace_ms = 750
dev.i2c.chassis_reservation.hold_ms = 1500
dev.i2c.chassis_reservation.wait_ms = 4500
dev.i2c.gilroy-debug-level = 3
dev.i2c.debug-level = 1
dev.i2c.pca9541-businit = 1
dev.i2c.pca9541-delay = 250
dev.i2c.bus2.write-cdelay = 100
dev.i2c.bus2.write-delay = 100
dev.i2c.bus1.write-cdelay = 30
dev.i2c.bus1.write-delay = 30
05-04-2017 07:27 PM
Well, I don't think the output looks that bad. Doesn't explain why the fans are 100% maxed. Will take some digging in the logs.
Once you get TAC to look at your case, do let us know what was the RCA.
Thanks,
Kirk...
02-06-2019 01:07 PM - edited 02-06-2019 01:26 PM
So I have a similar but different issue. All my UCS 5108 fans cycle from 0- to full within 10 min timespan. Has anybody encountered that behaviour??
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide