cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
2632
Views
0
Helpful
8
Replies

UCS Chassis 5108 fan speed

Richard Simmons
Level 3
Level 3

Hi,

Recently upgrade our test Fabric Interconnects and 5108 Chassis to 3.1(2e) and since this the 5108 Chassis fan's have all been running at full speed (9372). 

I can see there are bugs for other versions for fan speed but can't find one for this version, does anyone know the bug ID or a fix to reduce the fan speed?

Thanks,

Richard

8 Replies 8

Kirk J
Cisco Employee
Cisco Employee

Greetings.

Many of the issues resolved in firmware upgrades, still require a reseat of the PSUs and Fans in order for those devices to get the updated I2C timing programming.

If you haven't already, please do a reseat of the Fans and PSUs, one at a time, and confirm if the fan speeds go back down to normal ranges instead of the 'safemode' max speeds.

Thanks,

Kirk...

Hi Kirk,

Thanks for getting back so promptly, i will certainly test this tomorrow - the Chassis has been fully powered down since the update. I will let you know tomorrow how it goes.

Thanks,

Richard

Greetings.

The power down should have had the same result.

If you still have the fans maxed out after the reseats, you'll want to open a TAC case and let them look at your Chassis tech support logs to see if something still causing the FAN I2C segment to have a problem.

Thanks,

Kirk...

I'll see if i can get a case raised for this tomorrow, in the mean time is there any troubleshooting i can look into myself for this?

Thanks,

Richard

run from ucsm cli (connect to each FI node IP via ssh)

#connect iom 1 (or whatever chassis # is in question)

#show plat software cmcctrl fancontrol all

#show plat software cmcctrl showi2c

Thanks,

Kirk...

From FI-A

fex-1# show platform software cmcctrl fancontrol all
fex-1#
fex-1#
fex-1# show platform software cmcctrl showi2c
# I2C Bus Statistics Thu May 4 15:08:17 EDT 2017
# I2C Bus 1
busn=0 nseg=2
segment 0 local
segment 1 extended
wait_gt_deadline 27
error_pca9541_per_device:
# I2C Bus 2
busn=1 nseg=5
segment 0 local
segment 1 chassis
norxack 192
wait_gt_deadline 2
segment 2 blade
segment 3 fan
segment 4 psu
gilroy.counter.reserved 4366
gilroy.counter.released 4366
gilroy.counter.status_chassis_off 4518
error_pca9541_per_device:

# I2C Device Statistics
iom.fru={SUCCESS=12}
iom.rtc={SUCCESS=12}
NAME=iom.woodside
No such path /sys/devices/platform/fsl-i2c.1/i2c-0/0-0048
NAME=iom.dcdc0
detach_state driver iout iout_cal_offset iout_oc_fault_limit mfr_date mfr_id mfr_location mfr_model mfr_revision mfr_serial name ot_fault_limit ot_warn_limit temperature1 temperature2 vin vin_ov_fault_limit vin_uv_fault_limit vout vout_mode vout_n vout_ov_fault_limit vout_uv_fault_limit
NAME=iom.dcdc1
detach_state driver iout iout_cal_offset iout_oc_fault_limit mfr_date mfr_id mfr_location mfr_model mfr_revision mfr_serial name ot_fault_limit ot_warn_limit temperature1 temperature2 vin vin_ov_fault_limit vin_uv_fault_limit vout vout_mode vout_n vout_ov_fault_limit vout_uv_fault_limit
iom.gpio0={SUCCESS=1374}
iom.gpio1={SUCCESS=1374}
iom.gpio2={SUCCESS=1374}
iom.gpio3={SUCCESS=2814}
iom.temp.inlet1={SUCCESS=5251}
iom.temp.inlet2={SUCCESS=5251}
iom.temp.woodside={SUCCESS=4595}
c.fru={SUCCESS=1}
c.seeprom={SUCCESS=42}
f.fm0.fru={SUCCESS=1}
f.fm1.fru={SUCCESS=1}
f.fm2.fru={SUCCESS=1}
f.fm3.fru={SUCCESS=1}
f.fm4.fru={SUCCESS=1}
f.fm5.fru={SUCCESS=1}
f.fm6.fru={SUCCESS=1}
f.fm7.fru={SUCCESS=1}
p.psu3.psmi={SUCCESS=3}
p.psu0.fru={SUCCESS=1}
p.psu1.fru={SUCCESS=1}
p.psu1.psmi={SUCCESS=3}
p.psu2.fru={SUCCESS=1}
p.psu3.fru={SUCCESS=1}
c.gpio0={ENXIO=8}
c.gpio1={ENXIO=8}
c.gpio2={ENXIO=8}
c.gpio3={ENXIO=8}
# I2C Driver sysctl entries
sysctl: error: permission denied on key 'net.ipv4.route.flush'
sysctl: error: permission denied on key 'kernel.cad_pid'
sysctl: error: permission denied on key 'kernel.cap-bound'
dev.i2c.disconnect_retry = 3
dev.i2c.post_trigger = 64
dev.i2c.norxack_blink = 5
dev.i2c.norxack_blink = 5
dev.i2c.fixup_blink = 0
dev.i2c.pca9541-workaround = 16
dev.i2c.wait_deadline = 30
dev.i2c.chassis_reservation.demand = 0
dev.i2c.chassis_reservation.lock_state = 2
dev.i2c.chassis_reservation.auto_release = 1
dev.i2c.chassis_reservation.on_demand = 0
dev.i2c.chassis_reservation.pause_gilroy_thread = 0
dev.i2c.chassis_reservation.min_notheld_ms = 150
dev.i2c.chassis_reservation.wait_extra_ms = 3000
dev.i2c.chassis_reservation.grace_ms = 750
dev.i2c.chassis_reservation.hold_ms = 1500
dev.i2c.chassis_reservation.wait_ms = 4500
dev.i2c.gilroy-debug-level = 3
dev.i2c.debug-level = 1
dev.i2c.pca9541-businit = 1
dev.i2c.pca9541-delay = 250
dev.i2c.bus2.write-cdelay = 100
dev.i2c.bus2.write-delay = 100
dev.i2c.bus1.write-cdelay = 30
dev.i2c.bus1.write-delay = 30

From FI-B

fex-1# show platform software cmcctrl fancontrol all
Fan 0 Speed (FRONT): 9358
Fan 0 Speed (REAR): 9473
Fan 0 dutycycle: 100
Fan 1 Speed (FRONT): 9137
Fan 1 Speed (REAR): 9457
Fan 1 dutycycle: 100
Fan 2 Speed (FRONT): 9075
Fan 2 Speed (REAR): 9294
Fan 2 dutycycle: 100
Fan 3 Speed (FRONT): 8985
Fan 3 Speed (REAR): 9608
Fan 3 dutycycle: 100
Fan 4 Speed (FRONT): 9262
Fan 4 Speed (REAR): 9818
Fan 4 dutycycle: 100
Fan 5 Speed (FRONT): 9030
Fan 5 Speed (REAR): 9473
Fan 5 dutycycle: 100
Fan 6 Speed (FRONT): 8985
Fan 6 Speed (REAR): 9391
Fan 6 dutycycle: 100
Fan 7 Speed (FRONT): 9262
Fan 7 Speed (REAR): 9660
Fan 7 dutycycle: 100
fex-1#
fex-1#
fex-1# show platform software cmcctrl showi2c
# I2C Bus Statistics Thu May 4 15:13:37 EDT 2017
# I2C Bus 1
busn=0 nseg=2
segment 0 local
segment 1 extended
wait_gt_deadline 9
error_pca9541_per_device:
# I2C Bus 2
busn=1 nseg=5
segment 0 local
segment 1 chassis
norxack 199
wait_gt_deadline 2
segment 2 blade
segment 3 fan
norxack 7
wait_gt_deadline 205
segment 4 psu
wait_gt_deadline 114
gilroy.counter.reserved 4572
gilroy.counter.released 4572
gilroy.counter.status_chassis_off 4572
error_pca9541_per_device:

# I2C Device Statistics
iom.fru={SUCCESS=12}
iom.rtc={SUCCESS=12}
NAME=iom.woodside
No such path /sys/devices/platform/fsl-i2c.1/i2c-0/0-0048
NAME=iom.dcdc0
detach_state driver iout iout_cal_offset iout_oc_fault_limit mfr_date mfr_id mfr_location mfr_model mfr_revision mfr_serial name ot_fault_limit ot_warn_limit temperature1 temperature2 vin vin_ov_fault_limit vin_uv_fault_limit vout vout_mode vout_n vout_ov_fault_limit vout_uv_fault_limit
NAME=iom.dcdc1
detach_state driver iout iout_cal_offset iout_oc_fault_limit mfr_date mfr_id mfr_location mfr_model mfr_revision mfr_serial name ot_fault_limit ot_warn_limit temperature1 temperature2 vin vin_ov_fault_limit vin_uv_fault_limit vout vout_mode vout_n vout_ov_fault_limit vout_uv_fault_limit
iom.gpio0={SUCCESS=1454}
iom.gpio1={SUCCESS=1460}
iom.gpio2={SUCCESS=1454}
iom.gpio3={SUCCESS=2974}
iom.temp.inlet1={SUCCESS=5515}
iom.temp.inlet2={SUCCESS=5515}
iom.temp.woodside={SUCCESS=4826}
c.fru={SUCCESS=1}
c.seeprom={SUCCESS=76}
f.fm0.fc={SUCCESS=6040}
f.fm0.fru={SUCCESS=1}
f.fm1.fc={SUCCESS=6039}
f.fm1.fru={SUCCESS=1}
f.fm2.fc={SUCCESS=6039}
f.fm2.fru={SUCCESS=1}
f.fm3.fc={SUCCESS=6039}
f.fm3.fru={SUCCESS=1}
f.fm4.fc={SUCCESS=6041}
f.fm4.fru={SUCCESS=1}
f.fm5.fc={SUCCESS=6039}
f.fm5.fru={SUCCESS=1}
f.fm6.fc={SUCCESS=6039}
f.fm6.fru={SUCCESS=1}
f.fm7.fc={SUCCESS=6039}
f.fm7.fru={SUCCESS=1}
p.psu3.psmi={SUCCESS=18732}
p.psu0.fru={SUCCESS=4}
p.psu0.psmi={SUCCESS=18791}
p.psu1.fru={SUCCESS=4}
p.psu1.psmi={SUCCESS=18760}
p.psu2.fru={SUCCESS=4}
p.psu2.psmi={SUCCESS=18791}
p.psu3.fru={SUCCESS=4}
c.gpio0={ENXIO=8}
c.gpio1={ENXIO=8}
c.gpio2={ENXIO=8}
c.gpio3={ENXIO=8}
# I2C Driver sysctl entries
sysctl: error: permission denied on key 'net.ipv4.route.flush'
sysctl: error: permission denied on key 'kernel.cad_pid'
sysctl: error: permission denied on key 'kernel.cap-bound'
dev.i2c.disconnect_retry = 3
dev.i2c.post_trigger = 64
dev.i2c.norxack_blink = 5
dev.i2c.norxack_blink = 5
dev.i2c.fixup_blink = 0
dev.i2c.pca9541-workaround = 16
dev.i2c.wait_deadline = 30
dev.i2c.chassis_reservation.demand = 0
dev.i2c.chassis_reservation.lock_state = 1
dev.i2c.chassis_reservation.auto_release = 1
dev.i2c.chassis_reservation.on_demand = 0
dev.i2c.chassis_reservation.pause_gilroy_thread = 0
dev.i2c.chassis_reservation.min_notheld_ms = 150
dev.i2c.chassis_reservation.wait_extra_ms = 3000
dev.i2c.chassis_reservation.grace_ms = 750
dev.i2c.chassis_reservation.hold_ms = 1500
dev.i2c.chassis_reservation.wait_ms = 4500
dev.i2c.gilroy-debug-level = 3
dev.i2c.debug-level = 1
dev.i2c.pca9541-businit = 1
dev.i2c.pca9541-delay = 250
dev.i2c.bus2.write-cdelay = 100
dev.i2c.bus2.write-delay = 100
dev.i2c.bus1.write-cdelay = 30
dev.i2c.bus1.write-delay = 30

Well, I don't think the output looks that bad.  Doesn't explain why the fans are 100% maxed.  Will take some digging in the logs.

Once you get TAC to look at your case, do let us know what was the RCA.

Thanks,

Kirk...

pbiczek7639
Level 1
Level 1

So I have a similar but different issue. All my UCS 5108 fans cycle from 0- to full within 10 min timespan. Has anybody encountered that behaviour??

 

fex-1# show plat software cmcctrl fancontrol all
fex-1# show plat software cmcctrl showi2c
# I2C Bus Statistics Wed Feb  6 15:13:23 CST 2019
# I2C Bus 1
busn=0 nseg=2
segment 0 local
segment 1 extended
        wait_gt_deadline 1418
error_pca9541_per_device:
# I2C Bus 2
busn=1 nseg=5
segment 0 local
segment 1 chassis
        norxack 205
        wait_gt_deadline 122
segment 2 blade
segment 3 fan
segment 4 psu
        wait_gt_deadline 1
gilroy.counter.reserved 260467
gilroy.counter.released 260466
gilroy.counter.status_chassis_off 272088
error_pca9541_per_device:
 
# I2C Device Statistics
iom.fru={SUCCESS=12}
iom.rtc={SUCCESS=7}
NAME=iom.woodside
No such path /sys/devices/platform/fsl-i2c.1/i2c-0/0-0048
NAME=iom.dcdc0
        detach_state driver iout iout_cal_offset iout_oc_fault_limit mfr_date mfr_id mfr_location mfr_model mfr_revision mfr_serial name ot_fault_limit ot_warn_limit temperature1 temperature2 vin vin_ov_fault_limit vin_uv_fault_limit vout vout_mode vout_n vout_ov_fault_limit vout_uv_fault_limit
NAME=iom.dcdc1
        detach_state driver iout iout_cal_offset iout_oc_fault_limit mfr_date mfr_id mfr_location mfr_model mfr_revision mfr_serial name ot_fault_limit ot_warn_limit temperature1 temperature2 vin vin_ov_fault_limit vin_uv_fault_limit vout vout_mode vout_n vout_ov_fault_limit vout_uv_fault_limit
iom.gpio0={SUCCESS=84496}
iom.gpio1={SUCCESS=84496}
iom.gpio2={SUCCESS=84496}
iom.gpio3={SUCCESS=169058}
iom.temp.inlet1={SUCCESS=314787}
iom.temp.inlet2={SUCCESS=314787}
iom.temp.woodside={SUCCESS=275439}
c.fru={SUCCESS=1}
c.seeprom={SUCCESS=680}
f.fm0.fru={SUCCESS=1}
f.fm1.fru={SUCCESS=1}
f.fm2.fru={SUCCESS=1}
f.fm3.fru={SUCCESS=1}
f.fm4.fru={SUCCESS=1}
f.fm5.fru={SUCCESS=1}
f.fm6.fru={SUCCESS=1}
f.fm7.fru={SUCCESS=1}
p.psu3.psmi={SUCCESS=4}
p.psu0.fru={SUCCESS=1}
p.psu0.psmi={SUCCESS=4}
p.psu1.fru={SUCCESS=1}
p.psu1.psmi={SUCCESS=4}
p.psu2.fru={SUCCESS=1}
p.psu2.psmi={SUCCESS=4}
p.psu3.fru={SUCCESS=1}
c.gpio0={ENXIO=8}
c.gpio1={ENXIO=8}
c.gpio2={ENXIO=8}
c.gpio3={ENXIO=8}
# I2C Driver sysctl entries
sysctl: error: permission denied on key 'net.ipv4.route.flush'
sysctl: error: permission denied on key 'kernel.cad_pid'
sysctl: error: permission denied on key 'kernel.cap-bound'
dev.i2c.disconnect_retry = 3
dev.i2c.post_trigger = 64
dev.i2c.norxack_blink = 5
dev.i2c.norxack_blink = 5
dev.i2c.fixup_blink = 0
dev.i2c.pca9541-workaround = 16
dev.i2c.wait_deadline = 30
dev.i2c.chassis_reservation.demand = 0
dev.i2c.chassis_reservation.lock_state = 1
dev.i2c.chassis_reservation.auto_release = 1
dev.i2c.chassis_reservation.on_demand = 0
dev.i2c.chassis_reservation.pause_gilroy_thread = 0
dev.i2c.chassis_reservation.min_notheld_ms = 150
dev.i2c.chassis_reservation.wait_extra_ms = 3000
dev.i2c.chassis_reservation.grace_ms = 750
dev.i2c.chassis_reservation.hold_ms = 1500
dev.i2c.chassis_reservation.wait_ms = 4500
dev.i2c.gilroy-debug-level = 3
dev.i2c.debug-level = 1
dev.i2c.pca9541-businit = 1
dev.i2c.pca9541-delay = 250
dev.i2c.bus2.write-cdelay = 100
dev.i2c.bus2.write-delay = 100
dev.i2c.bus1.write-cdelay = 30
dev.i2c.bus1.write-delay = 30
fex-1#

Review Cisco Networking products for a $25 gift card