Re: Nexus 56128 Infrastructure - ASIC UCF issue

Sergej Barkovski · ‎09-30-2020

Hello,

we repeatedly got an error in the N5K-56128 logging:

2020 Sep 28 22:58:49 DE2C-NX-56128-2 %USER-2-SYSTEM_MSG: A fabric link on crossbar asic 0 has degraded - pacifica

2020 Sep 28 23:32:14 DE2C-NX-56128-2 %USER-2-SYSTEM_MSG: A fabric link on crossbar asic 0 has degraded - pacifica

---snip---

2020 Sep 30 08:56:18 DE2C-NX-56128-2 %USER-2-SYSTEM_MSG: A fabric link on crossbar asic 0 has degraded - pacifica

2020 Sep 30 09:29:42 DE2C-NX-56128-2 %USER-2-SYSTEM_MSG: A fabric link on crossbar asic 0 has degraded - pacifica

There are two N5K-56128 in vpc and the error are only seen on the one of them.

I read some articles trying to found how to check the UCF asic status and found these:

show platform fwm info asic-info all - shows all forwarding ASICs

show hardware internal bigsur all-ports - port-to-ASIC mapping

The output of last command shows me all ASIC's diag as pass (ASIC 0 also):

show hardware internal bigsur all-ports

Bigsur Port Info:

Port |asic|inst|inst|

---------+----+----+----+-----+----+----+---+---+--------+----+-----

xgb1/1 |0 |0 |0 | 0 - |0 |b3 |dis|dn |1a000000|pass| 0.00

xgb1/2 |0 |0 |0 | 1 p |1 |b3 |en |up |1a001000|pass| 0.00

xgb1/3 |0 |0 |0 | 2 - |2 |b3 |dis|dn |1a002000|pass| 0.00

xgb1/4 |0 |0 |0 | 3 - |3 |b3 |dis|dn |1a003000|pass| 0.00

xgb1/5 |0 |0 |0 | 4 p |4 |b3 |en |up |1a004000|pass| 0.00

xgb1/6 |0 |0 |0 | 5 p |5 |b3 |en |up |1a005000|pass| 0.00

1gb1/7 |0 |0 |0 | 6 p |6 |b3 |en |up |1a006000|pass| 0.00

xgb1/8 |0 |0 |0 | 7 p |7 |b3 |en |up |1a007000|pass| 0.00

xgb1/9 |0 |0 |0 | 8 - |8 |b3 |dis|dn |1a008000|pass| 0.00

xgb1/10 |0 |0 |0 | 9 p |9 |b3 |en |up |1a009000|pass| 0.00

xgb1/11 |0 |0 |0 |10 p |10 |b3 |en |up |1a00a000|pass| 0.00

xgb1/12 |0 |0 |0 |11 p |11 |b3 |en |up |1a00b000|pass| 0.00

xgb1/13 |1 |0 |1 | 0 p |12 |b3 |en |up |1a00c000|pass| 0.00

xgb1/14 |1 |0 |1 | 1 - |13 |b3 |dis|dn |1a00d000|pass| 0.00

xgb1/15 |1 |0 |1 | 2 - |14 |b3 |dis|dn |1a00e000|pass| 0.00

xgb1/16 |1 |0 |1 | 3 - |15 |b3 |dis|dn |1a00f000|pass| 0.00

xgb1/17 |1 |0 |1 | 4 - |16 |b3 |dis|dn |1a010000|pass| 0.00

xgb1/18 |1 |0 |1 | 5 - |17 |b3 |dis|dn |1a011000|pass| 0.00

xgb1/19 |1 |0 |1 | 6 - |18 |b3 |dis|dn |1a012000|pass| 0.00

xgb1/20 |1 |0 |1 | 7 - |19 |b3 |dis|dn |1a013000|pass| 0.00

xgb1/21 |1 |0 |1 | 8 - |20 |b3 |dis|dn |1a014000|pass| 0.00

xgb1/22 |1 |0 |1 | 9 - |21 |b3 |dis|dn |1a015000|pass| 0.00

xgb1/23 |1 |0 |1 |10 - |22 |b3 |dis|dn |1a016000|pass| 0.00

xgb1/24 |1 |0 |1 |11 - |23 |b3 |dis|dn |1a017000|pass| 0.00

xgb1/25 |2 |0 |2 | 0 - |24 |b3 |dis|dn |1a018000|pass| 0.00

xgb1/26 |2 |0 |2 | 1 - |25 |b3 |dis|dn |1a019000|pass| 0.00

xgb1/27 |2 |0 |2 | 2 - |26 |b3 |dis|dn |1a01a000|pass| 0.00

xgb1/28 |2 |0 |2 | 3 - |27 |b3 |dis|dn |1a01b000|pass| 0.00

xgb1/29 |2 |0 |2 | 4 - |28 |b3 |dis|dn |1a01c000|pass| 0.00

xgb1/30 |2 |0 |2 | 5 - |29 |b3 |dis|dn |1a01d000|pass| 0.00

xgb1/31 |2 |0 |2 | 6 - |30 |b3 |dis|dn |1a01e000|pass| 0.00

xgb1/32 |2 |0 |2 | 7 - |31 |b3 |dis|dn |1a01f000|pass| 0.00

xgb1/33 |2 |0 |2 | 8 - |32 |b3 |dis|dn |1a020000|pass| 0.00

xgb1/34 |2 |0 |2 | 9 - |33 |b3 |dis|dn |1a021000|pass| 0.00

xgb1/35 |2 |0 |2 |11 - |34 |b3 |dis|dn |1a022000|pass| 0.00

xgb1/36 |2 |0 |2 |10 - |35 |b3 |dis|dn |1a023000|pass| 0.00

xgb1/37 |3 |0 |3 | 0 - |36 |b3 |dis|dn |1a024000|pass| 0.00

xgb1/38 |3 |0 |3 | 1 - |37 |b3 |dis|dn |1a025000|pass| 0.00

xgb1/39 |3 |0 |3 | 2 - |38 |b3 |dis|dn |1a026000|pass| 0.00

xgb1/40 |3 |0 |3 | 3 - |39 |b3 |dis|dn |1a027000|pass| 0.00

xgb1/41 |3 |0 |3 | 4 p |40 |b3 |en |up |1a028000|pass| 0.00

xgb1/42 |3 |0 |3 | 5 p |41 |b3 |en |up |1a029000|pass| 0.00

xgb1/43 |3 |0 |3 | 6 p |42 |b3 |en |up |1a02a000|pass| 0.00

xgb1/44 |3 |0 |3 | 7 p |43 |b3 |en |up |1a02b000|pass| 0.00

xgb1/45 |3 |0 |3 | 8 p |44 |b3 |en |up |1a02c000|pass| 0.00

xgb1/46 |3 |0 |3 | 9 p |45 |b3 |en |up |1a02d000|pass| 0.00

xgb1/47 |3 |0 |3 |10 p |46 |b3 |en |up |1a02e000|pass| 0.00

xgb1/48 |3 |0 |3 |11 p |47 |b3 |en |up |1a02f000|pass| 0.00

40gb1/49 |4 |0 |4 | 2 p |48 |b3 |dis|dn |1a070000|pass| 0.00

40gb1/50 |4 |0 |4 | 1 p |49 |b3 |dis|dn |1a071000|pass| 0.00

40gb1/51 |5 |0 |5 | 2 p |50 |b3 |dis|dn |1a072000|pass| 0.00

40gb1/52 |5 |0 |5 | 1 p |51 |b3 |dis|dn |1a073000|pass| 0.00

sup0 |5 |0 |5 | 0 p |52 |b3 |en |up |15010000|pass| 0.00

I do not reall sure, if this is really a port-asic mapping of the UPC, and not of UCF.

can somebody advice, how can I check the status of the crossbar fabric asic 0?

Do the error logs on the top mean a hardware defect?

Thanks in advance,

Best regards,

Sergej

Kirk J · ‎10-13-2020

Might want to open a TAC case and collect the following info:

show hardware internal bigsur event-history errors

show hardware internal bigsur asic 0 registers | grep FI | grep PKT_CRC_ERR | no-more
show hardware internal bigsur asic 1 registers | grep FI | grep PKT_CRC_ERR | no-more
show hardware internal bigsur asic 2 registers | grep FI | grep PKT_CRC_ERR | no-more
show hardware internal bigsur asic 3 registers | grep FI | grep PKT_CRC_ERR | no-more

show hardware internal bigsur asic 0 instant-fi-errors | no-more
show hardware internal bigsur asic 1 instant-fi-errors | no-more
show hardware internal bigsur asic 2 instant-fi-errors | no-more
show hardware internal bigsur asic 3 instant-fi-errors | no-more

show hard internal pacifica asic 0 register PKT_CRC_ERR | no-more
show hard internal pacifica asic 1 register PKT_CRC_ERR | no-more
show hard internal pacifica asic 2 register PKT_CRC_ERR | no-more
show hard internal pacifica asic 3 register PKT_CRC_ERR | no-more

Thanks,

Kirk...