Found error in HQATM TCAM Space and not able to recover the error when ACL label 128 is in use

[udj]

Hi..

on IOS versions higher than 12.2(50) on Cisco 3560G-48TS I get this error/traceback, when I reach a certain number of Access-lists group'ed to "interface vlan", and the ACL inserted in the TCAM reaches acl label #128 (can be seen with : Show platform acl label 128)

I can see errors in the TCAM if I issue the command:

udj#sh platform tcam errors

TCAM Memory Consistency Checker Errors
--------------------------------------
TCAM Space          Values      Masks     Fixups    Retries   Failures
Unassigned               0          0          0          0          0
HFTM                     0          0          0          0          0
HQATM                   26          0         26          0          0

I have upgraded from 12.2(35), bacause this version dropped all traffic when I reached an ACL that was inserted in the TCAM as ACL label 128, and actually this is not the problem with later versions, but now they spit out these syslog messages very frequently and I am reluctant to put this into production

Anyone have any thoughts on what is going on and how I can circumvent this problem and have ALL my ACL's be functioning at the same time (hardware or software). This should not be a hard limit reached in the TCAM, so I am guessing it might be a (another) bug?

Thanks..

%HARDWARE-1-TCAM_ERROR: Found error in HQATM TCAM Space and not able to recover the error

-Traceback= 2320548z 1AD572Cz 1AD8B5Cz 1AD8E30z 18FB74Cz 1CC43E8z 1CBE25Cz

%HARDWARE-1-TCAM_ERROR: Found error in HQATM TCAM Space and not able to recover the error

-Traceback= 2320548z 1AD572Cz 1AD8B5Cz 1AD8E30z 18FB74Cz 1CC43E8z 1CBE25Cz

%HARDWARE-1-TCAM_ERROR: Found error in HQATM TCAM Space and not able to recover the error

-Traceback= 2320548z 1AD572Cz 1AD8B5Cz 1AD8E30z 18FB74Cz 1CC43E8z 1CBE25Cz

%HARDWARE-1-TCAM_ERROR: Found error in HQATM TCAM Space and not able to recover the error

-Traceback= 2320548z 1AD572Cz 1AD8B5Cz 1AD8E30z 18FB74Cz 1CC43E8z 1CBE25Cz2

000160: 00:05:23: platform assert failure: 0: ../src-hulc/src-sasq/hulc_tcam.c: 1109: htd_cam_value_write32

000161: 00:05:23: -Traceback= 2321FA0z 16E6D08z 18F6F30z 1AD5688z 1AD8B5Cz 1AD8E30z 18FB74Cz 1CC43E8z 1CBE25Cz

000162: 00:05:23: platform assert failure: 0: ../src-hulc/src-sasq/hulc_tcam.c: 1216: htd_cam_value_read32

000163: 00:05:23: -Traceback= 2321FA0z 16E6D08z 18F7274z 1AD56A0z 1AD8B5Cz 1AD8E30z 18FB74Cz 1CC43E8z 1CBE25Cz

000164: 00:05:23: platform assert failure: 0: ../src-hulc/src-sasq/hulc_tcam.c: 1109: htd_cam_value_write32

000165: 00:05:23: -Traceback= 2321FA0z 16E6D08z 18F6F30z 1AD5688z 1AD8B5Cz 1AD8E30z 18FB74Cz 1CC43E8z 1CBE25Cz

000166: 00:05:23: platform assert failure: 0: ../src-hulc/src-sasq/hulc_tcam.c: 1216: htd_cam_value_read32

000167: 00:05:23: -Traceback= 2321FA0z 16E6D08z 18F7274z 1AD56A0z 1AD8B5Cz 1AD8E30z 18FB74Cz 1CC43E8z 1CBE25Cz

000168: 00:05:23: platform assert failure: 0: ../src-hulc/src-sasq/hulc_tcam.c: 1109: htd_cam_value_write32

000169: 00:05:23: -Traceback= 2321FA0z 16E6D08z 18F6F30z 1AD5688z 1AD8B5Cz 1AD8E30z 18FB74Cz 1CC43E8z 1CBE25Cz

000170: 00:05:23: platform assert failure: 0: ..ulc_tcam.c: 1216: htd_cam_value_read32

000171: 00:05:23: -Traceback= 2321FA0z 16E6D08z 18F7274z 1AD56A0z 1AD8B5Cz 1AD8E30z 18FB74Cz 1CC43E8z 1CBE25Cz

000172: 00:05:23: platform assert failure: 0: ../src-hulc/src-sasq/hulc_tcam.c: 1109: htd_cam_value_write32

000173: 00:05:23: -Traceback= 2321FA0z 16E6D08z 18F6F30z 1AD5688z 1AD8B5Cz 1AD8E30z 18FB74Cz 1CC43E8z 1CBE25Cz

000174: 00:05:23: platform assert failure: 0: ../src-hulc/src-sasq/hulc_tcam.c: 1216: htd_cam_value_read32

000175: 00:05:23: -Traceback= 2321FA0z 16E6D08z 18F7274z 1AD56A0z 1AD8B5Cz 1AD8E30z 18FB74Cz 1CC43E8z 1CBE25Cz

000176: 00:05:23: platform assert failure: 0: ../src-hulc/src-sasq/hulc_tcam.c: 1109: htd_cam_value_write32

000177: 00:05:23: -Traceback= 2321FA0z 16E6D08z 18F6F30z 1AD5688z 1AD8B5Cz 1AD8E30z 18FB74Cz 1CC43E8z 1CBE25Cz

%HARDWARE-1-TCAM_ERROR: Found error in HQATM TCAM Space and not able to recover the error

-Traceback= 2320548z 1AD572Cz 1AD8B5Cz 1AD8E30z 18FB74Cz 1CC43E8z 1CBE25Cz

%HARDWARE-1-TCAM_ERROR: Found error in HQATM TCAM Space and not able to recover the error

-Traceback= 2320548z 1AD572Cz 1AD8B5Cz 1AD8E30z 18FB74Cz 1CC43E8z 1CBE25Cz

%HARDWARE-1-TCAM_ERROR: Found error in HQATM TCAM Space and not able to recover the error

-Traceback= 2320548z 1AD572Cz 1AD8B5Cz 1AD8E30z 18FB74Cz 1CC43E8z 1CBE25Cz

%HARDWARE-1-TCAM_ERROR: Found error in HQATM TCAM Space and not able to recover the error

-Traceback= 2320548z 1AD572Cz 1AD8B5Cz 1AD8E30z 18FB74Cz 1CC43E8z 1CBE25Cz2

000160: 00:05:23: platform assert failure: 0: ../src-hulc/src-sasq/hulc_tcam.c: 1109: htd_cam_value_write32

000161: 00:05:23: -Traceback= 2321FA0z 16E6D08z 18F6F30z 1AD5688z 1AD8B5Cz 1AD8E30z 18FB74Cz 1CC43E8z 1CBE25Cz

000162: 00:05:23: platform assert failure: 0: ../src-hulc/src-sasq/hulc_tcam.c: 1216: htd_cam_value_read32

000163: 00:05:23: -Traceback= 2321FA0z 16E6D08z 18F7274z 1AD56A0z 1AD8B5Cz 1AD8E30z 18FB74Cz 1CC43E8z 1CBE25Cz

000164: 00:05:23: platform assert failure: 0: ../src-hulc/src-sasq/hulc_tcam.c: 1109: htd_cam_value_write32

000165: 00:05:23: -Traceback= 2321FA0z 16E6D08z 18F6F30z 1AD5688z 1AD8B5Cz 1AD8E30z 18FB74Cz 1CC43E8z 1CBE25Cz

000166: 00:05:23: platform assert failure: 0: ../src-hulc/src-sasq/hulc_tcam.c: 1216: htd_cam_value_read32

000167: 00:05:23: -Traceback= 2321FA0z 16E6D08z 18F7274z 1AD56A0z 1AD8B5Cz 1AD8E30z 18FB74Cz 1CC43E8z 1CBE25Cz

000168: 00:05:23: platform assert failure: 0: ../src-hulc/src-sasq/hulc_tcam.c: 1109: htd_cam_value_write32

000169: 00:05:23: -Traceback= 2321FA0z 16E6D08z 18F6F30z 1AD5688z 1AD8B5Cz 1AD8E30z 18FB74Cz 1CC43E8z 1CBE25Cz

000170: 00:05:23: platform assert failure: 0: ..ulc_tcam.c: 1216: htd_cam_value_read32

000171: 00:05:23: -Traceback= 2321FA0z 16E6D08z 18F7274z 1AD56A0z 1AD8B5Cz 1AD8E30z 18FB74Cz 1CC43E8z 1CBE25Cz

000172: 00:05:23: platform assert failure: 0: ../src-hulc/src-sasq/hulc_tcam.c: 1109: htd_cam_value_write32

000173: 00:05:23: -Traceback= 2321FA0z 16E6D08z 18F6F30z 1AD5688z 1AD8B5Cz 1AD8E30z 18FB74Cz 1CC43E8z 1CBE25Cz

000174: 00:05:23: platform assert failure: 0: ../src-hulc/src-sasq/hulc_tcam.c: 1216: htd_cam_value_read32

000175: 00:05:23: -Traceback= 2321FA0z 16E6D08z 18F7274z 1AD56A0z 1AD8B5Cz 1AD8E30z 18FB74Cz 1CC43E8z 1CBE25Cz

000176: 00:05:23: platform assert failure: 0: ../src-hulc/src-sasq/hulc_tcam.c: 1109: htd_cam_value_write32

000177: 00:05:23: -Traceback= 2321FA0z 16E6D08z 18F6F30z 1AD5688z 1AD8B5Cz 1AD8E30z 18FB74Cz 1CC43E8z 1CBE25Cz

Kind regards,

Ulrik

[Richard Michael]

Hello Ulrik,

The Error message indicates the following, the switch cannot fix a ternary content addressable memory (TCAM) integrity error. [chars] is memory location with the error: Unassigned TCAM Space, HFTM TCAM Space (the ASIC forwarding TCAM manager space), or HQATM TCAM Space (the TCAM ASIC quality of service [QoS] and access control list [ACL] TCAM manager space)

The solution is to hard reboot the switch once and see if this error message is going away.

If error message appears after hard reboot please post the below ouput,

1) show platform tcam utilization

2) show platform tcam usage

3) show platform tcam log-results

Thanks,

Ricky Micky

*Rate Useful Posts

[udj]

The errors are persistent, so I get the same errors efter "power off -> power on"..

The TCAM usage is:

L3AccessSW10audj#sh platform tcam errors

TCAM Memory Consistency Checker Errors
--------------------------------------
TCAM Space          Values      Masks     Fixups    Retries   Failures
Unassigned               0          0          0          0          0
HFTM                     0          0          0          0          0
HQATM                   51          0         27        120         24

L3AccessSW10audj#sh platform tcam utilization

CAM Utilization for ASIC# 0                      Max            Used
                                             Masks/Values    Masks/values

Unicast mac addresses:                        528/4224         31/162
IPv4 IGMP groups + multicast routes:          152/1216          6/26
IPv4 unicast directly-connected routes:       528/4224         31/162
IPv4 unicast indirectly-connected routes:     280/2240         12/54
IPv4 policy based routing aces:               512/512           2/2
IPv4 qos aces:                                768/768         324/324
IPv4 security aces:                          1024/2048        306/1314

Note: Allocation of TCAM entries per feature uses
a complex algorithm. The above information is meant
to provide an abstract view of the current TCAM utilization

L3AccessSW10audj#sh platform tcam usage

=============================================================================
                                  TCAM Table
TCAM / SSRAM Table            TCAM            SSRAM
                                Start   Size X    Start   Size Y
=============================================================================
Local Forwarding Table:            0   1540 1        0   1540   4
Local Learning Table:              0   1540 1     5500   1540   2
Secondary Forwarding Table:     1080   1D80 1     7F80   1D80   8
QoS Table:                      2E00   1800 1    1F780   1800   4
ACL Table:                      4600   2000 1    25780   2000   4
IPV6 Secondary Forwarding Tabl  7D80    180 2    1F180     C0   8
IPV6 Classification Table:      7F00     80 2    2D780     40   4
IPV6 ACL Table:                 7F80     70 2    2D880     38   4
Station Table:                     0      0 0    2D980   1540   4
MAC Address Table:                 0      0 0    32E80   1000   8
Multicast Expansion Table:         0      0 0    3AE80    422   8
VLAN List Table:                   0      0 0    3E000    400  10
Equal Cost Route Table:            0      0 0    3D000     80  20

X - Number of 144-bit TCAM entries per descriptor
Y - Number of bytes per descriptor
=============================================================================
SDM template realignment scheme-1 in effect

L3AccessSW10audj#sh platform tcam log-results

================================================================
                       CAM Log Results
----------------------------------------------------------------
Total Number of PortASICs: 13

  ASIC 0
    Lookup "Invalid", value 0
    TCAM Index 94207, TCAM Table Index -1
    Cam Log Keys
      key-0: FF-FFFFFFFF-EFF7FFFF
      key-1: FF-FFFFFFFF-EFF7FFFF
      key-2: CF-FFFFFFFF-FDFFFFF7
      key-3: FF-BFFFFFF5-FFFFFFFF

  ASIC 1
    Lookup "Invalid", value 0
    TCAM Index 0, TCAM Table Index -1
    Cam Log Keys
      key-0: 09-10030020-30402CC1
      key-1: 09-10030020-30402CC1
      key-2: 40-00000404-91812040
      key-3: 47-15BCEF08-82E80940

  ASIC 2
    Lookup "Invalid", value 0
    TCAM Index 131071, TCAM Table Index -1
    Cam Log Keys
      key-0: 00-00400000-00000000
      key-1: 00-00000010-00000000
      key-2: 00-00000000-00000800
      key-3: 04-00000000-00000000

  ASIC 3
    Lookup "Invalid", value 0
    TCAM Index 131071, TCAM Table Index -1
    Cam Log Keys
      key-0: 00-00000200-00000000
      key-1: 00-00000200-00000000
      key-2: 00-00000000-00000000
      key-3: 00-00000000-00000000

  ASIC 4
    Lookup "Invalid", value 0
    TCAM Index 100560, TCAM Table Index -1
    Cam Log Keys
      key-0: 00-00000000-00000000
      key-1: 00-22000100-00200000
      key-2: 00-00000000-00080000
      key-3: 00-00000000-00000000

  ASIC 5
    Lookup "Invalid", value 0
    TCAM Index 131071, TCAM Table Index -1
    Cam Log Keys
      key-0: 80-0C000000-20000000
      key-1: 00-0A000000-10004000
      key-2: 0C-00200000-00000000
      key-3: 00-00000000-00000000

  ASIC 6
    Lookup "Invalid", value 0
    TCAM Index 4096, TCAM Table Index -1
    Cam Log Keys
      key-0: 00-870C0004-00200020
      key-1: 08-00000000-01200000
      key-2: 00-00000010-50000040
      key-3: 00-00000000-00400000

  ASIC 7
    Lookup "Invalid", value 0
    TCAM Index 131071, TCAM Table Index -1
    Cam Log Keys
      key-0: 00-00000400-00000000
      key-1: 00-04010000-00000000
      key-2: 00-00000000-00000000
      key-3: 00-00000000-00002000

  ASIC 8
    Lookup "Invalid", value 0
    TCAM Index 131071, TCAM Table Index -1
    Cam Log Keys
      key-0: 00-00000000-00000000
      key-1: 00-00000000-00000000
      key-2: 00-00000000-00000000
      key-3: 00-00000000-40000000

  ASIC 9
    Lookup "Invalid", value 0
    TCAM Index 131071, TCAM Table Index -1
    Cam Log Keys
      key-0: 00-00000000-00000000
      key-1: 00-00000000-00000000
      key-2: 00-00000000-00000000
      key-3: 50-00000000-00000000

  ASIC 10
    Lookup "Invalid", value 0
    TCAM Index 0, TCAM Table Index -1
    Cam Log Keys
      key-0: 00-00000000-00000000
      key-1: 00-00000000-00000000
      key-2: 00-00000000-00000000
      key-3: 00-00000000-00000000

  ASIC 11
    Lookup "Invalid", value 0
    TCAM Index 131071, TCAM Table Index -1
    Cam Log Keys
      key-0: 00-00200000-40040000
      key-1: 00-00000000-00040000
      key-2: 00-00000000-00000000
      key-3: 00-00000000-00000000

  ASIC 12
    Lookup "Invalid", value 0
    TCAM Index 131071, TCAM Table Index -1
    Cam Log Keys
      key-0: 00-00000000-00010000
      key-1: 00-28000000-00280000
      key-2: 00-00000000-000040C4
      key-3: 00-0000040C-00040004

  Notes:
    a) key-0 is most recent cam key
    b) key-0 contains lsb's and key-3 contains msb's
    c) watch for Lookup field in cam key for validating results
    d) TCAM Table Index -1 indicates invalid results
================================================================

L3AccessSW10audj#

[udj]

The switch is running the following verison at the moment (have testet various IOS versions)

Switch Ports Model              SW Version            SW Image

------ ----- -----              ----------            ----------

*    1 52    WS-C3560G-48TS     15.0(1)SE             C3560-IPSERVICESK9-M

[Richard Michael]

Hello,

As i said earlier, this error message is thrown when there is a TCAM corruption and the TCAM consistency check routines on the switch haven't been able to correct the corruption that has happened. Hence the syslog is thrown.

70% of the time rebooting the switch corrects this function incase if this doesn't gets corrected you would need a hardware replacement.

Thanks,

Richard

*Rate useful posts

[udj]

The thing is, that these errors only show, when ACL label 128 is in use .. everything is quiet untill I access-group the final Access-list, and then the above bitching and TCAM errors begin

It does seem like a firmware (or hardware) bug, it is probably not a coincidense that 0-127 works fine and the problem begins at 128.

Do I understand you correctly, when I hear you saying the Hardware is faulty, and should be replaced (with another C3560G)?

(I have only testet on a spare part we had in stock .. I'll see if I can find another C3560G to test.)

Thanks..

[horacio.tello]

Hello,

I have this problem too, but keep going after the reboot.

error:

 %HARDWARE-1-TCAM_ERROR: Found error in HQATM TCAM Spac
e and not able to recover the error
-Traceback= 888484 888E9C 88B418 88C0B8 6F1380 1515690 150C100
2090102: Aug  4 17:00:41: %HARDWARE-1-TCAM_ERROR: Found error in HQATM TCAM Spac
e and not able to recover the error
-Traceback= 888740 888F4C 88B418 88C0B8 6F1380 1515690 150C100
2090103: Aug  4 17:00:41: %HARDWARE-1-TCAM_ERROR: Found error in HQATM TCAM Spac
e and not able to recover the error
-Traceback= 888484 8893A8 88BAE0 88C0B8 6F1380 1515690 150C100
2090104: Aug  4 17:00:41: %HARDWARE-1-TCAM_ERROR: Found error in HQATM TCAM Spac
e and not able to recover the error
-Traceback= 888740 88991C 88BAF0 88C0B8 6F1380 1515690 150C100
2090105: Aug  4 17:00:41: %HARDWARE-1-TCAM_ERROR: Found error in HQATM TCAM Spac
e and not able to recover the error
-Traceback= 888484 889C50 88BB00 88C0B8 6F1380 1515690 150C100
2090106: Aug  4 17:00:41: %HARDWARE-1-TCAM_ERROR: Found error in HQATM TCAM Spac
e and not able to recover the error
-Traceback= 888484 889FB8 88BB10 88C0B8 6F1380 1515690 150C100
2090107: Aug  4 17:00:41: %HARDWARE-1-TCAM_ERROR: Found error in HQATM TCAM Spac
e and not able to recover the error
-Traceback= 888740 88A08C 88BB10 88C0B8 6F1380 1515690 150C100
2090108: Aug  4 17:00:41: %HARDWARE-1-TCAM_ERROR: Found error in HQATM TCAM Spac
e and not able to recover the error
-Traceback= 888740 88A08C 88BB10 88C0B8 6F1380 1515690 150C100
2090109: Aug  4 17:00:41: %HARDWARE-1-TCAM_ERROR: Found error in HQATM TCAM Spac
e and not able to recover the error

Here it's th output for the show command you required.

1) show platform tcam utilization

CAM Utilization for ASIC# 0                      Max            Used
                                             Masks/Values    Masks/values

 Unicast mac addresses:                       1040/8320        103/742
 IPv4 IGMP groups + multicast routes:           56/448           6/26
 IPv4 unicast routes:                            0/0             0/0
 IPv4 policy based routing aces:                 0/0             0/0
 IPv4 qos aces:                                384/384         260/260
 IPv4 security aces:                           384/384          39/39

Note: Allocation of TCAM entries per feature uses
a complex algorithm. The above information is meant
to provide an abstract view of the current TCAM utilization

2) 

show platform tcam usage

=============================================================================
                                  TCAM Table
 TCAM / SSRAM Table            TCAM            SSRAM
                                Start   Size X    Start   Size Y
=============================================================================
 Local Forwarding Table:            0   2240 1        0   2240   4
 Local Learning Table:              0   2240 1     8900   2240   2
 Secondary Forwarding Table:     2080    240 1     CD80    240   8
 QoS Table:                      22C0    C00 1     FC80    C00   4
 ACL Table:                      2EC0    C00 1    12C80    C00   4
 IPV6 Secondary Forwarding Tabl  3E40     C0 2     DF80     60   8
 IPV6 Classification Table:      3F00     80 2    15C80     40   4
 IPV6 ACL Table:                 3F80     70 2    15D80     38   4
 Station Table:                     0      0 0    15E80   2240   4
 MAC Address Table:                 0      0 0    1E780      2   8
 Multicast Expansion Table:         0      0 0    1E800    122   8
 VLAN List Table:                   0      0 0    1F180     40  10
 Equal Cost Route Table:            0      0 0    1F180      0  20

 X - Number of 144-bit TCAM entries per descriptor
 Y - Number of bytes per descriptor
=============================================================================
SDM template realignment scheme-2 in effect
Dummy SFD start: 0xEA80, entries 0x240, size 8

3) 

show platform tcam log-results

================================================================
                       CAM Log Results
----------------------------------------------------------------
Total Number of PortASICs: 12

  ASIC 0
    Lookup "Invalid", value 0
    TCAM Index 0, TCAM Table Index -1
    Cam Log Keys
      key-0: FF-FFF9F9C3-FEE17FBF
      key-1: FF-FFF9F9C3-FEE17FBF
      key-2: FF-FFFFFFFF-FFFFBFFF
      key-3: FF-FFFFFFFF-AFFFDFFF

  ASIC 1
    Lookup "Invalid", value 0
    TCAM Index 32768, TCAM Table Index -1
    Cam Log Keys
      key-0: FF-FFFFFFFF-FFFFFFFF
      key-1: FF-FFFFFFFF-FFFFFFFF
      key-2: FF-FFFFFFFF-FFFFFFFF
      key-3: 00-00000000-00000090

  ASIC 2
    Lookup "Invalid", value 0
    TCAM Index 0, TCAM Table Index -1
    Cam Log Keys
      key-0: FF-7FFFFF5F-FFFFFFFE
      key-1: FF-7FFFFF5F-FFFFFFFE
      key-2: FF-7FFFFF5F-FFFFFFFE
      key-3: FF-FFFF7FFF-FFFFFF7F

  ASIC 3
    Lookup "Invalid", value 0
    TCAM Index 0, TCAM Table Index -1
    Cam Log Keys
      key-0: FF-FBFF7FFF-FFBFFFFF
      key-1: FF-FBFF7FFF-FFBFFFFF
      key-2: D7-FFFFF47D-BDEFFFFF
      key-3: DB-BFFFF95F-9CEFFFE0

  ASIC 4
    Lookup "Invalid", value 0
    TCAM Index 0, TCAM Table Index -1
    Cam Log Keys
      key-0: FF-FFFFFFEF-FEFFDFE5
      key-1: FF-FFFFFFEF-FEFFDFE5
      key-2: D7-FFFFEF9D-F3FF7FFF
      key-3: DF-9F5FFBFF-EDFBEE7F

  ASIC 5
    Lookup "Invalid", value 0
    TCAM Index 0, TCAM Table Index -1
    Cam Log Keys
      key-0: FF-FFFFFFFF-FFFFFFFF
      key-1: FF-FFFFFFFF-FFFFFFFF
      key-2: FF-FFFFFFFF-FFFFFFFF
      key-3: FF-FFFFFEE8-37F3FFFF

  ASIC 6
    Lookup "Invalid", value 0
    TCAM Index 116735, TCAM Table Index -1
    Cam Log Keys
      key-0: F7-FFFFFFFF-B7FF7FEF
      key-1: 78-302A1023-8F69760C
      key-2: ED-FFF7EFFF-FFFD6DFF
      key-3: E8-FFD7FFFF-DFDFAFFF

  ASIC 7
    Lookup "Invalid", value 0
    TCAM Index 0, TCAM Table Index -1
    Cam Log Keys
      key-0: FF-FFBFFFFF-FFFFFFFE
      key-1: FF-FFBFFFFF-FFFFFFFE
      key-2: FF-FFBFFFFF-FFFFFFFE
      key-3: FF-7DFDF85F-F2E9FFF7

  ASIC 8
    Lookup "Invalid", value 0
    TCAM Index 131071, TCAM Table Index -1
    Cam Log Keys
      key-0: FF-FFFFFFFF-FFFFFFFF
      key-1: FF-FFFFFFFF-FFFFFFFF
      key-2: 2F-FFFEFFFF-FFFFFFDF
      key-3: FF-B9FEFFDF-DEFEBBFF

  ASIC 9
    Lookup "Invalid", value 0
    TCAM Index 0, TCAM Table Index -1
    Cam Log Keys
      key-0: FF-FFFFFFFE-FFFFFFFF
      key-1: FF-FFFFFFFE-FFFFFFFF
      key-2: FF-FFFFFFFE-FFFFFFFF
      key-3: FF-FBDFFE71-CFFFFFFF

  ASIC 10
    Lookup "Invalid", value 0
    TCAM Index 0, TCAM Table Index -1
    Cam Log Keys
      key-0: FF-FFFFFFFF-F7FFFFFF
      key-1: FF-FFFFFFFF-F7FFFFFF
      key-2: FF-7FFDFFFF-7FA6F7FD
      key-3: 9C-7FDFFFF6-7DF3FE75

  ASIC 11
    Lookup "Invalid", value 0
    TCAM Index 0, TCAM Table Index -1
    Cam Log Keys
      key-0: FF-FBFFFF7F-EFFFFFFF
      key-1: FF-FBFFFF7F-EFFFFFFF
      key-2: EF-FFFFFFFF-FFFFFFFB
      key-3: E3-EFFFFFF6-B7FFFFFF

  Notes:
    a) key-0 is most recent cam key
    b) key-0 contains lsb's and key-3 contains msb's
    c) watch for Lookup field in cam key for validating results
    d) TCAM Table Index -1 indicates invalid results
================================================================

Hope you can help us, I look forward yor answer.

Best regards,