Verified scalability guide, provides a verified number that is supported by both software and hardware, where the data sheet states what are values supported by hardware only (as table 3 headline states in the data sheet). If the user exceeds the verified limit, new entries will not be part of the table and traffic will be forwarded as an unknown unicast. More on this below:
When the packet/frame comes to the switch, this packet will come with unique a SMAC or DMAC, switch will not find DMAC in table (as table is full and not new entries are learned), that way switch will replicate this packet without changing mac addresses and send it to everybody, in this case this is not broadcast packet (explained about why is not) but unknown unicast (even does behave as broadcast in a sense it is flooded to everybody).
