SNS servers are effectively a one-trick-pony server - they will never boot any application other than ISE. How exactly Cisco does this is a mystery to me and many others. I don't have an old SNS server to try hack it to load ESXi or Linux etc. But if you trawl the web forums long enough you will find articles of folks who have tried. I thought it was as simple as UEFI Secure Boot ... just disable that and then you should be able to load any other OS. But it's not that simple. Many OS's will boot with UEFI Secure Boot enabled - yet, on an SNS they refuse to boot. If you login to the CIMC of the UCS you will see there are hundreds of settings you can tweak - but I also bet that there is something hard coded in the BIOS that ordinary users don't have easy access to.
As for, why ISE didn't boot on the generic UCS mainboard - most likely because that magic sauce was missing in the BIOS and the detection logic in the ISE bootloader refused to boot. Cisco will argue that they did this because they want to ensure that ISE only boots on a trusted platform - i.e. their super secure SNS server.
My personal opinion is that the SNS server should be allowed to be repurposed once newer ISE versions no longer boot on them. The current situation is that once a version of ISE no longer boots/installs on older hardware, you effectively have to scrap the server. That's a terrible situation for sustainability.
It's also important to note that if you plan to update the firmware of your CIMC/BIOS then you MUST always use the firmware on the ISE downloads page - do NOT use the firmware on the Cisco UCS page. You will make ISE unbootable.
I wonder if the TAC should have applied the latest ISE UCS HUU firmware on the generic mainboard - that might have got ISE to boot up. That would have been my suggestion.
