Well I have made some progress, after powering it down for awhile (3-5 minutes) the device boots properly.
If I try restarting the device immediately after, it begins to do the reboot loop until I power it off for awhile again.
I received the following for each port after getting the press Return to begin message:
POST: Loop back Test Failed on FastEthernet0/1. State:Disabled
Looking up the error, it appears that is a hardware issue.