I have a setup where - I have a cisco stack (4X SGE2010 Switches) trunking over to a 3COM switch. Both switches believe to be the "ROOT" of the network. Note The 3COM is running RSTP as opposed to the Cisco Stack which is running normal STP.
To my understanding of STP - Essentially STP is not functioning! Both switches believe to be the "ROOT" so they don't shut ports down. (We are currently having major issues with ports going up and down for seconds at a time on both switches)
My real question I guess is.... Does this sound like the issue? I'm due to turn on RSTP on the Cisco Stack and lower the priority for your info.
If you are running the default PVST on the Cisco Switch and 3COM is running RSTP, then each one of those will elect a root bridge. I had the same scenario recently but with two Cisco switches. I updated them all to rpvst and now there is only one root bridge.