With IPSec you establish a policy that says encrpyt a packet when it goes from this adreess to that address, and send the packet to this remote IP address.
So things like broadcast and mulicast dont work, which many routing protcols need.
Additionally because the encrytion address range is statically declared the network doesn't respond well to changes in its topology (usually cauused by device or link failure).
GRE over IPSec looks like a simple point to point link. Routing protocls can use it, and can route around failures.
Generally I use IPSec for small networks, and GRE over IPSec for medium to large networks.
