Re: reliability, depends much on the provider. Normally, one would expect it to be better for MPLS because providers tend to charge more for it and also offer some kind of SLA.
"Speed", everything else being equal, less overhead (I believe) with MPLS so it can provide more usable bandwidth per link and newer hardware can directly support it at very, very high rates.
Performance of VPN solutions can very much be limited by encryption/dencryption processing. Also with VPN, packet fragmentation, with tunnels, can be an issue.
Beyond processing overhead, MPLS often supports QoS and/or offers dedicated bandwidth, VPN running across the public Internet tends to support neither.
In summary, MPLS is generally technically better for supporting high performance. However, in the real-world (and keeping in mind dependencies on providers), VPN can, at "typical" WAN bandwidths, provide performance not all that distant from MPLS but usually at a much lower WAN bandwidth cost. Public Internet bandwidth is also much easier to obtain in many parts of the world where MPLS is hard to obtain.
