Since weight/local preference were the same I was thinking R2 was using R1s default route since it has a shorter AS Path. To get around that i had tried prepend the AS a couple of times so that R1's advertised default route would have a AS Path of 3333 3333 3333, but that didn't seem to work as when i issued a show ip bgp the AS path was still 3333.
route-map IN_PREPEND permit 10
set as-path prepend last-as 3
R1(config-route-map)#router bgp 12345
R1(config-router)# address-family ipv4
R1(config-router-af)# neighbor 10.0.0.2 route-map IN_PREPEND in
In terms of the iBGP/eBGP and local preference, you're saying because the local preference was 150 on R2, R1 will choose to prefer the iBGP route w/ a metric of 200 regardless of the eBGP route? If that's the case I understand the reason why the OSPF route would be installed as R1 has a static 0.0.0.0/0 route pointing to an interface w/ a metric of 200 which would explain the rib failure on the BGP route.
Correct re your last paragraph, that is why the OSPF route is chosen.
You could use weight because that is local to the router ie. it is not passed between routers so apply weight to the default route that R2 receives from it's ISP peer which would mean R1 uses it's default, R2 uses it's default but they would still send the IBGP routes to each other.
Using the weight attribute worked perfectly in the lab so I'll be updating that in production soon. I did have 1 question that was discovered during testing:
When we were testing that internet access was available through both R1 and R2 we noticed that if we did things in this order we had an outage. Disconnected R1 from it's upstream provider; it then learned it's route via OSPF from R2 as expected and routed out that connection. Trace route and looking up connections on BGP looking glasses showed traffic going through R2 ISP. But then when we reconnected R1 to it's upstream and it began learning the default route and sending traffic out that way. Trace routes seemed to indicate the correct path.
When we then disconnected R2 from it's upstream we expected traffic to fail back to R1. R2 did in fact have a default route of R1, but pings to the internet failed. Looking on BGP looking glasses and the networks that should have been advertised out of R1 were still showing as available from R2's ISP for quite some time. We plugged R2 back in after a few minutes as there was no internet available for anyone in the building. That restored service and some time later when i checked the looking glasses I was able to see the path had gone back over to the carrier for R1.
Shouldn't the route update from R1 to its ISP have been pushed out fairly quickly. R1 is advertising a /24 and R2 a /23 as well.