Hi,
what about trying to make the ISP to prefer your primary router?
You could send the BGP updates from your secondary router with a worse MED attribute or with your AS number prepended several times towards the ISP, e.g.
You would need your HSRP to track the ISP router(s) reachability on your primary router then, e.g., to decrease HSRP priority in a case it loses connectivity to the ISP router(s). But it's possible to configure.
So in a case of connectivity failure between your primary router and the ISP router(s):
a) The HSRP priority on your primary router would decrease, so the secondary rputer would become Active in HSRP (i.e., it would start to receive the traffic from the LAN).
b) The ISP would not receive BGP prefixes from your primary router anymore, so it would have to start using the prefixes advertised from your secondary router (with worse BGP attributes).
So finally traffic in both directions would take your secondary router.
When your primary router connection to the ISP router(s) would get recovered, the traffic would return to your primary router automatically.
HTH,
Milan