Im pretty sure you would need to filter upstream i.e. with an ACL on the router, and on the ASA you enable ISAKMP on the outside interface, so this behaviour is normal.
But having UDP 500 open is not really a vulnerability? if an attacker can't get a matching phase one policy then they can't get in (assuming you have the recent IKE vulnerability patched, see the following link)
https://tools.cisco.com/security/center/content/CiscoSecurityAdvisory/cisco-sa-20160210-asa-ike
Pete