After users connect for the first time using IP, I assume their profile is downloaded and the box on Anyconnect where you input the IP of the ASA is changed to the FQDN. This name will not resolve because it is not in any DNS except for my local.
Is there a way to remove the FQDN and just have the ASA's IP automatically inserted instead of the user replacing the FQDN with it IP every time?
You can use the AnyConnect profile editor to update the profile as @RJI suggested.
However if the certificate on the ASA references the FQDN and does not have the IP address as a Subject Alternative Name (SAN) then users will always get a certificate warning when connecting.