Yes, you can use an external AAA server with protocols as Radius or LDAP to perform the authorization part.
This authorization server can send a specific group-policy for the connection.
For example, if using Radius, the server can send attribute 25 which is for the group-policy assignment.
If LDAP, attribute ldap mapping can be used, link for config:
Then, you would configure the AAA server under the SAML tunnel-group with the command: "authorization-server-group".
Rate if it helps.
TAC - VPN Engineer.