Cisco ASA - Protected Access Credential (PAC) import/resilience questions

[Michal Bruncko]

Hello all,

there is not too much information about Protected Access Credential (PAC) file deployment and background for Cisco ASA firewalls except this and another official documentation.

there are stated following restrictions about importing PAC credentials:

• When the ASA is part of an HA configuration, you must import the PAC file to the primary ASA device.
• When the ASA is part of a clustering configuration, you must import the PAC file to the master
  device.

Questions:

• But what about in case the ASA is deployed with multi-context mode and PAC file has to be imported to specific context only:
  • In case active context is hosted on primary ASA unit, then I assume importing PAC file on active context is right approach, yes?
  • What in case the active context is hosted on standby ASA unit? Where to import that PAC file?
• About PAC file resiliency does stored/imported PAC file:
  • survives active context ASA reboot? I assume it should be copied over/used by standby context, right?
  • survives ASA failover cluter reboot (i.e. both ASA units will be reloaded for example due to power outage)?
  • survives ASA OS upgrade? 
• Where actually is the PAC credentials stored? I assume its not in stored in startup-configuration. 

thank you

michal

[Sheraz.Salim · Accepted solution]

Questions:

• But what about in case the ASA is deployed with multi-context mode and PAC file has to be imported to specific context only:
  • In case active context is hosted on primary ASA unit, then I assume importing PAC file on active context is right approach, yes?  -Correct.
  • What in case the active context is hosted on standby ASA unit? Where to import that PAC file? remember where ever the active context is PAC need to be hosted on that context. so for example Context A is active on Primary than PAC need to be on Primary and Context B is active on Secondary than PAC need to be on Secondary firewall. cisco documenation is not very clear on multix-context mode.
• About PAC file resiliency does stored/imported PAC file:
  • survives active context ASA reboot? I assume it should be copied over/used by standby context, right? -Correct
  • survives ASA failover cluter reboot (i.e. both ASA units will be reloaded for example due to power outage)? -Once the master (owner) back on line it take the priority.
  • survives ASA OS upgrade? -As a single unit the PAC stay intact so does in HA.
    
• Where actually is the PAC credentials stored? I assume its not in stored in startup-configuration. -The CTS credentials state retrievalisnot performed by the non volatile generation process(NVGEN)becausethe CTS credential information is saved in the keystore,not in the startup-config.Those credentialsare stored in the keystore,eliminating the need to save the running-config.
  

[Michal Bruncko]

Thank you very much Sheraz for your qualifed response. Now the real case based on which I've triggered this topic. We're using HA setup of two ASA firewalls with multi-context mode. In this particular case the primary unit hosts all active contexts. OS upgrade have been performed on this HA ASA solution. After OS upgrade the PAC credentials were no longer present at all... neither on active or standby context. Do you have any explanation why this could happen? Upgrade was done from 9.6(4) to 9.8(4).

What I can tell you more is that the backup unit was upgraded and reloaded first. This would mean that all the time of backup unit upgrade/reload the primary was active and I assume PAC credentials were in place. 

I would assume that during reload of upgraded primary unit, the PAC credentials *potentially* could not be used by already upgraded secondary unit due to theoretical OS version difference stateful failover incompatibility, but based on the "owner" status of PAC credentials by active context on primary unit I would assume it will be taken by active context from internal keystore and used like before upgrade. And therefore TrustSec security groups should be enumerated without issues. 

My only understanding is that there's a bug or whathever incompatibility for using keystore (structure change?) which breaks possibility to take the stored PAC credentials...

or anything else?

thank you

[Michal Bruncko]

One more hypotetical question: in case that administrator mistakenly import same PAC file (i.e. credentials created for specific device) to two different firewalls - could this be a reason for PAC credential expiration on both ASA firewalls?

[Sheraz.Salim]

Hi Michal.

sorry for the late responce I had some family commitment so i was away.

- Now the real case based on which I've triggered this topic. We're using HA setup of two ASA firewalls with multi-context mode. In this particular case the primary unit hosts all active contexts. OS upgrade have been performed on this HA ASA solution. After OS upgrade the PAC credentials were no longer present at all... neither on active or standby context. Do you have any explanation why this could happen? Upgrade was done from 9.6(4) to 9.8(4).

Primary Unit host all the active context and the secondary unit was keeping all the context as standby. once the software upgrade was performed the PAC credentials were no longer exist. so this confirms if there is a power outrage on the primary unit it will replicate everything aprat from PAC as you done the software upgrade and once the system boot up as standby (which was active prior to upgrade). might this not relevent but have you see this Bug

CSCvx27430

ASA: Unable to import PAC file if FIPS is enabled.

- What I can tell you more is that the backup unit was upgraded and reloaded first. This would mean that all the time of backup unit upgrade/reload the primary was active and I assume PAC credentials were in place.

You saying the backup unit (Secondary Firewall) was upgraded and reloaded first? If answer is yes. than once the Secondary firewall is upgrade to new version and once it come online did you check the pac was still on the secondary (now with new software) on it?

- would assume that during reload of upgraded primary unit, the PAC credentials *potentially* could not be used by already upgraded secondary unit due to theoretical OS version difference stateful failover incompatibility, but based on the "owner" status of PAC credentials by active context on primary unit I would assume it will be taken by active context from internal keystore and used like before upgrade. And therefore TrustSec security groups should be enumerated without issues.

I agree with you what you describe above.

- One more hypotetical question: in case that administrator mistakenly import same PAC file (i.e. credentials created for specific device) to two different firewalls - could this be a reason for PAC credential expiration on both ASA firewalls?

I guess if this was the case the ASA should have give warning or thown an error back to the CLI.