cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
5576
Views
4
Helpful
16
Replies

Upgrade to ise 2.3 failed

alyautdinov
Level 1
Level 1

Hello team.

We tryed to update ISE 2.2 to 2.3 and got such error:

UPGRADE STEP 1: Running ISE configuration database schema upgrade...
- Running db sanity to check and fix if any index corruption
- Auto Upgrading Schema for UPS Model
- Upgrading Schema completed for UPS Model
UPGRADE STEP 2: Running ISE configuration data upgrade...
- Data upgrade step 1/18, UPSUpgradeHandler(2.3.0.100)... .Failed.
% Error: ISE Global data upgrade failed!

After, we installed  2.3 from stratch version and tryed to restore configuration backup from ise 2.2 but still same error

Logs from dbupgrade-data-global-.log

Retrived the data from Handlercom.cisco.cpm.ups.upgrade.impl.PolicyUpgradeHandler]

com.cisco.cpm.infrastructure.upgrade.api.UpgradeFailureException: java.lang.NullPointerException

        at com.cisco.cpm.ups.upgrade.UpgradeHandler.exportAndImport(UpgradeHandler.java:41)

        at com.cisco.cpm.ups.upgrade.UpgradeHandler.execUpgrade(UpgradeHandler.java:29)

        at com.cisco.cpm.ups.upgrade.impl.UPSUpgradeHandler.upgrade(UPSUpgradeHandler.java:151)

        at com.cisco.cpm.infrastructure.upgrade.impl.UpgradeServiceRegistrar.UpgradeServices(UpgradeServiceRegistrar.java:132)

        at com.cisco.cpm.infrastructure.upgrade.impl.UpgradeServiceRegistrar.main(UpgradeServiceRegistrar.java:185)

Caused by: java.lang.NullPointerException

        at com.cisco.cpm.policy.configuration.upgrade.builder.AbstractUpgradePolicyDataBuilder.buildAuthenticationRuleResultDataForOuterDefaultRule(AbstractUpgradePolicyDataBuilder.java:284)

        at com.cisco.cpm.policy.configuration.upgrade.builder.AbstractUpgradePolicyDataBuilder.buildAuthenticationInnerRules(AbstractUpgradePolicyDataBuilder.java:182)

        at com.cisco.cpm.policy.configuration.upgrade.builder.UpgradePolicyDataBuilderRadius.buildUpgradeData(UpgradePolicyDataBuilderRadius.java:99)

        at com.cisco.cpm.policy.configuration.upgrade.PolicyUpgrade.upgradeLegacySetRadius(PolicyUpgrade.java:394)

        at com.cisco.cpm.policy.configuration.upgrade.PolicyUpgrade.upgradeLegacySet(PolicyUpgrade.java:337)

        at com.cisco.cpm.policy.configuration.upgrade.PolicyUpgrade.upgradeLegacySets(PolicyUpgrade.java:213)

        at com.cisco.cpm.ups.upgrade.impl.PolicyUpgradeHandler.importData(PolicyUpgradeHandler.java:67)

        at com.cisco.cpm.ups.upgrade.UpgradeHandler.exportAndImport(UpgradeHandler.java:38)

        ... 4 more

Error while applying changes in version: 2.3.0.100 class: com.cisco.cpm.ups.upgrade.impl.UPSUpgradeHandler

com.cisco.cpm.infrastructure.upgrade.api.UpgradeFailureException: Failed to upgrade to version 2.3.0.100: java.lang.NullPointerException

        at com.cisco.cpm.ups.upgrade.impl.UPSUpgradeHandler.upgrade(UPSUpgradeHandler.java:159)

        at com.cisco.cpm.infrastructure.upgrade.impl.UpgradeServiceRegistrar.UpgradeServices(UpgradeServiceRegistrar.java:132)

        at com.cisco.cpm.infrastructure.upgrade.impl.UpgradeServiceRegistrar.main(UpgradeServiceRegistrar.java:185)

ERROR! isedataupgrade.sh FAILED. ISE GLOBAL DATA UPGRADE FAILED

What is the reason?

1 Accepted Solution

Accepted Solutions

hslai
Cisco Employee
Cisco Employee

I am still waiting on our dev team's analysis, but I found two issues:

In the RADIUS policy sets, the Easy Connect policy set has the same condition for the policy set itself and for the non-default authentication policy rule MAB. And, the "Default Rule (if no match)" will never match. After combining the two rules into one -- the "Default Rule (if no match)" to use Default Network Access as the allowed protocols and Internal Endpoints as the ID source, URT able to complete the sanity tests for the RADIUS policy sets.

In the T+ policy sets, the Test_admin one has the condition "Network Access:Protocol EQUALS TACACS+". This is odd because it's always true for T+ auth. After merging it into one -- the "Default Rule (if no match)" to use Default Device Admin as the allowed protocols and PC_ISE_Ebusiness as the ID source, URT able to complete the sanity for T+ policy sets.

After these two edits, URT completed successfully.

View solution in original post

16 Replies 16

scamarda
Cisco Employee
Cisco Employee

I had the same issue with my 2 node 2.2P2 setup.  Same exact error on both nodes.  I am in the process of creating new 2.3 machines as well.

Timothy Abbott
Cisco Employee
Cisco Employee

Hi,

Please work with TAC to find out why the upgrade failed.

Regards,

-Tim

This is test environment. Lab version without license.

The failure seems something to do with authentication policy outer rules for Network Access. Could you post the screenshots of your policy sets, if any, and authentication policy rules?

hslai
Cisco Employee
Cisco Employee

Sam's failure has a different text so it's not the same as your failure.

paul
Level 10
Level 10

Rebuild with new version and restore backup is the best way to upgrade an ISE deploiyment.  I have done 50+ this way.  Every time I have tried either the CLI ugprade process something has blown up.  The rebuild/restore method is very predictable and offers maximum control over the upgrade process.

What you you are finding out is that when the automated process blows up you are going to spend days trying to fix it or investigate what went wrong when the whole process could have been down in a few hours using rebuild/restore method.

Have you tried the upgrade readiness tool to see what it says as well?

https://www.cisco.com/c/en/us/td/docs/security/ise/2-3/upgrade_guide/b_ise_upgrade_guide_23/b_ise_upgrade_guide_23_chapter_01.html

Yes we tried with URT:

com.cisco.cpm.ups.upgrade.impl.PolicyUpgradeHandler.importData(PolicyUpgradeHandler.java:67)

        at com.cisco.cpm.ups.upgrade.UpgradeHandler.exportAndImport(UpgradeHandler.java:38)

        ... 4 more

Error while applying changes in version: 2.3.0.100 class: com.cisco.cpm.ups.upgrade.impl.UPSUpgradeHandler

com.cisco.cpm.infrastructure.upgrade.api.UpgradeFailureException: Failed to upgrade to version 2.3.0.100: java.lang.NullPointerException

        at com.cisco.cpm.ups.upgrade.impl.UPSUpgradeHandler.upgrade(UPSUpgradeHandler.java:159)

        at com.cisco.cpm.infrastructure.upgrade.impl.UpgradeServiceRegistrar.UpgradeServices(UpgradeServiceRegistrar.java:132)

        at com.cisco.cpm.infrastructure.upgrade.impl.UpgradeServiceRegistrar.main(UpgradeServiceRegistrar.java:185)

ERROR! isedataupgrade.sh FAILED. ISE GLOBAL DATA UPGRADE FAILED

Please let us know whether you may provide the CFG backup for more investigation.

We tried to restore backup 2.2 on 2.3 version but have same error:

UPGRADE STEP 1: Running ISE configuration database schema upgrade...
- Running db sanity to check and fix if any index corruption
- Auto Upgrading Schema for UPS Model
- Upgrading Schema completed for UPS Model
UPGRADE STEP 2: Running ISE configuration data upgrade...
- Data upgrade step 1/18, UPSUpgradeHandler(2.3.0.100)... .Failed.
% Error: ISE Global data upgrade failed!

hslai
Cisco Employee
Cisco Employee

I am still waiting on our dev team's analysis, but I found two issues:

In the RADIUS policy sets, the Easy Connect policy set has the same condition for the policy set itself and for the non-default authentication policy rule MAB. And, the "Default Rule (if no match)" will never match. After combining the two rules into one -- the "Default Rule (if no match)" to use Default Network Access as the allowed protocols and Internal Endpoints as the ID source, URT able to complete the sanity tests for the RADIUS policy sets.

In the T+ policy sets, the Test_admin one has the condition "Network Access:Protocol EQUALS TACACS+". This is odd because it's always true for T+ auth. After merging it into one -- the "Default Rule (if no match)" to use Default Device Admin as the allowed protocols and PC_ISE_Ebusiness as the ID source, URT able to complete the sanity for T+ policy sets.

After these two edits, URT completed successfully.

Hi,

Unfortunately, URT failed for PS:Checkpoint.

For PS:Easyconnect Test migration was succesful:

@@@ PsUpgrade:  info- :***** Upgrade process for the legacy PS:Easyconnect Test was finished with the result:PolicyUpgradeResult status:SUCESS...Hooray!         Policy Id:84438d00-80cd-11e7-b4bf-02427242cd9c  Policy Name:Easyconnect Test


Full dbupgrade-data.log for PS:Checkpoint:

@@@ PsUpgrade:  info- :*** Starting an upgrade process for the Radius  legacy PS:Checkpoint

@@@ PsUpgrade:  debug- :Build PS level condition for PS: Checkpoint

@@@ PsUpgrade:  debug- :About to get condition RHS display value for Network Access with attribute Protocol

@@@ PsUpgrade:  debug- :Network Access:Protocol has allow values enumeration

@@@ PsUpgrade:  debug- : Found allow value for Network Access:Protocol0:RADIUS

@@@ PsUpgrade:  warn- :Couldn't buildConditionDataForNameValue for: lhsAttrId:DEVICE.Migrated_NDGs rhsString:Migrated_NDGs#All Migrated_NDGs#CheckPoint#CP_TEST, Will try to build it from rhs value

com.cisco.cpm.policy.pal.PalException: Value for attribute is not a permitted option

        at com.cisco.cpm.policy.pal.policyCondition.ConditionsData.validateAllowedValues(ConditionsData.java:510)

        at com.cisco.cpm.policy.pal.policyCondition.ConditionsData.initSimple(ConditionsData.java:425)

        at com.cisco.cpm.policy.pal.policyCondition.ConditionsData.<init>(ConditionsData.java:290)

        at com.cisco.cpm.policy.configuration.upgrade.PolicyUpgradeUtil.buildConditionDataForNameValue(PolicyUpgradeUtil.java:947)

        at com.cisco.cpm.policy.configuration.upgrade.builder.UpgradeNetAccessRuleBuilder.buildConditionDataClauseSimple(UpgradeNetAccessRuleBuilder.java:139)

        at com.cisco.cpm.policy.configuration.upgrade.builder.UpgradeNetAccessRuleBuilder.buildConditionDataClauses(UpgradeNetAccessRuleBuilder.java:99)

        at com.cisco.cpm.policy.configuration.upgrade.builder.UpgradeNetAccessRuleBuilder.buildRuleConditionData(UpgradeNetAccessRuleBuilder.java:70)

        at com.cisco.cpm.policy.configuration.upgrade.builder.AbstractUpgradePolicyDataBuilder.buildNetAccessRuleConditionData(AbstractUpgradePolicyDataBuilder.java:78)

        at com.cisco.cpm.policy.configuration.upgrade.builder.UpgradePolicyDataBuilderRadius.buildNetAccessRuleConditionData(UpgradePolicyDataBuilderRadius.java:200)

        at com.cisco.cpm.policy.configuration.upgrade.builder.AbstractUpgradePolicyDataBuilder.buildPSLevelConditionsData(AbstractUpgradePolicyDataBuilder.java:64)

        at com.cisco.cpm.policy.configuration.upgrade.builder.UpgradePolicyDataBuilderRadius.buildUpgradeData(UpgradePolicyDataBuilderRadius.java:76)

        at com.cisco.cpm.policy.configuration.upgrade.PolicyUpgrade.upgradeLegacySetRadius(PolicyUpgrade.java:394)

        at com.cisco.cpm.policy.configuration.upgrade.PolicyUpgrade.upgradeLegacySet(PolicyUpgrade.java:337)

        at com.cisco.cpm.policy.configuration.upgrade.PolicyUpgrade.upgradeLegacySets(PolicyUpgrade.java:213)

        at com.cisco.cpm.ups.upgrade.impl.PolicyUpgradeHandler.importData(PolicyUpgradeHandler.java:67)

        at com.cisco.cpm.ups.upgrade.UpgradeHandler.exportAndImport(UpgradeHandler.java:38)

        at com.cisco.cpm.ups.upgrade.UpgradeHandler.execUpgrade(UpgradeHandler.java:29)

        at com.cisco.cpm.ups.upgrade.impl.UPSUpgradeHandler.upgrade(UPSUpgradeHandler.java:151)

        at com.cisco.cpm.infrastructure.upgrade.impl.UpgradeServiceRegistrar.UpgradeServices(UpgradeServiceRegistrar.java:132)

        at com.cisco.cpm.infrastructure.upgrade.impl.UpgradeServiceRegistrar.main(UpgradeServiceRegistrar.java:185)

@@@ PsUpgrade:  debug- :Trying to rebuildConditionDataForNameValue  for: lhsAttrId:DEVICE.Migrated_NDGs rhsString:Migrated_NDGs#All Migrated_NDGs#CheckPoint#CP_TEST

@@@ PsUpgrade:  info- :Successfully rebuildConditionDataForNameValue for: lhsAttrId:DEVICE.Migrated_NDGs rhsString:All Migrated_NDGs#CheckPoint#CP_TEST

@@@ PsUpgrade:  debug- :Reading Authentication rules for Policy Set Checkpoint

@@@ PsUpgrade:  debug- :Reading Default Authentication rule for Policy Set Checkpoint

@@@ PsUpgrade:  debug- :Build authentication result data for default rule  of Policy Set  Checkpoint

isPolicySetModeActivated --> pss.getPolicySetMode() = POLICY_SET

-->validatePolicyMode, isArrivingFromPolicySetAPI= true

isPolicySetModeActivated --> pss.getPolicySetMode() = POLICY_SET

-->validatePolicyMode, PolicySetRestService.isPolicySetModeActivated() = true

isPolicySetModeActivated --> pss.getPolicySetMode() = POLICY_SET

@@@ PsUpgrade:  debug- :Built authentication result for rule Default with following attributes: Identity Source=DenyAccess, If Auth fail=REJECT, If Process fail=DROP, If User not found=REJECT

@@@ PsUpgrade:  debug- :Found 1 non default Authentication rules for Policy Set Checkpoint

@@@ PsUpgrade:  debug- :Reading Authentication rule Standard Rule 1  of Policy Set  Checkpoint

@@@ PsUpgrade:  debug- :Build authentication result data for rule Standard Rule 1 in Policy Set Checkpoint

isPolicySetModeActivated --> pss.getPolicySetMode() = POLICY_SET

-->validatePolicyMode, isArrivingFromPolicySetAPI= true

isPolicySetModeActivated --> pss.getPolicySetMode() = POLICY_SET

-->validatePolicyMode, PolicySetRestService.isPolicySetModeActivated() = true

isPolicySetModeActivated --> pss.getPolicySetMode() = POLICY_SET

@@@ PsUpgrade:  debug- :Build authentication rule result data for outer rule Standard Rule 1

@@@ PsUpgrade:  debug- :Reading authentication inner rules for PS: Checkpoint

@@@ PsUpgrade:  debug- :Build authentication rule result data for outer default rule

isPolicySetModeActivated --> pss.getPolicySetMode() = POLICY_SET

-->validatePolicyMode, isArrivingFromPolicySetAPI= true

isPolicySetModeActivated --> pss.getPolicySetMode() = POLICY_SET

-->validatePolicyMode, PolicySetRestService.isPolicySetModeActivated() = true

isPolicySetModeActivated --> pss.getPolicySetMode() = POLICY_SET

Retrived the data from Handlercom.cisco.cpm.ups.upgrade.impl.PolicyUpgradeHandler]

com.cisco.cpm.infrastructure.upgrade.api.UpgradeFailureException: java.lang.NullPointerException

        at com.cisco.cpm.ups.upgrade.UpgradeHandler.exportAndImport(UpgradeHandler.java:41)

        at com.cisco.cpm.ups.upgrade.UpgradeHandler.execUpgrade(UpgradeHandler.java:29)

        at com.cisco.cpm.ups.upgrade.impl.UPSUpgradeHandler.upgrade(UPSUpgradeHandler.java:151)

        at com.cisco.cpm.infrastructure.upgrade.impl.UpgradeServiceRegistrar.UpgradeServices(UpgradeServiceRegistrar.java:132)

        at com.cisco.cpm.infrastructure.upgrade.impl.UpgradeServiceRegistrar.main(UpgradeServiceRegistrar.java:185)

Caused by: java.lang.NullPointerException

        at com.cisco.cpm.policy.configuration.upgrade.builder.AbstractUpgradePolicyDataBuilder.buildAuthenticationRuleResultDataForOuterDefaultRule(AbstractUpgradePolicyDataBuilder.java:284)

        at com.cisco.cpm.policy.configuration.upgrade.builder.AbstractUpgradePolicyDataBuilder.buildAuthenticationInnerRules(AbstractUpgradePolicyDataBuilder.java:182)

        at com.cisco.cpm.policy.configuration.upgrade.builder.UpgradePolicyDataBuilderRadius.buildUpgradeData(UpgradePolicyDataBuilderRadius.java:99)

        at com.cisco.cpm.policy.configuration.upgrade.PolicyUpgrade.upgradeLegacySetRadius(PolicyUpgrade.java:394)

        at com.cisco.cpm.policy.configuration.upgrade.PolicyUpgrade.upgradeLegacySet(PolicyUpgrade.java:337)

        at com.cisco.cpm.policy.configuration.upgrade.PolicyUpgrade.upgradeLegacySets(PolicyUpgrade.java:213)

        at com.cisco.cpm.ups.upgrade.impl.PolicyUpgradeHandler.importData(PolicyUpgradeHandler.java:67)

        at com.cisco.cpm.ups.upgrade.UpgradeHandler.exportAndImport(UpgradeHandler.java:38)

        ... 4 more

Error while applying changes in version: 2.3.0.100 class: com.cisco.cpm.ups.upgrade.impl.UPSUpgradeHandler

com.cisco.cpm.infrastructure.upgrade.api.UpgradeFailureException: Failed to upgrade to version 2.3.0.100: java.lang.NullPointerException

        at com.cisco.cpm.ups.upgrade.impl.UPSUpgradeHandler.upgrade(UPSUpgradeHandler.java:159)

        at com.cisco.cpm.infrastructure.upgrade.impl.UpgradeServiceRegistrar.UpgradeServices(UpgradeServiceRegistrar.java:132)

        at com.cisco.cpm.infrastructure.upgrade.impl.UpgradeServiceRegistrar.main(UpgradeServiceRegistrar.java:185)

ERROR! isedataupgrade.sh FAILED. ISE GLOBAL DATA UPGRADE FAILED

Please provide a new CFG backup to the same dropbox location. I still have the link in my mail client.

You can download CFG backup from the same link.

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: