cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
7277
Views
16
Helpful
14
Replies

UCCX 11.6 Ontape failed to backup UCCX DB

cristian.munoz
Level 4
Level 4

Hi All

I have a issue with the backup of UCCX. The backup failing in CCXCOMPONENT and the error is "Ontape Backup failed."

Please see bellow:

 

020-06-07 23:00:26 :INFO: HA - Publisher Node. Backup will be initiated
2020-06-07 23:00:26 :INFO: Executing [/bin/touch /common/drf/ccx_comps/uccx_db/ontape_uccx_backup.gz] ...
2020-06-07 23:00:26 :INFO: Executing [source /opt/cisco/uccx/bin/uccx_ids.env ; onstat -] ...
2020-06-07 23:00:26 :INFO: Results of executing [source /opt/cisco/uccx/bin/uccx_ids.env ; onstat -], returnCode :[1280], Output :[
IBM Informix Dynamic Server Version 12.10.UC7X3 -- On-Line -- Up 123 days 05:05:13 -- 512000 Kbytes
]
2020-06-07 23:00:26 :INFO: Creating Ontape Backup of Unified CCX DB...
2020-06-07 23:00:26 :INFO: Executing [source /opt/cisco/uccx/bin/uccx_ids.env ; ontape -v -s -L 0 -t STDIO > /common/drf/ccx_comps/uccx_db/ontape_uccx_backup.gz] ...
2020-06-07 23:00:26 :INFO: Results of executing [source /opt/cisco/uccx/bin/uccx_ids.env ; ontape -v -s -L 0 -t STDIO > /common/drf/ccx_comps/uccx_db/ontape_uccx_backup.gz], returnCode :[256], Output :[Using the backup and restore filter /bin/gzip -c.
Archive failed - buc_fe.c : Archive API processing failed at line 205 for msglen

Program over.]
2020-06-07 23:00:26 :INFO: Ontape Backup failed.
2020-06-07 23:00:26 :INFO: ----- Exit Unified CCX DB DoBackup(db_do_backup.py) with returnCode = 24 : Unsucccessful -----
2020/06/07 23:00: DO_Backup DB Script failed to work
2020/06/07 23:00: Re-setting UCCX Archive Flags
2020/06/07 23:00: Executing cmd: /opt/cisco/uccx/drf/backup/backup_resetflags.py /common/drf/backup.log
2020/06/07 23:00: Re-setting UCCX Flags
2020/06/07 23:00: UCCX Archive flags has been re-set
2020/06/07 23:00: Re-Setting of Archive flags completed


----> BEGIN Standard Output

----> END Standard Output

 

TIA

Cristian

14 Replies 14

Konstantin Vaksin
Cisco Employee
Cisco Employee
Check DB replication.

It is most common problem

Hi Cristian, I have the same issue but I am using a single node UCCX - did you manage to fix it?

 

Thanks

Hi Jackson

I have the issue still.

 

 

Try to restart services

utils service restart Cisco Unified CCX Database

utils service restart Cisco DRF Master

utils service restart Cisco DRF Local


If you have HA, please do it on both sides

Kostia

thanks a lot, it works for me

neil wooloff
Level 1
Level 1

Hello,
Did you get a fix for this issue at all?

Neil

Hello Neil

No yet.

Hi Team, were we able to fix this issue? I have a similar issue with version 10.6.1.

Thanks for your help!

Hello,
Did anyone get a fix for this at all?

my customer is still having issues.

Neil

My customer is still getting this and cisco tac not being great.

anyone know what is causing this and how to fix please?
Neil

From the platform CLI, do a "show status". You may have an issue where the free disk space is too low to allow ontape to dump a backup.

mparra.fusionet
Level 1
Level 1

Old Thread but finally found the solution:

2023-10-14 18:36:21 :INFO: Results of executing [source /opt/cisco/uccx/bin/uccx_ids.env ; ontape -v -s -L 0 -t STDIO > /common/drf/ccx_comps/uccx_db/ontape_uccx_backup.gz], returnCode :[256], Output :[Using the backup and restore filter /bin/gzip -c.
Archive failed - ISAM error: An error has occurred during archive back up.

In short, our UCCX backup schedule was running at a time when there is a process that updates Statistics in the UCCX DB, this creates a DB problem that can't be caught with normal commands (utils dbreplication runtimestate - utils uccx dbreplication status), in fact your OS replication and UCCX replication will look good, but the back up does get affected, to fix it do this on the PUB and the issue will be solved:

utils service restart Cisco Unified CCX Database
utils service restart Cisco DRF Master
utils service restart Cisco DRF Local

Long Story: Our backups runs at 2:50AM daily and it takes a long while (close to 15min), a UCCX process named Update Database Statistics runs at 3AM during the weekends, in the UCCX operations guide it says not have a backup overlap when this process is happening, in older UCCX versions this process used to happen daily at 2AM and that is why we had the backup at 2:50 to avoid it, we thought we were smart outmaneuvering the issue by choosing this time but the timing of the backup changed.

I suspect this issue could happen even if your backup is running at a different time, as long as your VM is affected around 3AM during the weekend while this Update statistics process happens, a VM issue related to (CPU-RAM-HDD) could be a trigger also.

Another indication to know that this is the issue in question is in the log:

2023-10-14 18:36:21 :INFO: Results of executing [source /opt/cisco/uccx/bin/uccx_ids.env ; ontape -v -s -L 0 -t STDIO > /common/drf/ccx_comps/uccx_db/ontape_uccx_backup.gz], returnCode :[256], Output :[Using the backup and restore filter /bin/gzip -c.
Archive failed - ISAM error: An error has occurred during archive back up.

The return code 256 from the IBM documentation, points to a DB without transaction logs:

-256	Transaction not available.

You cannot begin a transaction in this database because it does not
have a transaction log. In order to support transactions, you must
start a transaction log. Refer to your Administrator's Guide for
information on how to start a transaction log.


https://www.ibm.com/docs/en/informix-servers/12.10?topic=informix-error-messages

Below the documentation about the Update Statistics  process

mparrafusionet_0-1697324914824.png

 

https://www.cisco.com/c/en/us/td/docs/voice_ip_comm/cust_contact/contact_center/crs/express_12_5_1_su1/maintain_and_operate/guide/uccx_b_1251su1admin-and-operations-guide/uccx_b_12_5_2admin-and-operations-guide_chapter_011011.html

 

In older versions 10 it used to run daily at 2AM
https://www.cisco.com/c/en/us/td/docs/voice_ip_comm/cust_contact/contact_center/crs/express_10_6/maintain_and_operate/guide/UCCX_BK_UAC60444_00_uccx-operations-guide-106.pdf

mparrafusionet_1-1697324914896.png

 

Hope this helps all other engineers who run into this error.

You are a star - thank you very much for this

Neil

Thank you so much...this solves my problem !