Automatic Restart before other events in Tidal for SAP

[jeronsanders1]

Here's the situation: We have a few jobs running in our SAP environment that will cancel in SAP and sit in Tidal with 'Completed Abnormally' status. With these specific jobs, we know they are running into locking issues and restarting the job always fixes the problem. I would like to be able to set the job to automatically restart when it fails, try X number of times, then trigger our Pager event that would page our support phone if it fails X times rather than on the first 'Completed Abnormally' status. I have been unable to find a way to do this on my own, is this possible? Or am I just wasting my time?

[tomdefelice · Accepted solution]

This is ugly but it works.  If you just need it to run a few reruns in the case of locking, timeouts, etc. Create multiple copies of the same job.

My_Job

My_Job_Rerun_1

My_Job_Rerun_2

My_Job_Rerun_3

Configure them all with the same schedule, but also configure rerun_1 with a dependency on the original job completing abnormally.  Configure rerun_2 with  a dependency on rerun_1 completing abnormally, and so on.  Remove all of the pager events from all except the last rerun.

If the original job succeeds, all the reruns will just time out at midnight and never run.  If the original job fails, the reruns will fire one after another until you run out of reruns and your pager event fires.  Rerun the original job (answering "Yes" when it asks you about rerunning the dependencies) and you start the chain all over.

Any jobs you may have that use your original job completing successfully as a dependency will now have to be modified to look for:

====================

My_Job

My_Job_Rerun_1

My_Job_Rerun_2

My_Job_Rerun_3

(At least one must be met)

====================

[Olaf Schroeder]

An ounce of prevention goes a long way.  Jobs that content with each others is the reason Tidal has a feature called Resources.  For ERPs, typically you would define a resource named the SAP_Table or SAP_Process that suffers from contentions. Then you set the resource quantity equal to 1. Then for each job that is having this locking contention issue relate to this table or process you set the resource on that job.  This will insure Tidal allows no more than one job regardless of schedule to be issued against your SAP instance at a time without effecting other SAP jobs that don't suffer from contention issues. Give it a try; you'll like it.

[tomdefelice]

We do use resources.  You're right....they are great.  They helped us limit contention for a poorly written web service we had multiple jobs fighting over.

I didn't recommend it here because it really didn't address the requirements of the original poster (retries / notification of failure).  I used the method I described above for some FTP job outside the firewall.  The vendor had a sleepy site that didn't always respond the first time we connected.  I only had a single job that used it, so there was no resource contention in Tidal.

[Frank ROGERS]

Another way to accomplish the same thing, is to create an action which will run a command line command to rerun the job.  The commandline tool is tricky to figure out, as the documentation is not great.   But it does work, once it is configured correctly.

[tomdefelice]

We do that for other situations, but that simply reruns a job over and over again until it succeeds.  The requirements here were to get it to execute x number of times and then fire the email/pager event.

Doing that with your method requires creating:

a variable for a counter

a Variable Action to increment the counter

a matching Job Event to call that increment action when the job fails

another Variable Action to zero the counter

a matching System Event to call that zero action each New Production Day

a Variable Event to fire the pager action when the variable reaches the threshold limit value.

That's a lot of objects to create.  I can see going through the trouble if you plan on having this job try to run a dozen times before it notifies you, but for somethig that will make 3-4 attempts....copying your job a few times and setting up the dependencies is easier IMHO.