cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
909
Views
0
Helpful
1
Replies
Beginner

TES: dependency on dynamically inserted job

I have following requirements:

  1. JobB depends on JobA
  2. JobC depends on JobA and JobB (if in schedule)
  3. Always execute JobA
  4. If exit code of JobA=1 execute JobB
  5. Always execute JobC when JobA and JobB (if in schedule) are done

During Tidal conversion we called this requirement “branching”. It was recommended by Tidal Professional services to accomplish branching via events/actions that would react to the exit code (event) and insert JobB (corresponding job action). The missing piece now is how to maintain proper dependencies downstream from the “branched” job, i.e. how to make sure JobC does not execute until JobB completes.

The way we implemented this is by adding an event to JobA that is triggered by exit code 1 and - if/when triggered - called action for JobB to be inserted.

JobB itself does not have a schedule and can only be inserted by a job action

JobC depends on both JobA and JobB (JobB dependency marked as "ignore if not in schedule").

The issue is that sometimes dependency on JobB is ignored even though it has been inserted and hence JobC starts prematurely.

I have been told that this will not work as the schedule is compiled ahead of time and the ad hoc job (JobB which does not have a calendar) will not be in schedule and hence will always be ignored.

Indeed in activity view for the future dates the dependency is not visible.

However, for many such sequences the activity view for the past dates will show the dependency just fine - but not always.

This to me is a kind of undeterministic behavior.

My thinking is that perhaps Tidal may not have enough time to consider the dependency before the JobC is triggered (as if consequences of JobB insertion were processed in parallel - on a parallel thread - to the main thread that inserts JobC). But I'm hesitant to be introducing arbitrary timers or additional variables and variable events/dependencies.

Any ideas appreciated.

Thanks

Dan

1 REPLY 1
Highlighted
Beginner

TES: dependency on dynamically inserted job

Its working for us now, case closed. As soon as I changed dependencies from match group to last occurrence in the "problem" jobs, they all started recognizing these dependencies (on dynamically inserted jobs) just fine.

I am sure variable based solution would have worked as well - just was reluctant to refactor and introduce more events/variables where it already seems like it should be working.

CreatePlease to create content