Reversing service record to previous state if a phase during RFM fails

[robert.lee2]

Assuming a RFM service is developed, one issue I am observing is -

1) The service record is modified with new input first recv'd in a modification/update request to an existing service instance.

2) The service implementation is committed - It kicks of cb-create, checks some plan state X - if not = reached returns

3) An action or some other task is triggered by a kicker, and if all goes well that action will set the status of plan state X to reached

4) A kicker monitoring plan-state X triggers a re-deploy of the service instance

5) Service create now sees that plan state X is reached and continues to apply a template and fulfill its purpose

If all steps go well from 1 to 5, then all is fine.

But if during step 3, the action/task fails, then even if I do the following

3.5) Task failed so set status of plan state X to 'failed'

4.5) kicker is triggered (re-deploys service instance on enter, when status = failed or reached)

5.5) Service create has a condition that says if plan state X status == failed, then raise Exception. 

But this does not cause the original update to the service record (made in step 1) to be reverted.  

Now if the consumer tries to retry the update - Service create says - no modifications found and will not proceed.

Any thoughts on how to work around this ?  

I am using python based service packages if that makes any difference....

[rogaglia]

The way it typically works is via actions in the service that implements the logic that you need. Sometimes (depending on your design) a “reactive-redeploy” works but frequently you need to create your own recovery action.

Roque

[robert.lee2]

A recovery action is straightforward in first-time 'create' scenarios as reversal of that change is essentially the removal of the service instance record. No changes were pushed to any network devices, since the first create failed.

Its much more challenging in a 'modify' scenario where a previous 'create' already successfully pushed changes to the network and there is an active service in prod.  In this latter scenario, to 'revert' using a custom action, one would need to reverse the service instance record to its previous values (otherwise you end up blowing away what was already created in the past).  Even if one were to use diff-iterate, leaf values modified in a modification update produce 'None' as old-value and marked a VALUE_SET.  Given this I am not sure how one can 'revert' the service instance in 'update' scenarios.

I was planning to look into nano services to see if it can provide more granular control vs RFM+plan-states.  But was hoping to avoid making things more complicated.