Azure Durable Functions

Azure durable functions help to define stateful workflow in code for stateless functions.

Durable Functions in a nutshell:

• An extension to Azure Functions
• Stateful functions in a serverless environment
• Define workflows in code

Key Benefits of Durable Functions

Some of the key benefits of Durable Functions are

• Parallel execution of functions (Fan-out > Fan-in)
• Central Error handling
• Easy to understand the dependencies among functions via “Orchestrator Function”

Overview

Durable Function overview

Starter Function: Simple Azure Function that starts the Orchestration by calling the Orchestrator function. It uses an OrchestrationClient binding.

OrchestrationClient is responsible for starting/stopping orchestrators and monitoring their status.

Orchestrator Function: Defines a stateful workflow in code and invokes the activity functions. Sleeps during activity invocation and replays when wakes up. Which also provides separation of concerns for defining the workflow and performing the actual activity.

The code in an orchestrator function MUST be deterministic because during the flow the code will be executed again and again till all activity functions finish. You declare a function as an orchestrator by using a OrchestrationTrigger

Orchestration Function limitations:

• Be Deterministic: No NewGuid(), Random, DateTime.Now, Http calls, etc.
• Be non-blocking: No I/O operations (e-g: table storage logic, etc.) , no Thread.sleep etc..
• Never initiate any async operations without using its context

Activity Functions: Simple Azure Functions that performs a single step in a workflow and can receive or return data. An Activity function uses an ActivityTrigger so that can be invoked by the orchestrator

Sub Orchestration:

Durable Functions also support Sub Orchestration. This feature enables us to create sub orchestrators that orchestrate several activities. An orchestrator function can call another orchestrator function using the CallSubOrchestratorAsync or the CallSubOrchestratorWithRetryAsync methods.

It could be handy for:

• Complex orchestration
• In cases where a retry mechanism is needed.
• In cases where a delay is needed in the activity function. Delay in the activity function will be considered as part of function execution time, thus it is billed, whereas timmer delay in orchestrator will not be billed as the orchestrator will go to sleep till timer is triggered.

Sub Orchestrator

Patterns

Durable Functions internal mechanism

Durable functions make use of storage queues. The message in the queue triggers the next function. State of orchestrations is saved in the storage table. It makes use of event sourcing to play the current events and to trigger the next event.

Durable Functions Storage Queues & Tables

Event source events:

During workflow execution, events are stored for the following activities:

• Orchestrator Started
• Activity 1 Scheduled
• Orchestrator Goes to Sleep (No explicit event is registered for it)
• Orchestrator Started (Wakes up) > Activity 1 Completed
• Activity 2 Scheduled
• Orchestrator Completed

Storage tables:

• DurableFunctionsHubInstances: Contains records for each orchestrator instance with its input, final output, and the RuntimeStatus
• DurableFunctionsHubHistory: Contains event source events for each orchestrator instance.

Durable Functions internal mechanism

Note: The tables & queues names mentioned above are default names for durable function; if storage is shared among different Function apps then it is advisable to specify custom hub name in host.json.

Function Version 2.0 host.json:
"extensions": {
     "durableTask": {
            "hubName": "PeppolSupportedDocumentUpdaterHub"
      }
}

Monitoring progress

Starter / OrchestrationClient with HttpTrigger can return a response with “CreateCheckStatusResponse” which contains the “statusQueryGetUri” that can be used to monitor the progress of the work flow.

statusQueryGetUri:
http://{BaseUri}/runtime/webhooks/durabletask/instances/{InstanceId}?taskHub=DurableFunctionsHub&connection=Storage&code={FunctionKey}

If there are sub tasks involved, then the progress / output of sub tasks can be monitored by adding following additional query parameters to “statusQueryGetUri”:

  &showHistoryOutput=true&showHistory=true

Delay & Retry Mechanism

context.CreateTimer, can be used to add delay between individual task execution in the chain.

var activityTask = context.CallActivityAsync<ActivityLog>(nameof(FunctionName), inputValueToFunction);
//add 3-sec delay between execution of tasks
var timeToStart = await context.CurrentUtcDateTime.AddSeconds(3);
var delayedActivityTask =  context.CreateTimer(timeToStart, CancellationToken.None).ContinueWith(t => activityTask);
tasks.Add(delayedActivityTask);
await Task.WhenAll(tasks);
var result = tasks.Select(task => task.Result).ToList();
return result;

Activity function can be retried in Orchestrator, and can also specify the delay between each retry. Optionally can also specify the exception for which to retry the activity.

Retry Activity Function for “InvalidOperationException”

Additional Resources:

Durable Functions Overview - Azure

Azure Best Practices for Durable Functions Patterns | Serverless360

Sub-orchestrations for Durable Functions - Azure