Skip to main content

Handle step errors

Every step has an on_error setting that decides what happens when it fails. Without one, a failed step fails the run, and the run waits in the exceptions queue for someone to retry or resolve it.

ModeShapeWhat happens when the step fails
Fail{ "mode": "fail" }The run fails and waits as an exception. This is the default
Skip{ "mode": "skip" }The step is recorded as skipped and the run continues
Retry{ "mode": "retry", "max_attempts": 3, "backoff": "30s" }The step is tried again, up to 10 attempts in total, waiting backoff (doubling each time) between tries. After the last attempt the run fails
Handler{ "mode": "handler", "steps": [...], "then": "continue" }A lane of handler steps runs, then the run continues or fails

Only failures that can succeed on another try are retried: timeouts, rate limits, 5xx responses, and outages. A step that fails for good (a 4xx response, invalid input, a record that does not exist) skips the remaining attempts.

Error handler lanes​

A handler lane is a list of steps that runs when its step fails for good, for example to alert someone, record the problem in another system, or fall back to a different approach.

{
"key": "push_order",
"type": "http.request",
"config": {
"url": "https://erp.example.com/orders",
"body": "{{ steps.fetch_order.output }}"
},
"on_error": {
"mode": "handler",
"retry": { "max_attempts": 3, "backoff": "1m" },
"then": "continue",
"steps": [
{
"key": "alert_ops",
"type": "http.request",
"config": {
"url": "https://hooks.example.com/ops",
"body": {
"text": "ERP push failed for {{ steps.fetch_order.output.order_number }}: {{ error.message }}"
}
}
}
]
}
}
FieldTypeDescription
mode"handler"Required
stepsarrayRequired. 1 to 20 steps that run in order
then"continue" or "fail"What the run does after every handler step succeeded. fail is the default
retryobjectOptional { "max_attempts", "backoff" }, applied to the step before the lane runs

What happens:

  1. The step fails for good, after its retry attempts if it has any.
  2. The handler steps run in order. Handler steps can read every earlier step and the error root: error.step (the failed step's key), error.type (its step type), and error.message, in references and in when guards (for example { "field": "error.message", "operator": "contains", "value": "429" }). A handler step that is skipped by its when guard or its own on_error skip counts as handled.
  3. When every handler step succeeded, then decides: continue moves on to the next step as if the failure had been handled, and the run can finish succeeded; fail fails the run with the step's original error, as it would have without a handler.
  4. When a handler step fails, the run fails with the original error of the step that started the lane, not the handler's, so the exception points at the real problem. The handler failure is recorded in the run's events. The one exception is a money step in the lane (such as a fallback label purchase): when it fails, the run fails with that step's error and the exception is opened against it at high priority, so the money failure is what triage sees.

The failed step keeps its failed attempts in the run either way, and a later step that references its output finds nothing there.

Rules​

  • Handler steps cannot have handler lanes of their own, so lanes are one level deep.
  • branch, wait.event, and respond cannot run in a handler lane.
  • Step keys are unique across the whole flow, handler lanes included.
  • Money steps (orderly.label.purchase and orderly.pickup.schedule) can only fail or retry; they cannot skip or have a handler lane. Inside a handler lane they must fail on error.
  • A handler lane runs at most once per failure. When you retry a run from the exceptions queue and the step fails again, handler steps that already succeeded in that run are not repeated.

Run events​

Handler lanes add three events to the run, each recorded against the step that failed:

EventPayload
step.handler_startederror: the step's error
step.handler_completedthen: continue or fail
step.handler_failedhandler_step_key and its error