Durable execution

Viewflow keeps all workflow state in the database and runs background steps through Celery. It targets a single Postgres + single Redis, scaled vertically — not partition tolerance. The settings below make that setup crash-safe.

Database

Use PostgreSQL. The process row is the coordination point (see Locking), and select ... for update needs a real transactional database. Run the workflow tables on a durable, non-replica connection.

DATABASES = {"default": env.db()}  # postgres://…

Locking

The lock serialises concurrent activations of the same process. The default is no_lock (a plain transaction) — not safe for flows with Join / Subprocess nodes, where two branches can finish at once. Set a real lock per flow:

from viewflow.workflow import lock

class MyFlow(flow.Flow):
    lock_impl = lock.select_for_update_lock   # Postgres, recommended
    # lock_impl = lock.CacheLock()            # only with memcached/Redis cache
    ...

select_for_update_lock is the safe default for Postgres. CacheLock is correct only on a cache with an atomic cross-process add (memcached / Redis); LocMemCache is per-process and unsafe. Avoid no_lock for any flow that joins branches.

Broker

Use a durable broker so scheduled and in-flight jobs survive a restart:

CELERY_BROKER_URL = "redis://localhost:6379/0"
CELERY_BROKER_TRANSPORT_OPTIONS = {"visibility_timeout": 3600}  # > longest task
CELERY_TASK_ACKS_LATE = True                 # redeliver if a worker dies
CELERY_TASK_REJECT_ON_WORKER_LOST = True

Enable Redis persistence (AOF), or use RabbitMQ, if you rely on celery.Timer / long Delay — an in-memory broker loses pending messages on restart. For long waits, prefer the database-backed flow.Timer: its due moment lives on the task row and survives any broker failure. Fire due timers with the workflow_timers management command from cron, or schedule workflow_fire_timers with celery beat.

Crash recovery

Worker dies mid-run. With acks_late (above) the broker redelivers the un-acked message and the task re-enters and finishes. This is at least once: the body may have partially run before the crash, so make Job side effects idempotent — guard on an external id:

@shared_task
def charge(activation_ref):
    with celery.Job.activate(activation_ref) as activation:
        if activation.process.charged:      # already done on a prior delivery
            return
        gateway.charge(activation.process.amount)
        activation.process.charged = True
        activation.process.save()

A task that failed or is partially applied is recovered by a human, not silently re-run: an ERROR task exposes undo → revive (both require the manage permission) so an operator can roll back and retry deliberately. Viewflow does not auto-reschedule running tasks — that would double-apply partial work.

Lost publish (rare). Scheduling publishes the Celery message after the database commits (connection.on_commit), so a crash in the small window between the commit and the publish can leave a task SCHEDULED with no message. It is uncommon on a single healthy host; if it matters for your workload, front the publish with a transactional outbox.