How to Give Millions of Workflows Quality Sleep

Ask ChatGPTAsk Claude

Often, durable workflows need to wait for something to happen in the real world: for a reviewer to approve an action, or for a customer to submit a form. These workflows can wait for a long time, sometimes days, weeks, or even months. As a result, it’s not uncommon for even a small application to have thousands of workflows waiting at the same time, and for a large application to have millions.

To make it easier to manage large numbers of long-waiting workflows, we recently released workflow suspension. This allows a workflow that’s sleeping or waiting for a notification to “suspend” itself until a deadline is reached or the notification arrives. A suspended workflow exists only as rows in your database; it doesn’t consume any resources like threads, memory, or queue slots. In this blog post, we’ll explain how workflow suspension works.

How DBOS Suspension Works

Using workflow suspension is easy: you pass suspend=True to DBOS.recv (which waits with a timeout for a notification) or DBOS.sleep (which sleeps for some length of time). For example, if you need to suspend a workflow until it receives human approval (waiting up to a timeout, for example one week), you call DBOS.recv on the “approval” topic with a timeout of one week and suspend=True:

Python code - an approval workflow calls DBOS.recv("approval", timeout_seconds=approval_timeout, suspend=True) to suspend while waiting for a decision

Under the hood, workflow suspension works by leveraging workflow durability. When a workflow is suspended, it first updates its own database record: it sets its status to “SUSPENDED”, its “delay deadline” to its scheduled wakeup time (the time at which its timeout expires) and its “suspend key” to the topic on which it is waiting for a notification, if any. Then, the workflow exits execution by throwing a special “suspension exception.” After exiting, the workflow exists only as database entries; it does not occupy a thread or consume memory.

Diagram - a suspending workflow updates its own row from PENDING to SUSPENDED, setting its delay deadline to now + 7 days and its suspend key to approval, then exits and frees its thread

The most challenging part of implementing workflow suspend is figuring out when to resume suspended workflows. If there are millions of suspended workflows in the database at once, we can’t simply poll all of them to see which ones are ready to resume execution. Instead, we need to build two different mechanisms for efficiently identifying workflows that are ready to resume: one for workflows whose waits have timed out, and another for workflows whose notifications have arrived.

To find workflows whose waits have timed out, we use a special database index. Recall that when a workflow suspends itself, it sets its “delay deadline” in the database to its wakeup time. We build a database index on that column, which essentially sorts all workflows from those that will wake up the soonest to those that will wake up the latest. A background thread periodically polls that index, finding the next workflows to wake up and waking them up. After waking up a workflow, it clears that workflow’s delay deadline, keeping the index small so that queries for the next workflows to wake up are always fast.

Diagram - a background poller reads only the front of the delay deadline index, re-enqueues workflows whose deadline has passed and clears their deadline, while millions of later workflows are never scanned until due

To find workflows woken up by a notification, we include some special logic in DBOS.send, the call that sends a notification. When sending a notification to a workflow, we also check if the workflow is suspended waiting for that notification: if its suspend_key in the database matches the topic of the notification. If it does, we set the workflow’s delay deadline to 0 (to avoid a race with this, workflows transactionally check if a notification has arrived before suspending themselves). This causes the background thread described previously to immediately wake the workflow up.

Diagram - DBOS.send writes the notification and, when the suspended workflow's suspend key matches the topic, sets its delay deadline to 0 so it moves to the front of the index

When a workflow is ready to wake up, we use its own durability to resume it by updating its status from SUSPENDED to ENQUEUED, re-enqueueing it for execution. The workflow then recovers from checkpoints and continues execution from where it was suspended, the same way it would recover from a crash, restart, or any other failure. This is the power of durable execution: a workflow’s state is completely described by its database records, so workflows can freely move between executing in memory and suspended in the database.

Diagram - on its first run the workflow runs prepare and send_for_review, then suspends in recv; after wake-up on any worker, completed steps are restored from checkpoints, recv gets the message, and execute and notify run

Learn More

If you like building scalable, reliable systems, we’d love to hear from you. At DBOS, our goal is to make Postgres-backed durable execution as simple and performant as possible. Check it out:

Insights

Recent articles

The latest in durable execution, AI workflows & more.

Postgres
Updated Nov 12, 2026

Postgres is All You Need for Durable Workflows

Peter Kraft
Product news
Oct 29, 2026

What's New in DBOS - October 2026

Workflow rewind, safer distributed execution, DBOS Transact Enterprise, and scaling to 1M+ workflows per second on sharded Postgres!
Qian Li
Postgres
Oct 28, 2026

1M Durable Workflows per Second on Postgres

How we sharded Postgres-backed durable workflows to scale linearly
Peter Kraft