bgt¶

Supervised background threads for Python

POV: you want a framework-agnostic way to reliably run a plain¹ function or method in the background, repeatedly, but not all the time.

bgt comes to the rescue with:

• A service that runs your code in a loop and waits between work units. It wakes up on a fixed time interval, or as soon as something wakes it.

• A supervisor that runs that loop in a background thread. If your code crashes, the supervisor restarts the loop after an exponential backoff. Write crash-only code, bgt takes care of the rest.

• Thorough instrumentation via structlog and Prometheus.

• Framework and platform independence.

bgt is the engine underneath pgbg, which adds PostgreSQL LISTEN / NOTIFY-driven wakeups and leader election with automatic failover on top. If your services should wake up on database events, or only one process at a time should do the work, that’s where to look.

Background services are not a worker queue. Common use cases include:

• Periodic cleanup duties for expired caches or sessions.
• Refreshing in-memory caches or configuration.
• Flushing buffered metrics or events.

Here’s a service that runs a work unit every two seconds and survives its own crashes:

import bgt


def do_work() -> bool:
    ...  # one bounded work unit

    return False  # nothing left to do: wait for the next wakeup


with bgt.SupervisedService.start(
    bgt.as_work_factory(do_work),
    name="example",
    wakeup=bgt.IntervalOnlyWakeup(),
    interval=2,
):
    ...  # do_work runs in the background until we leave this block

Return True from your work unit to be run again immediately. This keeps work units short, which makes shutdowns prompt.

Credits¶

bgt is written by Hynek Schlawack and distributed under the terms of the MIT license.

The development is kindly supported by my employer Variomedia AG and all my fabulous GitHub Sponsors.