Feature

Priority sub-queues

One model, several lines of customers. Sub-queues let paid users go before free ones, or interactive requests before batch jobs, with the same task code and the same loaded model.

A worker listening on inference serves all of its sub-queues from one Python process. Priorities decide which one it takes from next.

Declare the sub-queues

queues:
  inference:
    sub_queues:
      - name: premium
        priority: 10
      - name: standard
        priority: 3
    deployment:
      git_url: "https://github.com/you/models.git"
      branch: "main"
      startup_cmd: "python main.py"
      startup_timeout_secs: 300
# Your app picks the line per request
POST /queue/add
{"queue": "inference.premium", "data": {...}}

POST /queue/add
{"queue": "inference.standard", "data": {...}}

Workers started with RUNQY_QUEUES=inference serve both.

Two ways to apply them

Weighted

default

For every task, the order is drawn again: a sub-queue comes first with a probability proportional to its priority. With 10 and 3, when both have work waiting, premium is served about 77% of the time and standard 23%.

Nobody starves. An empty sub-queue is skipped, so a lone backlog runs at full speed.

Strict

strict_priority: true

Sub-queues are always polled from the highest priority down: a lower sub-queue is served only when every higher one is empty.

For background work that must never delay the others, at the risk of waiting a long time under constant load.

Timeouts, retries and retention per queue

A 2-second embedding job and a 20-minute video job need different rules. Set them on the queue; sub-queues inherit them.

queues:
  video:
    limits:
      active_timeout: "30m"
      pending_timeout: "10m"
      max_retry: 1
      ttl_completed: "1h"
      ttl_archived: "24h"
    deployment: ...
  • active_timeout Maximum run time. Past it, the task fails and is retried; a long-running Python process is restarted so the stuck work stops.
  • pending_timeout Maximum wait in the queue. A task picked up too late is archived without running, so nobody gets a stale result.
  • max_retry How many times a failed task is retried before it is archived.
  • ttl_completed / ttl_archived How long finished and failed tasks are kept, so Redis does not fill up.

A request can also set its own execution timeout. Changes apply to new tasks; tasks already queued keep their values.

Good to know

  • Weighted priorities are ratios: 6 and 4 give 60/40, not "6 first".
  • Priorities order the next task a worker takes. They do not pause a running task to start a more urgent one.
  • Weighted and strict ordering need runqy server and worker v0.4.0 or later.
  • Priorities compare the sub-queues of one parent queue. Between different parent queues on the same worker, the order is always weighted.

Read the guides

How priorities are applied Task lifecycle