Feature

Deploy from Git, update without SSH

A runqy worker needs three things to start: the server URL, an API key and a queue name. Everything else comes from the server: where your code lives, how to start it, which secrets it needs.

No Docker image to rebuild for each change, no SSH session on each GPU machine, no CI pipeline to keep in sync with your workers.

What a worker does at startup

  1. 1. Register

    It calls the server and receives its queue configuration, the Redis connection and its secrets.

  2. 2. Clone

    It clones the queue's repository and branch. Private repositories use a token stored in a vault.

  3. 3. Install

    It creates a virtualenv and installs requirements.txt or pyproject.toml.

  4. 4. Start

    It runs your startup command, waits for the ready signal, then takes tasks.

The server holds the config, the worker holds nothing

Queue config, on the server

queues:
  inference:
    deployment:
      git_url: "https://github.com/you/models.git"
      branch: "main"
      code_path: "sdxl"
      startup_cmd: "python main.py"
      git_token: "vault://github/models_pat"
      vaults: ["huggingface"]

Worker, on any machine

docker run --gpus all \
  -e RUNQY_SERVER_URL=https://runqy.example.com \
  -e RUNQY_API_KEY=worker-key \
  -e RUNQY_QUEUES=inference \
  ghcr.io/publikey/runqy-worker:inference

The same command on your desktop GPU, a lab server or a rented machine. The autoscaler runs it for you on RunPod and Vast.ai.

Remote control of running workers

Commands go from the server to the workers over Redis, from the CLI, the API or the dashboard. Target one worker, every worker of a queue, or all of them.

drain / resume

Stop taking new tasks while the running ones finish, then take tasks again. Use it before stopping a machine.

code-update

Move to the latest commit of the queue branch, reinstall dependencies, restart the task process. Disk caches, such as downloaded models, are kept.

worker-update

Install another runqy-worker release in the running container, verified against the release checksums, and restart into it with the same worker id.

config-reload

Ask the task process to reload its configuration through its @reload handler, with no restart: models stay loaded.

git push origin main
runqy control code-update --target inference
runqy worker info 3fc7e7ee1e37:1
# Last cmd: code_update done: updated inference main (40s ago)

A code update waits for the tasks in flight, and other queues on the same worker keep running meanwhile.

When your process crashes

The worker supervises your Python process. If it dies, the tasks it was running fail, the worker reports itself unhealthy and restarts the process with an exponential backoff. If it keeps crashing, a circuit breaker stops the restarts and marks the worker degraded, which can trigger an email alert.

Good to know

  • Commands are fire-and-forget: a worker offline at that moment misses them. The outcome shows in runqy worker info.
  • Remote control needs runqy server and worker v0.4.0 or later; config-reload needs runqy-python 0.3.0.
  • A code update restarts the task process, so models load again. Use config-reload for configuration only.
  • worker-update is not available on Windows workers.

Read the guides

Worker deployment Worker control