Comparison

Runqy vs RunPod Serverless

RunPod Serverless gives you a managed queue in front of GPU workers that start when jobs arrive and stop when they are idle. Runqy gives you the same pattern, self-hosted, with workers that can run anywhere, including on RunPod.

They are not mutually exclusive: Runqy's autoscaler can rent RunPod pods for a queue, next to the GPUs you already own. Runqy is our project; this page says when RunPod Serverless is the better choice.

What you get out of the box

Features available without writing glue code or adding another service.

Out of the box	RunPod Serverless	Runqy
Where workers run	RunPod GPUs only	Your machines, RunPod pods, Vast.ai, any server
Who runs the control plane	RunPod (managed)	You (self-hosted server + Redis)
Job queue	✓ managed, per endpoint	✓ Redis, sub-queues per queue
Submit jobs	✓ /run, /runsync, /status	✓ /queue/add, batch, CLI, Python client
Ship new code	Build and push a Docker image, or build from GitHub	Push to Git, then code-update: no image rebuild
Scale to zero	✓ idle timeout	✓ idle trigger, min_workers: 0
Scale up on demand	✓ queue delay or request count	✓ queue depth, latency, no workers
Scheduled pre-warming	~ active workers, always on	✓ cron schedules with a duration
Keeps a model loaded between jobs	✓ while the worker is warm	✓ @load, while the worker lives
Cold-start reduction	✓ FlashBoot, cached models	~ machines you keep, schedules
Execution timeout and job TTL	✓ per endpoint or request	✓ per queue; timeout also per request
Mix with machines you own	✗	✓
Billing	Per running worker, including cold starts and idle wait	Free software; you pay your machines and rentals

✓ built in · ~ partial or limited · ✗ not built in (you add a separate tool or write it yourself)

The same handler, both ways

RunPod Serverless

import runpod

pipe = load_pipeline()  # once per worker

def handler(event):
    prompt = event["input"]["prompt"]
    return save(pipe(prompt))

runpod.serverless.start({"handler": handler})
# Build a Docker image, push it,
# create an endpoint.

Runqy

from runqy_python import load, task, run

@load
def setup():
    return {"pipe": load_pipeline()}

@task
def generate(payload, ctx):
    return save(ctx["pipe"](payload["prompt"]))

run()
# Push to Git. Workers clone it;
# runqy control code-update rolls it out.

Choose RunPod Serverless when

You do not want to run any server: no Redis, no control plane, no updates.
Traffic is bursty and cold starts matter: FlashBoot and cached models help.
All your GPUs can live on RunPod.

Choose Runqy when

You already own GPUs and want rented ones only for peaks.
You want to compare providers: RunPod pods and Vast.ai from the same queue config.
You ship code often and do not want to rebuild an image each time.
Jobs of the same model need priorities, for example paid users before free ones.

Try Runqy

Open source and self-hosted. Start with your own machine, add rented GPUs when you need them.

Quickstart GPU autoscaling