Docs / Spot instances

Spot instances

Spot GPUs are cheaper and the provider can reclaim them at any time. A spot deployment on LLM Hangar recovers on its own: the reclaim is confirmed against the provider, the interval closes, and a fresh instance boots from your staged weights under the same endpoint. Spot requires deploy-time staging for that reason.

Choosing spot

At deploy, pick the spot tier and a policy. The ceiling you consent to still applies: a spot attempt is bounded by the shape's on-demand rate, never under-counted, and a fallback to on-demand needs a max hourly rate that covers it.

Policy	When spot has no capacity	Tradeoff
`fallback_on_demand` (default)	Recover on an on-demand instance of a consented shape, at the on-demand rate	Restores service after a new boot; costs more until spot returns
`wait_for_spot`	Retry spot every 2 minutes, then every 15, indefinitely	Avoids on-demand rates; the endpoint may remain unavailable

curl -X POST https://app.llmhangar.com/v1/deployments \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{ "name": "assistant", "model_id": "gemma-4-31b", "cloud_account_id": "...",
        "staging": true, "tier": "spot", "spot_policy": "fallback_on_demand",
        "reacquire": "next_boundary", "max_hourly_usd": 4.5 }'

Eviction, per provider

An eviction is confirmed, not inferred: the deployment's watchdog asks the provider every minute while the deployment is ready and declares an eviction only after two consecutive listings, ten seconds apart, show the compute gone. A listing error is never treated as an eviction.

AWS. Spot instances are launched one-time with terminate-on-interruption. A reclaim terminates the instance, which the listing by deployment tag reports.
RunPod. Interruptible pods disappear from the listing by name, or exit.
Nebius. Preemptible instances are stopped in place with their disks attached; a stopped instance under a ready deployment counts as evicted and the recovery creates a fresh one.
Verda. Spot instances are removed with the volumes kept detached; the recovery reattaches the stage.

The confirmed eviction closes the billing interval at detection time, releases the boot reservation, moves the deployment to recovering and sends deployment.evicted.

Recovery

Recovery runs under a fresh generation and walks your consented shapes again: every shape as spot first, then, under fallback_on_demand, every shape on demand. Success sends deployment.recovered and records the boot as a recovery in the timings the deployment page shows. Two exits are different on purpose:

A stop or destroy while recovering wins: the deployment lands in plain stopped (or is destroyed) and the recovery ends.
A recovery that exhausts its policy lands in stopped with a system hold, so the schedule cannot launch an ordinary resume around it. Retry recovery on the deployment page, or POST /deployments/{id}/recovery/retry, clears the hold and tries again.

Reacquiring spot

A spot deployment recovered on an on-demand fallback shows current_market: on_demand. With reacquire: next_boundary it goes back to spot at the next scheduled stop and start, because every resume walks spot first. With asap, or the Reacquire spot action, it stops the on-demand instance and resumes on spot right away: a brief interruption you asked for, with one active compute instance at a time.

What clients should do

Treat a connection error or a 5xx as retryable and retry with exponential backoff. On an edge placement the hostname answers a stable 503 with "status": "waking" and a Retry-After header during recovery; on the default placement the hostname has nothing to answer until the new instance is up. Requests that were in flight at the reclaim are lost; there is no queuing or replay.

Recovery times

The deployment page shows observed recovery times once it has measured them: medians of your own recoveries, dated. They depend on the provider having a GPU free and on your stage's warm boot, and they are never a service level. This site publishes recovery numbers only as dated measurements from real deployments; none are published yet.