LLM Hangar / Guides / Deploy an LLM on AWS
Deploy an LLM on AWS in your own account, no command line
Most guides for running an open model on AWS start with Terraform, an AMI and a Kubernetes cluster. For a single private endpoint that is the wrong shape. One EC2 GPU instance running an inference server, created in your own account, is enough: one instance, one line on the bill, nothing to uninstall afterwards. This guide covers which instance fits which model, what it costs by region, the two AWS-specific blockers that stop first deployments, and the four steps to a key-authenticated, OpenAI-compatible endpoint with LLM Hangar.
When one EC2 instance is the right answer
If you want a private endpoint for one model, used by your own applications, a single instance running vLLM or SGLang is the simplest thing that works. The model weights sit on the instance disk, the endpoint is served from the instance, and your prompts never leave your account. The alternatives earn their complexity in specific cases:
- EKS when you need to autoscale several models behind one gateway and already run Kubernetes.
- SageMaker when the model is one step in an existing MLOps pipeline with training jobs and registries.
- Bedrock when you do not want to run a model at all and its catalog has what you need. You do not choose the instance, and prompts are processed by the managed service rather than on a machine you own.
For everything else, the rest of this page is the simple path.
Which instance fits which model
The table below lists the shapes that matter for open models, what fits on them, and the prices published by two independent price trackers. The "what fits" column is derived from published weight sizes, not measured: a bf16 parameter is 2 bytes, a 4-bit parameter is about half a byte, and you need headroom for the KV cache.
| Instance | GPU and memory | What fits (derived) | On-demand / spot, per hour |
|---|---|---|---|
| g6e.xlarge | 1x NVIDIA L40S, 48 GB | An 8B model at bf16 (about 16 GB), or a 27B dense model at 4-bit (about 17 GB), with room for context | Stockholm (eu-north-1) $1.974 / $0.604 Aragon (eu-south-2) $1.961 / $0.722 Frankfurt (eu-central-1) $2.327 / $1.747 us-east-1 $1.861 / $1.725 |
| g6e.2xlarge | 1x NVIDIA L40S, 48 GB, more CPU and RAM | Same GPU as g6e.xlarge; the extra CPU and memory help with loading and batching | us-east-1 $2.242 on-demand |
| Two L40S-class GPUs, or one 80 GB GPU | 96 GB or 80 GB | A 70B model at 4-bit fits on one 48 GB GPU only with a small context; a second GPU, or an 80 GB H100-class GPU, is the comfortable shape | Shape and region dependent; shown in the LLM Hangar wizard before you confirm |
| p5en.48xlarge | 8x NVIDIA H200, 141 GB each | The largest open models. DeepSeek V4 Flash (284B MoE, 159.6 GB of fp8 weights) runs on 2x H200 in LLM Hangar's measured deployment; this shape is overkill for it | us-east-1 $63.30 on-demand (July 2026) |
Sources: g6e.xlarge EU prices from Spare Cores, fetched 2026-08-22; us-east-1 g6e.xlarge, g6e.2xlarge and p5en.48xlarge prices from Vantage. Spot prices move hourly and spot capacity can be reclaimed; treat them as the lower bound, not a plan. Two things stand out: Stockholm is the cheapest EU region for an L40S, and the Frankfurt spot price is almost three times Stockholm's.
How long a boot takes
We publish boot times only when we have measured them on a real deployment, and we have one so far: DeepSeek V4 Flash on 2x H200, launched through LLM Hangar into our own account on RunPod's European region, took 21 minutes 26 seconds from a fresh instance to a ready endpoint on 2026-08-11, and about 21 minutes on a repeat run a week later. Most of that is downloading 159.6 GB of weights. The full write-up is in the DeepSeek V4 Flash guide.
AWS g6e boot timings for smaller models will be published on this page when we have measured them. Nothing here is extrapolated from a spec sheet, which is also why the table above does not contain a tokens-per-second column.
The two blockers that stop first deployments on AWS
Your GPU quota is zero
A new AWS account has a default quota of 0 vCPUs for "Running On-Demand G and VT instances" and for "Running On-Demand P instances", and the same for their Spot equivalents (AWS EC2 instance quotas). The first GPU launch fails with a quota error until you request an increase through Service Quotas. Small requests are usually approved quickly; large ones take longer. If the first launch fails on quota, LLM Hangar explains the cause in plain language, removes whatever was created, and gives a direct link to the request form.
No capacity in the region
GPU instances are not always available in every availability zone.
Launches in eu-central-1 in particular can fail with
InsufficientInstanceCapacity for g6e shapes; there are
AWS
re:Post threads asking exactly that. The practical answer is to try
a neighbouring EU region or a different shape, and to know before you
commit. The live price table on the LLM Hangar homepage shows whether
each shape had capacity at the last market check, and a failed launch
is swept and reported, not left half-created.
Stop is not terminate
A stopped EC2 instance stops billing for compute but its EBS volume keeps billing, and a stopped instance is easy to forget. A forgotten running g6e.2xlarge at $2.242 per hour is about $1,600 a month. LLM Hangar deployments carry a hard budget cap and, optionally, a self-destruct timer; when either fires, or when you delete the deployment, the teardown is verified against AWS until every resource is confirmed gone. See budget caps and auto-destroy.
The four steps
- Connect your AWS account. A CloudFormation quick-create link deploys one IAM role that trusts the platform account, bound with a single-use external ID. No access keys. Delete the stack and the access is gone.
- Pick a model, a shape and a region. The wizard shows the shapes that fit the model with an hourly and monthly estimate for each. Tick EU-only if your data must stay in EU member-state regions.
- Set a budget cap and confirm. The confirmation page lists every resource that will be created. The plan is written to the audit log before anything exists.
- Copy the endpoint URL and key into your code. The endpoint is OpenAI-compatible, so the Python, JavaScript, LangChain and editor configurations you already have work by changing the base URL.
A screen recording of a trial deploy, start to first token, will be linked here once it is recorded.
Doing it yourself instead
The same result is reachable by hand. AWS publishes a Terraform sample for vLLM on EC2 that creates the instance, security group and inference server. You then own the quota request, the AMI and driver updates, the idle shutdown, the teardown check and the key handling. None of it is hard; all of it is yours to maintain. LLM Hangar is the version of that work that runs in your account through a role you can revoke, for a flat $39 a month on the Lab plan.
Frequently asked questions
SageMaker or EC2?
For a single private endpoint serving one open model, one EC2 GPU instance running vLLM is simpler and cheaper to reason about: one instance, one bill line, nothing to uninstall. SageMaker and EKS earn their complexity when you need autoscaling across many models or an existing MLOps pipeline. Bedrock is a managed API: you do not run the model and you choose from its catalog.
What is the cheapest EC2 instance that runs an open LLM?
For models up to roughly 27B parameters at 4-bit, or 8B at bf16, g6e.xlarge (one NVIDIA L40S with 48 GB) is the practical floor. Spare Cores listed it at $1.974 per hour on-demand and $0.604 spot in Stockholm (eu-north-1) on 2026-08-22; Vantage lists $1.861 on-demand in us-east-1. Smaller g5 and g4dn instances exist but their 24 GB and 16 GB GPUs rule out most current models.
Why is my AWS GPU quota zero?
New AWS accounts start with a default quota of 0 vCPUs for Running On-Demand G and VT instances and for P instances, so the first GPU launch fails until you request an increase through Service Quotas. If the launch fails on quota, LLM Hangar explains the cause and links directly to the request form.
Do I need Kubernetes or Terraform?
No. One instance with an inference server does not need an orchestrator. With LLM Hangar the instance, its disk, network and endpoint are created in your account through a role you can revoke; the DIY route with the aws-samples Terraform repository works too, but you maintain it.
Which models fit on one L40S (g6e)?
Derived from published weight sizes, not measured here: an 8B model at bf16 (about 16 GB) or a 27B dense model at 4-bit (about 17 GB) fits in 48 GB with room for context. A 70B model needs 4-bit quantization on 48 GB or a second GPU. DeepSeek V4 Flash, a 284B mixture of experts with 159.6 GB of fp8 weights, needs two H200s in LLM Hangar's measured deployment.