What's the difference between Spot GPUs and reserved GPUs?

Spot GPUs and reserved GPUs both reduce infrastructure costs compared to on-demand pricing, but they solve different problems. Spot GPUs offer the lowest prices in exchange for interruption risk, while reserved GPUs provide guaranteed capacity in return for a long-term commitment.

This guide explains the differences between spot and reserved GPUs, their tradeoffs, and how to choose the right pricing model for your AI workloads.

Although both pricing models reduce infrastructure costs compared to on-demand pricing, they solve different problems.

Spot GPUs prioritise lower costs by using spare cloud capacity that providers can reclaim when demand increases. Reserved GPUs prioritise guaranteed capacity and predictable pricing in exchange for a long-term commitment.

Spot GPUs are GPU instances sold using unused cloud capacity. Rather than leaving GPUs idle, cloud providers offer them at discounted rates until that capacity is needed elsewhere.

The tradeoff is that spot instances can be reclaimed with little warning. AWS Spot Instances, for example, typically provide a two-minute interruption notice before termination. Other providers have similar behaviour.

This means applications running on spot GPUs must be designed to recover from interruptions automatically. Training jobs should checkpoint frequently, long-running processes should save intermediate state, and orchestration systems should be able to restart workloads from the latest checkpoint.

Without this engineering, a single interruption can erase hours of compute.

• Distributed training with checkpointing
• Hyperparameter tuning
• Dataset preprocessing
• Offline batch inference
• Cost-sensitive research workloads

• Production inference
• Interactive AI applications
• Deadline-driven training jobs
• Workloads that cannot recover automatically

Reserved GPUs allow you to commit to GPU capacity for a fixed period, usually one to three years. In return, providers offer lower hourly rates and guarantee capacity is available when you need it. Unlike spot GPUs, reserved capacity cannot be reclaimed by the provider during the reservation period.

Reserved capacity works well for organisations running workloads continuously. When GPU utilisation remains consistently high, reserved pricing can significantly reduce long-term infrastructure costs.

The tradeoff is flexibility. You're committing to pay for capacity whether it's fully utilised or not.

If workload requirements change midway through a reservation, or utilisation drops unexpectedly, reserved GPUs may become more expensive overall than using on-demand capacity.

• Production inference platforms
• Continuous model training
• Stable enterprise AI workloads
• Predictable GPU demand

• Early-stage startups
• Research environments
• Seasonal workloads
• Teams with unpredictable GPU requirements

The right choice depends less on the price per hour and more on how predictable your workloads are.

• Choose spot GPUs if reducing infrastructure costs is your highest priority and your workloads can tolerate interruptions. Batch processing, distributed training, hyperparameter tuning, and offline inference are all good candidates because they can recover automatically when interrupted.
• Choose reserved GPUs if your workloads run continuously and require guaranteed capacity. Production inference services, recurring training pipelines, and enterprise AI platforms often benefit from predictable pricing and guaranteed availability.

If you're still discovering your workload patterns, starting with on-demand GPUs is often the safest option. Once utilisation becomes predictable, you can introduce reserved capacity while continuing to use spot GPUs for fault-tolerant workloads. In practice, many organisations reserve GPU capacity for predictable production workloads while using spot GPUs wherever interruption is acceptable.

Running GPU workloads in production is about more than accessing GPU compute. Most AI applications also need inference servers, databases, object storage, deployment pipelines, autoscaling, observability, secrets management, and networking to support training and inference.

Northflank brings all of these together in a single platform. Deploy GPU workloads across 18+ GPU types, including H100, H200, A100, L4, L40S, B200, and GB300, alongside your applications, managed databases, AI sandboxes, and inference services with no reserved commitments, long-term contracts, or vendor lock-in.

GPU workloads run in the same control plane as standard CPU services, microVM sandboxes, managed databases, and application deployments. You get the same RBAC, SSO, secrets management, audit logging, and BYOC across your entire stack without managing separate GPU infrastructure or billing relationships.

Key features of GPU workloads on Northflank:

• Auto-scaling from zero: Inference endpoints scale from zero to hundreds of GPUs based on traffic. Intelligent queuing prevents request drops during traffic spikes.
• GPU time-slicing: Run multiple workloads on single GPUs with NVIDIA time-slicing. Share expensive H100s across development, testing, and small inference workloads. Improve utilisation by 2–4x.
• Integrated storage: Co-locate GPU compute with managed databases, S3-compatible object storage, and persistent volumes for low-latency data access during training.
• Observability built in: Real-time GPU utilisation, memory usage, and temperature monitoring. Track inference latency, throughput, and cost per workload. Integrated logging for training runs.
• Same DevEx across every cloud: Deploy on AWS, GCP, Azure, Oracle, CoreWeave, or any GPU cloud via BYOC. Same UI, CLI, and API regardless of provider.

For enterprises that need GPU capacity inside their own cloud account, Northflank BYOC deploys into your existing AWS, GCP, Azure, Oracle, or CoreWeave account. GPU workloads run inside your own VPC with no markup on underlying compute. For the most isolated environments, Northflank also supports a forward-deployed control plane for defence technology companies, healthcare institutions, and financial services firms.

Spot GPUs usually offer the lowest hourly pricing but can be interrupted at any time. Reserved GPUs cost more per hour than spot but provide guaranteed availability and predictable pricing.

Generally no. Production inference requires consistent availability, making reserved or on-demand GPUs a better choice. Spot GPUs are better suited for batch workloads that can tolerate interruptions.

Usually not. Startups often have unpredictable infrastructure requirements and changing workload patterns. On-demand GPUs provide more flexibility until demand becomes consistent enough to justify reservations.

You continue paying for it. Reserved GPU contracts typically charge for the agreed commitment period regardless of how much of the capacity is actually used.

Yes. Many organisations reserve capacity for predictable production workloads while using spot GPUs for fault-tolerant batch processing and experimental workloads.

No. Spot GPUs usually have the lowest hourly price, but they can be interrupted at any time. Reserved GPUs cost more per hour than spot, but provide guaranteed availability and predictable pricing. The better choice depends on whether your workload can tolerate interruptions.

Spot and reserved GPUs solve different problems.

Spot GPUs minimise infrastructure costs but require workloads that can tolerate interruptions and recover automatically. Reserved GPUs provide guaranteed capacity and predictable pricing, making them the better choice for continuously running production workloads.

For most organisations, the decision isn't choosing one or the other. It's deciding which workloads benefit from each. Reserve GPU capacity where demand is predictable, and use spot GPUs wherever interruption is acceptable and cost savings matter most.

• How much does an NVIDIA L4 GPU cost?: Compare L4 specifications, pricing models, and Northflank deployment options.
• Top GPU sandboxes for AI agents: Compare isolated GPU execution platforms for agent workloads that need direct accelerator access.
• GPU workloads on Northflank: Review supported GPUs and deployment paths for inference, training, and compute-intensive workloads.
• How much does an NVIDIA A100 GPU cost? Compare cloud pricing for the 40 GB and 80 GB A100 variants.
• How much does an NVIDIA H100 GPU cost? Review H100 pricing for higher-throughput inference and training.
• How much does an NVIDIA B200 GPU cost? Understand pricing for NVIDIA Blackwell infrastructure.