Key Takeaways
- High Fixed Baseline: Self-hosting vLLM costs around ₹9 lakh/month for GPUs, engineering salaries, and infrastructure.
- Massive Crossover: Serverless is cheaper until you cross 12 to 25 billion tokens/month with 24/7 utilization.
- Save Engineering Time: Early-stage teams avoid managing CUDA drivers, OOM kills, and 2 AM paging alerts.
- Massive Savings at Scale: At 1B tokens/month, serverless remains roughly 26x cheaper than a self-hosted baseline.
- Hybrid Flexibility: Use serverless for spiky traffic and reserve dedicated clusters only for massive enterprise workloads.
If you're an ML or platform engineer in India trying to decide between serverless inference and self-hosted vLLM, you've probably hit a familiar wall: vendor pricing pages give you a neat per-token cost, but nobody tells you what a GPU box actually costs once you add infrastructure engineers, Kubernetes orchestration, real-time monitoring, and 2 AM paging alerts.
This guide breaks down the real, fully-loaded cost of both architectures, using precise rupee math across three distinct volume tiers, so you can identify your true financial crossover point instead of guessing.
Quick Answer: Which One Should You Pick?
- Choose Serverless Inference if: Your monthly traffic is below 1 to 2 billion tokens/month. For early-stage startups, MVPs, and scale-ups, serverless is significantly cheaper, faster to ship, and far lower risk.
- Choose Self-Hosted vLLM if: You operate at a massive scale (12 to 25+ billion tokens/month) with high, predictable 24/7 utilization, or have strict enterprise data-residency and compliance mandates that demand absolute hardware control.
What "Serverless Inference" and "Self-Hosted vLLM" Actually Mean
Before diving into the numbers, let's establish a clear technical foundation.
What is vLLM?
vLLM is an open-source, high-throughput inference and serving engine designed for Large Language Models (LLMs). Originally developed at UC Berkeley's Sky Computing Lab, it includes advanced performance features:
- Optimized Kernels: Native integration with FlashAttention and FlashInfer.
- Advanced Serving: Multiple decoding algorithms (beam search, parallel sampling) and tensor/pipeline parallelism for distributed inference.
- Hardware Compatibility: Supports NVIDIA GPUs, AMD CPUs/GPUs, Intel CPUs, Google TPUs, AWS Trainium/Inferentia, and IBM Power architectures.
- Quantization Support: Comprehensive formats including FP8, NVFP4, INT8/INT4, GPTQ/AWQ, GGUF, and TorchAO.
The Catch: "Open source and free" only applies to the software license. Running it reliably in production incurs heavy infrastructure and engineering costs.
What is Serverless Inference?
Serverless inference is a managed, pay-per-token model where a cloud or MLOps provider runs the GPU cluster, autoscaling engine, and on-call rotations. Platforms like Simplismart let teams deploy pre-configured open-source models via pay-as-you-go APIs, or manage custom models with complete resource control while outsourcing infrastructure overhead.
The Full Cost Stack of Self-Hosting vLLM in 2026
Evaluating self-hosted vLLM requires looking far beyond the raw rental price of a single graphics card.
The Self-Hosted Monthly Cost Stack
- 2x H100 GPUs (HA Setup): At ₹350/hour (a representative on-demand rate; discounted and subsidized options like IndiaAI Mission compute run as low as ₹92–₹217/hour) running 24/7, a single H100 costs ₹2,55,500/month. Production redundancy requires at least two GPUs (₹5,11,000/month) to handle maintenance, rolling updates, and failovers.
- Engineering Salaries: Managing OOM kills, KV cache allocations, and batch sizing requires specialized expertise. Factoring in two mid-level ML/platform engineers at a conservative ₹1.5L/month each brings this to ₹3,00,000/month.
- Kubernetes & Orchestration: Managed control plane fees, persistent model storage, node overhead, and network egress total roughly ₹50,000/month.
- Monitoring & Observability: Prometheus/Grafana or SaaS tracking for TTFT, latency, and GPU health runs about ₹22,000/month.
- On-Call Allowances: Direct allowances and burnout management add ₹15,000/month.
Baseline Financial Summary
Hidden Costs That Blow Up Budgets
Beyond fixed overheads, production teams frequently face insidious hidden expenses:
- Version Upgrades & CUDA Breaks: Frequent vLLM updates mean managing dependencies between PyTorch, CUDA toolkits, and GPU drivers. Upgrades require staging, regression testing, and accuracy checks, costing 2 to 4 engineering days per quarter.
- Cold Restart Downtime: Reloading tens of gigabytes of model weights into GPU memory after a pod crash creates multi-minute performance bottlenecks if redundancy isn't properly configured.
- Over-Provisioning: Without multi-tenant pooling, teams must either over-pay for idle GPUs to handle traffic spikes or under-provision and drop user requests
Serverless Inference Cost: The Per-Token Math
Serverless pricing scales linearly based on usage, eliminating fixed infrastructure minimums. Using a representative market rate of ₹35 per million tokens for an 8B parameter class open-source model (actual rates vary by provider and model, typically ₹15–₹40 per million tokens):
Volume Tier Breakdown
- Tier 1 (10 Million Tokens/Month - MVP Stage):
- Serverless Cost: 10M tokens × ₹35/M = ₹350/month
- Comparison: Self-hosting would waste ₹9L/month for a microscopic workload (a ~2,566x multiple).
- Tier 2 (100 Million Tokens/Month - Growing Product):
- Serverless Cost: 100M tokens × ₹35/M = ₹3,500/month
- Tier 3 (1 Billion Tokens/Month - Scale-Up):
- Serverless Cost: 1,000M tokens × ₹35/M = ₹35,000/month (vs. ₹8.98L self-hosted baseline).
Comprehensive Cost Comparison Table
The Crossover Point: When Self-Hosting Wins
Solving for the exact point where serverless expenditures match the fixed self-hosted baseline:
Crossover Volume = ₹8,98,000 ÷ ₹35 per million tokens ≈ 25.6 billion tokens/month
Sustaining 25.6 billion tokens/month requires handling roughly 9,900 tokens every single second around the clock, a volume reserved for heavy consumer-scale platforms. Even with a stripped-down, single-engineer, zero-redundancy self-hosted setup (₹4.45L/month), the crossover point remains near 12.7 billion tokens/month.
Decision Matrix for Indian AI Teams
Bottom Line
For the vast majority of Indian engineering teams processing under 1 to 2 billion tokens per month, serverless inference wins decisively on both financial outlay and operational sanity. Always model your specific engineering salaries, cloud rates, and actual token consumption profiles before committing capital to a complex internal Kubernetes infrastructure stack.
Conclusion: Making Your Decision
The choice between serverless inference and self-hosted vLLM ultimately comes down to a realistic assessment of scale, predictability, and engineering bandwidth.
While self-hosting offers deep infrastructure control, it introduces a heavy fixed financial baseline, exceeding ₹9 lakh per month once engineering salaries, high-availability redundancy, and operational overhead are factored in. For any team processing under 1 to 2 billion tokens monthly, locking into this fixed cost creates unnecessary financial drag and distracts engineering talent from core product development.
Key takeaways to guide your final choice:
- Protect Your Velocity: Early-stage and scale-up teams should prioritize shipping product over maintaining CUDA drivers, resolving memory leaks, and managing 2 AM PagerDuty alerts. Serverless inference absorbs these operational headaches while matching variable growth.
- Respect the Crossover Threshold: Do not transition to self-hosting prematurely. The economic scales only tip in favor of a dedicated internal vLLM cluster once you consistently push past 12 to 25+ billion tokens per month with high 24/7 GPU utilization.
- Consider a Hybrid Architecture: Mature organizations can leverage the best of both worlds by routing spiky, unpredictable, or lower-volume workloads through serverless endpoints while reserving dedicated infrastructure strictly for massive, predictable enterprise volumes or strict compliance requirements.
Before committing capital to a complex Kubernetes and GPU infrastructure stack, audit your actual token usage, factor in fully-loaded engineering hours, and match your architecture to your current stage of growth.
Frequently Asked Questions (FAQ)
Is vLLM actually free to use?
While the software license is open-source and free, running it reliably in production incurs high, ongoing costs for GPU hardware, engineering salaries, Kubernetes orchestration, monitoring, and on-call support, frequently exceeding ₹9 lakh/month.
What GPU utilization do I need for self-hosting to make sense?
Self-hosting becomes economically viable only when your GPUs maintain a consistent, high utilization rate of 60% to 80%+ around the clock. In contrast, spiky or unpredictable traffic is best absorbed by serverless multi-tenant pooling.
Does serverless inference support custom or fine-tuned models?
Yes, most managed serverless platforms support deploying custom or domain-specific fine-tuned models alongside pre-configured open-source options, though custom deployments may involve tailored resource allocation.
How often does vLLM require version upgrades?
Because vLLM is a rapidly evolving open-source project with frequent updates, self-hosted teams must regularly schedule engineering time for regression testing, validation, and managing breaking CUDA driver dependencies.
Can I switch from serverless to self-hosted later without much friction?
Yes, as long as your application integrates with standard OpenAI-compatible API endpoints, transitioning your underlying infrastructure later is primarily a deployment shift rather than a major code rewrite.
What is the real financial crossover point for Indian teams?
Factoring in fully-loaded engineering salaries and high-availability redundancy, the crossover point where self-hosting beats serverless ranges between 12.7 billion and 25.6 billion tokens per month.
How do hidden costs impact the self-hosted budget?
Hidden variables like cold restart downtime, over-provisioning for traffic spikes, and quarterly CUDA compatibility upgrades routinely inflate a basic "GPU-only" self-hosted budget by an extra 20% to 40%.
Ready to Skip the Infrastructure Overhead?
Stop spending engineering hours wrestling with CUDA drivers, Kubernetes pods, and 2 AM paging alerts. Whether you are validating an MVP or scaling an enterprise AI workload, Simplismart offers fully managed, high-performance serverless infrastructure and custom deployment options tailored for Indian teams.
- Instant Scale: Deploy pre-configured open-source models instantly or bring your own fine-tuned weights with zero cluster management required.
- Production-Grade Reliability: Built-in observability, low latency, and enterprise-grade uptime SLAs.
- Predictable Economics: Pay strictly for what you use without locking into heavy, fixed monthly GPU minimums.
Explore Simplismart Inference & Get Started Today
