Skip to content
Private AI

How to Deploy Ollama on a ₹800/Month VPS and Run LLMs for Free

Step-by-step guide for Indian developers: server selection, Ollama install, model benchmarks, and exposing a private OpenAI-compatible API.

By AI & Technology teamPublished Last updated 5 min read
How to Deploy Ollama on a ₹800/Month VPS and Run LLMs for Free

Step-by-step guide for Indian developers: server selection, Ollama install, model benchmarks, and exposing a private OpenAI-compatible API.

Running large language models used to require expensive GPU servers. Not anymore.

Ollama has changed the game for developers and businesses who want private, self-hosted AI. You can run Mistral 7B, LLaMA 3 8B, Gemma 2, and Phi-3 on a standard CPU-only VPS for ₹800–2,000 per month — with unlimited queries and zero per-token costs.

This guide covers everything: choosing the right VPS, installing Ollama, selecting the right model for your use case, and securing the API for production use.

Why Self-Host an LLM?

Three reasons dominate in India:

  1. Cost — No per-token pricing. Run 1 million queries per month for the same price as 10 queries on GPT-4.
  2. Privacy — Data stays on your server. Relevant for BFSI, healthcare, legal, and anyone under India's DPDP Act 2023.
  3. Reliability — No rate limits, no OpenAI outages, no API key suspensions.

The trade-off: open-source models at 7B parameters perform at roughly GPT-3.5 level. For document Q&A, customer support, classification, summarization, and form processing — this is entirely sufficient.

Choosing a VPS in India

For CPU-only inference (Mistral 7B), you need:

  • Minimum: 16GB RAM, 4 vCPUs
  • Recommended: 32GB RAM, 8 vCPUs
  • Storage: 20GB+ (model files are 4–8GB each)

Best VPS Options for Indian Developers

Hetzner Cloud (Most popular for self-hosted AI)

  • CX32: 8GB RAM, 4 vCPUs — €5.37/month (~₹490). Too small for Mistral 7B.
  • CX42: 16GB RAM, 8 vCPUs — €13.67/month (~₹1,250). Minimum viable.
  • CX52: 32GB RAM, 16 vCPUs — €25.72/month (~₹2,350). Recommended.
  • Servers in Falkenstein, Nuremberg, Helsinki (EU). Latency from India: 100–150ms.

DigitalOcean

  • 16GB/8vCPU Droplet: $96/month (~₹8,000). Expensive.
  • But: they have a Bangalore datacenter → ~10ms latency for India. Worth it for user-facing apps.

AWS EC2 (ap-south-1, Mumbai)

  • t3.xlarge: 16GB RAM — ~₹5,500/month. Best for compliance-sensitive apps that must stay in India.
  • Spot instances: 70% cheaper if you can handle interruptions.

Contabo (Best price-to-performance)

  • VPS M SSD: 16GB RAM, 6 vCPUs — €7.99/month (~₹730). Best value for budget-conscious teams.
  • Servers in Germany. Good for non-latency-sensitive back-office automation.

Our recommendation for most Indian teams: Contabo VPS M for development/internal tools, DigitalOcean Bangalore for customer-facing applications.

Installing Ollama

SSH into your VPS and run:

curl -fsSL https://ollama.ai/install.sh | sh
ollama --version
sudo systemctl start ollama
sudo systemctl enable ollama  # Start on boot
sudo systemctl status ollama

Ollama runs on port 11434 by default. It's now ready to pull models.

Choosing the Right Model

Here are the models we actually use in production, with honest assessments:

Mistral 7B Instruct (Our Default)

ollama pull mistral:7b-instruct-q4_K_M

Best for: Customer support chatbots, document Q&A, email drafting, data extraction, classification.

Benchmark on CX42 (16GB/8vCPU): ~5–8 tokens/second Quality vs GPT-3.5: Comparable on structured tasks. Slightly behind on open-ended generation.

LLaMA 3 8B Instruct (Better Reasoning)

ollama pull llama3:8b-instruct-q4_K_M

Best for: Workflows requiring stronger reasoning. Better at following complex multi-step instructions. Slightly better than Mistral on Hindi/regional language tasks.

Benchmark on CX42: ~4–6 tokens/second

Phi-3 Mini (Fast and Tiny)

ollama pull phi3:mini

Best for: Simple classification, yes/no decisions, keyword extraction, sentiment analysis. Extremely fast (~15–20 tokens/sec on CPU).

Gemma 2 9B (Best Overall Quality)

ollama pull gemma2:9b-instruct-q4_K_M

Best for: When you need the highest quality from a sub-10B model. Best instruction following we've tested at this size. Slower than Mistral.

nomic-embed-text (For RAG Embeddings)

ollama pull nomic-embed-text

Purpose: Generating embeddings for vector search (ChromaDB, Qdrant). This is NOT a chat model — it's used for RAG pipelines to convert text to vectors.

Running Your First Query

ollama run mistral:7b-instruct-q4_K_M
>>> Send a message (/? for help)
>>> Write a professional email declining a vendor proposal
[Mistral responds with a well-formatted email]
curl http://localhost:11434/api/generate -d '{
  "model": "mistral:7b-instruct-q4_K_M",
  "prompt": "Summarize this in 2 sentences: [your text]",
  "stream": false
}'

OpenAI-Compatible API Mode

Ollama exposes an OpenAI-compatible API endpoint. Any code written for OpenAI can use Ollama with minimal changes:


from openai import OpenAI
client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="not-needed"  # Required parameter but ignored by Ollama
)
response = client.chat.completions.create(
    model="mistral:7b-instruct-q4_K_M",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What are the key terms in this contract?"}
    ],
    max_tokens=500,
    temperature=0.3
)
print(response.choices[0].message.content)

This means you can build your application targeting the OpenAI API and simply change base_url to switch between OpenAI (cloud) and Ollama (local) — useful for development vs production.

Securing Ollama for Production

By default, Ollama only listens on localhost. To expose it to your application (running on the same server or another server), you need to configure it properly.

Option 1 — Application on the same server (simplest)

Your application and Ollama are on the same VPS. No network exposure needed. Use http://localhost:11434 directly.

Option 2 — Expose with Nginx and API key authentication

If your application is on a separate server, expose Ollama through Nginx with basic authentication:

server {
    listen 443 ssl;
    server_name ollama.yourdomain.com;
    ssl_certificate /etc/letsencrypt/live/yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/yourdomain.com/privkey.pem;
    location / {
        # Basic auth
        auth_basic "Ollama API";
        auth_basic_user_file /etc/nginx/.htpasswd;
        proxy_pass http://localhost:11434;
        proxy_set_header Host $host;
        # Allow streaming responses
        proxy_buffering off;
        proxy_read_timeout 300s;
    }
}

Generate a password file:

sudo apt install apache2-utils
sudo htpasswd -c /etc/nginx/.htpasswd youruser

Option 3 — Tailscale VPN (Our Preferred Approach)

For internal business tools, we use Tailscale — a zero-config VPN that lets servers communicate securely without exposing anything to the internet:

curl -fsSL https://tailscale.com/install.sh | sh
sudo tailscale up
curl -fsSL https://tailscale.com/install.sh | sh
sudo tailscale up

Tailscale is free for up to 3 devices (personal) and has affordable team plans.

Performance Benchmarks (Real Numbers from Our Tests)

Tested on Contabo VPS M (16GB RAM, 6 vCPUs, ~₹730/month):

Model Tokens/sec 200-token response time RAM Used
Phi-3 Mini (Q4) 18–22 ~10 sec 3.2GB
Mistral 7B (Q4_K_M) 5–8 ~28 sec 6.8GB
LLaMA 3 8B (Q4_K_M) 4–6 ~35 sec 7.4GB
Gemma 2 9B (Q4_K_M) 3–5 ~45 sec 9.1GB

For a customer-facing chatbot where response time matters, 28–45 seconds is too slow. Options:

  1. Use Phi-3 Mini (fast, less capable)
  2. Upgrade to a GPU-enabled VPS (10–20x faster)
  3. Use Ollama for back-office automation (where 30-second processing is fine) and an API for user-facing chat

Cost Comparison at Scale

Assuming 50,000 queries/month, average 500 tokens per response:

Option Monthly Cost
OpenAI GPT-4o ~₹1,80,000
OpenAI GPT-4o-mini ~₹12,000
Anthropic Claude Haiku ~₹6,000
Ollama (Contabo VPS) ₹730 fixed
Ollama (DigitalOcean Bangalore) ₹6,000 fixed

For back-office automation and internal tools at volume, Ollama on a cheap VPS is the clear winner.

What We Use This For

At Monk Media One Tech, we deploy Ollama-based systems for:

  • Private knowledge base chatbots — like the Cosmo Guru build (500+ daily queries)
  • Document processing pipelines — invoice extraction, contract review, report summarization
  • N8N AI nodes — LLM processing steps inside automation workflows, without per-execution API costs
  • Classification tasks — lead scoring, sentiment analysis, category tagging

For any workflow where data privacy matters or where volume makes cloud API costs prohibitive, Ollama is our default.

We're Monk Media One Tech — AI automation agency, Ahmedabad, India. We build private LLM systems, RAG chatbots, and automation pipelines on self-hosted infrastructure.

Book a free discovery call: monkmediaone.tech/contact 📞 +91 88668 19349 | hello@monkmediaone.tech

Related

Keep reading

AI you own. Systems that keep running.

We design, deploy and run private AI on your data and your systems.

example activity

Invoice INV-0412

Vendor GST and line items extracted

WhatsApp

GSTR-3B sent to a verified client

Reddit thread

Lead qualified and sent to the CRM

Shopify

24 product descriptions drafted for review

Inbound call

Answered in Gujarati, demo booked

Trend report

Eight emerging topics delivered

CRM

Follow-up sent, deal moved to Proposal

Line 3 sensor

Reading drifted, alert raised