Massed Compute

Load balancing distributes incoming HTTP requests across multiple backend servers to improve availability and performance. This guide walks through setting up Nginx as a reverse proxy load balancer with three algorithm options: round-robin (default), least-connections, and IP-hash sticky sessions.

Nginx’s open-source version provides passive health checks that automatically remove failing backends from rotation. For active health monitoring, you’ll need Nginx Plus or a custom health check script.

Technology Stack

Requirements

Massed Compute VM Pricing

The cpu_mini_amd_epyc handles moderate traffic loads well. Scale up to cpu_small_amd_epyc or larger for high-throughput scenarios with thousands of concurrent connections.

Step-by-Step Deployment

curl -X POST "https://vm.massedcompute.com/api/v1/instances" \
  -H "Authorization: Bearer MC_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "product": "cpu_mini_amd_epyc",
    "image": "ubuntu-22.04",
    "sshKeys": ["YOUR_KEY_NAME"]
  }'

ssh ubuntu@YOUR_VM_IP

sudo apt-get update
sudo apt-get install -y nginx

for U in backend1.example.com:8080 backend2.example.com:8080; do
    echo "=== $U ==="
    curl -sS -o /dev/null -w "%{http_code}\n" --connect-timeout 3 "http://$U/"
done

sudo tee /etc/nginx/sites-available/lb > /dev/null << 'EOF'
upstream backends {
    # Choose ONE algorithm (or omit for round-robin):
    # least_conn;
    # ip_hash;

    server backend1.example.com:8080 max_fails=3 fail_timeout=30s;
    server backend2.example.com:8080 max_fails=3 fail_timeout=30s;
    server backend3.example.com:8080 max_fails=3 fail_timeout=30s;
}

server {
    listen 80 default_server;
    listen [::]:80 default_server;

    location / {
        proxy_pass http://backends;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        proxy_next_upstream error timeout http_502 http_503 http_504;
    }
}
EOF

sudo ln -sf /etc/nginx/sites-available/lb /etc/nginx/sites-enabled/lb
sudo rm -f /etc/nginx/sites-enabled/default

sudo nginx -t
sudo systemctl reload nginx

for i in $(seq 1 10); do
    curl -sS -o /dev/null -w "%{http_code}\n" http://127.0.0.1/
done

Performance Tuning by VM Size

For higher-capacity VMs, adjust Nginx worker settings to fully utilize available resources:

Apply these settings by editing /etc/nginx/nginx.conf and updating the events block and worker_rlimit_nofile directive.

Troubleshooting

All Requests Return 502 Bad Gateway

This indicates all upstream servers are unreachable. Re-run the connectivity test from step 4. Check security groups, firewalls, and verify upstream service ports are correct.

Inter-VM Connectivity Issues

Many cloud providers firewall traffic between customer VMs at the hypervisor level. SSH (port 22) typically works, but custom ports may be blocked. Test with:

timeout 2 bash -c "
If blocked, use VPC networking, SSH tunneling, or co-locate services on the same VM.

Uneven Traffic Distribution
With round-robin, slower upstream servers can cause uneven distribution as Nginx advances based on completed requests. This is normal behavior—consider switching to least_conn algorithm.

IP Hash Stickiness Testing
IP hash is working correctly if all requests from your test IP go to the same backend. Test from multiple source IPs to see proper distribution across backends.

Health Check Timing
After fixing an upstream issue, Nginx may still mark it as down during the fail_timeout period (30 seconds by default). Wait or reload Nginx to reset immediately.

Skip All of This: Deploy with an AI Agent

This entire load balancer setup exists as a tested, machine-readable recipe in the Massed Compute MCP. Instead of running commands manually, connect an AI agent to deploy automatically.

Add this server to your MCP client configuration:

{
  "mcpServers": {
    "massed-compute": {
      "type": "http",
      "url": "https://vm.massedcompute.com/api/mcp",
      "headers": { "Authorization": "Bearer MC_TOKEN" }
    }
  }
}

Then say:


"Deploy an Nginx HTTP load balancer with round-robin distribution across three upstream services: api1.internal:8080, api2.internal:8080, and api3.internal:8080. Use passive health checks with 3 max failures and 30-second timeout."


The agent matches your request against the recipe catalog, provisions the right VM shape, installs and configures Nginx with your specific upstreams and algorithm choice, tests connectivity to each backend, validates the load balancer is working, and reports back with the results. If any step fails, it stops and provides detailed error information.

This recipe was tested on June 2, 2026.


  
Ready to Deploy Your Load Balancer?
  
Launch your Nginx load balancer on high-performance AMD EPYC instances. Get started in under 2 minutes with our streamlined deployment process.
  
Think it. Build it. Scale it.
  

    Launch VM Instance
    View Pricing
  


Quick Setup Guide


  
Deploy Ubuntu 22.04 VM with 2+ vCPU and 2GB+ RAM
  
Install Nginx: sudo apt-get update && sudo apt-get install -y nginx
  
Test upstream connectivity with curl
  
Create load balancer config at /etc/nginx/sites-available/lb
  
Choose algorithm: round-robin (default), least_conn, or ip_hash
  
Enable config: sudo ln -sf /etc/nginx/sites-available/lb /etc/nginx/sites-enabled/lb
  
Remove default site: sudo rm -f /etc/nginx/sites-enabled/default
  
Validate and reload: sudo nginx -t && sudo systemctl reload nginx
  
Test load balancing with multiple requests
  
Monitor logs and adjust performance settings as needed


Frequently Asked Questions


  
01What's the difference between the three load balancing algorithms?
  
Round-robin (default) sends each request to the next server in rotation—best for equal-capacity servers with stateless requests. Least-connections routes to the server with fewest active connections—ideal when response times vary significantly. IP-hash ensures requests from the same client IP always go to the same backend—use when backends have session state and you can't implement shared session storage.



  
02How do passive health checks work in Nginx?
  
The max_fails=3 fail_timeout=30s parameters create passive health monitoring. If a backend returns 3 connection errors or timeouts within 30 seconds, Nginx marks it unhealthy and excludes it from rotation for the next 30 seconds. This only triggers on actual request failures—there's no proactive health checking. For active monitoring (periodic GET requests to /health endpoints), you need Nginx Plus or a custom script.



  
03Can I weight backends differently for unequal server capacities?
  
Yes, add weight=N to any server line: server backend1.example.com:8080 weight=3; makes that backend receive 3x as many requests as a weight=1 server. This works with all algorithms and is useful when backends have different CPU, memory, or processing capabilities.



  
04Why do I get connection refused errors between VMs on the same network?
  
Most cloud providers implement security policies that firewall traffic between customer VMs at the hypervisor level, even within the same subnet. SSH (port 22) is typically allowed, but custom application ports are blocked by default. Solutions include using VPC/private networking features, SSH port forwarding, or deploying services on the same VM where localhost traffic always works.



  
05How do I add HTTPS termination to the load balancer?
  
Configure SSL certificates on the Nginx load balancer and modify the server block to listen 443 ssl with your certificate paths. The backends can remain HTTP since communication between the load balancer and upstreams typically happens over a private network. This setup terminates SSL at the load balancer and forwards plain HTTP internally, reducing CPU load on backend servers.