Massed Compute

Launch GPU VM

# Launch via Massed Compute dashboard or API
# Product: gpu_1x_pro_6000_blackwell
# Image: 184 (Ubuntu Server 24.04 w/ Drivers)
# SSH Key: attach your public key
# Instance name: hermes-agent-local-llm

Wait until the instance is running and has an IP.

Verify the GPU

ssh -o IdentitiesOnly=yes Ubuntu@YOUR_VM_IP \
  'nvidia-smi --query-gpu=name,memory.total,driver_version --format=csv,noheader'

This run printed NVIDIA RTX PRO 6000 Blackwell Server Edition, 97887 MiB, 580.126.16.

Install Ollama on loopback and pull Llama 3.1 70B

The GPU workload is Ollama. Bind it to 127.0.0.1 so the model API is not on the public interface. Hermes talks to OpenAI-compat /v1 on that same host.

ssh -o IdentitiesOnly=yes Ubuntu@YOUR_VM_IP 'bash -s' <<'EOF'
set -euo pipefail
curl -fsSL https://ollama.com/install.sh | sh
sudo mkdir -p /etc/systemd/system/ollama.service.d
sudo tee /etc/systemd/system/ollama.service.d/override.conf >/dev/null <<'CONF'
[Service]
Environment="OLLAMA_HOST=127.0.0.1:11434"
Environment="OLLAMA_MAX_LOADED_MODELS=1"
Environment="OLLAMA_NUM_CTX=65536"
CONF
sudo systemctl daemon-reload
sudo systemctl enable --now ollama
sudo systemctl restart ollama
ss -ltn | grep 11434
curl -sS http://127.0.0.1:11434/api/tags
ollama --version
EOF

You should see 127.0.0.1:11434 and ollama version is 0.33.0 (or the version you actually installed that day).

Pull a tool-capable 70B. Fallback if the pull OOMs: llama3.1:8b (not run on this VM).

ssh -o IdentitiesOnly=yes Ubuntu@YOUR_VM_IP 'bash -s' <<'EOF'
set -euo pipefail
ollama pull llama3.1:70b
ollama show llama3.1:70b
EOF

This pull was 42.5 GB. ollama show reported architecture llama, 70.6B, Q4_K_M, context length 131072, capabilities completion and tools. Ollama-only setup, without Hermes, is already covered in Deploy LLMs with Ollama on GPU Cloud.

The systemd drop-in set OLLAMA_NUM_CTX=65536. This load still reported 131072 in ollama ps (the model max). Size VRAM for the context you actually get, not only the weights.

Install Hermes Agent

ssh -o IdentitiesOnly=yes Ubuntu@YOUR_VM_IP 'bash -s' <<'EOF'
set -euo pipefail
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
# shellcheck disable=SC1091
source ~/.bashrc
export PATH="$HOME/.local/bin:$PATH"
hermes --version
EOF

This VM installed Hermes Agent v0.20.5 (2026.8.19) from git b3a2065f, Python 3.11.16, OpenAI SDK 2.24.0. Binary: ~/.local/bin/hermes (symlink to the installer venv). Pin the version you print from hermes --version. The official installer’s Node/browser step can fail and still leave a working CLI.

Create Kanban team profiles and start the gateway

Four profiles: orchestrator, researcher, writer, reviewer. Each gets the same local /v1 endpoint. Persistence is ~/.hermes (kanban.db, profiles/, config.yaml).

ssh -o IdentitiesOnly=yes Ubuntu@YOUR_VM_IP 'bash -s' <<'EOF'
set -euo pipefail
export PATH="$HOME/.local/bin:$PATH"
OLLAMA_ENV=$'OPENAI_BASE_URL=http://127.0.0.1:11434/v1\nOPENAI_API_KEY=ollama'
for profile in orchestrator researcher writer reviewer; do
  hermes profile create "$profile" 2>/dev/null || true
  mkdir -p "$HOME/.hermes/profiles/${profile}"
  printf '%s\n' "$OLLAMA_ENV" >"$HOME/.hermes/profiles/${profile}/.env"
  hermes -p "$profile" config set model.provider custom
  hermes -p "$profile" config set model.base_url "http://127.0.0.1:11434/v1"
  hermes -p "$profile" config set model.name llama3.1:70b
done
hermes profile describe orchestrator --text "Orchestrator: decomposes goals into Kanban tasks and routes work to researcher, writer, and reviewer."
hermes profile describe researcher --text "Researcher: gathers facts and posts findings to Kanban tasks."
hermes profile describe writer --text "Writer: drafts prose from research cards."
hermes profile describe reviewer --text "Reviewer: checks outputs for completeness."
hermes profile use orchestrator
hermes kanban init
EOF

Enable the gateway as a systemd unit so the Kanban dispatcher survives logout. This run had no messaging platforms (Telegram/Discord) enabled. hermes gateway status showed PID 5417. There was no extra public HTTP Control UI — only SSH on the public interface.

ssh -o IdentitiesOnly=yes Ubuntu@YOUR_VM_IP 'bash -s' <<'EOF'
set -euo pipefail
sudo tee /etc/systemd/system/hermes-gateway.service >/dev/null <<'UNIT'
[Unit]
Description=Hermes Agent gateway (Kanban dispatcher)
After=network.target ollama.service

[Service]
Type=simple
User=Ubuntu
Environment=PATH=/home/Ubuntu/.local/bin:/usr/local/bin:/usr/bin:/bin
Environment=HOME=/home/Ubuntu
WorkingDirectory=/home/Ubuntu
ExecStart=/home/Ubuntu/.local/bin/hermes gateway run
Restart=on-failure
RestartSec=10

[Install]
WantedBy=multi-user.target
UNIT
sudo systemctl daemon-reload
sudo systemctl enable --now hermes-gateway
systemctl is-active hermes-gateway
EOF

Firewall

Default incoming deny. Only SSH is public. Ollama 11434 stays on loopback.

ssh -o IdentitiesOnly=yes Ubuntu@YOUR_VM_IP 'bash -s' <<'EOF'
set -euo pipefail
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 22/tcp
echo y | sudo ufw enable
sudo ufw status
EOF

If you later expose a Hermes UI or a messaging webhook, put HTTP basic auth in front (and TLS before it faces the internet). The OpenClaw post already shows that loopback nginx gate — do not bind Ollama or the gateway to 0.0.0.0.

Run a real Kanban swarm

Not “hello”. Smoke-chat the orchestrator, then dispatch researcher / writer / reviewer against one goal.

ssh -o IdentitiesOnly=yes Ubuntu@YOUR_VM_IP 'bash -s' <<'EOF'
set -euo pipefail
export PATH="$HOME/.local/bin:$PATH"
hermes -p orchestrator chat --quiet -q "Reply with exactly: smoke ok"
hermes kanban swarm "Smoke: summarize why local Llama workers help Hermes" \
  --worker researcher:"Gather three facts about local Llama inference" \
  --worker writer:"Draft a two-sentence summary from research" \
  --verifier reviewer \
  --synthesizer orchestrator \
  --json
hermes kanban list
EOF

This Blackwell produced:

• Orchestrator smoke reply matched smoke.
• Swarm JSON created task IDs (t_672724cb root, plus researcher / writer / reviewer / synthesizer).
• ~/.hermes/kanban.db persisted five tasks, comments, and run summaries.
• Researcher t_4a8c7e74 done.
• Writer t_cfa402c6 blocked (plain-text reply is not a terminal Kanban state — the worker called kanban_block).
• Reviewer and synthesizer stayed todo, waiting on the writer.

Transcript snippet from the researcher run summary:

Local Llama inference is a decentralized, peer-to-peer network...
I have gathered three facts: (1) Local Llama is designed to be efficient...

That summary is what 70B Q4 actually wrote on this smoke prompt — not a polished wiki. The point of the demo is the board + tools + persistence, not the prose quality. Orchestrator blackboard comment stored the topology JSON (root_id, worker_ids, verifier_id).

Persistence: ~/.hermes/kanban.db, ~/.hermes/profiles/{orchestrator,researcher,writer,reviewer}/, ~/.hermes/config.yaml. Gateway: systemd hermes-gateway.service.

Confirm GPU utilization

ssh -o IdentitiesOnly=yes Ubuntu@YOUR_VM_IP \
  'nvidia-smi --query-gpu=name,memory.used,utilization.gpu --format=csv,noheader; ollama ps; nvidia-smi'

During the swarm this Blackwell showed 81911 MiB used of 97887 (model load). After verify: 81943 MiB, 0% util. The compute process was ollama llama-server (~81902 MiB). ollama ps: llama3.1:70b, 85 GB, 100% GPU, context 131072. The Hermes gateway is a Python process on CPU. That is the split: GPU = model, CPU = agent orchestration.