Hermes Agent on DigitalOcean Serverless Inference

Published on May 11, 2026

By James Skelton
AI/ML Technical Content Strategist

Table of contents

Most self-hosted AI agents eventually run into the same problem: the model layer becomes infrastructure glue code. Coding tasks want one model, summarization wants another, vision needs multimodal support, and suddenly your “simple agent” is juggling API keys, routing logic, retries, and provider quirks before it even becomes useful.

Hermes Agent makes this especially obvious by combining coding, reasoning, memory, delegation, tool use, and shell access within a single agent loop. Instead of manually orchestrating multiple providers, DigitalOcean Inference lets Hermes communicate with a single OpenAI-compatible endpoint, while the platform handles model selection and routing behind the scenes.

In this tutorial, we’ll connect Hermes Agent to DigitalOcean Serverless Inference and then use the Inference Router to automatically optimize which models handle different agent workloads - without building custom routing infrastructure yourself.

Key Takeaways

• One API key, one endpoint, many models. A single DigitalOcean model access key and the fixed base URL https://inference.do-ai.run/v1 gives Hermes Agent access to 70+ models — no per-vendor key juggling or provider-specific code changes required.
• The Inference Router eliminates manual routing logic. Instead of hardcoding which model handles coding vs. summarization vs. vision, you define a task pool once in the Control Panel and let the router classify and dispatch each request automatically — optimizing for cost, speed, or a manual ranking you set.
• Auxiliary task overrides are where the real cost savings hide. By default, Hermes routes vision, compression, session search, and web extraction through your main model. Pinning those to cheaper models in the auxiliary config block can dramatically reduce token spend without sacrificing quality on tasks that don’t need frontier models.

Hermes with DigitalOcean: A Powerful Pairing

Hermes is model-agnostic by design. It supports about 19 first-class providers, plus any OpenAI-compatible custom endpoint like DigitalOcean’s, and is switchable to the hermes model at any time with no code changes and no lock-in. The agent core handles tool calling, skills, memory, and sub-agent delegation, and it supports 18+ messaging gateways (e.g., Telegram, Discord, Slack, WhatsApp, Signal, Email, Home Assistant). The LLM is just a pluggable backend.

DigitalOcean’s Inference Engine bundles four things behind one OpenAI- and Anthropic-compatible endpoint:

• Serverless Inference — pay-per-token access to 70+ open-source and frontier models (NVIDIA, DeepSeek, Qwen, MiniMax, Moonshot/Kimi, OpenAI, Anthropic, Mistral, and more) with scale-to-zero pricing.
• Inference Router — middleware that classifies each prompt and routes it to the best model in a pool you define, optimizing for cost or latency. Public preview, no extra charge during preview; you pay only for the models that actually serve each request.
• Dedicated Inference — reserved GPU endpoints for steady, high-throughput workloads.
• Batch Inference — async jobs at up to 50% off real-time pricing.

For a Hermes setup, the two relevant pieces are Serverless Inference (your direct path to specific models) and the Inference Router (your way to get smart per-request model selection without writing routing logic yourself).

Demo Prerequisites

You will need:

• A DigitalOcean account with access to the Inference product (visible in the Control Panel sidebar under INFERENCE)
• A model access key in sk-do-… format. Create one in the Control Panel under Inference → Model Access Keys. You can scope a key to specific foundation models, to inference routers, and optionally restrict it to a VPC
• A working Hermes Agent install. On Linux, macOS, WSL2, or Android (Termux)

curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bash

• The installer pulls uv, Python 3.11, Node.js, ripgrep, ffmpeg, and the Hermes source. Native Windows is supported via a PowerShell one-liner but is still early beta — WSL2 is the more battle-tested path.

• A model with ≥64K context. Hermes refuses to start with anything smaller, because tool schemas plus system prompt plus working memory fill a smaller window before the conversation even begins. Most catalog models on DigitalOcean meet this easily.

The 60-second path: Serverless Inference as Hermes’ main provider

This is the minimum viable setup. You’ll point Hermes’ custom-endpoint provider at DigitalOcean and pick a single model.

Export the key

echo 'export MODEL_ACCESS_KEY="sk-do-..."' >> ~/.zshrc
source ~/.zshrc

Use ~/.bashrc (bash) or ~/.config/fish/config.fish (fish) as appropriate.

Find a model ID

DigitalOcean uses simple model slugs (e.g. llama3.3-70b-instruct, anthropic-claude-4.6-sonnet, openai-gpt-4.1). List what’s available with the key you just exported:

curl -s -H "Authorization: Bearer $MODEL_ACCESS_KEY" \
  https://inference.do-ai.run/v1/models \
  | jq '.data[].id'

Pick one. For a Hermes main model, prefer something agentic and ≥64K context — Claude Sonnet/Opus, GPT-4.1, Llama 3.3 70B, DeepSeek V3.2, Qwen 3.5, or MiniMax M2 are all solid choices.

Configure the provider in Hermes

hermes model

In the interactive picker:

• Choose Custom endpoint (self-hosted / VLLM / etc.) — or, depending on your Hermes version, More providers… → Custom endpoint (enter URL manually).
• API base URL: https://inference.do-ai.run/v1
• API key: paste your sk-do-… key.
• Model name: the model slug you picked, e.g. llama3.3-70b-instruct.
• Context length: leave blank for auto-detect.

Hermes will verify the endpoint by hitting /v1/models and writes the choice to ~/.hermes/config.yaml.

Confirm everything works with a smoke test

hermes

Then in the chat: Summarize this repo in 3 bullets and show me the main entrypoint. If you get a coherent answer and Hermes’ tool indicators light up, you’re done with the basics.

A Step Further: Using the Inference Router as Hermes’ Engine

Instead of pinning Hermes to a single model, you let DigitalOcean route each request to the best-fit model in a pool.

What an Inference Router actually is

An Inference Router is a policy you configure once in the Control Panel and address by name. Every request to the router is analyzed against your task definitions; the router picks a model from the pool you specified for the matching task, applying your priority (cost, latency, or a manual ranking). If a chosen model rate-limits or 5xx’s, the router falls back to the next candidate automatically - no dropped calls. Every response carries metadata about which model was selected and which task was detected, so you can see exactly what ran and why. This has profound implications for running an application. Specifically, it allows for even deeper optimization for cost as we scale, and more efficient model application in general.

Create a router

In the Control Panel (on the left hand sidebar) select Inference → Inference Router, and then click Create Router in the activity screen.

Two ways to start:

• Default / Recommended router. Click See Default Routers on the Getting Started tab. DigitalOcean ships pre-configured routers for common agentic patterns: writing and content development, software engineering, and document intelligence. Pick one if you want a one-click setup; you can always inspect, duplicate, and edit later.
• Custom router. Click Create Router to define a pool from scratch.

For a Hermes-shaped workload, a custom router is usually worth the few extra minutes: Hermes mixes coding, summarization, reasoning, planning, and tool dispatch in a single agent loop, and you’ll want explicit routes for each.

Point Hermes at the router

Using a router is a drop-in replacement for a model call: same endpoint, same auth, just a different value in the model field: router:<router-name> instead of a raw model slug. The simplest way is to re-run hermes model and update only the model name:

hermes model
# Custom endpoint
# API base URL:   https://inference.do-ai.run/v1   (same as before)
# API key:        sk-do-...                         (same as before)
# Model name:     router:my-hermes-router           ← the only change

Verify with hermes config get model and hermes status. Then start a session.

Hermes Features and Capabilities

A few things worth knowing before you push this into a long-running setup.

• Tool calling. Hermes depends heavily on tool/function calling. OpenAI and Anthropic models on DigitalOcean support this reliably, as do most large open-source models in the catalog. If tool calls suddenly stop working after a router change, check which model the router selected.
• Built-in tools. DigitalOcean’s Chat Completions API supports server-side tools including RAG (knowledge_base_retrieval), MCP integration, and web search. Hermes does not expose these directly by default, but they can be injected through custom plugins.
• Prompt caching. Supported models automatically cache repeated prompt prefixes, reducing cost for Hermes’ large recurring system prompts and skill definitions.
• Reasoning controls. Reasoning-capable models support configurable reasoning effort levels, and Hermes’ agent.reasoning_effort setting maps directly onto them.
• Privacy and observability. Open-weight model traffic is not logged or used for training. DigitalOcean also exposes detailed metrics for latency, throughput, errors, token usage, and router decisions through the Control Panel.
• Pricing. Serverless Inference is pay-per-token, and the Inference Router currently adds no additional cost during public preview. In practice, this lets Hermes route most requests to cheaper models while reserving frontier models for harder tasks.