Building a Per-Message Model Switcher Plugin for Hermes Gateway

The Problem

I wanted to be able to switch AI models on the fly during a WhatsApp conversation with my Hermes agent — specifically, to dip into GPT-4o Mini (for vision, creative writing, second opinions on bugs) without restarting the gateway or permanently changing my default config.

The naive approach — changing config.yaml and restarting the gateway — took ~5 seconds and required me to resend my question. Not great UX.

The Goal

Build a gateway plugin that intercepts /gpt <message>, /pro <message>, and /flash <message> commands at the pre-dispatch stage and routes just that message through the specified model, with no gateway restart.

Architecture Deep Dive

Plugin System in Hermes

Hermes has a built-in plugin system at ~/.hermes/hermes-agent/hermes_cli/plugins.py. Plugins can hook into various lifecycle events in both the agent and the gateway. The key hook for our use case is pre_gateway_dispatch, which fires before authentication and agent dispatch when a new message arrives.

Plugin structure:

~/.hermes/plugins/model-switcher/
├── plugin.yaml        # Manifest — declares hooks it uses
└── __init__.py        # Plugin code — registers hook callbacks

How plugins are loaded:

• Plugins in ~/.hermes/plugins/ (user) or ~/.hermes/hermes-agent/plugins/ (bundled)
• Listed in config.yaml under plugins.enabled
• Loaded when the gateway starts — requires restart to pick up new plugins

The Gateway Message Pipeline

When a message arrives on WhatsApp:

WhatsApp message arrives
  → GatewayRunner._handle_message()
    → Plugin hook: pre_gateway_dispatch fires
      → Plugin can inspect/modify the event or handle it entirely
    → Gateway resolves session, auth, agent runtime
    → Agent processes the message with its assigned model
    → Response sent back to WhatsApp

The pre_gateway_dispatch hook receives:

• event — a MessageEvent object with text, source, etc.
• gateway — the GatewayRunner instance (full access to internal state)
• session_store — the session store

The hook can return:

• None or {"action": "allow"} → normal dispatch
• {"action": "rewrite", "text": "..."} → modify the message text and continue
• {"action": "skip", "reason": "..."} → drop the message entirely (no reply)

The Model Override Mechanism

The GatewayRunner has a dict called _session_model_overrides that maps session keys to model config overrides:

_session_model_overrides: dict[str, dict] = {
    "whatsapp:[chat-id]": {
        "model": "gpt-4o-mini",
        "provider": "openai-api",
        "api_key": "...",
        "base_url": "https://api.openai.com/v1",
        "api_mode": "codex_responses",
    }
}

When the gateway resolves the model for a session (in _resolve_session_agent_runtime), it checks if there’s an override:

override = self._session_model_overrides.get(session_key)
if override:
    override_model = override.get("model", model)
    override_runtime = {
        "provider": override.get("provider"),
        "api_key": override.get("api_key"),
        "base_url": override.get("base_url"),
        "api_mode": override.get("api_mode"),
    }
    if override_runtime.get("api_key"):
        # FAST PATH: use the override directly, skip config.yaml entirely
        return override_model, override_runtime
    # Otherwise: fall through to config.yaml resolution

⚠️ The Critical Bug We Hit

When I first wrote the plugin, the override dict did not include api_key:

# WRONG — missing api_key
MODELS = {
    "gpt-4o-mini": {
        "model": "gpt-4o-mini",
        "provider": "openai-api",
        "base_url": "https://api.openai.com/v1",
        "api_mode": "codex_responses",
    },
}

Without api_key, the gateway hit the slow path:

1. Called _resolve_runtime_agent_kwargs() which resolves credentials from config.yaml
2. Config.yaml says provider: deepseek — so it resolved DeepSeek’s API key
3. Then _apply_session_model_override() overwrote provider to openai-api but kept DeepSeek’s API key
4. The agent tried to call OpenAI’s API with DeepSeek’s API key → authentication failure

The fix: Read OPENAI_API_KEY from the environment and inject it into the override:

api_key = os.environ.get("OPENAI_API_KEY")
if api_key:
    override["api_key"] = api_key

With api_key present, the gateway takes the fast path — uses the override directly without touching config.yaml at all.

The Plugin Code Explained

plugin.yaml — The Manifest

name: model-switcher
version: 1.0.0
description: "Switch AI models on the fly via /gpt and /flash commands."
hooks:
  - pre_gateway_dispatch

This tells the plugin system: “I want to listen to the pre_gateway_dispatch hook.”

__init__.py — The Logic

Step 1: Register the hook

def register(ctx):
    ctx.register_hook("pre_gateway_dispatch", _on_pre_gateway_dispatch)
    logger.info("model-switcher: registered (pre_gateway_dispatch)")

The register() function is called by the plugin system at startup. It tells Hermes “call my _on_pre_gateway_dispatch function whenever a message is about to be dispatched.”

Step 2: Define the model configs

MODELS = {
    "gpt-4o-mini": {
        "model": "gpt-4o-mini",
        "provider": "openai-api",
        "base_url": "https://api.openai.com/v1",
        "api_mode": "codex_responses",
    },
    "o3-mini": { ... },
    "o4-mini": { ... },
}

Each entry is a complete model config that the gateway can drop directly into _session_model_overrides.

Step 3: The hook handler

def _on_pre_gateway_dispatch(**kwargs) -> dict | None:
    event = kwargs["event"]
    gateway = kwargs["gateway"]
    text = (event.text or "").strip()
    
    # Match /gpt [model] <message>
    m = re.match(r"^/gpt(?:\s+(\S+))?\s*(.*)", text, re.DOTALL)
    if m:
        model_slug = m.group(1)       # e.g., "o3-mini" or None
        user_msg = m.group(2).strip()  # the actual message
        
        # Resolve which model config to use
        if model_slug and model_slug in MODELS:
            override = dict(MODELS[model_slug])
        elif model_slug:
            # Unrecognized slug → treat as part of the message
            user_msg = f"{model_slug} {user_msg}".strip()
            override = dict(MODELS[_DEFAULT_GPT])
        else:
            override = dict(MODELS[_DEFAULT_GPT])
        
        # CRITICAL: inject the API key
        api_key = os.environ.get("OPENAI_API_KEY")
        if api_key:
            override["api_key"] = api_key
        
        # Set the session override
        session_key = gateway._session_key_for_source(event.source)
        gateway._session_model_overrides[session_key] = override
        
        # Rewrite the event text (strip the /gpt prefix)
        event.text = user_msg or "."
        return {"action": "rewrite", "text": event.text}

Step 4: The /flash handler — simply pops the override:

    m = re.match(r"^/flash\s*(.*)", text, re.DOTALL)
    if m:
        session_key = gateway._session_key_for_source(event.source)
        gateway._session_model_overrides.pop(session_key, None)
        event.text = user_msg or "."
        return {"action": "rewrite", "text": event.text}

The Data Flow

User in WhatsApp: "/gpt analyze this image"
  └─▶ Gateway receives text="/gpt analyze this image"
       └─▶ pre_gateway_dispatch hook fires
            └─▶ Plugin matches /gpt pattern
                 ├─▶ Parses: model_slug=None (use default), user_msg="analyze this image"
                 ├─▶ Loads gpt-4o-mini config from MODELS dict
                 ├─▶ Injects OPENAI_API_KEY from environment
                 ├─▶ Computes session_key = "whatsapp:[chat-id]"
                 ├─▶ Sets: _session_model_overrides[session_key] = {model, provider, api_key, ...}
                 └─▶ Returns: {"action": "rewrite", "text": "analyze this image"}
       └─▶ Gateway sees rewritten text and active session override
            └─▶ _resolve_session_agent_runtime()
                 └─▶ Detects override with api_key → FAST PATH
                      └─▶ Returns model=gpt-4o-mini, runtime={provider: openai-api, api_key: sk-...}
            └─▶ Creates agent with GPT-4o Mini config
            └─▶ Agent processes "analyze this image" using GPT-4o Mini
            └─▶ Response sent back to WhatsApp from GPT-4o Mini

Debugging Tips

If the plugin fails, check:

1. Config: Is model-switcher in plugins.enabled?

   grep -A3 "enabled:" ~/.hermes/config.yaml

2. Plugin loaded? Check agent.log:

   grep "model-switcher" ~/.hermes/logs/agent.log

3. API key available? The gateway process needs OPENAI_API_KEY in its environment (from ~/.hermes/.env):

   grep OPENAI_API_KEY ~/.hermes/.env

4. Auth errors? Real API errors are in agent.log, not gateway.log (gateway scrubs provider errors):

   grep -i "error\|400\|401\|fail" ~/.hermes/logs/agent.log | tail -20

5. Session override active? Add debug logging to the plugin or check runtime:

   grep -i "session model override\|switcher" ~/.hermes/logs/agent.log

Lessons Learned

1. Gateway plugins are powerful — the pre_gateway_dispatch hook gives you access to the entire gateway instance, including internal state like _session_model_overrides. With great power comes great responsibility.

2. The “fast path” vs “slow path” matters — session overrides without api_key fall through to config.yaml resolution, which resolves the default provider, not the override provider. Always include api_key to use the fast path.

3. Configuration via config.yaml is authoritative — _resolve_runtime_agent_kwargs() only reads config.yaml, not ~/.hermes/.env or environment variables for behavioral config. Env vars are only for secrets.

4. Session-scoped overrides persist — The override stays active until /flash, /new, or gateway restart. It’s not a true one-shot switch, but it’s close enough for practical use.

5. Plugin loading requires restart — New plugins or plugin changes need a gateway restart to take effect. After that, switching models is instant.

---

📁 Back to Research