A Small Service Mesh for My Macs and Supercomputers

How I run local dashboards, separate personal and agent knowledge vaults, shared agent memory, email, terminal history, model gateways, and persistent agent sessions across two Macs and ALCF systems -- with one health checker for the whole stack.

I have accumulated a surprising number of small web services around my daily work: a training dashboard, terminal history, separate personal and agent knowledge vaults, an agent memory API, an email API, a model gateway, and persistent agent sessions. None is a large application. The interesting part is making each one available in the right places without making everything public.

This post is an inventory of that system: what each service does, how the network paths fit together, and the commands and launchd patterns I use to bring them up again. It is the implementation-level companion to Working From Anywhere, which explains the larger remote-work design.

The topology

The always-on MacBook, mbph, is the hub and the public edge for the one service published through Cloudflare. A second MacBook, mbpr, is often on campus networks and acts as a client. Aurora and CELS are remote compute environments behind SSH bastions.

                      mbpr (mobile Mac)
                        │           │
          SSH forwards  │           │ Tailcat forwards
                        ▼           ▼
                       mbph (home Mac)
      ┌────────────┬──────────┴─────────┬─────────────────┐
      │            │                    │                 │
  Tailscale    localhost            SSH jump          herdr
    Serve      services               hosts           server
      │            │                    │                 │
      ▼            ▼                    ▼                 ▼
  browsers      agents            Aurora / CELS    Heeler / relay

Here is the concrete inventory. Ports are included because they make debugging far easier than descriptions like “the notes service.”

The table contains both applications and multiple routes to some applications. That distinction is useful: an application owns its local port; a transport merely decides who can reach it. I can replace Tailscale with an SSH forward without changing the application.

The common service recipe

Every local application starts life on loopback:

my-service --host 127.0.0.1 --port 9000
curl --fail http://127.0.0.1:9000/healthz

Binding to 127.0.0.1 means the process is not accidentally exposed on Wi-Fi, Ethernet, or a VPN interface. I then make persistence explicit with a macOS LaunchAgent:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>dev.example.my-service</string>
  <key>ProgramArguments</key>
  <array>
    <string>/Users/me/.local/bin/my-service</string>
  </array>
  <key>RunAtLoad</key>
  <true/>
  <key>KeepAlive</key>
  <true/>
  <key>StandardOutPath</key>
  <string>/Users/me/Library/Logs/my-service.log</string>
  <key>StandardErrorPath</key>
  <string>/Users/me/Library/Logs/my-service.err.log</string>
</dict>
</plist>

Install it once, or restart it after a change:

label=dev.example.my-service
plist="$HOME/Library/LaunchAgents/$label.plist"

plutil -lint "$plist"
launchctl bootstrap "gui/$(id -u)" "$plist"     # first install
launchctl kickstart -k "gui/$(id -u)/$label"    # restart
launchctl print "gui/$(id -u)/$label"           # inspect

RunAtLoad starts the job at login. KeepAlive restarts it if it exits. Logs go to stable files instead of disappearing with a terminal window.

Private web apps with Tailscale Serve

Tailscale Serve terminates HTTPS on the tailnet hostname and proxies to a loopback HTTP server. These are the public-safe route definitions from mbph; the agent-vault route stays in private configuration:

tailscale serve --bg --https=10443 http://127.0.0.1:8766   # Scrollback
tailscale serve --bg --https=9443  http://127.0.0.1:3000   # SilverBullet
tailscale serve --bg --https=8443  http://127.0.0.1:49375  # ai-memory
tailscale serve --bg --https=8720  http://127.0.0.1:8720   # AGPT dashboard
tailscale serve --bg --https=443   http://127.0.0.1:8891   # VaultServe
tailscale serve status

Serve configuration persists in Tailscale, so these commands define routes; they do not need to remain running in a shell. The backend processes still need their own supervision.

This is Tailscale Serve, not Funnel. Serve is tailnet-only. Funnel would put a service on the public internet, which is not what I want for terminal history, notes, dashboards, or agent memory.

Scrollback: searchable terminal history

Scrollback turns captured terminal output into a small searchable web application. This is useful when I remember seeing an error or command but not which terminal, host, or session contained it. It is also much easier to read long output on a phone than through a terminal multiplexer.

My wrapper allows the private Tailscale hostname while keeping the socket local:

#!/usr/bin/env python3
import uvicorn
from scrollback.web.app import create_app

uvicorn.run(
    create_app(allowed_hosts=["my-mac.my-tailnet.ts.net"]),
    host="127.0.0.1",
    port=8766,
)

Run the wrapper from a LaunchAgent, then add the :10443 Serve route shown above. The hostname allowlist matters: the proxy preserves an HTTP host that a default local-only configuration may reject.

SilverBullet: a human view of agent memory

SilverBullet is a Markdown knowledge base with wiki links, search, and a pleasant browser editor. I point it at a mounted/synchronized view of my agent-memory wiki:

mount-agent-memory
silverbullet --single \
  --hostname 127.0.0.1 \
  --port 3000 \
  "$HOME/AgentMemory"

This gives humans a useful complement to programmatic retrieval: I can browse a project’s decisions, edit a durable page, or follow links across related notes.

SilverBullet in this configuration does not add application-level authentication. Tailscale makes it private to the tailnet, not private to one person inside that tailnet. On a shared tailnet I would add a restrictive Tailscale grant/ACL for port 9443, and preferably application authentication as defense in depth.

ai-memory: one memory service for every agent

ai-memory stores captured agent sessions plus a Git-backed Markdown wiki. Every supported coding harness points at the same MCP endpoint, so Claude Code, Codex, OpenCode, and other clients can retrieve the same project history and hand work to one another.

A minimal local launch looks like:

export AI_MEMORY_AUTH_TOKEN="$(openssl rand -hex 32)"
ai-memory serve \
  --transport http \
  --bind 127.0.0.1:49375 \
  --enable-web

I put the token in the LaunchAgent environment or a protected environment file, never in a public repository. The service is exposed through Tailscale on :8443; clients use the /mcp path and send the bearer token.

/mcp is a machine endpoint, not a web page. Opening it in a normal browser correctly returns 401 because the browser did not send the bearer token. That response proves only that the route reached ai-memory and that authentication ran; it does not prove an MCP client can log in. A complete health check sends an authenticated initialize request and verifies the JSON-RPC response:

curl --fail-with-body \
  -H "Authorization: Bearer $AI_MEMORY_AUTH_TOKEN" \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  --data '{
    "jsonrpc": "2.0",
    "id": 1,
    "method": "initialize",
    "params": {
      "protocolVersion": "2025-06-18",
      "capabilities": {},
      "clientInfo": {"name": "healthcheck", "version": "1.0"}
    }
  }' \
  https://my-mac.my-tailnet.ts.net:8443/mcp

A healthy response is HTTP 200 with result.serverInfo.name equal to ai-memory. The optional human web interface is a separate /web route and needs browser/session authentication configured before it is useful as a page.

For a new client, the project README’s installers are preferable to hand-editing every harness:

ai-memory install-mcp --client claude-code --apply
ai-memory install-hooks --agent claude-code --apply

AGPT dashboard: training state without an SSH terminal

The AGPT dashboard is a small local web app for current and historical AuroraGPT training runs. Its backend merges cached run metadata with a live data API forwarded from Aurora. I start the dashboard itself on mbph:

cd ~/agpt-dash
uv run python3 server.py --port 8720
curl --fail http://127.0.0.1:8720/

Tailscale Serve makes it available to my tailnet at HTTPS port 8720. On mbpr, I additionally keep the same dashboard at a predictable localhost URL:

ssh -N \
  -o BatchMode=yes \
  -o ControlMaster=no \
  -o ControlPath=none \
  -o ExitOnForwardFailure=yes \
  -o ServerAliveInterval=30 \
  -o ServerAliveCountMax=3 \
  -L 127.0.0.1:8720:127.0.0.1:8720 \
  mbph

That SSH command lives in a script supervised by launchd on mbpr. Disabling SSH connection sharing is intentional: a multiplexed master can accept the forward and detach from the process that launchd is supervising, leaving job state and actual port ownership out of sync.

VaultServe: read-only notes in a browser

VaultServe is my intentionally small, read-only Obsidian viewer. It renders Markdown, resolves wiki links and embedded media, and searches the vault without giving the browser a write API.

VAULT_ROOT="$HOME/Obsidian/Notes" \
VAULT_BIND=127.0.0.1 \
VAULT_PORT=8891 \
  ~/.hermes/vaultserve/.venv/bin/python \
  ~/.hermes/vaultserve/server.py

This is useful on devices where I want to consult notes without installing or synchronizing the full Obsidian vault. The default Tailscale HTTPS route proxies to it, but it remains tailnet-only.

Two vaults, two trust boundaries

The personal vault and the agent vault use the same small Python server, but they are separate services with separate roots, repositories, ports, and sync rules.

The personal service on 8891 reads my Obsidian tree. Automation treats that tree as read-only. The agent service on 8892 serves a private Git repository containing curated memory, compiled session evidence, project histories, and a generated index of recently modified notes. Its daily sync job stages only the paths it owns; it cannot sweep an unrelated hand-edited history page into an automated commit.

This split is simpler than teaching one application which pages are personal, generated, editable, or publishable. A route now implies one source tree and one policy.

Hermes, AgentMail, and Herdr

The browser services are only half of the stack. Hermes runs the agent control plane: model routing, MCP servers, scheduled jobs, and notifications. A dedicated AgentMail inbox is connected through an MCP subprocess with a narrow tool allowlist. I verified the full loop with one outbound message, one inbound reply, and one threaded reply. Credentials stay in a protected environment file; the mailbox address and provider identifiers do not belong in public configuration.

Herdr owns persistent terminal panes and reports process state to Heeler. Current Hermes processes start through a Python bootstrap, so process detection has to recognize the bootstrap signature rather than only a binary named hermes. The local detector now does that without treating arbitrary Python source containing hermes_cli as an agent. The three HPC panes remain attached while Herdr reports each as hermes instead of unknown.

Bringing HPC services home with SSH

Some services cannot originate on either Mac. They run behind institutional login nodes, so SSH is the transport.

Aurora data for the dashboard

The training-data service listens on Aurora’s loopback port 8712. A local forward makes it look like an mbph service:

ssh -N \
  -o ExitOnForwardFailure=yes \
  -o ServerAliveInterval=30 \
  -L 127.0.0.1:8712:127.0.0.1:8712 \
  aurora

curl --fail http://127.0.0.1:8712/api/backbone

The dashboard only knows about 127.0.0.1:8712; it does not need to know about Aurora’s bastions or network topology. This is the same indirection principle as Tailscale Serve, in the opposite direction.

Argo through CELS

ALCF’s Argo service is HTTPS inside the CELS network. An SSH jump host carries that endpoint back to a local TLS port:

ssh -N -f \
  -o BatchMode=yes \
  -o ExitOnForwardFailure=yes \
  -o ServerAliveInterval=15 \
  -J "$USER@logins.cels.anl.gov" \
  -L 127.0.0.1:25939:apps.inside.anl.gov:443 \
  "$USER@compute-01.cels.anl.gov"

My actual entry point is argo-shim, which wraps the tunnel and the authentication details:

uvx --no-cache argo-shim --host compute-01.cels.anl.gov

Local model gateways can now speak to 127.0.0.1:25939 while preserving the correct upstream TLS server name. A useful acceptance test is a complete TLS handshake, not merely seeing the SSH process:

openssl s_client \
  -connect 127.0.0.1:25939 \
  -servername apps.inside.anl.gov \
  -brief </dev/null

A fallback data plane with Tailcat

Tailscale is normally the cleanest path between my Macs, but one campus network has blocked its control connection. I wanted the services above to depend on “a port exists here,” not on one particular VPN implementation, so I added a second point-to-point transport using Tailcat.

On mbph, the server is restricted to the peer’s public node key:

tailcat serve --allow='nodekey:<MBPR_PUBLIC_NODE_KEY>' all

On mbpr, one client process creates two loopback forwards:

tailcat forward '<MBPH_TAILCAT_ADDRESS>' \
  2222:22 \
  8443:49375

The first mapping makes mbph SSH available at mbpr:2222; the second makes the ai-memory API available at mbpr:8443. Both commands run under LaunchAgents with KeepAlive and a short throttle interval.

The node key and Tailcat address are capabilities. I keep the real values in private configuration and use placeholders here. The key design point is that the services above do not change: SSH still speaks SSH and ai-memory still speaks HTTP/MCP. Only the bytes’ route between the Macs changes.

One carefully public route with Cloudflare Tunnel

Everything so far assumes the client can join my tailnet or reach an SSH path. The exception is Herdr’s browser relay, which lets me attach to a persistent terminal/agent session from an ordinary browser. That needs a public hostname, so it gets a separate and more explicit boundary.

The relay binds to loopback on mbph:

HERDR_RELAY_PORT=8375 uv run ./relay/herdr_relay.py
curl -I http://127.0.0.1:8375/

This one detail matters more than it looks: the relay shells out to the local herdr binary, so it can only ever report the sessions on the machine it runs on. The relay has to live wherever the agents live. It originally ran on mbpr, and once most of my agents had moved to mbph the public URL was faithfully serving the wrong machine’s terminals. Moving it was the fix.

A named Cloudflare Tunnel publishes only that backend:

tunnel: herdr-relay
credentials-file: /Users/me/.cloudflared/<TUNNEL-ID>.json

ingress:
    - hostname: relay.example.com
      service: http://localhost:8375
    - service: http_status:404

Create and run it with:

cloudflared tunnel login
cloudflared tunnel create herdr-relay
cloudflared tunnel route dns herdr-relay relay.example.com
cloudflared tunnel \
  --config "$HOME/.cloudflared/config-herdr.yml" \
  run herdr-relay

The final catch-all 404 prevents the tunnel from becoming an accidental general-purpose proxy. In production I also put the hostname behind Cloudflare Access; the relay retains its own token as a fallback. The tunnel credential, access audience, and relay token live in private files and never in the LaunchAgent plist or repository.

Two LaunchAgents supervise this path independently:

1. the Herdr relay on 127.0.0.1:8375;
2. cloudflared, which maintains outbound connections to Cloudflare.

Splitting them makes failures legible. I can test the local relay first, then the authenticated public URL, and know which half is broken.

Moving the relay to another machine

Because the relay reports on whatever machine it runs on, “which host serves the public URL” is a decision I expect to revisit. The useful property of a named tunnel is that moving it needs no DNS change at all: relay.example.com is a CNAME pointing at the tunnel’s UUID, not at a host. Moving the connector is invisible from the outside.

The move is a cutover, not a parallel run. Two copies of the relay on one LAN collide on mDNS (both register the same hardcoded herdr-remote._herdr-remote._tcp.local. name), and two connectors for one tunnel is not a state worth reasoning about. So: stop the old host first.

Before touching anything, get any uncommitted work off the old machine. Mine had an unpushed feature sitting in the checkout, which a migration is an excellent way to lose:

git -C ~/projects/herdr-remote status --short

Install the prerequisites on the new host — cloudflared, uv, and herdr itself — then clone the relay at the same commit the old host was running.

Copy the credentials and configuration. The tunnel credential and the cert.pem are what let the new host claim the same tunnel; without them cloudflared has no identity:

scp ~/.cloudflared/cert.pem newhost:~/.cloudflared/cert.pem
scp ~/.cloudflared/<TUNNEL-ID>.json newhost:~/.cloudflared/
ssh newhost 'chmod 600 ~/.cloudflared/cert.pem; chmod 400 ~/.cloudflared/*.json'

If the two machines have different usernames — mine do — every absolute path in the config, the env files, and the LaunchAgent plists has to be rewritten, and then verified, because a plist with a bad path fails quietly:

sed 's#/Users/olduser#/Users/newuser#g' config.env > /tmp/config.env
ssh newhost 'for p in $(grep -oE "/Users/[^ ]*" ~/.config/herdr-remote/config.env); do
  [ -e "$p" ] && echo "OK   $p" || echo "MISS $p"
done'

Then cut over. Unload on the old host, confirm the port is actually released, and only then load on the new one:

# Old host
launchctl unload ~/Library/LaunchAgents/com.herdr-remote.{relay,tunnel}.plist
lsof -nP -iTCP:8375 -sTCP:LISTEN   # must be empty

# New host
launchctl load ~/Library/LaunchAgents/com.herdr-remote.{relay,tunnel}.plist

Verifying this is where it gets interesting, because the obvious check is useless. curl https://relay.example.com/ returns the same 302 before and after the move — that is Cloudflare Access redirecting to SSO, and it would keep returning 302 even if I had migrated nothing. The public URL cannot tell me which machine is behind it.

Two checks actually can. The tunnel’s own connector list reports the connector’s cloudflared version and creation time, so if the new host runs a different build the version alone identifies it:

cloudflared tunnel info herdr-relay

I expect exactly one connector, created at cutover time. And then the only check that really matters — ask the relay what it is actually serving, and confirm the session paths belong to the new machine:

ssh newhost 'herdr pane list' | head

Finally, keep the old host’s LaunchAgents on disk but renamed, so they do not silently reload at next login while remaining available for rollback:

mv com.herdr-remote.relay.plist com.herdr-remote.relay.plist.migrated-YYYYMMDD

One log line will look alarming and is not: the relay throws a zeroconf NonUniqueNameException if anything else on the LAN still advertises that mDNS name. It runs in a daemon thread, and the relay logs relay on :8375 and Polling: local immediately afterwards. Read the lines after the traceback before concluding anything is broken.

Starting and checking the whole stack

I used to keep this as a list of launchctl kickstart lines per machine, and copy-pasted the relevant block. That was a bad habit for the reason this whole post keeps circling: kickstarting an agent only asks launchd to run something. It says nothing about whether the service came back. The list also drifted — I added the spool drainer and the export job and never updated the snippet.

So restart commands live in one start-services script, while a separate check-services.py verifies the steady state without changing it:

start-services            # restart everything, then verify
start-services --check    # verify only, change nothing
start-services --serve    # also re-declare the Tailscale Serve routes
~/.hermes/scripts/check-services.py
~/.hermes/scripts/check-services.py --json

It picks its service list from hostname -s, so the same script is correct on both machines. Each entry is a LaunchAgent label, an optional probe URL, and a description:

read -r -d '' SERVICES_MBPH <<'EOF'
dev.saforem2.scrollback|http://127.0.0.1:8766/|Scrollback terminal history
dev.saforem2.silverbullet-agent-memory|http://127.0.0.1:3000/|SilverBullet
com.github.akitaonrails.ai-memory|http://127.0.0.1:49375/mcp|ai-memory server
dev.saforem2.vaultserve|http://127.0.0.1:8891/|VaultServe
sh.samf.tailcat-serve||Tailcat listener
com.herdr-remote.relay|http://127.0.0.1:8375/|Herdr relay
com.herdr-remote.tunnel||Cloudflare tunnel
dev.saforem2.ai-memory-hook-drain||ai-memory spool drainer
dev.saforem2.ai-memory-silverbullet-sync||ai-memory -> SilverBullet export
EOF

The verdict logic is the part worth stealing. 401 and 403 count as healthy: they prove TLS, routing, and the authorization boundary all ran, and the service is correctly refusing an unauthenticated probe. Only 000 — nothing answered at all — is an unambiguous failure:

case "$code" in
  2*|3*|401|403) printf '  %-34s ok (HTTP %s)\n' "$desc" "$code" ;;
  000)           printf '  %-34s NO RESPONSE\n'  "$desc"; fail=$((fail+1)) ;;
  *)             printf '  %-34s HTTP %s\n'      "$desc" "$code"; fail=$((fail+1)) ;;
esac

Two smaller details matter. A periodic agent such as the spool drainer shows - instead of a PID between runs, which is healthy rather than missing. And the probe URL has to be a path the service actually routes: ai-memory answers 404 on / and 401 on /mcp, so probing / reports a false failure.

The checker has explicit contracts for the Hermes gateway, AgentMail MCP, ai-memory MCP, llm-rosetta, both vaults, SilverBullet, the required Tailscale Serve routes, Herdr, the three HPC agent panes, the Hermes cron scheduler, the AGPT dashboard, and Scrollback. The local Hermes dashboard is optional. A required failure exits 1; an optional failure does not change the exit status.

The important distinction is protocol health versus port health. The two MCP checks perform real MCP connection tests. The Tailscale check parses the route table and confirms each backend is listening. The Herdr check requires a compatible live server, then verifies that the Aurora, Sunspot, and Polaris panes are still classified as hermes rather than unknown.

The current result is 14 OK, 0 warnings, 0 failures:

OK   Hermes gateway
OK   AgentMail MCP
OK   ai-memory MCP
OK   llm-rosetta
OK   personal vault
OK   agent vault
OK   SilverBullet
OK   Tailscale
OK   Herdr
OK   Hermes HPC panes
OK   Hermes cron
OK   AGPT dashboard
OK   Scrollback viewer
OK   Hermes dashboard [optional]
SUMMARY ok=14 warn=0 fail=0

The same checker runs twice daily. Its wrapper prints nothing when every contract is healthy, so the notification path stays quiet unless there is a warning or failure. The JSON form carries the same verdict for other automation.

Verify paths, not processes

This stack recently produced five instructive failures:

• Tailscale Serve was healthy but proxied the AGPT URL to stale port 8726 instead of the live dashboard on 8720.
• VaultServe still accepted TCP connections on 8890 but closed every HTTP request without sending a response.
• An SSH control master owned mbpr:8720, while the LaunchAgent that was supposed to own the forward was not actually running.
• The Herdr relay served relay.sf.onl correctly from the wrong machine for weeks. Every probe passed, because the public URL returns an identical Cloudflare Access 302 no matter which host is behind the tunnel.
• llm-rosetta answered correctly on IPv4 while a stale review server owned the same port on IPv6. 127.0.0.1 returned the model list; localhost resolved to the other process and returned 404.

All five could look “up” in a process list. The acceptance checks need to cross the same boundary as the real consumer:

# Backend first
curl --fail http://127.0.0.1:8891/healthz

# Then the tailnet route
curl --fail https://my-mac.my-tailnet.ts.net/

# Reachability/auth-boundary check only; this is not a full health check
curl -sS -o /dev/null -w '%{http_code}\n' \
  https://my-mac.my-tailnet.ts.net:8443/mcp

# Test the forwarded dashboard on the consuming Mac, not the server
ssh mbpr 'curl --fail http://127.0.0.1:8720/'

# Inspect ownership as well as presence
lsof -nP -iTCP:8720 -sTCP:LISTEN
launchctl print gui/$(id -u)/sh.samf.agpt-forward-mbph

# Check both address families when localhost and 127.0.0.1 disagree
curl --fail http://127.0.0.1:8765/v1/models
curl --fail http://[::1]:8765/v1/models

For an authenticated API, 401 proves that TLS, routing, proxying, and the authorization boundary ran. It does not prove valid credentials or protocol initialization; use the authenticated MCP request above for that. Conversely, a Tailscale-generated 502 means the route exists but its backend is unavailable or failed to produce HTTP.

Security boundaries

The useful question is not “is it encrypted?” Every path here is encrypted. The question is who is allowed to arrive at the application after decryption?

There are a few rules I now apply consistently:

1. Bind local unless public is deliberate. No application here needs 0.0.0.0.
2. Treat tailnet-only as private, not secret. Every authorized tailnet member can potentially reach an unauthenticated Serve route unless grants say otherwise.
3. Keep credentials out of examples, repositories, and process arguments. Use protected environment files or the platform’s credential store.
4. Publish one hostname, not a network. The Cloudflare tunnel has one ingress rule and a terminating 404 rule.
5. Test from the consumer. A browser route is only healthy if a browser-side HTTP request works; a forwarded API is only healthy if the client host can use it.

What this buys me

The obvious payoff is convenience: terminal history on my phone, training state without an Aurora login shell, notes from a browser, and agent context that does not depend on which laptop or coding harness I opened.

The larger payoff is replaceability. Each application sees a local port. Each consumer sees a stable URL or local port. Between them I can choose Tailscale, SSH, Tailcat, or Cloudflare according to the trust boundary and the network I am currently on. When one transport fails, I do not have to redesign the service.

That is enough “service mesh” for two Macs and a few supercomputers: not a cluster orchestrator, just small processes with explicit ownership, narrow network paths, persistent supervision, and health checks that exercise the whole route.