Where the agents live: rebalancing the AI fleet for a 64 GB Mac Studio

The Oddbyte agent fleet has quietly outgrown the machine it lives on. Nineteen Hermes profiles, five live agents, three model servers and roughly 227 GB of model weights all share one 24 GB Mac mini — the same box that also answers chat messages, runs the schedules and holds the work board. Measured this morning: 11% of system memory free, 18.8 of 19.4 GB of swap in use, and 34 GiB of disk left.

A 64 GB M5 Max Mac Studio arrives next, and the obvious question — which model should it run? — is the wrong one. Memory buys capacity; it does not decide which agent belongs on which machine. That is a placement question, and placement is what this post is about: where the fleet is today, where it should be, and the six phases that get it there. The design decisions below are now made, not proposed, and the last section records them.

TL;DR

• Today the whole fleet is one 24 GB Mac mini. Five live agents, nineteen profiles, a 27-build model library — and swap almost fully consumed.
• Two of the three local workers are not really local. One worker’s gateway runs on the mini while its model runs on a Windows workstation; another points at a model port that is not running at all.
• 64 GB of unified memory buys roughly 46 GiB of usable model memory. That is the 26–36B class at long context — the tier the mini cannot hold.
• The control plane is smaller than anyone assumes. Every live gateway together costs about 0.6 GB of memory. The swap is model serving, not the agents.
• The fix is separation of roles, not a bigger version of the same box: a light always-on control plane, a heavy inference hub, a GPU node for experiments, and a cloud tier for judgement-heavy work.
• Security boundaries have to be enforced, not agreed. Three trust tiers, each with an operating-system boundary; the detailed remediation plan stays internal.
• Six phases, each with a gate. Nothing moves on until the previous gate passes, and every phase is reversible.

Where we are

The honest starting point is a measurement, not an impression. Everything below was read from the live fleet on the morning of 24 September 2026.

• One host runs everything. A 24 GB M4 Mac mini hosts the shared Hermes installation: nineteen profiles, five running gateways (the principal, the operational worker, the two local workers, and the kid-facing bot) and fourteen parked profiles.
• Its model server is the fleet’s model server. Ollama on that mini exposes 27 builds whose weight files total about 303 GiB; stored once with shared layers counted once, they occupy 227 GB — the single largest consumer on a data volume that is 92% full.
• It has no headroom left. 11% of system memory free, swap at 18.8 of 19.4 GB, 34 GiB of disk remaining. A 12B-parameter model at 131k context was resident when I measured. This is a machine that is working, not a machine with room to spare.
• The Windows workstation is the real local-inference host. A discrete GPU runs an Ollama server (10 builds, about 169 GiB) and a Vulkan llama.cpp server that answers one worker at 64k context. The estate inventory still records that machine’s hardware as unverified, which is its own small lesson about documenting what you have actually confirmed.
• The VMware host runs no inference. The ESXi hypervisor virtualises identity, logging, databases and web services. No GPU is passed through, which makes it a good home for always-on software and a useless one for models.
• Two cloud models do the architectural thinking. The principal runs a frontier API model; the operational worker runs a mid-tier API model. Neither is local, and for those two roles that is deliberate.

Three defects fell out of the same pass, and they are worth naming, because each one is a symptom of the same illness: no host owns a role.

• A live worker profile whose model endpoint is not running. The profile is configured, the gateway is up, and the model port answers nothing.
• A parked profile sharing the default profile’s chat-bot credential. One bot token cannot belong to two profiles, so the duplicate blocks gateway consolidation until it is cleaned up.
• Placement drifts silently. Model assignments live in per-profile configuration files. One published description of this fleet already names a model that the agent in question no longer runs — nobody edited the document; the fleet changed underneath it.

What the fleet is actually made of

Strip the names away and there are three planes, each with a completely different appetite:

• The control plane — gateways, schedules, chat bots, the work board. Always on, latency-sensitive, and almost memory-free. It should never share a machine with a 30B model.
• The inference plane — the model servers. Memory-hungry, throughput-sensitive, and the only part that actually needs the big machine.
• The identity and coordination plane — one account per agent, one board, one agreed way to hand work over. This is what makes a fleet a fleet rather than a pile of chat windows.

Today all three planes are the same Mac mini. That is the whole problem: a reboot to free memory for a model takes the agents offline, and an agent loop re-reading a long instruction block competes with the model that is doing the reasoning. The coordination plane has the same weakness in miniature — an agent that both implements a change and approves it is not a review, and a work board with no owner column is a to-do list.

It is worth being concrete about what the control plane is, because it is not a separate product and it is not a mystery box. It is the always-on agent process set that owns the chat connections, the scheduler, the work board and its dispatcher, the session store and the secret-injection step — and that asks a model endpoint for every turn of thinking. The interesting part is that this split already exists in the fleet: one worker’s gateway runs on the mini while its model runs on the GPU workstation. The target design simply generalises that pattern, so repointing an agent at the Studio is a base-URL change in a profile, not a migration.

What 64 GB actually buys

Memory is a gate, not a multiplier. macOS holds roughly a quarter of unified memory back from the GPU by default, so a 64 GB machine has about 46 GiB to spend on weights plus cache, and a 24 GB machine about 17 GiB. Against the real weight files already sitting on the fleet’s own servers, that line falls in a useful place.

• Below the line already: the 9B–14B class and a 20B mixture-of-experts build, which is why the mini can keep serving small models and the kid-facing bot without difficulty.
• Newly above the line: gemma4 31B at 18.5 GiB, qwen3.5 35B at 22.2 GiB and qwen3.6 36B at 22.3 GiB. These are the models the upgrade is actually for, and none of them fit comfortably in 24 GB.
• Context is not free. Those figures are weights alone. A long working context adds several more GiB of key-value cache on top, so the number to plan against is weights plus the context the agent actually needs — not the model’s advertised maximum.

For how the Studio’s own configurations compare with each other — and what the memory-bandwidth trade does to speed — the earlier posts on what the first shipping M5 Studio reviews actually measure and M5 Ultra vs M5 Max vs DGX Spark cover that ground in detail. This post is about a different question: given the memory, who should live where.

Where we’re going

Four roles, four hosts, and an explicit rule for what happens when one of them dies.

• Control plane → the 24 GB Mac mini. It stays the always-on box: every gateway, the work board, the schedules, the small models and the kid-facing bot. It never loads a 30B model again.
• Inference hub → the 64 GB Mac Studio. The 26–36B tier at 64–128k context, served to the fleet. Both local workers point here, and so does any profile that needs a long-context reasoning step.
• GPU node → the Windows workstation. It keeps its Vulkan server for models that fit its own video memory and becomes the benchmark bench, where candidate models are measured before the fleet depends on them. It is not a 24/7 dependency: a desktop that gets rebooted and used for games is not a server.
• Cloud tier → the frontier APIs. The principal keeps the strongest reasoning available and the operational worker keeps a mid-tier model. Cloud is for judgement and for the work that has to be right, not for the kid-facing bot.
• And a fourth host waiting in the wings. The ESXi hypervisor has spare capacity — exactly the right shape for an always-on agent host for the cloud-model agents, and the natural failover home for the control plane.

One placement deserves its reasoning spelled out, because “put the agents on the spare server” sounds obvious and is half right. Profiles: yes. Models: no. That hypervisor is a 2018 desktop processor (8 cores, four channels of DDR4) virtualising roughly thirty guests, with no GPU exposed to any of them. Decoding is memory-bandwidth-bound: a 20B model at 4-bit streams about 12 GB of weights per token, so the arithmetic ceiling on shared desktop memory bandwidth lands near 4 tokens per second and the realistic figure is one or two — against roughly 30–50 on the Studio for the same model. That is a ten-to-thirty-fold gap with no upgrade path short of hardware, and it would put inference’s memory spikes next to the identity, logging and web services that everything else depends on. The one honest exception is the tiny end of the range: an embedding model or a small classifier would run there perfectly well if a workflow ever wants one beside those services.

Two rules run through the whole design. Every agent has a declared fallback: if the cloud is unreachable, local agents keep working on local models, and if the Studio is down, the workers fall back to a small model on the mini rather than stopping. And nothing that matters runs only on a machine that can be switched off.

Trust boundaries

There is a second reason to spread the fleet across hosts, and it is not memory. Some of the agents hold credentials, and today the separation between those credentials is designed rather than enforced: profiles have their own vault accounts, but they all run under one machine login, so anything running as that user can read a sibling’s unlock credential. Separate accounts under a shared login are a promise between agents, not a boundary — and a promise is exactly what a prompt-injected agent breaks.

So the target adds a rule to the placement decision: each trust tier gets an operating-system boundary, not a convention.

• Tier 1 — principals. Own always-on host, own operating-system user, own vault identity, and the only tier holding estate-wide credentials. Their reasoning stays on frontier cloud models, so nothing about them depends on local accelerator memory.
• Tier 2 — workers. Own operating-system user on the always-on host, own vault identity, no estate-wide items, models reached over the network from the inference hub.
• Tier 3 — the kid-facing bot. Own operating-system user, no vault account at all, and a single scoped chat token. Strict first: a local small model, no internet tools, an empty tool allowlist, and transcripts a parent can read.

Three principles carry the whole scheme. A boundary the operating system does not enforce is not a boundary — same user account means same secrets, however the files are labelled. Secrets should be brokered rather than possessed, so that no agent holds an unlock credential opening more than its own material. And blast radius should equal role: the kid tier holds nothing, the workers hold their own tokens, and only the principals can touch the estate. The detailed remediation plan — the sequence of rotations, the per-tier isolation mechanics and the acceptance tests that prove each boundary — is deliberately not published here.

How we get there

Six phases, each with a gate. No phase begins before the previous gate passes.

1. Freeze and baseline. Stop adding profiles. Record, for each one, what it runs, where its model lives, and what it is allowed to touch. Gate: every live profile has a declared model endpoint and a named owner.
2. Bring the Studio online. Install the runtime, pull one 30–35B build, serve it to the network behind an allowlist, and measure it rather than trusting a specification. Gate: measured tokens per second and time-to-first-token at the target context.
3. Move the local workers. Repoint one worker at the Studio; restore the other’s model server on the Studio with its own context profile. The mini keeps only small models. Gate: both agents answer on the intended model, and the mini’s swap stays near zero under load.
4. Split control from inference. Gateways and coordination stay on the always-on host; add a fallback model to every profile so an outage degrades instead of stopping work. Gate: power down the Studio mid-task and the fleet still replies.
5. Harden the kid-facing bot. A local small model, no internet tools, an allowlisted toolset, a separate bot identity, and transcripts a parent can read. Gate: a set of adversarial prompts fails to reach the network.
6. Fleet coordination. One work board as the ledger, a message board for long-form discussion, per-agent identities with no shared tokens, and the implement/review split enforced. Gate: a task handed between two agents is traceable from card to result.

Phase one has a second job beyond the baseline: consolidation. Of nineteen profiles, five are genuinely live, six are parked but declare a real purpose and need that purpose written down where the router can see it, and eight are candidates for retirement — each of them running the stock default instruction set, with no sessions, no schedule and no declared role. They get archived rather than deleted, and the retirement is recorded as a decision rather than left as an absence.

Beyond that, the interesting things — agents that write code in parallel, agents that trade a paper portfolio — stay behind a hard limit, a review gate and an audit trail. Autonomy is the reward for a fleet that can be measured, not the starting condition.

Decisions taken

These started as open questions and are now settled, so the design above reflects them:

1. The principal stays on a frontier cloud model. Reasoning quality matters more than locality for the agent that designs the fleet.
2. The installation does not follow the models. The control plane stays on the always-on small host and only model traffic goes to the Studio. Moving the whole install there would re-create today’s failure at a larger scale: one reboot would take every agent offline, the credential store would live beside the software being experimented on, and the memory budget bought for models would be spent on gateways.
3. The hypervisor becomes a third always-on host — for agents, not models, and only once the two prerequisites it needs are met: a credential path that does not assume a desktop operating system’s keychain, and a work board that is reachable from more than one host.
4. The kid-facing bot is strict first, then loosened on evidence. Local small model, no network tools, one capability added at a time, each addition recorded.
5. The parked profiles are consolidated, not carried. Five live, six documented, eight archived as deliberate retirements.
6. Nothing changes during planning. This session produced measurements and decisions; the estate was not touched. The phases are gate-driven and undated, and phase one starts the day the Studio arrives.

The measurement that started this post is the argument for it. A machine at 11% free memory, swapping 18.8 of 19.4 GB and holding a 92% full disk is not one upgrade away from comfort — it is one role away. The Mac Studio does not fix that by existing. Moving the right things off the mini does, and the order matters more than the hardware.