Claude Is Selling Your Local Open Source Models Short

I’ve spent a lot of hours, and a lot of tokens, building local agents with Claude. I’ve built enough of them to spot the mistakes Claude makes repeatedly when designing and building local agent automations and workflows.

If you’re using Claude, or any frontier model, to help you work out whether open source models on your own hardware can do the job you need, you probably aren’t getting a fair picture. Claude’s mistakes create a false perception of capability (or lack thereof) and there are two things working against you out of the gate.

Its knowledge is already out of date

Every model has a training cutoff, so what it knows is always a little out of date. For most subjects that’s fine. For agent architecture, it means a build based on what Claude already knows can never be current. This field moves faster than models are trained. The engines, the models, the harnesses and the ways people run them locally change month to month, sometimes week to week, so part of what Claude learned about agents was already stale the day it shipped. It’s building from a snapshot of a field that has moved on, and it won’t know what it’s missing until you give it that context.

Its habits come from the cloud

Claude learned to build agents for cloud models, where context is huge, memory is someone else’s problem and answers come back fast. Those habits don’t carry over. On your own hardware they fill the context window, ignore the memory you actually have and cut the model off before it’s finished. Then Claude looks at the result and tells you the local model couldn’t handle the task.

I’m here to tell you that Claude is probably wrong about that, and there’s a high probability Claude introduced one of the following into the build that caused the agent to fail mechanically.

The blind spots, biases and odd little tics

Here’s what I watch for. If you’ve built a local agent with Claude, some of these will look familiar.

It turns reasoning off. When something looks slow, or a reply comes back empty, Claude’s first move is to switch off the model’s thinking. With a reasoning model, the thinking is where the work happens. You get a faster answer from a dumber model, and nobody mentions that’s the trade. Claude will sometimes do this silently, so you don’t even realize that the reasoning was turned off.

It overloads the context, then blames the model. Claude puts the whole document, the full chat history and every tool definition into the window, then decides the model “struggles with long inputs.” Tool definitions alone are sent again on every turn and can run to thousands of tokens.

It reads an empty reply as an answer. When a model runs out of output budget, the server says so. Claude often skips that check and takes whatever came back. One of my invoice extractors spent 7,550 tokens thinking about a single small invoice, mostly debating whether to wrap its answer in markdown. That filled its context, so it returned nothing. Once I showed it the exact output I wanted and told it code would check the arithmetic, the same call took about 2,500 tokens, with thinking still on.

It times out the model mid-thought. A local model writing 30 tokens a second is slow, not broken. Claude puts a timeout on it anyway. One of my judge agents was cut off mid-decision by a five-minute limit, the loop around it crashed, and an outside watcher phoned me to say the watcher had died. The fix was taking the timeout off the model.

It ends the work on a count. Claude likes a loop that runs ten times and calls the job done. One of my fixer agents hit its turn limit and was logged as a dead run for twelve nights in a row. Resuming the same session finished the work, and the fix shipped.

It doesn’t check where the model is running. Engines quietly fall back to the CPU when the GPU is full or the driver loads late. That looks exactly like a weak model, and Claude rarely checks. This is a configuration issue, not a model failure.

It writes prompts as rule lists. NEVER, ALWAYS, MUST, with no reasons. A small model follows those to the letter, and two rules without reasons can box an agent into doing nothing. My own publishing agent left an approved post unreadable for seventeen days that way.

There’s a skill for that

After months of fixing the same things, I turned what I’d learned into a skill called local-agents. With it loaded, Claude stops guessing. It measures your machine first, with a script that reports your free memory, how much context the model really has, and whether it’s on the GPU. It sizes the context on purpose and ends loops when the model says it’s finished, not when a counter runs out. It leaves thinking on, and it tests on the real model before calling anything done. When something goes wrong, it checks whether the harness took something away before deciding the model can’t do the task.

If you’re paying for frontier tokens, there’s a side benefit. Every job a local model can carry is one you aren’t paying per token for, and this gives you a reliable way to find out whether local agents can handle those jobs.

Get it

It’s free under the MIT license.

• Claude Code: run /plugin marketplace add ghosthat-studio/marketplace, then /plugin install local-agents@ghosthat-studio
• Claude.ai, desktop or the API: download the skill and upload it under Customize → Skills.
• The code: on GitHub at ghosthat-studio/local-agents.
• More detail: the local-agents page in our Toolbox.

If you try it, I’d like to hear which of these you’d already run into.