Should You Ditch Claude for Qwen3.8 27B? What influencers don't tell you about running local

Qwen3.8 27B is a free model you can download and run on your own machine, and it is good enough to make people ask a serious question: why keep paying a monthly bill to a cloud provider? It holds 262,144 tokens at once out of the box, which is roughly a long novel's worth of text in front of it at one time. It can look at images and video on its own, without bolting on a separate tool to do it. And it has a built in trick for generating text faster. On paper, the math says cancel the subscription, buy a good workstation, and bring everything in house.

The Model Is Ready. The Software Around It Is Not.

The model is genuinely great. It has a potentially massive context window. It is multi-modal (meaning it can see pictures), and its benchmark scores put it in the same league as the frontier cloud models we were all running in February. You should get this model and you should run it. But you need to be aware that there are some limitations the benchmarks and social media influencers don't tell you about. Some of these will bite hard if you don't see them coming.

Open source agent frameworks like Hermes, OpenCode, and OpenClaw promise you the world: they remember, they teach themselves new skills, they come with a nice interface. All of that is true, but there are hidden costs and traps in some of those features.

The biggest problem is that agent frameworks, even ones like Hermes that are built to run on your own hardware from the start, inherited their habits from the cloud. They weren't built exclusively for local models. They assume answers come back almost instantly. They assume context is stable and handled by the cloud model. The benefits of running local are real and proven: your data stays yours, your costs are transparent (and significantly lower), and you can see exactly what your agents are doing. What you take on is the housekeeping the cloud was doing behind the scenes.

Four Ways Local Agent Software Mishandles Local Models

1. Slowness is sometimes caused by the harness

When an AI agent uses a tool (looking something up, running a search, calling another program), there is a pause while the model works out what to ask for. Cloud services hide that pause behind smooth typing animations and status messages, so your brain fills in the waiting time with UI candy.

Local models are, as a rule, genuinely slower than cloud models, but much of that gap is caused by how the agent software you run them in handles tool connections, system prompts, and general administrative cruft surrounding the model. Tool definitions in particular get resent on every single turn. A lean OpenCode setup sends somewhere around 7,000 tokens before you type a word, and a loaded one with a full set of skills and tools has been measured at ten times that. Every one of those tokens is work your machine does before it starts on your actual question.

The fix is to apply progressive disclosure and discovery on tool calls. Don't give your agent all of the tools all of the time. And ruthlessly prune what you put into the prompts and rules. If you're letting Claude or another cloud model write your system prompts and rules, you should do an edit pass over those to make sure they aren't too verbose and don't add unnecessary directives.

2. What they don't tell you about context limits on local hardware

For those who don't know what context is, it's the immediate session memory that a model holds while you are carrying on a conversation. The model will keep up that conversation and coherently remember what you've talked about for the whole session until it hits that context limit. For most modern cloud models, this tops out at one million tokens. That's enough for Claude to hold an entire codebase in its context and run a refactor job.

Qwen3.8 27B can theoretically support up to one million tokens too, using a stretching technique called YaRN. This is genuinely unprecedented for a model that fits on your laptop. But there's a catch, and it's the one nobody puts in the demo video: the context has to live in your memory alongside the model itself, and you are the one who has to budget for it.

Here is the arithmetic on a 64GB machine. Say the model takes 35GB and the operating system wants 10GB or so, which leaves you roughly 19GB for context. On this model, holding one token of context costs about 64KB. Three out of every four layers use a design whose memory doesn't grow as the conversation gets longer, which is why that number is as low as it is. So 19GB buys you north of 250,000 tokens, which covers the full native 262,144 window with a little room to spare. It gets you nowhere near a million. A million tokens would need roughly 61GB of context memory on its own, which is more than the entire machine. The million token number is real, and it is not real on a 64GB box.

Budget for the overhead too. The model and the context aren't the only things competing for that memory, and on a Mac the GPU won't touch the full 64GB without you raising the limit yourself. So manage that budget on purpose. Boil older parts of the conversation down to short summaries. Throw out debug output once you have read it. Keep what the model needs for the job in front of it and file the rest away. Don't give your model all the tools at once. And don't run a memory or tool system that compounds into the system prompt: at 10k of prompts and tool definitions per turn, you are spending a chunk of your window every turn before you have said a word.

3. Memory use can degrade reasoning over time

The frameworks that brag about remembering everything and teaching themselves new skills are ignoring the physics of how LLMs work. As you build up memory over time, your agent will prioritize memory in context. For an agent with a small context window, that means you aren't getting reasoning based on the 27 billion parameter library that powers the model. You are getting reasoning polluted by the hardcoded memory system you're stuffing into the system prompt at every turn.

The best fix for this is to build your system so that memory isn't stuffed into the system prompt at every turn, and to create searchable data that gets pulled on demand rather than aggressively pushed into context.

And prune the memory. You should always have visibility into what is going into the agent's memory. If you don't, you need to build this into your local infrastructure along with tools that let you remove things that shouldn't be there.

4. The speed trick that can cost you speed

This model has a feature, speculative decoding, that guesses several words ahead and then checks its own guesses all at once, rather than working out one word at a time. On a proper server, that makes it noticeably faster.

On a machine under your desk, the checking step is extra work for the same graphics card that is already busy, and it runs hotter. Chips protect themselves from heat by slowing down. So the trick that was supposed to make things faster makes the machine hot, the machine slows itself down, and you end up worse off than if you had left it alone. Most frameworks never notice, because they never look at how much memory or headroom your card actually has.

Turn the dial down to fit your hardware. Have it guess fewer words ahead when memory is tight, limit how much checking happens at once, and watch the temperature on long runs. Speed you cannot sustain is not speed.

Build the Plumbing, Do Not Bolt It On

Moving your agents in house requires you to be much more aware of the environment you run in. At Ghost Hat Studio, that's what we love about running local. Everything is within our purview to either monitor or control. Qwen3.8 27B settles the question of whether an open model you can download is good enough to run as your daily agent. It is all of that and more.

Key advice: pay attention to consumption patterns, power draw, and memory usage, and build for your custom setup. Assume things take time and design around it. Keep the context clean. Be deliberate about what you put in memory. Set the speed features to match your hardware instead of the numbers on the spec sheet. The good news is that testing it is free and you can run unlimited iterative passes to get it right.

The trade is straightforward. You give up the convenience of paying someone else to handle this, and you get a new set of challenges that only you can solve. What you get back is your own data, costs you can predict, and the ability to see exactly how your agents think. For a lot of people that is worth it, as long as you go in knowing you will need to engage with it to make it yours.