the best local llm for mac in 2026, sized to your memory

the best local llm for mac, as of september 2026, is gemma 4 for most people and qwen3.8-27b if you have 32 gb or more. on a 16 gb machine pick gemma 4 12b. on 96 gb and up, mistral small 4 is our pick for a general model. all of these are apache 2.0. none of them match a frontier cloud model on hard reasoning, and you should know exactly where that gap bites before you switch.

the rest of this post is the working: memory, speed, license, and a verdict per mac.

which open weight models can you run on a mac right now?

six model families matter for a mac in 2026, and five of them ship something a mac can hold, though "can hold" varies a lot.

google released gemma 4 on april 2, 2026 in e2b, e4b, 26b moe and 31b dense sizes, and later added a 12b multimodal model to the family. google says the 31b ranked #3 among open models on the arena ai text leaderboard at launch.

alibaba shipped qwen3.8 in august 2026. the flagship is a 2.4 trillion parameter moe, but the one you care about is qwen3.8-27b, a dense 27b model with 262,144 tokens of native context under apache 2.0.

openai's gpt-oss 20b, released august 5, 2025, is still relevant. it has 21b parameters with 3.6b active, so only 17% of the model works on each token. its moe weights ship in mxfp4, and openai says it runs within 16 gb of memory.

mistral released mistral small 4 on march 16, 2026: 119b total parameters, 6b active per token, a 256k context window, apache 2.0.

meta's newest open model is muse glimmer, released august 10, 2026. it is a dense 30b model (29.6b including its vision encoder) under apache 2.0, distilled from muse spark and built to run in 24 or 32 gb. the older llama 4 scout, from april 5, 2025, is 109b total with 17b active under meta's own license.

deepseek v4 is open under mit, but v4 pro is 1.6t parameters with 49b active. that is a server, not a laptop.

how much unified memory does each model need?

this is the column that decides everything. a mac's gpu shares one pool of unified memory with the os, so a model that "fits" at exactly your ram size will not run well.

the gemma figures are google's own q4_0 estimates from the gemma 4 model overview, which include 20% loading overhead and come to about 25% of the bf16 size. the gpt-oss figure is from apple's mlx benchmark below. the qwen3.8, llama 4 scout and mistral small 4 figures are the summed weight files of the mlx-community 4 bit conversions, and muse glimmer's is meta's own gguf. none of them include context, so treat them as a floor.

the practical tiers:

• 16 gb: gemma 4 12b, or gpt-oss 20b with everything else closed.
• 24 to 32 gb: gemma 4 26b a4b, gemma 4 31b, qwen3.8-27b, muse glimmer.
• 64 gb: the same models with long context or at 8 bits. google lists gemma 4 31b at 34.9 gb in 8 bit.
• 96 gb and up: mistral small 4 or llama 4 scout, with room left for context.

how fast are local models on apple silicon?

apple silicon llm speed is mostly a function of memory bandwidth. generating each token means reading the active weights, which is why moe models with few active parameters feel quick for their size.

apple published the cleanest first party numbers on november 19, 2025. on a macbook pro with m5 and 24 gb, token generation ran 19 to 27% faster than m4, tracking the 28% bandwidth jump from 120 gb/s to 153 gb/s. time to first token was 3.33x to 4.06x faster thanks to the m5's gpu neural accelerators. apple notes a 24 gb machine holds a 30b moe at 4 bits under 18 gb.

for raw tokens per second across chips, the long running llama.cpp apple silicon thread uses one fixed test, llama 7b at q4_0, a 3.56 gib file. text generation for each chip's largest gpu:

1. m1 pro, 16 core gpu: 36.41 tokens/sec, the baseline
2. m2 max, 38 core gpu: 65.95 tokens/sec, 1.8x the m1 pro
3. m4 max, 40 core gpu: 83.06 tokens/sec, 2.3x
4. m2 ultra, 76 core gpu: 94.27 tokens/sec, 2.6x
5. m5 max, 40 core gpu: 119.92 tokens/sec, 3.3x

one caveat: the m5 rows were run on a 2026 llama.cpp build, while the older chips used a 2023 one. the thread's own m2 ultra rerun on that newer build, with flash attention, reaches 125.21, so part of the m5's lead is software.

to run llm locally on mac hardware at the 30b size, divide by the bytes. a 4 bit dense 27b to 30b model is 16 to 17 gb of weights against 3.8 gb for the 7b test, about 4.2 to 4.4x the bytes per token, so expect roughly a quarter of those numbers. meta's own figure agrees: muse glimmer ran 23.7 tokens/sec on an m4 max without speculative decoding, 29% of that chip's 83.06, though on a different runtime.

mlx or llama.cpp: which runtime?

you need a runtime to load the weights. two open source ones do the heavy lifting on a mac.

mlx is apple's array framework built for unified memory, and it is what apple benchmarks on. llama.cpp is the portable c/c++ engine behind the gguf format, with quantization from 1.5 to 8 bits. google listed both as day one runtimes for gemma 4.

our verdict: use mlx builds when they exist for your model, and llama.cpp when you need a specific quant or a model mlx has not caught up with.

where do local models fall short of frontier cloud models?

be honest about this, because the gap is real and it sits in exactly the tasks people want to hand off.

on the artificial analysis intelligence index, the leading open weights models were within 3 to 6 points of the leading proprietary ones on april 30, 2026. a year earlier the gap was about 13 points. open models have closed most of the gap on benchmarks, but benchmarks are not your inbox, and the same report found the gap still wide on the hardest reasoning and agentic coding evaluations.

and those open leaders are data center models of a trillion parameters or more. the ones that fit on your mac sit well below them. so the realistic comparison is a 27b model on your desk against a frontier model in the cloud.

where local loses:

• long agentic coding runs across a whole repo
• hard multi step reasoning and math
• very long context in practice, since a 256k window still needs memory you may not have
• anything that benefits from live web search built in

where local wins: privacy of the prompt, zero marginal cost, offline use, and steady latency. summarizing, rewriting, extraction, classification and questions over retrieved documents are well inside a 12b to 31b model's range. that last one is the job we cover in what is rag.

what should you actually install?

• macbook air, 16 gb: gemma 4 12b. good writing, sees images, 6.7 gb.
• 24 to 36 gb, mostly writing and chat: gemma 4 31b.
• 24 to 36 gb, mostly code: qwen3.8-27b, for the long context and apache license.
• 24 to 36 gb, local agents: muse glimmer, which meta built for tool use on this class of machine.
• you want reasoning with tool calls on 16 to 24 gb: gpt-oss 20b.
• 96 gb or more: mistral small 4. the 6b active parameters keep it quicker than its size suggests.
• anything hard or high stakes: a frontier cloud model. do not be precious about it.

local models are one half of a private setup

swapping models is easy in 2026, and the thing that is hard to swap is context: what you worked on, read and heard. if that lives inside one vendor's chat history, every model change costs you your memory. we covered that trap in ai memory compared.

remynd keeps that context on your side. it is a mac app that captures the focused window, runs ocr locally with apple vision, and keeps the searchable index on your mac. you can exclude apps and sites, and recordings retention defaults to 30 days. history access is read only.

two parts line up with this post. call transcription already runs on device with mlx. and settings let you add a custom api endpoint that speaks the openai responses api. we have not published a tested local model setup for it, so treat pointing it at a local server as an experiment rather than a supported recipe.

remynd also ships an agent window that runs your own claude code or codex cli, so the tool you ask can change while your history stays put. sign in and cloud agents do reach the network, so the honest claim is that the archive lives on your mac. details are on our security page and in private ai on your mac. if you are weighing context windows for local models, start with what is a context window.

download remynd for mac and keep your context when you change models.