With the new M6 chip, Apple puts local AI compute front and center. But how well does the M6 Mac – specifically the new Mac mini – actually handle running large language models (LLMs) directly on your own machine? The answer comes down to two numbers, not the marketing message.
Related: Apple unveils the new Mac mini with M6 chip – here we break down what it means for local LLMs.
Why run LLMs locally on a Mac at all?
A locally running language model has three concrete advantages: your data never leaves the machine (privacy), there are no per-request API costs, and it works entirely offline. For sensitive documents, coding help, or simply experimenting, that is appealing – provided the hardware keeps up. And this is exactly where the marketing message parts ways with practice.
Two numbers decide it – not the Neural Engine
For LLM practice, two metrics matter most:
- Unified memory (RAM): it determines which models fit into memory at all. A model must be loaded in full – if it does not fit in RAM, it either will not run or crawls via the SSD.
- Memory bandwidth (GB/s): it determines how fast text is generated. LLM inference is almost always bandwidth-limited: for every single token, the entire model has to be read through memory once.
A useful rule of thumb for the speed ceiling: tokens/second ≈ memory bandwidth ÷ model size in RAM. In reality you reach about 60–80 percent of that, because the KV cache, attention, and kernel overhead consume additional bandwidth.
What the M6 Mac mini actually offers
The new Mac mini starts at 16 GB of unified memory with 153 GB/s of bandwidth ($899). Upgrading to 24 or 32 GB raises bandwidth to 170 GB/s. The maximum is 32 GB – and that is the decisive limit for local LLMs.
- 16 GB (153 GB/s): after macOS, roughly 10–12 GB remain usable. Realistic are 7B/8B models (4-bit quantized, ~5 GB) – ideal for chat, summaries, and simple coding help.
- 24 GB (170 GB/s): now 14B models (~8–9 GB) fit comfortably, with headroom for longer context.
- 32 GB (170 GB/s): the ceiling. This even loads 30B/32B models (4-bit, ~18–20 GB) – tight, but doable. Models in the 70B class (~40 GB) no longer fit.
How fast is it? A few reference points
Applied to the M6’s 170 GB/s (with a realistic 70 percent discount), you roughly get:
- 8B model (~5 GB): about 20–25 tokens/s – faster than reading speed, very comfortable.
- 14B model (~8.5 GB): about 12–16 tokens/s – well usable for more serious tasks.
- 32B model (~19 GB): about 5–7 tokens/s – noticeably slower, but fine for quality answers if you are not waiting on every word.
For context: the base M6 at 153–170 GB/s is solid but no bandwidth monster. A Mac Studio with an M-Max or Ultra chip reaches 400 to over 800 GB/s and up to 192 GB of RAM – that is where 70B models and higher speeds become possible.
Does the dual Neural Engine help with LLMs?
Apple markets the M6 with a new dual Neural Engine (two 16-core units). Important to know: the common LLM tools such as Ollama, LM Studio, or llama.cpp compute almost exclusively on the GPU, not on the Neural Engine (ANE). The ANE mainly accelerates Apple’s own on-device features and CoreML models. So for classic local chatbots, the second Neural Engine brings less than the name suggests – the bottleneck remains memory bandwidth.
The right software
- Ollama – the easiest entry: one command, the model loads and runs. Ideal for a first test.
- LM Studio – a graphical interface with a model browser, great for beginners without a terminal.
- MLX – Apple’s own framework, optimized specifically for Apple Silicon and often a bit faster than generic runtimes.
- llama.cpp – the lean foundation many tools build on; maximum control for advanced users.
Mac mini, Studio, or Pro – what fits for local LLMs?
In short: the M6 Mac mini is the ideal entry point for local LLMs up to about 14B, and with the 32 GB variant up to ~32B. Anyone regularly working with 70B models, loading several models in parallel, or wanting high speeds should reach for the Mac Studio with an M-Max or Ultra chip (far more RAM and bandwidth). There is no “Mac Pro with M6” – the top models still rely on the Ultra chips. For the vast majority of home and developer setups, though, the Mac mini with 32 GB is the price-performance sweet spot.
Frequently asked questions (FAQ)
Is 16 GB enough for local LLMs? For 7B/8B models, yes – they cover chat, summaries, and simple coding tasks well. For 14B and larger, choose the 24 or 32 GB variant.
Which model is a good start? A current 8B model (e.g. from the Llama, Qwen, or gpt-oss family) in 4-bit quantization runs smoothly on any M6 Mac mini and already delivers surprisingly good results.
What does “4-bit quantized” mean? A compression that roughly quarters memory usage with only a small loss in quality. It is the standard way to make models usable on consumer hardware.
Can I run 70B models on the Mac mini? Practically no – they need around 40 GB and more, and the Mac mini tops out at 32 GB. That calls for a Mac Studio with more unified memory.
Is a Mac mini or a gaming PC with a graphics card better? A dedicated GPU with lots of VRAM is often faster on bandwidth, but expensive and power-hungry. The Mac mini scores with very low power draw, quiet operation, and the ability to load large models at all thanks to unified memory.
Sources: 9to5Mac, AppleInsider, Apple Newsroom. Speed figures are approximations based on the bandwidth rule of thumb and vary with model, quantization, and context length.
