The Coming of Local Models
Slides of my code.talks 2026 talk on local AI, plus the long version with all the slides I had to cut
Yesterday I gave a talk at code.talks in Hamburg about local models. In February 2026 running a model locally was mostly a fun experiment: Gemma 3 27B or GPT OSS at around 12 tokens per second on my M4 Max. Since then a lot happened. New inference engines, new quantizations, new architectures, and the same model now runs at 88 tokens per second instead of 12. Local models are roughly six months behind the frontier, and for many tasks that is good enough.
The talk was only 20 minutes, so I had to cut a lot of slides. There are two versions of the deck:
- The slides as presented (PDF, 61 MB)
- The long version, including all the skipped slides (PDF, 80 MB)
The long version also covers the things I didn’t have time for, such as steering, weight editing, batching, embedding models in apps, finetunes and hardware.
Both decks were made with Hyperdeck, of course. If you have questions, find me on x.com or LinkedIn.