nibbles
Local LLMs are going to be a real thing.
Local LLMs are going to be a real thing.
I had an interesting experience spinning up Open Hands and the newly announced Open Hands 32b (AWQ variant) trained coding model yesterday. Seeing an all local AI agent/coding tool running against a local LLM and providing a rudimentary but serviceable experience gives me a sense that we’ll have some all-local AI coding options soon. I don’t think they’ll replace the big LLMs for most things, but they may make a lot of sense in specific cases (strict privacy/confidentiality cases, certain kinds of prototyping, code on a plane, etc…) Some wild times ahead, and it’s becoming a little more fun and a little less scary to be near the front of the curve.