Contrary to the belief that special hardware is needed to work with large language models (LLMs), your desktop system is likely capable of running a wide range of LLMs, including chatbots like Mistral or source code generators like Codellama. With tools like Ollama, LM Suite, and Llama.cpp, it’s relatively easy to get these models running on your system. Ollama, which works across Windows, Linux, and Macs, offers native support for Nvidia and Apple’s M-series GPUs. If you don’t have a supported graphics card, Ollama will still run on an AVX2-compatible CPU, albeit slower. Installing Ollama is straightforward, and working with it is largely the same across operating systems. Despite some performance and compatibility considerations, running an LLM on your PC is achievable in less than 10 minutes.
read more > www.theregister.com