CodeLlama for DevOps massive productivity hack.
Codellama LLM for DevOps Using the OpenWebUI Well, it happened. Thanks to quantized models and the specs of a Raspberry Pi, it is possible ( but not advisable) to get LLM working on a Raspberry Pi's 4B 8GB or RAM. I did some testing and got about 0.5 tokens per second on my Raspberry Pi. I will be getting a Pi 5 ( completed doubled the speed to just over 1tps). I ran it on My MacBook M2, and it was closer to 27 tokens per second, much faster. Codellama 7B/13B is a game changer for DevOps Productivity as it can do everything from Switch to configuration examples, Ansible playbooks, Terraform, and Python code. I was able to generate solid working examples of all of the above. Still, with it being very slow, it's helpful because you can ask it to run and get an example of code or configuration. Come back in 10-20 minutes. It has it. You can do it while in a meeting or working on something else. I had some doubts, like it was talking to a backend or something, ...