DevOps LLM shootout TLDR
DevOps LLM shootout TLDR The experiment is to take four everyday DevOps tasks and test them with the open and free-accessible LLMs available to compare them and see which solution is preferred based on several factors for DevOps Engineers. We will rate them by the following: Correctness of solution Explaining the Answer Elegance of results A quick note on correctness. Things can be less than 100% accurate, but give the user the correct format, fields, examples, and explanations to fill in the blanks. We will score them 1-10 on each with a max score of 120 and share the results. We used ChatGBT 3.5 Free, Google Gemini, and Meta CodeLlama 13B models to ask the following prompts to compare here are the prompts: Write Ansible Playbook to install httpd, copy a template from a file, ensure the service runs, and post a simple Hello World webpage. Create terraform for an EKS cluster of 5 nodes using terraform with t3.medium nodes. Write a Kubernetes manifest for Minecraft wit...