Litecoin Inference Kubernetes Cluster

 When I started this project, I had been looking at small Kubernetes clusters to make for my home lab. The issue was the size and power required to run them. In my bachelor days, I once owned a fully enclosed computer rack on wheels, which lived in the corner of my bachelor pad dining room. It was full of salvaged HP, Cisco, SGI, and Sun hardware from the dot.com crash. It's a true story; running it costs me about $300 extra monthly in power. When I got married, the spousal acceptance level of such a setup was a non-starter. Which made me rethink the whole X86 architecture or other rack-mounted hardware for a home lab. Linux has been running on ARM for a long time, and most significant projects have arm ports. This is fueled by the AWS Graviton platform and Apple's move to ARM-based M1 and M2 chips.  

In 2019 Raspberry Pi 4B option came out with some decent specs:

Cortex-A72 (ARM v8) 64-bit SoC @ 1.5GHz, 8GB RAM, 2 USB 3.0, and a fast Micro-SD card slot.

Also, Google released a very cool little project called Coral TPU.

It's a USB 3.0 TPU

https://coral.ai/products/accelerator

Google Edge TPU coprocessor:

4 TOPS (int8); 2 TOPS per watt 

With the USB 3.0 support, unleashing the TPU's power made sense. It's like running Nivida GTX 1080 on a Rasberry Pi. Here is a link to a great benchmark article. So now I had a good home lab hardware platform to start my experiments on. 

I have followed Picocluster Raspberry Pi cluster for a few years. I liked the elegant design of the cases and setups, including the power supply, network switch, chassis, and cooling system. So I took the plunge last year. I have a five-node Rasberry Pi 4B 8GB cluster running 64bit Pi Lite OS on my desk in front of me, costing about 1200 dollars. I did do a minor upgrade. I replaced the stock Picocluster 80mm with a Noctua NF-R8 fan, which was quieter.




 The first node is my gateway and command model that allows SSH access and port forwarding to my kids (Minecraft server, Jnotebook, Grafana running in my cluster, and the Python jobs). 

The first node also holds the marina DB for all the storage for the jobs I will run. That leaves me with four nodes to run as a K3s cluster. A Kubernetes distro by Rancher Labs owned by Suse Linux. 

K3s is an excellent kubernets cluster. I have worked with Minikube, MicroK8s, and K3s and found K3s to be more sturdy and stable than MicroK8s. Of the two with multinode support, I chose K3s over MicroK8s. I am considering trying out Sidero Metal, but that will likely be for a different project. 

I started using Argocd for cluster state/deployment management and Airflow for CI/CD (tech I use at work ) but found that I was doing too much and using too much of my limited resources, so I settled on Python scripts and Kubernetes cron jobs for job execution. If a job failed, it would launch on another node, more reliable than the services it was pulling from. A minor two-day outage of the sentiment API around Christmas, a Twitter hiccup here and there, and a Yahoo finance update required a pip lib update to get the script going. After 6 months, I had a training data set, and my Twitter API stopped because of policy Twitter API Term of service changes. So my account only captured via free API account 8 months of data.  

Now let's work on getting that data!





 

Comments

Post a Comment

Popular posts from this blog

Litecoin Data Collection and saved data features

Legal Disclaimer:  The Content is for informational purposes only. You should not construe such information or other material as legal, tax, investment, financial, or other advice . Nothing contained on our Site constitutes a solicitation, recommendation, endorsement, or offer by myself or any third-party service provider to buy or sell any securities or other financial instruments in this or in any other jurisdiction in which such solicitation or offer would be unlawful under the securities laws of such jurisdiction. I have already talked about the two scripts, one that pulls the financial data from Yahoo Finance hourly, then looks for the TA signals in that data. The other script searches one thousand tweets daily, forty-two tweets every hour for Litecoin, and uses hugging face to analyze sentiment.  Let's talk about the data features we wanted to save and the tables we kept them in.  Transactions: We collect the following metrics open, high, low, close, adj close, volu...
Read more

CodeLlama for DevOps massive productivity hack.

Image
Codellama LLM for DevOps Using the OpenWebUI  Well, it happened. Thanks to quantized models and the specs of a Raspberry Pi, it is possible ( but not advisable) to get LLM working on a Raspberry Pi's 4B 8GB or RAM. I did some testing and got about 0.5 tokens per second on my Raspberry Pi. I will be getting a Pi 5 ( completed doubled the speed to just over 1tps). I ran it on My MacBook M2, and it was closer to 27 tokens per second, much faster.   Codellama 7B/13B is a game changer for DevOps Productivity as it can do everything from Switch to configuration examples, Ansible playbooks, Terraform, and Python code. I was able to generate solid working examples of all of the above.  Still, with it being very slow, it's helpful because you can ask it to run and get an example of code or configuration. Come back in 10-20 minutes. It has it. You can do it while in a meeting or working on something else.  I had some doubts, like it was talking to a backend or something, ...
Read more

DevOps LLM shootout TLDR

  DevOps LLM shootout TLDR The experiment is to take four everyday DevOps tasks and test them with the open and free-accessible LLMs available to compare them and see which solution is preferred based on several factors for DevOps Engineers.  We will rate them by the following: Correctness of solution Explaining the Answer Elegance of results A quick note on correctness. Things can be less than 100% accurate, but give the user the correct format, fields, examples, and explanations to fill in the blanks. We will score them 1-10 on each with a max score of 120 and share the results. We used ChatGBT 3.5 Free, Google Gemini, and Meta CodeLlama 13B models to ask the following prompts to compare here are the prompts: Write Ansible Playbook to install httpd, copy a template from a file, ensure the service runs, and post a simple Hello World webpage. Create terraform for an EKS cluster of 5 nodes using terraform with t3.medium nodes. Write a Kubernetes manifest for Minecraft wit...
Read more