Sam Stoelinga articles - 5 total
Don't use a K8s Service for LLM Serving!
Relying solely on standard Kubernetes Services for load balancing can lead to suboptimal performance...
Tutorial: Deploying Llama 3.1 405B on GKE Autopilot with 8 x A100 80GB
Tutorial on how to deploy the Llama 3.1 405B model on GKE Autopilot with 8 x A100 80GB GPUs using...
Infinity embeddings on Kubernetes with KubeAI
Just merged and released the Infinity support PR in KubeAI, adding Infinity as an embedding engine....
Introducing KubeAI: Open AI Inference Operator
We recently launched KubeAI. The goal of KubeAI is to get LLMs, embedding models and Speech to text...
How I deployed a global website speed checking service to 25 locations while costing less than $5/yr using GCP Cloud Run
I created https://websu.io an open source webpage speed monitoring tool and a key feature was the...