Distributed Multi-Node Model Inference Using the LeaderWorkerSet API- Abdullah Gharaibeh, Rupeng Liu
Similar Tracks
ARM-Wrestling: Overcoming CPU Migration Challenges to Reduce Costs- Laurent Bernaille, Eric Mountain
CNCF [Cloud Native Computing Foundation]
Accelerate Your GenAI Model Inference with Ray and Kubernetes - Richard Liu, Google Cloud
CNCF [Cloud Native Computing Foundation]
Building Massive-Scale Generative AI Services with Kubernetes and Open Source - John McBride
CNCF [Cloud Native Computing Foundation]
DRAcon: Demystifying Dynamic Resource Allocation - from Myths to Facts - Kevin Klues & Patrick Ohly
CNCF [Cloud Native Computing Foundation]
Best Practices for Deploying LLM Inference, RAG and Fine Tuning Pipelines... M. Kaushik, S.K. Merla
CNCF [Cloud Native Computing Foundation]
Better Together! GPU, TPU and NIC Topological Alignment with DRA - John Belamaric & Patrick Ohly
CNCF [Cloud Native Computing Foundation]
Enhancing the Kubernetes Scheduler for Diverse Workloads in Large Clusters - Yuan Chen & Yan Xu
CNCF [Cloud Native Computing Foundation]
Resilient Multi-Cloud Strategies: Harnessing Kubernetes, Cluster API, and... T. Rahman & J. Mosquera
CNCF [Cloud Native Computing Foundation]
The State of GenAI & ML in the Cloud Native Ecosystem - Alejandro Saucedo & Bartosz Ocytko, Zalando
CNCF [Cloud Native Computing Foundation]
Production Multi-node Jobs with Gang Scheduling, K8s, GPUs... Madhukar Korupolu & Sanjay Chatterjee
CNCF [Cloud Native Computing Foundation]
Distributed Inference with Multi-Machine & Multi-GPU Setup | Deploying Large Models via vLLM & Ray !
sheepcraft7555