Scaling Inference Deployments with NVIDIA Triton Inference Server and Ray Serve | Ray Summit 2024

Scaling Inference Deployments with NVIDIA Triton Inference Server and Ray Serve | Ray Summit 2024
Share:

Download MP3


Similar Tracks

Ray at Scale: Apple's Approach to Elastic GPU Management | Ray Summit 2024 Anyscale
Scaling Ray to 10K NPUs: Huawei's Hyperscale Journey | Ray Summit 2024 Anyscale
Deploying Many Models Efficiently with Ray Serve Anyscale
How to pick a GPU and Inference Engine? Trelis Research
Transforming Healthcare with AI: Practical Examples and Business Impact Alex Gurbych, PhD | AI Automation
The mind behind Linux | Linus Torvalds | TED TED
Scaling AI at Autodesk with Ray and Metaflow | Ray Summit 2024 Anyscale
NVIDIA Triton Inference Server and its use in Netflix's Model Scoring Service Outerbounds
Visualising software architecture with the C4 model - Simon Brown, Agile on the Beach 2019 Agile on the Beach
Optimizing vLLM Performance through Quantization | Ray Summit 2024 Anyscale
Klaviyo's Journey to Robust Model Serving with Ray Serve | Ray Summit 2024 Anyscale
Multi-tenant Data Processing with Ray: Phaidra's Approach to Industrial AI | Ray Summit 2024 Anyscale
Setup HuggingFace VLM on Triton Inference Server with Docker Ambesh Shekhar
Fast LLM Serving with vLLM and PagedAttention Anyscale
Understanding the LLM Inference Workload - Mark Moyou, NVIDIA PyTorch
you need to learn Kubernetes RIGHT NOW!! NetworkChuck
Akash Network: Powering Ray on an Open Source Cloud | Ray Summit 2024 Anyscale
How to Build an Inference Service Trelis Research
Building Scalable AI Infrastructure with Kuberay and Kubernetes | Ray Summit 2024 Anyscale
Building a GPU Cloud with Ray at io.net | Ray Summit 2024 Anyscale