Optimizing vLLM Performance through Quantization | Ray Summit 2024

Optimizing vLLM Performance through Quantization | Ray Summit 2024
Share:

Download MP3


Similar Tracks

Scaling Inference Deployments with NVIDIA Triton Inference Server and Ray Serve | Ray Summit 2024 Anyscale
Greg Brockman on Founding OpenAI and Systems for AI | Ray Summit 2022 Anyscale
But what are Hamming codes? The origin of error correction 3Blue1Brown
Databricks' vLLM Optimization for Cost-Effective LLM Inference | Ray Summit 2024 Anyscale
NYT Reporter on Trump’s Crypto Empire: “No Precedent in Modern U.S. History” | Amanpour and Company Amanpour and Company
Louis-Vincent Gave on AI, ANT IPO and trade war negotiations CNBC International Live
Scaling AI at Autodesk with Ray and Metaflow | Ray Summit 2024 Anyscale
But what is a neural network? | Deep learning chapter 1 3Blue1Brown
The Investment Case for Bitcoin: Chris Kuiper, Fidelity Digital Assets | Bitcoin for Corps '25 Bitcoin Magazine
The State of vLLM | Ray Summit 2024 Anyscale
Trump Thanks Qatar for Their Generous Jet Bribe & Accidentally Does a Socialism | The Daily Show The Daily Show
Scaling LLMs on Google Cloud: Synergy Between Ray, TPU, and GKE | Ray Summit 2024 Anyscale
Computer Scientist Explains One Concept in 5 Levels of Difficulty | WIRED WIRED
Powell Says Fed Can Wait to Adjust Rates, Sees Tariff Risks: The Fed Decides Bloomberg Podcasts
vLLM Office Hours - FP8 Quantization Deep Dive - July 9, 2024 Neural Magic
LoRA explained (and a bit about precision and quantization) DeepFindr
Building a GPU Cloud with Ray at io.net | Ray Summit 2024 Anyscale
Multi-tenant Data Processing with Ray: Phaidra's Approach to Industrial AI | Ray Summit 2024 Anyscale
Transformers (how LLMs work) explained visually | DL5 3Blue1Brown