Demo: Optimizing Gemma inference on NVIDIA GPUs with TensorRT-LLM

Demo: Optimizing Gemma inference on NVIDIA GPUs with TensorRT-LLM
Share:

Download MP3


Similar Tracks

From model weights to API endpoint with TensorRT LLM: Philip Kiely and Pankaj Gupta AI Engineer
Understanding the LLM Inference Workload - Mark Moyou, NVIDIA PyTorch
Exploring the Latency/Throughput & Cost Space for LLM Inference // Timothée Lacroix // CTO Mistral MLOps.community
Andrew Ng Explores The Rise Of AI Agents And Agentic Reasoning | BUILD 2024 Keynote Snowflake Inc.
The Evolution of Multi-GPU Inference in vLLM | Ray Summit 2024 Anyscale
Long-Context LLM Extension Sasha Rush
TensorRT-LLM: Quantization and Benchmarking Long's Short-Term Memory
Understanding LLM Inference | NVIDIA Experts Deconstruct How AI Works DataCamp
Demo: Gemma on-device with MediaPipe and TensorFlow Lite Google for Developers
Mastering AI Prompting: Get the Results You Actually Want! 🚀 Kim Gaskill
Deep Dive: Optimizing LLM inference Julien Simon
Quantization vs Pruning vs Distillation: Optimizing NNs for Inference Efficient NLP
Beyond the Algorithm with NVIDIA: TensorRT-LLM Goes GitHub First NVIDIA Developer
TensorRT for Beginners: A Tutorial on Deep Learning Inference Optimization Long's Short-Term Memory
Training LLMs with Synthetic Data (How NVIDIA Trained Nemotron) What's AI by Louis-François Bouchard
NVAITC Webinar: Deploying Models with TensorRT NVIDIA Developer
Vision Transformer Quick Guide - Theory and Code in (almost) 15 min DeepFindr
Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads Gabriel Mongaras
Fast LLM Serving with vLLM and PagedAttention Anyscale