Paper Club with Peter - ZeRO: Memory Optimizations Toward Training Trillion Parameter Models

Paper Club with Peter - ZeRO: Memory Optimizations Toward Training Trillion Parameter Models
Share:

Download MP3


Similar Tracks

How Fully Sharded Data Parallel (FSDP) works? Ahmed Taha
Distributed ML Talk @ UC Berkeley Sourish Kundu
Naomi presents Enhancing Text Classification through LLM-Driven Active Learning and Human Annotation nPlan
Lecture 1: Introduction to Power Electronics MIT OpenCourseWare
From Chaos to Control: The Easy AI Governance Blueprint by Matthias Muhlert CYBERSEC - European Cybersecurity Forum
Ultimate Guide To Scaling ML Models - Megatron-LM | ZeRO | DeepSpeed | Mixed Precision Aleksa Gordić - The AI Epiphany
Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM | Jared Casper @Scale
Transformers (how LLMs work) explained visually | DL5 3Blue1Brown
Ben presents: Flow Matching for Generative Modeling nPlan
CppCon 2017: Teresa Johnson “ThinLTO: Scalable and Incremental Link-Time Optimization” CppCon
Microsoft DeepSpeed introduction at KAUST KAUST Supercomputing Laboratory
Understanding GD&T The Efficient Engineer
Gerard presents: The Surprising Effectiveness of Test-Time Training for Abstract Reasoning nPlan
Harvard Professor Explains Algorithms in 5 Levels of Difficulty | WIRED WIRED
Peter presents: Bayesian Learning via Stochastic Gradient Langevin Dynamics (LD Part 1) nPlan
NMR Spectroscopy for Visual Learners Chemistorian
Attention is all you need (Transformer) - Model explanation (including math), Inference and Training Umar Jamil
📊 How to Build Excel Interactive Dashboards Kevin Stratvert