Video by Hugging Face via YouTube
This video features Dr. Sara Hooker, co-founder of Adaption Labs, discussing the transition from monolithic model scaling to an era of adaptive intelligence. Addressing the Hugging Face ML Club India, she explores why the "bigger is better" race in compute is reaching an inflection point and what this means for the future of research.
Timestamps:
* 00:00 – Introduction of Dr. Sara Hooker
* 02:37 – Personal anecdote: Using ChatGPT to generate presentation slides
* 04:03 – The problem with monolithic AI and "one-size-fits-all" models
* 05:27 – Analyzing the "Bitter Lesson" and the limits of scaling
* 07:08 – How the belief in scaling has shaped the AI ecosystem
* 09:28 – Evidence against monolithic models: Performant small models and weight redundancy
* 11:00 – Recent disappointments in massive model scaling
* 12:20 – Moving toward post-training and test-time scaling
* 14:14 – Optimization in the data space and Adapt Data
* 15:38 – Auto Scientist: Automating end-to-end model adaptation
* 19:26 – The pillars of research at Adaption Labs
* 21:30 – Q&A: Can you explicitly optimize for adaptability?
* 23:44 – Q&A: Relationship between pre-training, post-training, and test-time scaling
* 28:46 – Q&A: Defining adaptive intelligence vs. continual learning
* 31:32 – Q&A: Efficiency as a pillar for data and compute
* 35:12 – Q&A: Why do labs still invest in LLMs over alternatives?
* 37:20 – Q&A: The impact of the "Hardware Lottery"
* 40:30 – Q&A: The future of adaptive user interfaces
* 43:03 – Q&A: Accuracy and limitations of public "scaling laws"
* 44:28 – Q&A: Undervalued research domains (Sparsity and alternative architectures)
* 48:50- Q&A: Evolution of advice for machine learning beginners
* 53:16 – Q&A: The "Strawberry" problem and tokenization issues
* 56:44 – Q&A: The importance of research communities
* 57:50 – Q&A: pertinent theoretical research in optimization
