About this episode
Seventy3: 用NotebookLM将论文生成播客，让大家跟着AI一起进步。
今天的主题是：
Scaling Laws for Precision
Summary
This research paper investigates the impact of precision in training and inference on the performance of large language models. The authors explore how precision affects the effective parameter count and propose scaling laws that predict performance degradation due to low-precision training and post-training quantization. They find that overtrained models are more sensitive to post-training quantization, and that training larger models in lower precision might be computationally optimal. Their unified scaling law accounts for both training and post-training effects and predicts loss in varied precision settings, ultimately suggesting that the standard practice of training models in 16-bit might be suboptimal.
原文链接：https://arxiv.org/abs/2411.04330
解读链接：https://www.jiqizhixin.com/articles/2024-11-13-9
前往小宇宙评论区与主播互动
Get every episode summarized
Each time Seventy3 publishes, we email you a written briefing from the transcript — the topics, who appeared, and any specific claims, with the ad reads skipped.
Email me new episodesFree for 3 shows. No card needed.
No transcript yet
This episode has not been transcribed. Request it and it moves to the front of the queue.
More episodes
