Goodput, Not Just Throughput: Prefill, Decode, and Rethinking AI Inference
About this episode
We unpack the core bottleneck in streaming AI: the split between heavy pre-fill computations and fast, memory-light decoding. From chunked prefill to physical separation (DissServe) and logical isolation (DuetServe), we explore how phase isolation eliminates interference, delivering 2x–4.5x better goodput and transforming cost efficiency. Join us as we translate GPU architecture ideas into scalable, user-friendly AI services, with practical takeaways for builders, operators, and decision-makers.
Note: This podcast was AI-generated, and sometimes AI can make mistakes. Please double-check any critical information.
Sponsored by Embersilk LLC
Get every episode summarized
Each time Intellectually Curious publishes, we email you a written briefing from the transcript — the topics, who appeared, and any specific claims, with the ad reads skipped.
Email me new episodesFree for 3 shows. No card needed.
Hosts & guests
No transcript yet
This episode has not been transcribed. Request it and it moves to the front of the queue.
More episodes
More from Intellectually Curious
GPT-6 Astra: The Autonomous AI Operator Redefining Science and Workflows
Intellectually Curious
Claude Commerce: The One-Brain AI Reimagining Digital Shopping
Intellectually Curious
Zero-Friction Innovation: AI, Activation Energy, and the Long-Tail Frontier
Intellectually Curious
Momentum Exchange Tethers and Orbital Skyhooks
Intellectually Curious