Unlocking AI Potential with AMD's ROCm Stack

Summary
In this episode of the AI Engineering podcast Anush Elangovan, VP of AI software at AMD, discusses the strategic integration of software and hardware at AMD. He emphasizes the open-source nature of their software, fostering innovation and collaboration in the AI ecosystem, and highlights AMD's performance and capability advantages over competitors like NVIDIA. Anush addresses challenges and opportunities in AI development, including quantization, model efficiency, and future deployment across various platforms, while also stressing the importance of open standards and flexible solutions that support efficient CPU-GPU communication and diverse AI workloads.

Announcements

• Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems
• Your host is Tobias Macey and today I'm interviewing Anush Elangovan about AMD's work to expand the playing field for AI training and inference

Interview

• Introduction
• How did you get involved in machine learning?
• Can you describe what your work at AMD is focused on?
• A lot of the current attention on hardware for AI training and inference is focused on the raw GPU hardware. What is the role of the software stack in enabling and differentiating that underlying compute?
• CUDA has gained a significant amount of attention and adoption in the numeric computation space (AI, ML, scientific computing, etc.). What are the elements of platform risk associated with relying on CUDA as a developer or organization?
• The ROCm stack is the key element in AMD's AI and HPC strategy. What are the elements that comprise that ecosystem?
  • What are the incentives for anyone outside of AMD to contribute to the ROCm project?
• How would you characterize the current competitive landscape for AMD across the AI/ML lifecycle stages? (pre-training, post-training, inference, fine-tuning)
• For teams who are focused on inference compute for model serving, what do they need to know/care about in regards to AMD hardware and the ROCm stack?
• What are the most interesting, innovative, or unexpected ways that you have seen AMD/ROCm used?
• What are the most interesting, unexpected, or challenging lessons that you have learned while working on AMD's AI software ecosystem?
• When is AMD/ROCm the wrong choice?
• What do you have planned for the future of ROCm?

Contact Info

• LinkedIn

Parting Question

• From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?

Closing Announcements

• Thank you for listening! Don't forget to check out our other shows. The Data Engineering Podcast covers the latest on modern data management. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used.
• Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
• If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.
• To help other people find the show please leave a review on iTunes and tell your friends and co-workers.

Links

• ImageNet
• AMD
• ROCm
• CUDA
• HuggingFace
• Llama 3
• Llama 4
• Qwen
• DeepSeek R1
• MI300X
• Nokia Symbian
• UALink Standard
• Quantization
• HIPIFY
• ROCm Triton
• AMD Strix Halo
• AMD Epyc
• Liquid Networks
• MAMBA Architecture
• Transformer Architecture
• NPU == Neural Processing Unit
• llama.cpp
• Ollama
• Perplexity Score
• NUMA == Non-Uniform Memory Access
• vLLM
• SGLang

The intro and outro music is from Hitman's Lovesong feat. Paola Graziano by The Freak Fandango Orchestra/CC BY-SA 3.0