Exploring Python's Internals By Rewriting Them In Rust

Summary

The CPython interpreter has been the primary implementation of the Python runtime for over 20 years. In that time other options have been made available for different use cases. The most recent entry to that list is RustPython, written in the memory safe language Rust. One of the added benefits is the option to compile to WebAssembly, offering a browser-native Python runtime. In this episode core maintainers Windel Bouwman and Adam Kelly explain how the project got started, their experience working on it, and the plans for the future. Definitely worth a listen if you are curious about the inner workings of Python and how you can get involved in a relatively new project that is contributing to new options for running your code.

Announcements

• Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
• When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With 200 Gbit/s private networking, scalable shared block storage, node balancers, and a 40 Gbit/s public network, all controlled by a brand new API you’ve got everything you need to scale up. And for your tasks that need fast computation, such as training machine learning models, they just launched dedicated CPU instances. Go to pythonpodcast.com/linode to get a $20 credit and launch a new server in under a minute. And don’t forget to thank them for their continued support of this show!
• You listen to this show to learn and stay up to date with the ways that Python is being used, including the latest in machine learning and data analysis. For even more opportunities to meet, listen, and learn from your peers you don’t want to miss out on this year’s conference season. We have partnered with organizations such as O’Reilly Media, Dataversity, and the Open Data Science Conference. Go to pythonpodcast.com/conferences to learn more and take advantage of our partner discounts when you register.
• Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
• To help other people find the show please leave a review on iTunes and tell your friends and co-workers
• Join the community in the new Zulip chat workspace at pythonpodcast.com/chat
• Your host is Tobias Macey and today I’m interviewing Adam Kelly and Windel Bouwman about RusPython, a project to implement a new Python interpreter in Rust

Interview

• Introduction
• How did you get introduced to Python?
• Can you start by explaining what Rust is for anyone who isn’t familiar with it?
• How did RustPython got started and what are your goals for the project?
• Can you discuss what is involved in implementing a fully compliant Python interpreter?
• What are some of the challenges that you face in replicating the capabilities of the CPython interpreter?
  • Are you attempting to maintain bug parity?
  • How much of the stdlib needs to be reimplemented?
  • Can you compare and contrast the benefits of Rust vs C?
  • Will the end result be compatible with libraries that rely on C extensions such as NumPy?
• What is the current state of the project?
  • What are some of the notable missing features?
• Can you talk through your vision of how the WebAssembly support will manifest and the types of applications that it will enable?
  • How much effort have you put into size optimization for the webassembly target to reduce client-side load time?
  • Are there any existing options for minification of Python code so that it can be delivered to users with less bandwidth?
• What have been some of the most interesting/challenging/unexpected aspects of implementing a Python runtime?
• What do you have planned for the future of the project?
• What are the risks that you anticipate which could derail the project before it becomes production ready?

Contact Info

• Windel
  • windelbouwman on GitHub
  • Website
  • @windelbouwman on Twitter
  • @[email protected] on Mastodon
• Adam
  • cthulahoops on GitHub
  • @cthulahoops on Twitter

Picks

• Tobias
  • Oysterhead
• Adam
  • FZF fuzzy finder
• Windel
  • TQDM Python progress bar

Links

• RustPython
• Windel Presentation EuroPython
• Rust
• C++
• Rust Memory Safety
• MicroPython
  • Podcast Episode
• PyPy
• Ouroboros – Pure Python standard library
• WebAssembly
• lalrpop – Rust parser generator
• Rust Crates
• PickItUp in-browser Python game engine
• QuickSilver Game Engine
• PEP 441
• JIT (Just-In-Time) Compilation

The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA