9/9 Research Software Engineering with Python (COMP233) - Performance

In this last episode of this course, I talk to Itamar Turner-Trauring who created the website PythonSpeed and spent a considerable time on finding ways to make Python code faster and more efficient. Python and its ecosystem also have great tools how you can measure performance.

Links:

• https://pythonspeed.com a set of articles and recommendations on how to improve your performance
• https://blog.sentry.io/python-performance-testing-a-comprehensive-guide/ a general blog post on performance testing
• https://uwpce-pythoncert.github.io/SystemDevelopment/profiling.html 
  • https://uwpce-pythoncert.github.io/SystemDevelopment/index.html
• https://en.wikipedia.org/wiki/Computer_performance
• https://python-102.readthedocs.io/en/latest/performance.html
• https://docs.python.org/3/tutorial/datastructures.html
• https://www.green-algorithms.org
• https://doi.org/10.1145/356635.356640 Donald Knuth's paper on over optimisation
• https://wiki.python.org/moin/TimeComplexity
• https://blog.jetbrains.com/dataspell/2023/08/polars-vs-pandas-what-s-the-difference/ comparing Polars with Panda

Profiling tools

• https://pyinstrument.readthedocs.io/en/latest/
• https://docs.python.org/3/library/profile.html
• https://docs.python.org/3/library/time.html the time function in Python
• https://docs.python.org/3/library/timeit.html another function to measure time in Python
• https://jiffyclub.github.io/snakeviz/ a graphic profile viewer
• https://bloomberg.github.io/memray/ flexible memory profiler
• https://github.com/benfred/py-spy
  • https://www.wrighters.io/profiling-python-code-with-py-spy/
• https://github.com/P403n1x87/austin-python The Python wrapper for the Austin profiler

Don't be shy - say Hi

This podcast is brought to you by the Advanced Research Computing Centre of the University College London, UK.
Producer and Host: Peter Schmidt