Weather forecasting with AI, Kaggle tips and tricks, dealing with missing data, deep learning with Jesper Dramsch, The Data Scientist Show #040

Jesper Dramsch is a scientist for machine learning at the European Centre for Medium-Range Weather forecasts. They have a phd in applied Machine Learning to Geoscience from Technical University of Denmark. They are a Kaggle Kernals Expert and TPU star, ranking at top 81/100k worldwide. We talked about weather forecasting, things they learned from Kaggle, how to deal with missing data and ourliers, deep learning, Keras vs Pytorch, XGBoost, their struggles as a phd student, working in the EU vs US. Follow @DalianaLiu for more updates on data science and this show.

(00:01:27) how he got into in ML

(00:09:10) how he handled missing data

(00:28:34) Transformers are eating the world

(00:49:36) Hoover Loss is a fantastic metric to deal with extreme values

(00:54:48) his experience with Kaggle competition

(01:02:59) Kaggle tricks that helped his models perform better

(01:08:18) PyTorch vs Keras

(01:30:30) working in different countries and cultures

Resources shared by Jesper:

The newsletter with missing data:

https://buttondown.email/jesper/archive/towels-have-quite-a-dry-sense-of-humor/

The paper by Gael about missing data: