In this episode Gudrun speaks with Nadja Klein and Moussa Kassem Sbeyti who work at the Scientific Computing Center (SCC) at KIT in Karlsruhe.

Since August 2024, Nadja has been professor at KIT leading the research group Methods for Big Data (MBD) there. She is an Emmy Noether Research Group Leader, and a member of AcademiaNet, and Die Junge Akademie, among others. In 2025, Nadja was awarded the Committee of Presidents of Statistical Societies (COPSS) Emerging Leader Award (ELA). The COPSS ELA recognizes early career statistical scientists who show evidence of and potential for leadership and who will help shape and strengthen the field. She finished her doctoral studies in Mathematics at the Universität Göttingen before conducting a postdoc at the University of Melbourne as a Feodor-Lynen fellow by the Alexander von Humboldt Foundation. Afterwards she was a Professor for Statistics and Data Science at the Humboldt-Universität zu Berlin before joining KIT.

Moussa joined Nadja's lab as an associated member in 2023 and later as a postdoctoral researcher in 2024. He pursued a PhD at the TU Berlin while working as an AI Research Scientist at the Continental AI Lab in Berlin. His research primarily focuses on deep learning, developing uncertainty-based automated labeling methods for 2D object detection in autonomous driving. Prior to this, Moussa earned his M.Sc. in Mechatronics Engineering from the TU Darmstadt in 2021.

The research of Nadja and Moussa is at the intersection of statistics and machine learning. In Nadja's MBD Lab the research spans theoretical analysis, method development and real-world applications. One of their key focuses is Bayesian methods, which allow to incorporate prior knowledge, quantify uncertainties, and bring insights to the “black boxes” of machine learning. By fusing the precision and reliability of Bayesian statistics with the adaptability of machine and deep learning, these methods aim to leverage the best of both worlds. The KIT offers a strong research environment, making it an ideal place to continue their work. They bring new expertise that can be leveraged in various applications and on the other hand Helmholtz offers a great platform in that respect to explore new application areas. For example Moussa decided to join the group at KIT as part of the Helmholtz Pilot Program Core-Informatics at KIT (KiKIT), which is an initiative focused on advancing fundamental research in informatics within the Helmholtz Association. Vision models typically depend on large volumes of labeled data, but collecting and labeling this data is both expensive and prone to errors. During his PhD, his research centered on data-efficient learning using uncertainty-based automated labeling techniques. That means estimating and using the uncertainty of models to select the helpful data samples to train the models to label the rest themselves. Now, within KiKIT, his work has evolved to include knowledge-based approaches in multi-task models, eg. detection and depth estimation — with the broader goal of enabling the development and deployment of reliable, accurate vision systems in real-world applications.

Statistics and data science are fascinating fields, offering a wide variety of methods and applications that constantly lead to new insights. Within this domain, Bayesian methods are especially compelling, as they enable the quantification of uncertainty and the incorporation of prior knowledge. These capabilities contribute to making machine learning models more data-efficient, interpretable, and robust, which are essential qualities in safety-critical domains such as autonomous driving and personalized medicine. Nadja is also enthusiastic about the interdisciplinarity of the subject — repeatedly changing the focus from mathematics to economics to statistics to computer science. The combination of theoretical fundamentals and practical applications makes statistics an agile and important field of research in data science.

From a deep learning perspective, the focus is on making models both more efficient and more reliable when dealing with large-scale data and complex dependencies. One way to do this is by reducing the need for extensive labeled data. They also work on developing self-aware models that can recognize when they're unsure and even reject their own predictions when necessary. Additionally, they explore model pruning techniques to improve computational efficiency, and specialize in Bayesian deep learning, allowing machine learning models to better handle uncertainty and complex dependencies. Beyond the methods themselves, they also contribute by publishing datasets that help push the development of next-generation, state-of-the-art models. The learning methods are applied across different domains such as object detection, depth estimation, semantic segmentation, and trajectory prediction — especially in the context of autonomous driving and agricultural applications. As deep learning technologies continue to evolve, they’re also expanding into new application areas such as medical imaging.

Unlike traditional deep learning, Bayesian deep learning provides uncertainty estimates alongside predictions, allowing for more principled decision-making and reducing catastrophic failures in safety-critical application. It has had a growing impact in several real-world domains where uncertainty really matters. Bayesian learning incorporates prior knowledge and updates beliefs as new data comes in, rather than relying purely on data-driven optimization. In healthcare, for example, Bayesian models help quantify uncertainty in medical diagnoses, which supports more risk-aware treatment decisions and can ultimately lead to better patient outcomes. In autonomous vehicles, Bayesian models play a key role in improving safety. By recognizing when the system is uncertain, they help capture edge cases more effectively, reduce false positives and negatives in object detection, and navigate complex, dynamic environments — like bad weather or unexpected road conditions — more reliably. In finance, Bayesian deep learning enhances both risk assessment and fraud detection by allowing the system to assess how confident it is in its predictions. That added layer of information supports more informed decision-making and helps reduce costly errors. Across all these areas, the key advantage is the ability to move beyond just accuracy and incorporate trust and reliability into AI systems.

Bayesian methods are traditionally more expensive, but modern approximations (e.g., variational inference or last layer inference) make them feasible. Computational costs depend on the problem — sometimes Bayesian models require fewer data points to achieve better performance. The trade-off is between interpretability and computational efficiency, but hardware improvements are helping bridge this gap.

Their research on uncertainty-based automated labeling is designed to make models not just safer and more reliable, but also more efficient. By reducing the need for extensive manual labeling, one improves the overall quality of the dataset while cutting down on human effort and potential labeling errors. Importantly, by selecting informative samples, the model learns from better data — which means it can reach higher performance with fewer training examples. This leads to faster training and better generalization without sacrificing accuracy. They also focus on developing lightweight uncertainty estimation techniques that are computationally efficient, so these benefits don’t come with heavy resource demands. In short, this approach helps build models that are more robust, more adaptive to new data, and significantly more efficient to train and deploy — which is critical for real-world systems where both accuracy and speed matter.

Statisticians and deep learning researchers often use distinct methodologies, vocabulary and frameworks, making communication and collaboration challenging. Unfortunately, there is a lack of Interdisciplinary education: Traditional academic programs rarely integrate both fields. It is necessary to foster joint programs, workshops, and cross-disciplinary training can help bridge this gap.

From Moussa's experience coming through an industrial PhD, he has seen how many industry settings tend to prioritize short-term gains — favoring quick wins in deep learning over deeper, more fundamental improvements.
To overcome this, we need to build long-term research partnerships between academia and industry — ones that allow for foundational work to evolve alongside practical applications. That kind of collaboration can drive more sustainable, impactful innovation in the long run, something we do at methods for big data.

Looking ahead, one of the major directions for deep learning in the next five to ten years is the shift toward trustworthy AI. We’re already seeing growing attention on making models more explainable, fair, and robust — especially as AI systems are being deployed in critical areas like healthcare, mobility, and finance. The group also expect to see more hybrid models — combining deep learning with Bayesian methods, physics-based models, or symbolic reasoning. These approaches can help bridge the gap between raw performance and interpretability, and often lead to more data-efficient solutions. Another big trend is the rise of uncertainty-aware AI. As AI moves into more high-risk, real-world applications, it becomes essential that systems understand and communicate their own confidence. This is where uncertainty modeling will play a key role — helping to make AI not just more powerful, but also more safe and reliable.

The lecture "Advanced Bayesian Data Analysis" covers fundamental concepts in Bayesian statistics, including parametric and non-parametric regression, computational techniques such as MCMC and variational inference, and Bayesian priors for handling high-dimensional data. Additionally, the lecturers offer a Research Seminar on Selected Topics in Statistical Learning and Data Science.

The workgroup offers a variety of Master's thesis topics at the intersection of statistics and deep learning, focusing on Bayesian modeling, uncertainty quantification, and high-dimensional methods. Current topics include predictive information criteria for Bayesian models and uncertainty quantification in deep learning. Topics span theoretical, methodological, computational and applied projects. Students interested in rigorous theoretical and applied research are encouraged to explore our available projects and contact us for further details.

The general advice of Nadja and Moussa for everybody interested to enter the field is: "Develop a strong foundation in statistical and mathematical principles, rather than focusing solely on the latest trends.
Gain expertise in both theory and practical applications, as real-world impact requires a balance of both. Be open to interdisciplinary collaboration. Some of the most exciting and meaningful innovations happen at the intersection of fields — whether that’s statistics and deep learning, or AI and domain-specific areas like medicine or mobility. So don’t be afraid to step outside your comfort zone, ask questions across disciplines, and look for ways to connect different perspectives. That’s often where real breakthroughs happen. With every new challenge comes an opportunity to innovate, and that’s what keeps this work exciting. We’re always pushing for more robust, efficient, and trustworthy AI. And we’re also growing — so if you’re a motivated researcher interested in this space, we’d love to hear from you."

Literature and further information

• Webpage of the group
• G. Nuti, Lluis A.J. Rugama, A.-I. Cross: Efficient Bayesian Decision Tree Algorithm, arxiv Jan 2019
• Wikipedia: Expected value of sample information
• C. Howson & P. Urbach: Scientific Reasoning: The Bayesian Approach (3rd ed.). Open Court Publishing Company. ISBN 978-0-8126-9578-6, 2005.
• A.Gelman e.a.: Bayesian Data Analysis Third Edition. Chapman and Hall/CRC. ISBN 978-1-4398-4095-5, 2013.
• Yu, Angela: Introduction to Bayesian Decision Theory cogsci.ucsd.edu, 2013.
• Devin Soni: Introduction to Bayesian Networks, 2015.
• G. Nuti, L. Rugama, A.-I. Cross: Efficient Bayesian Decision Tree Algorithm, arXiv:1901.03214 stat.ML, 2019.
• M. Carlan, T. Kneib and N. Klein: Bayesian conditional transformation models, Journal of the American Statistical Association, 119(546):1360-1373, 2024.
• N. Klein: Distributional regression for data analysis , Annual Review of Statistics and Its Application, 11:321-346, 2024
• C.Hoffmann and N.Klein: Marginally calibrated response distributions for end-to-end learning in autonomous driving, Annals of Applied Statistics, 17(2):1740-1763, 2023
• Kassem Sbeyti, M., Karg, M., Wirth, C., Klein, N., & Albayrak, S. (2024, September). Cost-Sensitive Uncertainty-Based Failure Recognition for Object Detection. In Uncertainty in Artificial Intelligence (pp. 1890-1900). PMLR.
• M. K. Sbeyti, N. Klein, A. Nowzad, F. Sivrikaya and S. Albayrak: Building Blocks for Robust and Effective Semi-Supervised Real-World Object Detection pdf. To appear in Transactions on Machine Learning Research, 2025

Podcasts

• Learning, Teaching, and Building in the Age of AI Ep 42 of Vanishing Gradient, Jan 2025.
• O. Beige, G. Thäter: Risikoentscheidungsprozesse, Gespräch im Modellansatz Podcast, Folge 193, Fakultät für Mathematik, Karlsruher Institut für Technologie (KIT), 2019.