Aaron Klein

I lead a research group at the ELLIS Institute Tübingen as part of OpenEuroLLM OpenEuroLLM, a European initiative developing a family of foundation models for European languages. Our team focuses on advancing automated methods for large-scale pre-training of large language models. I also serve as a scientific advisor to distil labs, a startup developing customized small language models.

I am a Member of the ELLIS Network and co-host of the virtual AutoML Seminar.

My research focuses on developing automated, efficient methods to train deep learning models at scale. More specifically my research interests include:

• Automated hyperparameter optimization

• Neural architecture search for designing resource-efficient neural network architectures

• Black-box optimization, with a focus on Bayesian optimization methods

• Efficient pre-training of large language models

Previously, I headed the AutoML research group at ScaDS.AI (Center for Scalable Data Analytics and Artificial Intelligence) in Leipzig.

Until 2024, I worked as an senior scientist at AWS, where I was part of the long-term science team behind SageMaker, AWS’s machine learning cloud platform, as well as the science team for Amazon Q, AWS’s generative AI assistant.

I earned my PhD at the University of Freiburg in 2019 in the machine learning lab under the supervision of Frank Hutter. In 2022, my co-authors and I received the Best Paper Award at the AutoML Conference. Earlier, in 2015, my collaborators from the University of Freiburg and I won the ChaLearn AutoML Challenge.

I have co-organized the Neural Architecture Search workshop at ICRL 2020 and ICLR 2021, and served as the local chair for the AutoML Conference 2023. I regularly serve as reviewer for top-tier venues (ICML, NeurIPS, ICLR) and as action editor for TMLR.