Karsten Roth
CV     @confusezius   @karsten-roth   @Confusezius   Karsten Roth

  About me (click to collapse)

I am currently an IMPRS-IS & ELLIS PhD Student at the Cluster for Excellence in Machine Learning at the University of Tuebingen , co-supervised by Zeynep Akata (University of Tuebingen) and Oriol Vinyals (Google Deepmind, London).

My current research focuses on effective generalization in representation learning, for example through self-supervised, contrastive or metric learning. A particular focus is placed on developing systems capable of strong out-of-distribution generalization for applications ranging from zero- to few-shot and continual learning tasks.

I have also spent time as a research scientist intern at Meta AI Paris [Diane Bouchacourt, Pascal Vincent & Mark Ibrahim], and as a research intern at Amazon AWS [Peter Gehler, Thomas Brox], Vector [Marzyeh Ghassemi] and MILA [Joseph Paul Cohen, Yoshua Bengio].

Pre-ML I completed my Master and Bachelor of Physics at Heidelberg University with initial focus on Medical and Solid State Physics.

Reviewed at MICCAI, IJCAI, MIDL, AutoML, wNeurIPS, CVIU, CVPR, ECCV, TPAMI.
Outstanding reviewer CVPR 2022, ECCV 2022.


I'm always open to collaborations or project supervisions! Just drop me a message :).

Worked/Currently working with
Jessica Bader (Master Student): Sketch-based Image Retrieval.
Madhav Iyengar (Master Student): Self-Supervised Learning.
Yavuz Durmazkeser (Master Student): Deep Metric Learning.
Lukas Thede (Research Scientist): Continual Distillation.
Stefan Wezel (Master Student): Anomaly Detection.
Zafir Stojanovski (Master Student): Continual Learning and Adaptation.

  Papers (click to collapse)

Disentanglement of Correlated Factors via Hausdorff Factorized Support
We tackle the problem of disentangled representation learning without the unrealistic assumption of independent factors of variation, i.e. we allow for correlation in the training data. We achieve this through a novel, Hausdorff-distance based objective by factorizing the support instead of the distribution over latents.
Karsten Roth, Mark Ibrahim, Zeynep Akata, Pascal Vincent*, Diane Bouchacourt*
ICLR, 2023
arXiv
Momentum-based Weight Interpolation of Strong Zero-Shot Models for Continual Learning
This work showcases that foundation models suffer from continual adaptation, which can be fixed surprisingly easily by retaining a weight-interpolated momentum version, without the need to re-integrate the momentum network back into training.
Zafir Stojanovski*, Karsten Roth*, Zeynep Akata
INTERPOLATE Workshop @ NeurIPS 2022 (Best Paper)
arXiv
A Non-isotropic Probabilistic Take on Proxy-based Deep Metric Learning
In this work, we propose a novel probabilistic approach to Deep Metric Learning by describing embeddings and proxies as (non-isotropic) distributions, and the problem of metric learning as that of distribution matching.
Michael Kirchhof*, Karsten Roth*, Zeynep Akata, Enkelejda Kasneci
ECCV, 2022
arXiv
(ORAL) Integrating Language Guidance into Vision-based Deep Metric Learning
We showcase the benefits of re-aligning visual similarity spaces using language semantics, without the need of additional expert supervision and with significant improvements in generalization performance.
Karsten Roth, Oriol Vinyals, Zeynep Akata
CVPR, 2022
arXiv | code
Non-isotropy Regularization for Proxy-based Deep Metric Learning
This work showcases the benefits of resolving local structures in proxy-based Deep Metric Learning without sample-to-sample relations. Doing so retains incredibly fast convergence speeds while ensuring strong generalization performance.
Karsten Roth, Oriol Vinyals, Zeynep Akata
CVPR, 2022
arXiv | code
Towards Total Recall in Industrial Anomaly Detection
We develop PatchCore - an anomaly detection method for visual product inspection, which is scalable, fast, extremely accurate, interpretable and usable without expert knowledge. Using Coreset Memories, PatchCore has retained the state-of-the-art for more than a year now.
Karsten Roth, Latha Pemula, Joaquin Zepeda, Bernhard Schoelkopf, Thomas Brox, Peter Gehler
CVPR, 2022
arXiv | code
Improving the Fairness of Chest X-ray Classifiers
Fairness methods for chest x-ray classifier that account for worst-group performance do not outperform simple data balancing, and methods which achieve group fairness do so by worsening performance for all groups.
Haoran Zhang, Natalie Dullerud, Karsten Roth, Lauren Oakden-Rayner, Stephen Pfohl, Marzyeh Ghassemi
CHIL, 2022
arXiv | proceedings | code | bibtex
Is fairness only metric deep? evaluating and addressing subgroup gaps in deep metric learning
This work proposes finDML, the fairness in non-balanced DML benchmark to characterize representation fairness. We find bias in DML representations to propagate to common downstream classification tasks, even when training data in the downstream task is re-balanced, and propose a regularizer to tackle this.
Natalie Dullerud, Karsten Roth, Kimia Hamidieh, Nicolas Papernot, Marzyeh Ghassemi
ICLR, 2022
arXiv | proceedings | bibtex
Temporal control of the integrated stress response by a stochastic molecular "switch"
Five year interdisciplinary project studying and evaluating integrated cellular stress response.
Philipp Klein, Stefan M.Kallenberger, Hanna Roth, Karsten Roth, Thi Bach Nga Ly-Hartig, Vera Magg, Janez Aleš, Soheil Rastgou Talemi, Yu Qiang, Steffen Wolf, Olga Oleksiuk, Roma Kurilov, Barbara Di Ventura, Ralf Bartenschlager, Roland Eils, Karl Rohr, Fred A. Hamprecht, Thomas Höfer, Oliver T. Fackler, Georg Stoecklin, Alessia Ruggieri
Science Advances, 2022
Science
Characterizing generalization under out-of-distribution shifts in deep metric learning
This work proposes a new and controllable benchmark for more principled studies of the Out-of-Distribution performance of Deep Metric Learning Models.
Timo Milbich*, Karsten Roth*, Samarth Sinha, Ludwig Schmidt, Marzyeh Ghassemi*, Bjoern Ommer*
NeurIPS, 2021
arXiv | proceedings | bibtex | code
Simultaneous Similarity-based Self-Distillation for Deep Metric Learning
Based on the insight that the dimensionality of metric spaces faciliates regularization, we introduce S2SD to implicitly incorporate high-dimensional rankings into low-dimensional embedding spaces.
Karsten Roth, Timo Milbich, Bjoern Ommer, Joseph Paul Cohen, Marzyeh Ghassemi
ICML, 2021
arXiv | proceedings | bibtex | code
DiVA: Diverse Visual Feature Aggregation for Deep Metric Learning
DiVA introduces a plethora of feature mining extensions including concurrent self-supervised learning with no additional supervision to improve the generalization performance of supervised visual similarity spaces.
Timo Milbich*, Karsten Roth*, Homanga Bharadhwaj, Samarth Sinha, Yoshua Bengio, Bjoern Ommer, Joseph Paul Cohen
ECCV, 2020
arXiv | proceedings | bibtex | code
Revisiting Training Strategies and Generalization Performance in Deep Metric Learning
A seminal project that highlights significant performance saturation in Deep Metric Learning research and the reasons, as well as an initial study into structural drivers of generalization in Deep Metric Learning.
Karsten Roth*, Timo Milbich*, Samarth Sinha, Prateek Gupta, Bjoern Ommer, Joseph Paul Cohen
ICML, 2020
arXiv | proceedings | bibtex | code
Pads: Policy-adapted sampling for visual similarity learning
Using Reinforcement Learning, PADS introduces a tuple mining heuristic that presents the network the right tuples to learn from at the right time.
Karsten Roth*, Timo Milbich*, Bjoern Ommer
CVPR, 2020
arXiv | proceedings | bibtex | code
Predicting covid-19 pneumonia severity on chest x-ray with deep learning
This study presents a severity score model for COVID-19 pneumonia from frontal chest X-ray images.
Joseph Paul Cohen, Lan Dao, Karsten Roth, Paul Morrison, Yoshua Bengio, Almas F Abbasi, Beiyi Shen, Hoshmand Kochi Mahsa, Marzyeh Ghassemi, Haifang Li, Tim Duong
Cureus, 2020
Cureus
Covid-19 image data collection: Prospective predictions are the future
This was the first chest x-ray image data collection for COVID-19, manually aggregated to provide tools for a first exploratory analysis into COVID-19 prediction from chest x-ray.
Joseph Paul Cohen, Paul Morrison, Lan Dao, Karsten Roth, Tim Q Doung, Marzyeh Ghassemi
MELBA, 2020
arXiv
Diffuse domain method for needle insertion simulations
We present a new strategy for needle insertion simulations without the necessity of meshing utilizing a diffuse domain approach.
Katharina I Jerg, René Phillip Austermühl, Karsten Roth, Jonas Große Sundrup, Guido Kanschat, Jürgen W Hesser, Lisa Wittmayer
International Journal for Numerical Methods in Biomedical Engineering, 2020
Wiley
MIC: Mining Interclass Characteristics for Improved Metric Learning
We extend standard supervised Deep Metric Learning with a intra-class feature mining step that allows the network to explain away feature variances within classes for improved generalization.
Karsten Roth*, Biagio Brattoli*, Bjoern Ommer
ICCV, 2019
arXiv | proceedings | bibtex | code
Boosting Liver and Lesion Segmentation from CT Scans by Mask Mining
We propose a novel boosting procedure to improve liver and lesion segmentation from CT scans for U-Net based models based on an iterative error prediction and refinement setup.
Karsten Roth*, Jürgen Hesser, Tomasz Konopczyński
Medical Imaging Workshop NeurIPS, 2019
Paper

  Events (click to collapse)

IMPRS-IS Interview Symposium [Jan 2023]
Presented my research on representation learning under distribution shifts to PhD applicants at the IMPRS-IS PhD Interview Symposium.
Remote

ELLIS [Oct 2022]
Talked about my ELLIS PhD as part of the official ELLIS PhD Program trailer.
Germany

Finalist Qualcomm Innovation Fellowship [June 2022]
Finalist for the Qualcomm Innovation PhD Fellowship award 2022.
Amsterdam, Netherlands

Pioneer AI Center [May 2022]
Presentation of CVPR 2022 publications on language guidance and normalizing flows for deep metric learning.
Copenhagen, Danemark

EMVA Young Professional Award [May 2022]
Recipient of the Young Professional Award 2022 endowed by the European Machine Vision Association to honor the outstanding and innovative work of a student or a young professional in the field of machine vision or image processing.
Brussels, Belgium

G-Research [March 2022]
Introduction to Deep Metric Learning and presentation of interesting research directions in this field.
Remote

Ruhr University Bochum [Jan 2022]
On Out-of-Distribution Generalization in zero-shot Deep Metric Learning.
Remote

Bright Machines [Oct 2021]
Autonomous Industrial Anomaly Detection using PatchCore.
Remote

Chaudhari Group, Stanford University [Jul 2021]
Addressing the shortcomings in Deep Metric Learning research.
Remote

MLSS Tuebingen [Jun/Jul 2020]
Selected as participant to the Machine Learning Summer School in Tuebingen, Germany.
Remote