Momentum-based Weight Interpolation of Strong Zero-Shot Models for Continual Learning
This work showcases that foundation models suffer from continual adaptation, which can be fixed surprisingly easily by retaining a weight-interpolated momentum version, without the need to re-integrate the momentum network back into training.
Zafir Stojanovski*, Karsten Roth*, Zeynep Akata
INTERPOLATE Workshop @ NeurIPS 2022 (Spotlight)
arXiv

Disentanglement of Correlated Factors via Hausdorff Factorized Support
We tackle the problem of disentangled representation learning without the unrealistic assumption of independent factors of variation, i.e. we allow for correlation in the training data. We achieve this through a novel, Hausdorff-distance based objective by factorizing the support instead of the distribution over latents.
Karsten Roth, Mark Ibrahim, Zeynep Akata, Pascal Vincent*, Diane Bouchacourt*
Under submission.
arXiv

A Non-isotropic Probabilistic Take on Proxy-based Deep Metric Learning
In this work, we propose a novel probabilistic approach to Deep Metric Learning by describing embeddings and proxies as (non-isotropic) distributions, and the problem of metric learning as that of distribution matching.
Michael Kirchhof*, Karsten Roth*, Zeynep Akata, Enkelejda Kasneci
ECCV, 2022
arXiv

(ORAL) Integrating Language Guidance into Vision-based Deep Metric Learning
We showcase the benefits of re-aligning visual similarity spaces using language semantics, without the need of additional expert supervision and with significant improvements in generalization performance.
Karsten Roth, Oriol Vinyals, Zeynep Akata
CVPR, 2022
arXiv | code

Non-isotropy Regularization for Proxy-based Deep Metric Learning
This work showcases the benefits of resolving local structures in proxy-based Deep Metric Learning without sample-to-sample relations. Doing so retains incredibly fast convergence speeds while ensuring strong generalization performance.
Karsten Roth, Oriol Vinyals, Zeynep Akata
CVPR, 2022
arXiv | code

Towards Total Recall in Industrial Anomaly Detection
We develop PatchCore - an anomaly detection method for visual product inspection, which is scalable, fast, extremely accurate, interpretable and usable without expert knowledge. Using Coreset Memories, PatchCore has retained the state-of-the-art for more than a year now.
Karsten Roth, Latha Pemula, Joaquin Zepeda, Bernhard Schoelkopf, Thomas Brox, Peter Gehler
CVPR, 2022
arXiv | code

Is fairness only metric deep? evaluating and addressing subgroup gaps in deep metric learning
This work proposes finDML, the fairness in non-balanced DML benchmark to characterize representation fairness. We find bias in DML representations to propagate to common downstream classification tasks, even when training data in the downstream task is re-balanced, and propose a regularizer to tackle this.
Natalie Dullerud, Karsten Roth, Kimia Hamidieh, Nicolas Papernot, Marzyeh Ghassemi
ICLR, 2022
arXiv | proceedings | bibtex

Temporal control of the integrated stress response by a stochastic molecular "switch"
Five year interdisciplinary project studying and evaluating integrated cellular stress response.
Philipp Klein, Stefan M.Kallenberger, Hanna Roth, Karsten Roth, Thi Bach Nga Ly-Hartig, Vera Magg, Janez Aleš, Soheil Rastgou Talemi, Yu Qiang, Steffen Wolf, Olga Oleksiuk, Roma Kurilov, Barbara Di Ventura, Ralf Bartenschlager, Roland Eils, Karl Rohr, Fred A. Hamprecht, Thomas Höfer, Oliver T. Fackler, Georg Stoecklin, Alessia Ruggieri
Science Advances, 2022
Science

Characterizing generalization under out-of-distribution shifts in deep metric learning
This work proposes a new and controllable benchmark for more principled studies of the Out-of-Distribution performance of Deep Metric Learning Models.
Timo Milbich*, Karsten Roth*, Samarth Sinha, Ludwig Schmidt, Marzyeh Ghassemi*, Bjoern Ommer*
NeurIPS, 2021
arXiv | proceedings | bibtex | code

Simultaneous Similarity-based Self-Distillation for Deep Metric Learning
Based on the insight that the dimensionality of metric spaces faciliates regularization, we introduce S2SD to implicitly incorporate high-dimensional rankings into low-dimensional embedding spaces.
Karsten Roth, Timo Milbich, Bjoern Ommer, Joseph Paul Cohen, Marzyeh Ghassemi
ICML, 2021
arXiv | proceedings | bibtex | code

DiVA: Diverse Visual Feature Aggregation for Deep Metric Learning
DiVA introduces a plethora of feature mining extensions including concurrent self-supervised learning with no additional supervision to improve the generalization performance of supervised visual similarity spaces.
Timo Milbich*, Karsten Roth*, Homanga Bharadhwaj, Samarth Sinha, Yoshua Bengio, Bjoern Ommer, Joseph Paul Cohen
ECCV, 2020
arXiv | proceedings | bibtex | code

Revisiting Training Strategies and Generalization Performance in Deep Metric Learning
A seminal project that highlights significant performance saturation in Deep Metric Learning research and the reasons, as well as an initial study into structural drivers of generalization in Deep Metric Learning.
Karsten Roth*, Timo Milbich*, Samarth Sinha, Prateek Gupta, Bjoern Ommer, Joseph Paul Cohen
ICML, 2020
arXiv | proceedings | bibtex | code

Pads: Policy-adapted sampling for visual similarity learning
Using Reinforcement Learning, PADS introduces a tuple mining heuristic that presents the network the right tuples to learn from at the right time.
Karsten Roth*, Timo Milbich*, Bjoern Ommer
CVPR, 2020
arXiv | proceedings | bibtex | code

Predicting covid-19 pneumonia severity on chest x-ray with deep learning
This study presents a severity score model for COVID-19 pneumonia from frontal chest X-ray images.
Joseph Paul Cohen, Lan Dao, Karsten Roth, Paul Morrison, Yoshua Bengio, Almas F Abbasi, Beiyi Shen, Hoshmand Kochi Mahsa, Marzyeh Ghassemi, Haifang Li, Tim Duong
Cureus, 2020
Cureus

Covid-19 image data collection: Prospective predictions are the future
This was the first chest x-ray image data collection for COVID-19, manually aggregated to provide tools for a first exploratory analysis into COVID-19 prediction from chest x-ray.
Joseph Paul Cohen, Paul Morrison, Lan Dao, Karsten Roth, Tim Q Doung, Marzyeh Ghassemi
MELBA, 2020
arXiv

Diffuse domain method for needle insertion simulations
We present a new strategy for needle insertion simulations without the necessity of meshing utilizing a diffuse domain approach.
Katharina I Jerg, René Phillip Austermühl, Karsten Roth, Jonas Große Sundrup, Guido Kanschat, Jürgen W Hesser, Lisa Wittmayer
International Journal for Numerical Methods in Biomedical Engineering, 2020
Wiley

MIC: Mining Interclass Characteristics for Improved Metric Learning
We extend standard supervised Deep Metric Learning with a intra-class feature mining step that allows the network to explain away feature variances within classes for improved generalization.
Karsten Roth*, Biagio Brattoli*, Bjoern Ommer
ICCV, 2019
arXiv | proceedings | bibtex | code

Boosting Liver and Lesion Segmentation from CT Scans by Mask Mining
We propose a novel boosting procedure to improve liver and lesion segmentation from CT scans for U-Net based models based on an iterative error prediction and refinement setup.
Karsten Roth*, Jürgen Hesser, Tomasz Konopczyński
Medical Imaging Workshop NeurIPS, 2019
Paper