Shreyas Padhy
Email: sp2058 (at) cam.ac.uk
         

I am a PhD student in the Machine Learning Group at the University of Cambridge, supervised by Dr. Jośe Miguel Hernández-Lobato. My research interests lie in massively scalable Bayesian inference and probabilistic machine learning, uncertainty estimation and out-of-distribution detection, and diffusion models and sampling.

Previously, I was an AI Resident in the Google Brain team at Cambridge, MA, where I worked with Jasper Snoek and Balaji Lakshminarayanan on probabilistic machine learning methods.

I received my Masters in Biomedical Engineering from Johns Hopkins University, where I worked with Dr, Michael I. Miller on using machine learning and computational anatomy to understand the pathology of neurodegenerative diseases such as Alzheimers and schizophrenia using diffeomorphic mapping of brain MRI scans. I received my Bachelors in Engineering Physics at IIT Delhi, where I worked with Dr. Uday Khankhoje on stochastic solutions to forward and inverse imaging.

Recent Updates

June 2024: I will be a Research Scientist intern in Meta Reality Labs in Burlingame!

June 2024: We have some really exciting SOTA inverse problem results with our new method DEFT, a way to finetune diffusion models for conditional generation [paper] [code]

June 2024: We show that you can speed up marginal likelihood optimization of GPs by over 30x using warm starts and pathwise sampling [paper]

May 2024: I presented a poster at ICLR 2024 in Vienna on Controlled Monte Carlo Diffusions [paper] [code]

May 2024: I presented a poster at ICLR 2024 in Vienna on Improved Stochastic Gradient Descent for GPs [paper] [code]

February 2024: I gave an invited talk at the SIAM UQ Conference in Trieste on SGD for large-scale Bayesian ML [event] [slides]

February 2024: I gave an invited talk at Atinary Technologies in Lausanne on SGD for GPs [venue] [slides]

February 2024: I gave a guest lecture on Conditioning in SDEs [slides]

December 2023: I gave an invited talk at the NeurIPS@Cambridge meetup on our NeurIPS Oral [event] [paper]

Selected Publications
Conferences

Transport meets Variational Inference: Controlled Monte Carlo Diffusions
Francisco Vargas*, Shreyas Padhy*, Denis Blessing, Nikolas Nüsken.
ICLR 2024 [paper] [code]

Stochastic Gradient Descent for Gaussian Processes Done Right
Shreyas Padhy*, Jihao Andreas Lin*, Javier Antorán*, Austin Tripp, Alexander Terenin, Csaba Szepesvári, José Miguel Hernández-Lobato, David Janz
ICLR 2024 [paper] [code]

Sampling from Gaussian Process Posteriors using Stochastic Gradient Descent
Shreyas Padhy*, Jihao Andreas Lin*, Javier Antorán*, David Janz, José Miguel Hernández-Lobato, Alexander Terenin.
NeurIPS 2023 (Oral) [code] [paper]

Sampling-based inference for large linear models, with application to linearised Laplace
Shreyas Padhy*, Javier Antorán*, Riccardo Barbano, Eric Nalisnick, David Janz, José Miguel Hernández-Lobato
ICLR 2023 [code] [paper] [slides] [poster]

Simple and principled uncertainty estimation with deterministic deep learning via distance awareness
Jeremiah Zhe Liu, Zi Lin, Shreyas Padhy, Dustin Tran, Tania Bedrax-Weiss, and Balaji Lakshminarayanan.
NeurIPS 2020 [code] [paper]

Preprints

DEFT: Efficient Finetuning of Conditional Diffusion Models by Learning the Generalised h-transform
Shreyas Padhy*, Alexander Denker*, Francisco Vargas*, Kieran Didi*, Simon Mathis*, Vincent Dutordoir, Riccardo Barbano, Emile Mathieu, Urszula Julia Komorowska, Pietro Lio
[paper]

Improving Linear System Solvers for Hyperparameter Optimisation in Iterative Gaussian Processes
Jihao Andreas Lin, Shreyas Padhy, Bruno Mlodozeniec, Javier Antorán, José Miguel Hernández-Lobato
[paper]

Kernel Regression with Infinite-Width Neural Networks on Millions of Examples
Ben Adlam, Jaehoon Lee, Shreyas Padhy, Zachary Nado, Jasper Snoek.
[paper]

Journals

A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awareness
Jeremiah Zhe Liu*, Shreyas Padhy*, Jie Ren*, Zi Lin, Yeming Wen, Ghassen Jerfel, Zack Nado, Jasper Snoek, Dustin Tran, Balaji Lakshminarayanan
JMLR 2023 [code] [paper] [poster]

Workshops

Revisiting One-vs-All Classifiers for Predictive Uncertainty and Out-of-Distribution Detection in Neural Networks
Shreyas Padhy, Zachary Nado, Jie Ren, Jeremiah Liu, Jasper Snoek, and Balaji Lakshminarayanan.
ICML 2020 Workshop on Uncertainty and Robustness in Deep Learning.
[code] [paper]

A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detection
Jie Ren, Stanislav Fort, Jeremiah Liu, Abhijit Guha Roy, Shreyas Padhy, and Balaji Lakshminarayanan
ICML 2021 Workshop on Uncertainty and Robustness in Deep Learning.
[paper]

Evaluating prediction-time batch normalization for robustness under covariate shift
Zachary Nado, Shreyas Padhy, D. Sculley, Alexander D'Amour, Balaji Lakshminarayanan, and Jasper Snoek.
ICML 2020 Workshop on Uncertainty and Robustness in Deep Learning.
[code] [paper]

Uncertainty Baselines: Benchmarks for Uncertainty & Robustness in Deep Learning
Zachary Nado at al.
Bayesian Deep Learning Workshop, 2021.
[code] [paper]

Template: Everyone here, but specifically this version