Research
I'm interested in machine learning, computational neuroscience, information theory, computer vision, optimization, and cycling.
My current focus is on generative models that enable new creative applications.
Your browser does not support the video tag.
ReconFusion: 3D Reconstruction with Diffusion Priors
Rundi Wu *,
Ben Mildenhall *,
Keunhong Park ,
Philipp Henzler ,
Ruiqi Gao ,
Daniel Watson ,
Dor Verbin ,
Pratul Srinivasan ,
Jonathan T. Barron ,
Ben Poole ,
Aleksander Holynski *
Preprint 2023
project page /
arXiv
3D reconstruction of real-world scenes from only a few photos
Diffusion Self-Guidance for Controllable Image Generation
Dave Epstein , Allan Jabri , Ben Poole , Alexei A. Efros , Aleksander Holynski
NeurIPS 2023
project page / arXiv
Self-guidance is a method for controllable image generation that guides sampling using only the attention and activations of a pretrained diffusion model.
Your browser does not support the video tag.
DreamBooth3D: Subject-Driven Text-to-3D Generation
Amit Raj , Srinivas Kaza , Ben Poole , Michael Niemeyer , Nataniel Ruiz ,
Ben Mildenhall , Shiran Zada , Kfir Aberman , Michael Rubinstein ,
Jonathan T. Barron , Yuanzhen Li , Varun Jampani
ICCV 2023
project page /
arXiv
Combining DreamBooth (personalized text-to-image) and DreamFusion (text-to-3D) yields high-quality, subject-specific 3D assets with text-driven modifications
Learning a Diffusion Prior for NeRFs
Guandao Yang ,
Abhijit Kundu ,
Leonidas Guibas ,
Jonathan Barron ,
Ben Poole
ICLR Neural Fields Workshop 2023
Learn a regularized set of NeRFs in parallel, then learn a 3D diffusion model that can generate new NeRFs.
Your browser does not support the video tag.
DreamFusion: Text-to-3D using 2D Diffusion
Ben Poole ,
Ajay Jain ,
Jonathan T. Barron ,
Ben Mildenhall
ICLR 2023 (Outsanding Paper Award)
project page
/
arXiv
/
gallery
We optimize a NeRF from scratch using a pretrained text-to-image diffusion model to do text-to-3D generative modeling.
Your browser does not support the video tag.
Imagen Video: High Definition Video Generation with Diffusion Models
Jonathan Ho *,
William Chan *,
Chitwan Saharia *,
Jay Whang *,
Ruiqi Gao ,
Alexey Gritsenko ,
Diederik P. Kingma ,
Ben Poole ,
Mohammad Norouzi ,
David J. Fleet ,
Tim Salimans *
Preprint 2022
arXiv / project page
A general framework for training and sampling from score-based models enabling likelihood computation and controllable generaiton.
Your browser does not support the video tag.
Journey to the BAOAB-limit: finding effective MCMC samplers for score-based models
Ajay Jain *,
Ben Poole *
NeurIPS 2022 Score-Based Models Workshop
project page
/
paper
Sometimes bugs are effective MCMC samplers for score-based models.
Your browser does not support the video tag.
Zero-Shot Text-Guided Object Generation with Dream Fields
Ajay Jain ,
Ben Mildenhall ,
Jonathan T. Barron ,
Pieter Abbeel ,
Ben Poole
CVPR 2022
project page
/
arXiv
/
video
Supervising the CLIP embeddings of NeRF renderings lets you to generate 3D objects from text prompts.
Autoregressive Diffusion Models
Emiel Hoogeboom ,
Alexey Gritsenko ,
Jasmijn Bastings ,
Ben Poole ,
Rianne van den Berg ,
Tim Salimans
ICLR 2022
arXiv
A new model class for discrete variables encompassing order agnostic autoregressive models and absorbing discrete diffusion.
VeLO: Training Versatile Learned Optimizers by Scaling Up
Luke Metz , James Harrison , C. Daniel Freeman , Amil Merchant , Lucas Beyer , James Bradbury , Naman Agarwal , Ben Poole , Igor Mordatch , Adam Roberts , Jascha Sohl-Dickstein
Preprint 2022
arXiv / code
A general purpose learned optimizer.
Variational Diffusion Models
Diederik P. Kingma *,
Tim Salimans *,
Ben Poole ,
Jonathan Ho
NeurIPS 2021
arXiv / code
SOTA likelihood using diffusion models with learnable noise schedule
Score-Based Generative Modeling through Stochastic Differential Equations
Yang Song ,
Jascha Sohl-Dickstein ,
Diederik P. Kingma ,
Abhishek Kumar ,
Stefano Ermon ,
Ben Poole
ICLR 2021 (Outstanding Paper Award)
arXiv / code
A general framework for training and sampling from score-based models enabling likelihood computation and controllable generaiton.
Learning Energy-Based Models by Diffusion Recovery Likelihood
Ruiqi Gao ,
Yang Song ,
Ben Poole ,
Ying Nian Wu
Diederik P. Kingma
ICLR 2021
arXiv
Tractably learn and sample from a sequence of EBMs based on a diffusion process. High sample quality and stable long-run MCMC chains.
What Makes for Good views for Contrastive Learning?
Yonglong Tian ,
Chen Sun ,
Ben Poole ,
Dilip Krishnan ,
Cordelia Schmid ,
Philip Isola
NeurIPS 2020
arXiv / project page
Weakly-Supervised Disentanglement Without Compromises
Francesco Locatello ,
Ben Poole ,
Gunnar Rätsch ,
Bernhard Schölkopf ,
Olivier Bachem ,
Michael Tschannen
ICML 2020
arXiv
With a causality-inspired twist, disentangled representations are identifiable in theory and practice.
On Implicit Regularization in β-VAEs
Abhishek Kumar ,
Ben Poole
ICML 2020
arXiv
Heuristics in VAEs can lead to uniqueness and beneficial regularization.
VIB is Half Bayes
Alex Alemi ,
Warren Morningstar ,
Ben Poole ,
Ian Fischer ,
Josh Dillon
AABI Symposium 2020
arXiv
The Variational Information Bottleneck can rederived as Half-Bayesian.
On variational bounds of mutual information
Ben Poole , Sherjil Ozair , Aäron van den Oord , Alex Alemi , George Tucker
ICML 2019
arXiv / colab / video / slides / poster
Old, new, and improved estimators of mutual information w/neural nets.
Discrete Flows: Invertible Generative
Models of Discrete Data
Dustin Tran , Keyon Vafa , Kumar Krishna Agrawal , Laurent Dinh , Ben Poole
ICLR Deep GenStruct Workshop 2019
arXiv
Fast sampling generative models for discrete data.
Preventing posterior collapse with delta-VAEs
Ali Razavi , Aäron van den Oord , Ben Poole , Oriol Vinyals
ICLR 2019
OpenReview / arXiv / poster
Avoid posterior collapse by lower bounding the rate.
Neuronal Dynamics Regulating Brain and Behavioral State Transitions
Aaron Andalman, Vanessa Burns, Matthew Lovett-Barron, Michael Broxton , Ben Poole , Samuel Yang, Logan Grosenick , Talia Lerner, Ritchie Chen, Tyler Benster, Philippe Mourrai, Marc Levoy , Kanaka Rajan , Karl Deisseroth
Cell 2019
Fixing a Broken ELBO
Alex Alemi , Ben Poole , Ian Fischer ,
Joshua V. Dillon, Rif A. Saurous , Kevin Murphy
ICML , 2018
arXiv
Understanding tradeoffs in VAE models through the lens of information theory.
Continuous relaxation training of discrete latent-variable image models
Casper Kaae Sønderby* , Ben Poole *, Andriy Mnih
NIPS Bayesian Deep Learning Workshop , 2017
PDF
Continuous relaxation training of discrete latent-variable models can flexibly capture both continuous and discrete aspects of natural data.
Identification Of Cellular-Activity Dynamics Across Large Tissue Volumes In The Mammalian Brain
Logan Grosenick *, Michael Broxton *, Christina Kim*, Conor Liston*, Ben Poole , Samuel Yang, Aaron Andalman, Edward Scharff, Noy Cohen, Ofer Yizhar, Charu Ramakrishnan, Surya Ganguli , Patrick Suppes, Marc Levoy , Karl Deisseroth
*equal contribution
bioRxiv
Large-scale cellular-level imaging in the mammalian brain using lightfield microscopy. 1x1x0.5mm3 @ 100Hz.
Continual Learning through Synaptic Intelligence
Friedemann Zenke *, Ben Poole *, Surya Ganguli
*equal contribution
ICML , 2017
arXiv / code
Learns to solve tasks sequentially without forgetting by learning which weights are important.
On the expressive power of deep neural networks
Maithra Raghu , Ben Poole , Jon Kleinberg , Surya Ganguli , Jascha Sohl-Dickstein
ICML , 2017
arXiv
Random neural networks show exponential growth in activation patterns and more.
Time-warped PCA: simultaneous alignment and dimensionality reduction of neural data
Ben Poole , Alex Williams , Niru Maheswaranathan , Byron Yu , Gopal Santhanam, Stephen Ryu, Stephen Baccus , Krishna Shenoy , Surya Ganguli
Computational Systems Neuroscience (COSYNE) , 2017
abstract / poster / code
Extends dimensionality reduction techniques to account for trial-to-trial variability in timing.
Categorical Reparameterization with Gumbel-Softmax
Eric Jang , Shane Gu , Ben Poole
ICLR , 2017
arXiv
/
blog post
Efficient gradient estimator for categorical variables.
Unrolled Generative Adversarial Networks
Luke Metz , Ben Poole , David Pfau , Jascha Sohl-Dickstein
ICLR , 2017
arXiv
/
code
Stabilize GANs by defining the generator objective with respect to an unrolled optimization of the discriminator.
Adversarially learned inference
Vincent Dumoulin , Ishmael Belghazi, Ben Poole , Alex Lamb, Martin Arjovsky, Olivier Mastropietro, Aaron Courville
ICLR , 2017
arXiv
/
project page
Jointly learn a generative model and an inference network through an adversarial process.
Exponential expressivity in deep neural networks through transient chaos
Ben Poole , Subhaneil Lahiri , Maithra Raghu , Jascha Sohl-Dickstein , Surya Ganguli
Neural Information Processing Systems (NIPS) , 2016
arXiv
/
code
/
poster
Random neural networks have curvature that grows exponentially with depth.
Improved generator objectives for GANs
Ben Poole , Alex Alemi , Jascha Sohl-Dickstein , Anelia Angelova
NIPS Workshop on Adversarial Training , 2016
arXiv
/
poster
A new take on the Generative Adversarial Network training procedure.
Direction Selectivity in Drosophila Emerges from Preferred-Direction Enhancement and Null-Direction Suppression
Jonathan Leong*, Jennifer Esch*, Ben Poole *, Surya Ganguli , Thomas Clandinin
* equal contribution
The Journal of Neuroscience , 2016
Fruit flies detect motion using a very similar algorithm to humans.
The Fast Bilateral Solver
Jonathan T. Barron , Ben Poole
European Conference on Computer Vision (ECCV) , 2016 (oral presentation)
arXiv
/
supplement
/
code
/
bibtex
Fast and accurate edge-aware smoothing. Differentiable for all your deep learning needs.
Fast large-scale optimization by unifying stochastic gradient and quasi-newton methods
Jascha Sohl-Dickstein , Ben Poole , Surya Ganguli
International Conference on Machine Learning (ICML) , 2014
arXiv /
code
Speed up quasi-newton methods by maintaining a low-dimensional approximation of the Hessian for each minibatch.
Analyzing noise in auotoencoders and deep networks
Ben Poole , Jascha Sohl-Dickstein , Surya Ganguli
NIPS Workshop on Deep Learning , 2013
arXiv
Derives analytic regularizers for different forms of noise injection, and shows how alternative types of additive noise can improve over dropout.
Brain Regions Engaged by Part- and Whole-task Performance in a Video Game: A Model-based Test of the Decomposition Hypothesis
John Anderson, Daniel Bothell, Jon Fincham, Abraham Anderson, Ben Poole , Yulin Qin
Journal of Cognitive Neuroscience , 2011
Complex tasks, like the Space Fortress video game, can be decomposed into a set of independent reusable components.
Robust non-rigid alignment of volumetric calcium imaging data
Ben Poole , Logan Grosenick , Michael Broxton , Karl Deisseroth , Surya Ganguli
Computational Systems Neuroscience (COSYNE) , 2015
poster
Correct for translations and rotations of noisy volumetric data without a clean reference volume.
Connecting scene statistics to probabilistic population codes and tuning properties of V1 neurons
Ben Poole , Ian Lenz, Grace Linsday, Jason Samonds, Tai Sing Lee
Society for Neuroscience (SFN) , 2010 (oral presentation)