Research
I'm interested in machine learning, computational neuroscience, information theory, computer vision, optimization, and cycling.
My current focus is on generative models that enable new creative applications.
Diffusion Self-Guidance for Controllable Image Generation
Dave Epstein , Allan Jabri , Ben Poole , Alexei A. Efros , Aleksander Holynski
Preprint 2023
project page / arXiv
Self-guidance is a method for controllable image generation that guides sampling using only the attention and activations of a pretrained diffusion model.
Your browser does not support the video tag.
DreamBooth3D: Subject-Driven Text-to-3D Generation
Amit Raj , Srinivas Kaza , Ben Poole , Michael Niemeyer , Nataniel Ruiz ,
Ben Mildenhall , Shiran Zada , Kfir Aberman , Michael Rubinstein ,
Jonathan T. Barron , Yuanzhen Li , Varun Jampani
ICCV 2023
project page /
arXiv
Combining DreamBooth (personalized text-to-image) and DreamFusion (text-to-3D) yields high-quality, subject-specific 3D assets with text-driven modifications
Learning a Diffusion Prior for NeRFs
Guandao Yang ,
Abhijit Kundu ,
Leonidas Guibas ,
Jonathan Barron ,
Ben Poole
ICLR Neural Fields Workshop 2023
Learn a regularized set of NeRFs in parallel, then learn a 3D diffusion model that can generate new NeRFs.
Your browser does not support the video tag.
DreamFusion: Text-to-3D using 2D Diffusion
Ben Poole ,
Ajay Jain ,
Jonathan T. Barron ,
Ben Mildenhall
ICLR 2023 (Outsanding Paper Award)
project page
/
arXiv
/
gallery
We optimize a NeRF from scratch using a pretrained text-to-image diffusion model to do text-to-3D generative modeling.
Your browser does not support the video tag.
Imagen Video: High Definition Video Generation with Diffusion Models
Jonathan Ho *,
William Chan *,
Chitwan Saharia *,
Jay Whang *,
Ruiqi Gao ,
Alexey Gritsenko ,
Diederik P. Kingma ,
Ben Poole ,
Mohammad Norouzi ,
David J. Fleet ,
Tim Salimans *
Preprint 2022
arXiv / project page
A general framework for training and sampling from score-based models enabling likelihood computation and controllable generaiton.
Your browser does not support the video tag.
Zero-Shot Text-Guided Object Generation with Dream Fields
Ajay Jain ,
Ben Mildenhall ,
Jonathan T. Barron ,
Pieter Abbeel ,
Ben Poole
CVPR 2022
project page
/
arXiv
/
video
Supervising the CLIP embeddings of NeRF renderings lets you to generate 3D objects from text prompts.
Autoregressive Diffusion Models
Emiel Hoogeboom ,
Alexey Gritsenko ,
Jasmijn Bastings ,
Ben Poole ,
Rianne van den Berg ,
Tim Salimans
ICLR 2022
arXiv
A new model class for discrete variables encompassing order agnostic autoregressive models and absorbing discrete diffusion.
Variational Diffusion Models
Diederik P. Kingma *,
Tim Salimans *,
Ben Poole ,
Jonathan Ho
NeurIPS 2021
arXiv / code
SOTA likelihood using diffusion models with learnable noise schedule
Score-Based Generative Modeling through Stochastic Differential Equations
Yang Song ,
Jascha Sohl-Dickstein ,
Diederik P. Kingma ,
Abhishek Kumar ,
Stefano Ermon ,
Ben Poole
ICLR 2021 (oral presentation)
arXiv / code
A general framework for training and sampling from score-based models enabling likelihood computation and controllable generaiton.
Learning Energy-Based Models by Diffusion Recovery Likelihood
Ruiqi Gao ,
Yang Song ,
Ben Poole ,
Ying Nian Wu
Diederik P. Kingma
ICLR 2021
arXiv
Tractably learn and sample from a sequence of EBMs based on a diffusion process. High sample quality and stable long-run MCMC chains.
What Makes for Good views for Contrastive Learning?
Yonglong Tian ,
Chen Sun ,
Ben Poole ,
Dilip Krishnan ,
Cordelia Schmid ,
Philip Isola
NeurIPS 2020
arXiv / project page
Weakly-Supervised Disentanglement Without Compromises
Francesco Locatello ,
Ben Poole ,
Gunnar Rätsch ,
Bernhard Schölkopf ,
Olivier Bachem ,
Michael Tschannen
ICML 2020
arXiv
With a causality-inspired twist, disentangled representations are identifiable in theory and practice.
On Implicit Regularization in β-VAEs
Abhishek Kumar ,
Ben Poole
ICML 2020
arXiv
Heuristics in VAEs can lead to uniqueness and beneficial regularization.
VIB is Half Bayes
Alex Alemi ,
Warren Morningstar ,
Ben Poole ,
Ian Fischer ,
Josh Dillon
AABI Symposium 2020
arXiv
The Variational Information Bottleneck can rederived as Half-Bayesian.
On variational bounds of mutual information
Ben Poole , Sherjil Ozair , Aäron van den Oord , Alex Alemi , George Tucker
ICML 2019
arXiv / colab / video / slides / poster
Old, new, and improved estimators of mutual information w/neural nets.
Discrete Flows: Invertible Generative
Models of Discrete Data
Dustin Tran , Keyon Vafa , Kumar Krishna Agrawal , Laurent Dinh , Ben Poole
ICLR Deep GenStruct Workshop 2019
arXiv
Fast sampling generative models for discrete data.
Preventing posterior collapse with delta-VAEs
Ali Razavi , Aäron van den Oord , Ben Poole , Oriol Vinyals
ICLR 2019
OpenReview / arXiv / poster
Avoid posterior collapse by lower bounding the rate.
Neuronal Dynamics Regulating Brain and Behavioral State Transitions
Aaron Andalman, Vanessa Burns, Matthew Lovett-Barron, Michael Broxton , Ben Poole , Samuel Yang, Logan Grosenick , Talia Lerner, Ritchie Chen, Tyler Benster, Philippe Mourrai, Marc Levoy , Kanaka Rajan , Karl Deisseroth
Cell 2019
Fixing a Broken ELBO
Alex Alemi , Ben Poole , Ian Fischer ,
Joshua V. Dillon, Rif A. Saurous , Kevin Murphy
ICML , 2018
arXiv
Understanding tradeoffs in VAE models through the lens of information theory.
Continuous relaxation training of discrete latent-variable image models
Casper Kaae Sønderby* , Ben Poole *, Andriy Mnih
NIPS Bayesian Deep Learning Workshop , 2017
PDF
Continuous relaxation training of discrete latent-variable models can flexibly capture both continuous and discrete aspects of natural data.
Identification Of Cellular-Activity Dynamics Across Large Tissue Volumes In The Mammalian Brain
Logan Grosenick *, Michael Broxton *, Christina Kim*, Conor Liston*, Ben Poole , Samuel Yang, Aaron Andalman, Edward Scharff, Noy Cohen, Ofer Yizhar, Charu Ramakrishnan, Surya Ganguli , Patrick Suppes, Marc Levoy , Karl Deisseroth
*equal contribution
bioRxiv
Large-scale cellular-level imaging in the mammalian brain using lightfield microscopy. 1x1x0.5mm3 @ 100Hz.
Continual Learning through Synaptic Intelligence
Friedemann Zenke *, Ben Poole *, Surya Ganguli
*equal contribution
ICML , 2017
arXiv / code
Learns to solve tasks sequentially without forgetting by learning which weights are important.
On the expressive power of deep neural networks
Maithra Raghu , Ben Poole , Jon Kleinberg , Surya Ganguli , Jascha Sohl-Dickstein
ICML , 2017
arXiv
Random neural networks show exponential growth in activation patterns and more.
Time-warped PCA: simultaneous alignment and dimensionality reduction of neural data
Ben Poole , Alex Williams , Niru Maheswaranathan , Byron Yu , Gopal Santhanam, Stephen Ryu, Stephen Baccus , Krishna Shenoy , Surya Ganguli
Computational Systems Neuroscience (COSYNE) , 2017
abstract / poster / code
Extends dimensionality reduction techniques to account for trial-to-trial variability in timing.
Categorical Reparameterization with Gumbel-Softmax
Eric Jang , Shane Gu , Ben Poole
ICLR , 2017
arXiv
/
blog post
Efficient gradient estimator for categorical variables.
Unrolled Generative Adversarial Networks
Luke Metz , Ben Poole , David Pfau , Jascha Sohl-Dickstein
ICLR , 2017
arXiv
/
code
Stabilize GANs by defining the generator objective with respect to an unrolled optimization of the discriminator.
Adversarially learned inference
Vincent Dumoulin , Ishmael Belghazi, Ben Poole , Alex Lamb, Martin Arjovsky, Olivier Mastropietro, Aaron Courville
ICLR , 2017
arXiv
/
project page
Jointly learn a generative model and an inference network through an adversarial process.
Exponential expressivity in deep neural networks through transient chaos
Ben Poole , Subhaneil Lahiri , Maithra Raghu , Jascha Sohl-Dickstein , Surya Ganguli
Neural Information Processing Systems (NIPS) , 2016
arXiv
/
code
/
poster
Random neural networks have curvature that grows exponentially with depth.
Improved generator objectives for GANs
Ben Poole , Alex Alemi , Jascha Sohl-Dickstein , Anelia Angelova
NIPS Workshop on Adversarial Training , 2016
arXiv
/
poster
A new take on the Generative Adversarial Network training procedure.
Direction Selectivity in Drosophila Emerges from Preferred-Direction Enhancement and Null-Direction Suppression
Jonathan Leong*, Jennifer Esch*, Ben Poole *, Surya Ganguli , Thomas Clandinin
* equal contribution
The Journal of Neuroscience , 2016
Fruit flies detect motion using a very similar algorithm to humans.
The Fast Bilateral Solver
Jonathan T. Barron , Ben Poole
European Conference on Computer Vision (ECCV) , 2016 (oral presentation)
arXiv
/
supplement
/
code
/
bibtex
Fast and accurate edge-aware smoothing. Differentiable for all your deep learning needs.
Fast large-scale optimization by unifying stochastic gradient and quasi-newton methods
Jascha Sohl-Dickstein , Ben Poole , Surya Ganguli
International Conference on Machine Learning (ICML) , 2014
arXiv /
code
Speed up quasi-newton methods by maintaining a low-dimensional approximation of the Hessian for each minibatch.
Analyzing noise in auotoencoders and deep networks
Ben Poole , Jascha Sohl-Dickstein , Surya Ganguli
NIPS Workshop on Deep Learning , 2013
arXiv
Derives analytic regularizers for different forms of noise injection, and shows how alternative types of additive noise can improve over dropout.
Brain Regions Engaged by Part- and Whole-task Performance in a Video Game: A Model-based Test of the Decomposition Hypothesis
John Anderson, Daniel Bothell, Jon Fincham, Abraham Anderson, Ben Poole , Yulin Qin
Journal of Cognitive Neuroscience , 2011
Complex tasks, like the Space Fortress video game, can be decomposed into a set of independent reusable components.
Robust non-rigid alignment of volumetric calcium imaging data
Ben Poole , Logan Grosenick , Michael Broxton , Karl Deisseroth , Surya Ganguli
Computational Systems Neuroscience (COSYNE) , 2015
poster
Correct for translations and rotations of noisy volumetric data without a clean reference volume.
Connecting scene statistics to probabilistic population codes and tuning properties of V1 neurons
Ben Poole , Ian Lenz, Grace Linsday, Jason Samonds, Tai Sing Lee
Society for Neuroscience (SFN) , 2010 (oral presentation)