Speakers
Dr. Mauricio A Álvarez
Advances in scalable inference for Gaussian Processes-Variational Autoencoders (GPVAEs)
Gaussian Processes (GP) Variational Autoencoders (GPVAEs) assign a GP prior over the latent space of a VAE. There are several examples in the literature showing why a GP prior helps the VAE in several representation learning tasks. However, scaling GPVAEs to long sequences is challenging. In this talk, I’ll introduce GPVAEs and discuss a recent couple of research directions in our group looking to reduce GPVAEs’ computational complexity.
Prof. Christophe Andrieu
Geometry informed selection in the conditional SMC sampler
The cSMC is a specialised MCMC kernel particularly suitable for inference in state-space models. We show how a simple modification of the selection procedure can lead to significant performance improvements.
Joint work with Yuan Chen.
Dr. Paris Giampouras
Talk title to be confirmed
Abstract to follow.
Dr. Avetik Karagulyan
Talk title to be confirmed
Abstract to follow.
Prof. Anthony Lee
Monte Carlo, reproducing kernels and neural networks: explicit integral representations and quantitative bounds for two-layer ReLU networks
An approach to construct explicit integral representations for two-layer ReLU networks is presented, which provides relatively simple representations for functions in the reproducing kernel Hilbert space with reproducing kernel K(x,y) = exp(⟨x,y⟩). Quantitative bounds demonstrate that functions can be approximated with errors that do not depend explicitly on dimension or degree, but rather the coefficients of their monomial expansions and the “data” distribution over inputs to the function. One perspective is that one can represent the function in this relatively standard RKHS, while Monte Carlo is used to approximate the representation using an activation function, inner products and biases, and an optimal change of measure is used to produce a good distribution for random ReLU networks.
Prof. Benedict Leimkuhler
Position-adaptive Langevin
I will discuss the design of Langevin sampling algorithms based on using a projected position-dependent friction as a type of preconditioner. I will show with several numerical studies that this approach can accelerate convergence while also improving accuracy and overall robustness. I will also discuss the combination of position-adaptive Langevin with variable stepsizes implemented using the SamAdams framework.
Joint work with Peter Whalley.
Dr. Samuel Livingstone
Skew-symmetric numerical schemes for stochastic differential equations: strong convergence and multi-level extension
I will discuss recent work fusing together two strands of the applied mathematics and statistics literature, one concerned with developing flexible probability distributions for data that rely on a small number of parameters, and another concerned with developing numerical integration schemes to simulate stochastic processes. The specific case that I will focus on uses the skew-symmetric family of probability distributions introduced by Adelchi Azzalini and co-authors to approximate the transition kernels of diffusion processes over small time steps, producing alternative numerical schemes to the classical Euler–Maruyama approach. Applying the scheme to the overdamped Langevin diffusion leads to an unadjusted version of the Barker proposal Metropolis–Hastings algorithm. In earlier work weak accuracy was established over finite and infinite time scales, crucially without needing a globally Lipschitz assumption on the drift of the stochastic differential equation. I will review this and then discuss more recent work establishing strong convergence in the mean-squared sense using a novel coupling between the numerical and exact processes. This also enables the development of a multi-level Monte Carlo scheme, which I will discuss the merits of with particular focus on the superlinear drift case, as compared to Euler and Tamed Euler alternatives.
Joint work with Yuga Iguchi, Giorgos Vasdekis and Rui-Yang Zhang.
Prof. Ruth Misener
Bayesian optimization for mixed feature spaces using tree kernels and graph kernels
Bayesian optimization is effectively a two-step iterative process that first trains a surrogate model using continuous optimization over hyperparameter space and then optimizes the acquisition function over the search space. We investigate Bayesian optimization for mixed-feature search spaces using both tree kernels and graph kernels for Gaussian processes. With respect to trees kernels, our Bayesian Additive Regression Trees Kernel (BARK) uses tree agreement to define a posterior over sum-of-tree functions. With respect to graph kernels, our acquisition function with shortest paths encoded allows us to optimize over graphs, for instance to find the best graph structure and/or node features. We formulate both acquisition functions using mixed-integer optimization and show applications to a variety of challenges in molecular design, engineering and machine learning.
Joint work with Toby Boyne, Alexander Thebelt, Yilin Xie, Shiqiang Zhang, Jixiang Qing, Jose Folch, Robert Lee, Nathan Sudermann-Merx, David Walz, Behrang Shafei and Calvin Tsay.
Dr. Siddharth Narayanaswamy
Talk title to be confirmed
Abstract to follow.
Dr. Clarice Poon
On the limit points of gradient flows for wide shallow models beyond homogeneous nonlinearities
A surprising phenomenon in the training of neural networks is the ability of gradient descent to find global minimizers of the training loss despite its non-convexity. Following earlier work, we investigate this behavior for wide shallow models. Existing global convergence results primarily concern models with positively one-homogeneous nonlinearities, such as ReLU activations, and models with scalar output weights and bounded nonlinearities, such as sigmoid activations. We study a broader class of models, including multi-head attention layers and two-layer networks with bounded or asymptotically positively one-homogeneous activations and vector output weights. We prove that, in the limit of many hidden neurons or attention heads, non-global minimizers of the training loss are unstable under mean-field gradient flow dynamics by constructing “escape regions” in the parameter space. Our global convergence statements are conditional in the following sense: if the mean-field gradient flow converges in W2, then its limit must be a global minimizer.
Joint work with Romain Petit and Gabriel Peyré.
Prof. Gareth Roberts
Challenges in Bayesian Privacy
Differential privacy (DP) and its generalisations have become the gold standard for assessing privacy protection of published data. This talk will discuss some of the challenges this raises for Bayesian inference. As well as considering DP, we shall also work with the more flexible and more statistically intuitive f-Differential Privacy (fDP). As Bayesian analyses often result in the generation of random samples from the posterior distribution, we begin with the basic problem of how to retain (f-)DP from publication of posterior draws while retaining as much statistical information as possible. We then consider how commonly used accept/reject algorithms (for example rejection sampling) can be equally protected in fDP. The latter work will concentrate on decentralised algorithms which use homomorphic encryption.
Joint work with Shenggang Hu, Louis Aslett, Hongsheng Dai and Murray Pollock.
Prof. Chris Sherlock
Robust, partially alive particle Metropolis-Hastings via the Frankenfilter
When a hidden Markov model permits the conditional likelihood of an observation given the hidden process to be zero, all particle simulations from one observation time to the next could produce zeros. If so, the filtering distribution cannot be estimated and the estimated parameter likelihood is zero. The alive particle filter addresses this by simulating a random number of particles for each inter-observation interval, stopping after a target number of non-zero conditional likelihoods. For outlying observations or poor parameter values, a non-zero result can be extremely unlikely, and computational costs prohibitive. We introduce the Frankenfilter, a principled, partially alive particle filter that targets a user-defined amount of success whilst fixing lower and upper bounds on the number of simulations. The Frankenfilter produces unbiased estimators of the likelihood, suitable for pseudo-marginal Metropolis–Hastings (PMMH). We demonstrate that PMMH with the Frankenfilter is more robust to outliers and mis-specified initial parameter values than PMMH using standard particle filters, and is typically at least 2–3 times more efficient. We also provide advice for choosing the amount of success. In the case of n exact observations, this is particularly simple: target n successes.
Dr. Tim Sullivan
Talk title to be confirmed
Abstract to follow.