Speakers

Challenges in Bayesian Privacy

Differential privacy (DP) and its generalisations have become the gold standard for assessing privacy protection of published data. This talk will discuss some of the challenges this raises for Bayesian inference. As well as considering DP, we shall also work with the more flexible and more statistically intuitive f-Differential Privacy (fDP). As Bayesian analyses often result in the generation of random samples from the posterior distribution, we begin with the basic problem of how to retain (f-)DP from publication of posterior draws while retaining as much statistical information as possible. We then consider how commonly used accept/reject algorithms (for example rejection sampling) can be equally protected in fDP. The latter work will concentrate on decentralised algorithms which use homomorphic encryption.

Joint work with Shenggang Hu, Louis Aslett, Hongsheng Dai and Murray Pollock.

Bayesian optimization for mixed feature spaces using tree kernels and graph kernels

Bayesian optimization is effectively a two-step iterative process that first trains a surrogate model using continuous optimization over hyperparameter space and then optimizes the acquisition function over the search space. We investigate Bayesian optimization for mixed-feature search spaces using both tree kernels and graph kernels for Gaussian processes. With respect to trees kernels, our Bayesian Additive Regression Trees Kernel (BARK) uses tree agreement to define a posterior over sum-of-tree functions. With respect to graph kernels, our acquisition function with shortest paths encoded allows us to optimize over graphs, for instance to find the best graph structure and/or node features. We formulate both acquisition functions using mixed-integer optimization and show applications to a variety of challenges in molecular design, engineering and machine learning.

Joint work with Toby Boyne, Alexander Thebelt, Yilin Xie, Shiqiang Zhang, Jixiang Qing, Jose Folch, Robert Lee, Nathan Sudermann-Merx, David Walz, Behrang Shafei and Calvin Tsay.

Advances in scalable inference for Gaussian Processes-Variational Autoencoders (GPVAEs)

Gaussian Processes (GP) Variational Autoencoders (GPVAEs) assign a GP prior over the latent space of a VAE. There are several examples in the literature showing why a GP prior helps the VAE in several representation learning tasks. However, scaling GPVAEs to long sequences is challenging. In this talk, I’ll introduce GPVAEs and discuss a recent couple of research directions in our group looking to reduce GPVAEs’ computational complexity.

Geometry informed selection in the conditional SMC sampler

The cSMC is a specialised MCMC kernel particularly suitable for inference in state-space models. We show how a simple modification of the selection procedure can lead to significant performance improvements.

Joint work with Yuan Chen.

Semantic Alignment for Robust Representations in Deep Generative Models

Deep generative models learn rich representations of complex data and have emerged as powerful priors for conditional generation and inverse problems. Reliably using these representations, however, requires more than generating realistic samples: the extracted representation must capture the semantic structure of the unknown signal, while the resulting reconstruction remains consistent with the conditioning information or available measurements. This talk studies reliable representation extraction for inverse problems with deep generative models through semantic alignment at inference time.

I will introduce Align & Invert, an inference-time method that guides diffusion- and flow-based inverse solvers by aligning their internal representations with semantic features from pretrained encoders. Because the clean target is unavailable, the method constructs and iteratively refines a proxy for its semantic representation. This guidance improves perceptual reconstruction quality and reduces the number of sampling steps without compromising measurement consistency. I will also discuss theoretical connections between representation alignment, distribution matching, and contraction toward the clean latent representation. Finally, I will show empirical results validating the benefits of our approach on various inverse imaging problems.

Improved Guarantees for Langevin Monte Carlo with Average Smoothness

We establish improved nonasymptotic bounds for Langevin Monte Carlo in the strongly log-concave setting, when the error is measured by the Wasserstein distance. The main result shows that the discretization error is governed by an average coordinate-wise smoothness constant, rather than by the usual global smoothness constant. The proof is short and probabilistic, and relies on a refined use of the synchronous coupling. We further show that the same ideas lead to improved bounds for variable step sizes, for potentials whose Laplacian is Lipschitz-continuous, and for finite-sum problems sampled by stochastic-gradient Langevin dynamics with fixed point control variates. In the Laplacian-smooth case, the usual Hessian-Lipschitz contribution is replaced by a weaker trace-type third-order smoothness quantity. In the finite-sum setting, the resulting SGLD bound improves the dependence on the root mean square smoothness of the component functions. Applications to generalized linear models with Gaussian design show that these refinements can yield substantial, dimension-dependent improvements over previously known bounds, especially for correlated covariates.

Monte Carlo, reproducing kernels and neural networks: explicit integral representations and quantitative bounds for two-layer ReLU networks

An approach to construct explicit integral representations for two-layer ReLU networks is presented, which provides relatively simple representations for functions in the reproducing kernel Hilbert space with reproducing kernel K(x,y) = exp(⟨x,y⟩). Quantitative bounds demonstrate that functions can be approximated with errors that do not depend explicitly on dimension or degree, but rather the coefficients of their monomial expansions and the “data” distribution over inputs to the function. One perspective is that one can represent the function in this relatively standard RKHS, while Monte Carlo is used to approximate the representation using an activation function, inner products and biases, and an optimal change of measure is used to produce a good distribution for random ReLU networks.

Position-adaptive Langevin

I will discuss the design of Langevin sampling algorithms based on using a projected position-dependent friction as a type of preconditioner. I will show with several numerical studies that this approach can accelerate convergence while also improving accuracy and overall robustness. I will also discuss the combination of position-adaptive Langevin with variable stepsizes implemented using the SamAdams framework.

Joint work with Peter Whalley.

Skew-symmetric numerical schemes for stochastic differential equations: strong convergence and multi-level extension

I will discuss recent work fusing together two strands of the applied mathematics and statistics literature, one concerned with developing flexible probability distributions for data that rely on a small number of parameters, and another concerned with developing numerical integration schemes to simulate stochastic processes. The specific case that I will focus on uses the skew-symmetric family of probability distributions introduced by Adelchi Azzalini and co-authors to approximate the transition kernels of diffusion processes over small time steps, producing alternative numerical schemes to the classical Euler–Maruyama approach. Applying the scheme to the overdamped Langevin diffusion leads to an unadjusted version of the Barker proposal Metropolis–Hastings algorithm. In earlier work weak accuracy was established over finite and infinite time scales, crucially without needing a globally Lipschitz assumption on the drift of the stochastic differential equation. I will review this and then discuss more recent work establishing strong convergence in the mean-squared sense using a novel coupling between the numerical and exact processes. This also enables the development of a multi-level Monte Carlo scheme, which I will discuss the merits of with particular focus on the superlinear drift case, as compared to Euler and Tamed Euler alternatives.

Joint work with Yuga Iguchi, Giorgos Vasdekis and Rui-Yang Zhang.

Can LLMs talk in MCMC?

LLMs are great for language modelling. Their core competence is really sequence to sequence modelling. Prompt sequence goes in, task relevant sequence comes out. Here, I will discuss some preliminary work on leveraging this core competence for probabilistic inference. Model, data, and states go in and transformed states come out. How well does this work? Can we actually do interesting things with it?

Inverse optimal transport

In this talk, I will discuss a particular inverse problem arising from Optimal transport (OT). OT is now a central modeling tool to compare and couple probability distributions, with applications spanning economics, imaging, generative modeling, and computational biology. In many modern pipelines, however, the transport cost is not known a priori and must be inferred from data. This leads to inverse optimal transport (iOT): recover the ground cost (or metric parameters) from an observed optimal coupling. On the other hand, modern computational pipelines typically exploit an entropic regularization variant (eOT) of OT. I will discuss iOT in a regime that is both mathematically delicate and practically unavoidable: the entropic regularization level is small (approaching the unregularized OT model), while the coupling is observed through a finite number of samples. I will discuss well-posedness of this inverse problem as well as present conditions where iOT has statistically efficient bounds.

Robust, partially alive particle Metropolis-Hastings via the Frankenfilter

When a hidden Markov model permits the conditional likelihood of an observation given the hidden process to be zero, all particle simulations from one observation time to the next could produce zeros. If so, the filtering distribution cannot be estimated and the estimated parameter likelihood is zero. The alive particle filter addresses this by simulating a random number of particles for each inter-observation interval, stopping after a target number of non-zero conditional likelihoods. For outlying observations or poor parameter values, a non-zero result can be extremely unlikely, and computational costs prohibitive. We introduce the Frankenfilter, a principled, partially alive particle filter that targets a user-defined amount of success whilst fixing lower and upper bounds on the number of simulations. The Frankenfilter produces unbiased estimators of the likelihood, suitable for pseudo-marginal Metropolis–Hastings (PMMH). We demonstrate that PMMH with the Frankenfilter is more robust to outliers and mis-specified initial parameter values than PMMH using standard particle filters, and is typically at least 2–3 times more efficient. We also provide advice for choosing the amount of success. In the case of n exact observations, this is particularly simple: target n successes.

arXiv:2601.23173

Autoencoders in function space

High-resolution approximations of functional data arise frequently in scientific computing. While discretisation makes problems finite dimensional, designing algorithms in function space first enables smooth operation between resolutions. We propose function-space versions of autoencoders—machine-learning methods for dimension reduction and generative modelling—in both their deterministic (FAE) and variational (FVAE) forms. This enables training and evaluation on data discretised at arbitrary resolutions, and unlocks new applications such as inpainting, superresolution, and generative modelling. We demonstrate this on scientific data sets, including data from Navier–Stokes fluid flow simulations.

Joint work with Justin Bunker and Mark Girolami (Cambridge), Hefin Lambley (Warwick), and Andrew M. Stuart (Caltech).