IMSS Annual Lecture

Stein’s method is a powerful tool from probability theory for bounding the distance between probability distributions. In this talk, I’ll describe how this tool designed to prove central limit theorems can be adapted to assess and improve the quality of practical inference procedures. Along the way, I’ll highlight applications to Markov chain Monte Carlo sampler selection, goodness-of-fit testing, generative modeling, de novo sampling, post-selection inference, distribution compression, bias correction, and nonconvex optimization, and I’ll close with opportunities for future work.

We study the problem of private estimation for stochastic block models, where the observation comes in the form of an undirected graph, and the goal is to partition the nodes into unknown, underlying communities. We consider a notion of differential privacy known as node differential privacy, meaning that two graphs are treated as neighbors if one can be transformed into the other by changing the edges connected to exactly one node. The goal is to develop algorithms with optimal misclassification error rates, subject to a certain level of differential privacy. We present several algorithms based on private eigenvector extraction, private low-rank matrix estimation, and private SDP optimization. A key contribution of our work is a method for converting a procedure which is differentially private and has low statistical error on degree-bounded graphs to one that is differentially private on arbitrary graph inputs, while maintaining good accuracy (with high probability) on typical inputs. This is achieved by considering a certain smooth version of a map from the space of all undirected graphs to the space of bounded-degree graphs, which can be appropriately leveraged for privacy. We discuss the relative advantages of the algorithms we introduce and also provide some lower-bounds for the performance of any private community estimation algorithm. This is joint work with Laurentiu Marchis, Ethan D'souza, and Tomas Flidr.

Recent advances such as self-consistency and test-time reinforcement learning (TTRL) improve the reliability of large language models (LLMs) without additional supervision, yet their underlying mechanisms and statistical guarantees remain poorly understood. We present a unified framework for certifiable inference in LLMs, showing that majority voting provides a statistical certificate of self-consistency: under mild assumptions, the aggregated answer coincides with the mode of the model’s terminal distribution with high probability. We derive finite-sample and anytime-valid concentration bounds that quantify this confidence, and introduce the Martingale Majority Certificate (MMC), a sequential stopping rule that adaptively determines when sufficient samples have been drawn. We further prove that label-free post-training methods such as TTRL implicitly sharpen the answer distribution by exponentially tilting it toward its mode, thereby reducing the number of samples required for certification. Building on this insight, we propose new post-training objectives that explicitly optimise this trade-off between sharpness and bias. Together, these results explain and connect two central test-time scaling strategies, self-consistency and TTRL, within a single statistical framework for label-free, certifiable reliability in reasoning LLMs. This work has been done in collaboration with Andrew Duncan.

Assessing the quality of statistical models is challenging when likelihoods are unnormalised or when data lie on non-Euclidean spaces, as classical distances between distributions often become computationally intractable. In this talk, we show how Stein discrepancies exploit the curvature of the target distribution to provide a computable alternative. We discuss recent results demonstrating that suitably constructed kernel Stein discrepancies can characterise Wasserstein convergence. We then connect these ideas to inference on spherical data, where these discrepancies enable robust inference without requiring normalising constants.