The following papers cover some of the background to Probabilistic AI.

The importance of probabilistic AI

• Probabilistic machine learning and artificial intelligence Z. Ghahramani (2015) Nature 521, 452-459

• Position Paper: Bayesian Deep Learning in the Age of Large-Scale AI T. Papamarkou et al. (2024) arXiv:2402.00809

Scalable sampling and generalised Bayesian ideas

• Scalable Monte Carlo for Bayesian Learning P Fearnhead, C Nemeth, CJ Oates and C Sherlock arXiv:2407.12751. A book length introduction to recent advances in Monte Carlo.

• Stochastic gradient Markov chain Monte Carlo. P. Fearnhead and C. Nemeth (2019) Journal of the American Statistical Association 433, 533-550

• Martingale posterior distribution. E. Fong, C. Holmes and S. Walker (2023) Journal of the Royal Statistical Association, Series B 85, 1358-1391

• Stochastic normalising flows. H. Wu, J. Kohler and F. Noe (2020) NeurIPS 34

Dynamical systems view of AI

• Bayesian Learning for Neural Networks. R. Neal (2012) Springer

• Gaussian process behaviour in wide deep neural networks A. Matthews, M. Rowland, J. Hiron, R. Turner and Z. Ghahramani (2018) arXiv:1804.11271

• Neural Processes. M. Garnelo et al. (2018) arXiv:1807.01622

• Deep limits of residual neural networks M. Thorpe and Y. van Gennip. (2018) arXiv:1810.11741

  Mathematical underpinnings of generative models

• Denoising Diffusion Probabilistic Models in Six Simple Steps. R. Turner et al. (2024) arXiv:2402.04384

• Diffusion Schrödinger Bridges for Bayesian Computation Jeremy Heng, Valentin De Bortoli, Arnaud Doucet (2024) Statistical Science 39, 90-99

• From Denoising Diffusions to Denoising Markov Models. J Benton et al. (2022) arXiv:2211.03595

  Structure-constrained and informed AI

• Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. M. Raissi, P Perdikaris and G Karniadakis (2019) Journal of Computational Physics 378, 686-707

• Testing the manifold hypothesis C. Fefferman, S. Mitter and H. Narayanan (2016), Journal of the American Mathematical Society 29, 983-1049

  Probabilistic and Uncertainty-aware AI

• Convergence of Gaussian process regression with estimated hyper-parameters and applications in Bayesian inverse problems. A. Teckentrup SIAM-ASA JUQ 8, 1310-1337