• Compression, inversion, and approximate PCA of dense kernel matrices at near-linear computational complexity(1706.02205)

March 22, 2019 math.NA, math.PR, cs.DS, cs.CC
Dense kernel matrices $\Theta \in \mathbb{R}^{N \times N}$ obtained from point evaluations of a covariance function $G$ at locations $\{ x_{i} \}_{1 \leq i \leq N}$ arise in statistics, machine learning, and numerical analysis. For covariance functions that are Green's functions of elliptic boundary value problems and homogeneously-distributed sampling points, we show how to identify a subset $S \subset \{ 1 , \dots , N \}^2$, with $\# S = O ( N \log (N) \log^{d} ( N /\epsilon ) )$, such that the zero fill-in incomplete Cholesky factorisation of the sparse matrix $\Theta_{ij} 1_{( i, j ) \in S}$ is an $\epsilon$-approximation of $\Theta$. This factorisation can provably be obtained in complexity $O ( N \log( N ) \log^{d}( N /\epsilon) )$ in space and $O ( N \log^{2}( N ) \log^{2d}( N /\epsilon) )$ in time; we further present numerical evidence that $d$ can be taken to be the intrinsic dimension of the data set rather than that of the ambient space. The algorithm only needs to know the spatial configuration of the $x_{i}$ and does not require an analytic representation of $G$. Furthermore, this factorization straightforwardly provides an approximate sparse PCA with optimal rate of convergence in the operator norm. Hence, by using only subsampling and the incomplete Cholesky factorization, we obtain, at nearly linear complexity, the compression, inversion and approximate PCA of a large class of covariance matrices. By inverting the order of the Cholesky factorization we also obtain a solver for elliptic PDE with complexity $O ( N \log^{d}( N /\epsilon) )$ in space and $O ( N \log^{2d}( N /\epsilon) )$ in time.
• Strong convergence rates of probabilistic integrators for ordinary differential equations(1703.03680)

Probabilistic integration of a continuous dynamical system is a way of systematically introducing model error, at scales no larger than errors introduced by standard numerical discretisation, in order to enable thorough exploration of possible responses of the system to inputs. It is thus a potentially useful approach in a number of applications such as forward uncertainty quantification, inverse problems, and data assimilation. We extend the convergence analysis of probabilistic integrators for deterministic ordinary differential equations, as proposed by Conrad et al.\ (\textit{Stat.\ Comput.}, 2017), to establish mean-square convergence in the uniform norm on discrete- or continuous-time solutions under relaxed regularity assumptions on the driving vector fields and their induced flows. Specifically, we show that randomised high-order integrators for globally Lipschitz flows and randomised Euler integrators for dissipative vector fields with polynomially-bounded local Lipschitz constants all have the same mean-square convergence rate as their deterministic counterparts, provided that the variance of the integration noise is not of higher order than the corresponding deterministic integrator. These and similar results are proven for probabilistic integrators where the random perturbations may be state-dependent, non-Gaussian, or non-centred random variables.
• Equivalence of weak and strong modes of measures on topological vector spaces(1708.02516)

Aug. 18, 2018 math.PR, math.FA, math.ST, stat.TH, math.OC
A strong mode of a probability measure on a normed space $X$ can be defined as a point $u$ such that the mass of the ball centred at $u$ uniformly dominates the mass of all other balls in the small-radius limit. Helin and Burger weakened this definition by considering only pairwise comparisons with balls whose centres differ by vectors in a dense, proper linear subspace $E$ of $X$, and posed the question of when these two types of modes coincide. We show that, in a more general setting of metrisable vector spaces equipped with measures that are finite on bounded sets, the density of $E$ and a uniformity condition suffice for the equivalence of these two types of modes. We accomplish this by introducing a new, intermediate type of mode. We also show that these modes can be inequivalent if the uniformity condition fails. Our results shed light on the relationships between among various notions of maximum a posteriori estimator in non-parametric Bayesian inference.
• Random forward models and log-likelihoods in Bayesian inverse problems(1712.05717)

We consider the use of randomised forward models and log-likelihoods within the Bayesian approach to inverse problems. Such random approximations to the exact forward model or log-likelihood arise naturally when a computationally expensive model is approximated using a cheaper stochastic surrogate, as in Gaussian process emulation (kriging), or in the field of probabilistic numerical methods. We show that the Hellinger distance between the exact and approximate Bayesian posteriors is bounded by moments of the difference between the true and approximate log-likelihoods. Example applications of these stability results are given for randomised misfit models in large data applications and the probabilistic solution of ordinary differential equations.
• Quasi-invariance of countable products of Cauchy measures under non-unitary dilations(1611.10289)

Jan. 29, 2018 math.PR, math.ST, stat.TH
Consider an infinite sequence $(U_n)_{n\in\mathbb{N}}$ of independent Cauchy random variables, defined by a sequence $(\delta_n)_{n\in\mathbb{N}}$ of location parameters and a sequence $(\gamma_n)_{n\in\mathbb{N}}$ of scale parameters. Let $(W_n)_{n\in\mathbb{N}}$ be another infinite sequence of independent Cauchy random variables defined by the same sequence of location parameters and the sequence $(\sigma_n\gamma_n)_{n\in\mathbb{N}}$ of scale parameters, with $\sigma_n\neq 0$ for all $n\in\mathbb{N}$. Using a result of Kakutani on equivalence of countably infinite product measures, we show that the laws of $(U_n)_{n\in\mathbb{N}}$ and $(W_n)_{n\in\mathbb{N}}$ are equivalent if and only if the sequence $(\vert \sigma_n\vert-1)_{n\in\mathbb{N}}$ is square-summable.
• Well-posedness of Bayesian inverse problems in quasi-Banach spaces with stable priors(1710.05610)

Oct. 16, 2017 math.NA, math.PR, math.ST, stat.TH
The Bayesian perspective on inverse problems has attracted much mathematical attention in recent years. Particular attention has been paid to Bayesian inverse problems (BIPs) in which the parameter to be inferred lies in an infinite-dimensional space, a typical example being a scalar or tensor field coupled to some observed data via an ODE or PDE. This article gives an introduction to the framework of well-posed BIPs in infinite-dimensional parameter spaces, as advocated by Stuart (Acta Numer. 19:451--559, 2010) and others. This framework has the advantage of ensuring uniformly well-posed inference problems independently of the finite-dimensional discretisation used for numerical solution. Recently, this framework has been extended to the case of a heavy-tailed prior measure in the family of stable distributions, such as an infinite-dimensional Cauchy distribution, for which polynomial moments are infinite or undefined. It is shown that analogues of the Karhunen--Lo\eve expansion for square-integrable random variables can be used to sample such measures on quasi-Banach spaces. Furthermore, under weaker regularity assumptions than those used to date, the Bayesian posterior measure is shown to depend Lipschitz continuously in the Hellinger and total variation metrics upon perturbations of the misfit function and observed data.
• Well-posed Bayesian inverse problems and heavy-tailed stable quasi-Banach space priors(1605.05898)

June 30, 2017 math.NA, math.PR, math.ST, stat.TH
This article extends the framework of Bayesian inverse problems in infinite-dimensional parameter spaces, as advocated by Stuart (Acta Numer. 19:451--559, 2010) and others, to the case of a heavy-tailed prior measure in the family of stable distributions, such as an infinite-dimensional Cauchy distribution, for which polynomial moments are infinite or undefined. It is shown that analogues of the Karhunen--Lo\eve expansion for square-integrable random variables can be used to sample such measures on quasi-Banach spaces. Furthermore, under weaker regularity assumptions than those used to date, the Bayesian posterior measure is shown to depend Lipschitz continuously in the Hellinger metric upon perturbations of the misfit function and observed data.
• Cameron-Martin theorems for sequences of symmetric Cauchy-distributed random variables(1608.03784)

Dec. 1, 2016 math.PR, math.ST, stat.TH
Given a sequence of Cauchy-distributed random variables defined by a sequence of location parameters and a sequence of scale parameters, we consider another sequence of random variables that is obtained by perturbing the location or scale parameter sequences. Using a result of Kakutani on equivalence of infinite product measures, we provide sufficient conditions for the equivalence of laws of the two sequences.
• Optimal uncertainty quantification for legacy data observations of Lipschitz functions(1202.1928)

April 13, 2013 math.PR, cs.NA, stat.CO, math.ST, stat.TH
We consider the problem of providing optimal uncertainty quantification (UQ) --- and hence rigorous certification --- for partially-observed functions. We present a UQ framework within which the observations may be small or large in number, and need not carry information about the probability distribution of the system in operation. The UQ objectives are posed as optimization problems, the solutions of which are optimal bounds on the quantities of interest; we consider two typical settings, namely parameter sensitivities (McDiarmid diameters) and output deviation (or failure) probabilities. The solutions of these optimization problems depend non-trivially (even non-monotonically and discontinuously) upon the specified legacy data. Furthermore, the extreme values are often determined by only a few members of the data set; in our principal physically-motivated example, the bounds are determined by just 2 out of 32 data points, and the remainder carry no information and could be neglected without changing the final answer. We propose an analogue of the simplex algorithm from linear programming that uses these observations to offer efficient and rigorous UQ for high-dimensional systems with high-cardinality legacy data. These findings suggest natural methods for selecting optimal (maximally informative) next experiments.
• Stratified Graphene-Noble Metal Systems for Low-Loss Plasmonics Applications(1301.5620)

We propose a composite layered structure for tunable, low-loss plasmon resonances, which con- sists of a noble-metal thin film coated in graphene and supported on a hexagonal boron nitride (hBN) substrate. We calculate electron energy loss spectra (EELS) for these structures, and nu- merically demonstrate that bulk plasmon losses in noble-metal films can be significantly reduced, and surface coupling enhanced, through the addition of a graphene coating and the wide-bandgap hBN substrate. Silver films with a trilayer graphene coating and hBN substrate demonstrated sur- face plasmon-dominant spectral profiles for metallic layers as thick as 34 nm. A continued-fraction expression for the effective dielectric function, based on a specular reflection model which includes boundary interactions, is used to systematically demonstrate plasmon peak tunability for a variety of configurations. Variations include substrate, plasmonic metal, and individual layer thickness for each material. Mesoscale calculation of EELS is performed with individual layer dielectric functions as input to the effective dielectric function calculation, from which the loss spectra are directly determined.
• Thermalization of rate-independent processes by entropic regularization(1209.3619)

Sept. 17, 2012 math-ph, math.MP, math.PR, math.DS
We consider the effective behaviour of a rate-independent process when it is placed in contact with a heat bath. The method used to "thermalize" the process is an interior-point entropic regularization of the Moreau--Yosida incremental formulation of the unperturbed process. It is shown that the heat bath destroys the rate independence in a controlled and deterministic way, and that the effective dynamics are those of a non-linear gradient descent in the original energetic potential with respect to a different and non-trivial effective dissipation potential.