• Many computer vision pipelines involve dynamic programming primitives such as finding a shortest path or the minimum energy solution in a tree-shaped probabilistic graphical model. In such cases, extracting not merely the best, but the set of M-best solutions is useful to generate a rich collection of candidate proposals that can be used in downstream processing. In this work, we show how M-best solutions of tree-shaped graphical models can be obtained by dynamic programming on a special graph with M layers. The proposed multi-layer concept is optimal for searching M-best solutions, and so flexible that it can also approximate M-best diverse solutions. We illustrate the usefulness with applications to object detection, panorama stitching and centerline extraction. Note: We have observed that an assumption in section 4 of our paper is not always fulfilled, see the attached corrigendum for details.
  • We propose to optimize the activation functions of a deep neural network by adding a corresponding functional regularization to the cost function. We justify the use of a second-order total-variation criterion. This allows us to derive a general representer theorem for deep neural networks that makes a direct connection with splines and sparsity. Specifically, we show that the optimal network configuration can be achieved with activation functions that are nonuniform linear splines with adaptive knots. The bottom line is that the action of each neuron is encoded by a spline whose parameters (including the number of knots) are optimized during the training procedure. The scheme results in a computational structure that is compatible with the existing deep-ReLU and MaxOut architectures. It also suggests novel optimization challenges, while making the link with $\ell_1$ minimization and sparsity-promoting techniques explicit.
  • Current algorithmic approaches for piecewise affine motion estimation are based on alternating motion segmentation and estimation. We propose a new method to estimate piecewise affine motion fields directly without intermediate segmentation. To this end, we reformulate the problem by imposing piecewise constancy of the parameter field, and derive a specific proximal splitting optimization scheme. A key component of our framework is an efficient one-dimensional piecewise-affine estimator for vector-valued signals. The first advantage of our approach over segmentation-based methods is its absence of initialization. The second advantage is its lower computational cost which is independent of the complexity of the motion field. In addition to these features, we demonstrate competitive accuracy with other piecewise-parametric methods on standard evaluation benchmarks. Our new regularization scheme also outperforms the more standard use of total variation and total generalized variation.
  • We consider linear inverse problems that are formulated in the continuous domain. The object of recovery is a function that is assumed to minimize a convex objective functional. The solutions are constrained by imposing a continuous-domain regularization. We derive the parametric form of the solution (representer theorems) for Tikhonov (quadratic) and generalized total-variation (gTV) regularizations. We show that, in both cases, the solutions are splines that are intimately related to the regularization operator. In the Tikhonov case, the solution is smooth and constrained to live in a fixed subspace that depends on the measurement operator. By contrast, the gTV regularization results in a sparse solution composed of only a few dictionary elements that are upper-bounded by the number of measurements and independent of the measurement operator. Our findings for the gTV regularization resonates with the minimization of the $l_1$ norm, which is its discrete counterpart and also produces sparse solutions. Finally, we find the experimental solutions for some measurement models in one dimension. We discuss the special case when the gTV regularization results in multiple solutions and devise an algorithm to find an extreme point of the solution set which is guaranteed to be sparse.
  • In this paper, we study the compressibility of random processes and fields, called generalized L\'evy processes, that are solutions of stochastic differential equations driven by $d$-dimensional periodic L\'evy white noises. Our results are based on the estimation of the Besov regularity of L\'evy white noises and generalized L\'evy processes. We show in particular that non-Gaussian generalized L\'evy processes are more compressible in a wavelet basis than the corresponding Gaussian processes, in the sense that their $n$-term approximation error decays faster. We quantify this compressibility in terms of the Blumenthal-Getoor index of the underlying L\'evy white noise.
  • In this paper, we characterize the local smoothness and the asymptotic growth rate of L\'evy white noises. We do so by identifying the weighted Besov spaces in which they are localized. We extend known results in two ways. First, we obtain new bounds for the local smoothness via the Blumenthal-Getoor indices of the L\'evy white noise. We deduce the critical local smoothness when the two indices coincide, which is true for symmetric-alpha-stable, compound Poisson and symmetric-gamma white noises. Second, we express the critical asymptotic growth rate in terms of the moments properties of the L\'evy white noise. Previous analysis only provided lower bounds for both the local smoothness and the asymptotic growth rate. Showing the sharpness of these bounds requires to determine in which Besov spaces a given L\'evy white noise is (almost surely) not. Our methods are based on the wavelet-domain characterization of Besov spaces and precise moment estimates for the wavelet coefficients of the noise.
  • Consider a random process s solution of the stochastic partial differential equation Ls = w with L a homogeneous operator and w a multidimensional L\'evy white noise. In this paper, we study the asymptotic effect of zooming in or zooming out of the process s. More precisely, we give sufficient conditions on L and w so that the rescaled versions of s converges in law to a self-similar process of order H at coarse scales and at fine scales. The parameter H depends on the homogeneity order of the operator L and the Blumenthal-Getoor indices associated to the L\'evy white noise w. Finally, we apply our general results to several notorious classes of random processes and random fields.
  • In this survey paper, we review recent uses of convolution neural networks (CNNs) to solve inverse problems in imaging. It has recently become feasible to train deep CNNs on large databases of images, and they have shown outstanding performance on object classification and segmentation tasks. Motivated by these successes, researchers have begun to apply CNNs to the resolution of inverse problems such as denoising, deconvolution, super-resolution, and medical image reconstruction, and they have started to report improvements over state-of-the-art methods, including sparsity-based techniques such as compressed sensing. Here, we review the recent experimental work in these areas, with a focus on the critical design decisions: Where does the training data come from? What is the architecture of the CNN? and How is the learning problem formulated and solved? We also bring together a few key theoretical papers that offer perspective on why CNNs are appropriate for inverse problems and point to some next steps in the field.
  • In applications that involve interactive curve and surface modeling, the intuitive manipulation of shapes is crucial. For instance, user interaction is facilitated if a geometrical object can be manipulated through control points that interpolate the shape itself. Additionally, models for shape representation often need to provide local shape control and they need to be able to reproduce common shape primitives such as ellipsoids, spheres, cylinders, or tori. We present a general framework to construct families of compactly-supported interpolators that are piecewise-exponential polynomial. They can be designed to satisfy regularity constraints of any order and they enable one to build parametric deformable shape models by suitable linear combinations of interpolators. They allow to change the resolution of shapes based on the refinability of B-splines. We illustrate their use on examples to construct shape models that involve curves and surfaces with applications to interactive modeling and character design.
  • The detection of landmarks or patterns is of interest for extracting features in biological images. Hence, algorithms for finding these keypoints have been extensively investigated in the literature, and their localization and detection properties are well known. In this paper, we study the complementary topic of local orientation estimation, which has not received similar attention. Simply stated, the problem that we address is the following: estimate the angle of rotation of a pattern with steerable filters centered at the same location, where the image is corrupted by colored isotropic Gaussian noise. For this problem, we use a statistical framework based on the Cram\'{e}r-Rao lower bound (CRLB) that sets a fundamental limit on the accuracy of the corresponding class of estimators. We propose a scheme to measure the performance of estimators based on steerable filters (as a lower bound), while considering the connection to maximum likelihood estimation. Beyond the general results, we analyze the asymptotic behaviour of the lower bound in terms of the order of steerablility and propose an optimal subset of components that minimizes the bound. We define a mechanism for selecting optimal subspaces of the span of the detectors. These are characterized by the most relevant angular frequencies. Finally, we project our template to a basis of steerable functions and experimentally show that the prediction accuracy achieves the predicted CRLB. As an extension, we also consider steerable wavelet detectors.
  • We present a new method for image reconstruction which replaces the projector in a projected gradient descent (PGD) with a convolutional neural network (CNN). CNNs trained as high-dimensional (image-to-image) regressors have recently been used to efficiently solve inverse problems in imaging. However, these approaches lack a feedback mechanism to enforce that the reconstructed image is consistent with the measurements. This is crucial for inverse problems, and more so in biomedical imaging, where the reconstructions are used for diagnosis. In our scheme, the gradient descent enforces measurement consistency, while the CNN recursively projects the solution closer to the space of desired reconstruction images. We provide a formal framework to ensure that the classical PGD converges to a local minimizer of a non-convex constrained least-squares problem. When the projector is replaced with a CNN, we propose a relaxed PGD, which always converges. Finally, we propose a simple scheme to train a CNN to act like a projector. Our experiments on sparse view Computed Tomography (CT) reconstruction for both noiseless and noisy measurements show an improvement over the total-variation (TV) method and a recent CNN-based technique.
  • Optical diffraction tomography relies on solving an inverse scattering problem governed by the wave equation. Classical reconstruction algorithms are based on linear approximations of the forward model (Born or Rytov), which limits their applicability to thin samples with low refractive-index contrasts. More recent works have shown the benefit of adopting nonlinear models. They account for multiple scattering and reflections, improving the quality of reconstruction. To reduce the complexity and memory requirements of these methods, we derive an explicit formula for the Jacobian matrix of the nonlinear Lippmann-Schwinger model which lends itself to an efficient evaluation of the gradient of the data- fidelity term. This allows us to deploy efficient methods to solve the corresponding inverse problem subject to sparsity constraints.
  • In Optical diffraction tomography, the multiply scattered field is a nonlinear function of the refractive index of the object. The Rytov method is a linear approximation of the forward model, and is commonly used to reconstruct images. Recently, we introduced a reconstruction method based on the Beam Propagation Method (BPM) that takes the nonlinearity into account. We refer to this method as Learning Tomography (LT). In this paper, we carry out simulations in order to assess the performance of LT over the linear iterative method. Each algorithm has been rigorously assessed for spherical objects, with synthetic data generated using the Mie theory. By varying the RI contrast and the size of the objects, we show that the LT reconstruction is more accurate and robust than the reconstruction based on the linear model. In addition, we show that LT is able to correct distortion that is evident in Rytov approximation due to limitations in phase unwrapping. More importantly, the capacity of LT in handling multiple scattering problem are demonstrated by simulations of multiple cylinders using the Mie theory and confirmed by experimental results of two spheres.
  • We propose a data-driven algorithm for the maximum a posteriori (MAP) estimation of stochastic processes from noisy observations. The primary statistical properties of the sought signal is specified by the penalty function (i.e., negative logarithm of the prior probability density function). Our alternating direction method of multipliers (ADMM)-based approach translates the estimation task into successive applications of the proximal mapping of the penalty function. Capitalizing on this direct link, we define the proximal operator as a parametric spline curve and optimize the spline coefficients by minimizing the average reconstruction error for a given training set. The key aspects of our learning method are that the associated penalty function is constrained to be convex and the convergence of the ADMM iterations is proven. As a result of these theoretical guarantees, adaptation of the proposed framework to different levels of measurement noise is extremely simple and does not require any retraining. We apply our method to estimation of both sparse and non-sparse models of L\'{e}vy processes for which the minimum mean square error (MMSE) estimators are available. We carry out a single training session and perform comparisons at various signal-to-noise ratio (SNR) values. Simulations illustrate that the performance of our algorithm is practically identical to the one of the MMSE estimator irrespective of the noise power.
  • In our recent work, the sampling and reconstruction of non-decaying signals, modeled as members of weighted-$L_p$ spaces, were shown to be stable with an appropriate choice of the generating kernel for the shift-invariant reconstruction space. In this paper, we extend the Strang-Fix theory to show that, for $d$-dimensional signals whose derivatives up to order $L$ are all in some weighted-$L_p$ space, the weighted norm of the approximation error can be made to go down as $O(h^L)$ when the sampling step $h$ tends to $0$. The sufficient condition for this decay rate is that the generating kernel belongs to a particular hybrid-norm space and satisfies the Strang-Fix conditions of order $L$. We show that the $O(h^L)$ behavior of the error is attainable for both approximation schemes using projection (when the signal is prefiltered with the dual kernel) and interpolation (when a prefilter is unavailable). The requirement on the signal for the interpolation method, however, is slightly more stringent than that of the projection because we need to increase the smoothness of the signal by a margin of $d/p+\varepsilon$, for arbitrary $\varepsilon >0$. This extra amount of derivatives is used to make sure that the direct sampling is stable.
  • We consider $L^1$-TV regularization of univariate signals with values on the real line or on the unit circle. While the real data space leads to a convex optimization problem, the problem is non-convex for circle-valued data. In this paper, we derive exact algorithms for both data spaces. A key ingredient is the reduction of the infinite search spaces to a finite set of configurations, which can be scanned by the Viterbi algorithm. To reduce the computational complexity of the involved tabulations, we extend the technique of distance transforms to non-uniform grids and to the circular data space. In total, the proposed algorithms have complexity $\mathscr{O}(KN)$ where $N$ is the length of the signal and $K$ is the number of different values in the data set. In particular, the complexity is $\mathscr{O}(N)$ for quantized data. It is the first exact algorithm for TV regularization with circle-valued data, and it is competitive with the state-of-the-art methods for scalar data, assuming that the latter are quantized.
  • The theory of sparse stochastic processes offers a broad class of statistical models to study signals. In this framework, signals are represented as realizations of random processes that are solution of linear stochastic differential equations driven by white L\'evy noises. Among these processes, generalized Poisson processes based on compound-Poisson noises admit an interpretation as random L-splines with random knots and weights. We demonstrate that every generalized L\'evy process-from Gaussian to sparse-can be understood as the limit in law of a sequence of generalized Poisson processes. This enables a new conceptual understanding of sparse processes and suggests simple algorithms for the numerical generation of such objects.
  • Splines come in a variety of flavors that can be characterized in terms of some differential operator L. The simplest piecewise-constant model corresponds to the derivative operator. Likewise, one can extend the traditional notion of total variation by considering more general operators than the derivative. This leads us to the definition of the generalized Beppo-Levi space M, which is further identified as the direct sum of two Banach spaces. We then prove that the minimization of the generalized total variation (gTV) over M, subject to some arbitrary (convex) consistency constraints on the linear measurements of the signal, admits nonuniform L-spline solutions with fewer knots than the number of measurements. This shows that non-uniform splines are universal solutions of continuous-domain linear inverse problems with LASSO, L1, or TV-like regularization constraints. Remarkably, the spline-type is fully determined by the choice of L and does not depend on the actual nature of the measurements.
  • In this paper, we propose a novel deep convolutional neural network (CNN)-based algorithm for solving ill-posed inverse problems. Regularized iterative algorithms have emerged as the standard approach to ill-posed inverse problems in the past few decades. These methods produce excellent results, but can be challenging to deploy in practice due to factors including the high computational cost of the forward and adjoint operators and the difficulty of hyper parameter selection. The starting point of our work is the observation that unrolled iterative methods have the form of a CNN (filtering followed by point-wise non-linearity) when the normal operator (H*H, the adjoint of H times H) of the forward model is a convolution. Based on this observation, we propose using direct inversion followed by a CNN to solve normal-convolutional inverse problems. The direct inversion encapsulates the physical model of the system, but leads to artifacts when the problem is ill-posed; the CNN combines multiresolution decomposition and residual learning in order to learn to remove these artifacts while preserving image structure. We demonstrate the performance of the proposed network in sparse-view reconstruction (down to 50 views) on parallel beam X-ray computed tomography in synthetic phantoms as well as in real experimental sinograms. The proposed network outperforms total variation-regularized iterative reconstruction for the more realistic phantoms and requires less than a second to reconstruct a 512 x 512 image on GPU.
  • We propose a wavefront-based method to estimate the PSF over the whole field of view. This method estimate the aberrations of all the mirrors of the telescope using only field stars. In this proof of concept paper, we described the method and present some qualitative results.
  • In this paper, we study the Besov regularity of d-dimensional L\'evy white noises. More precisely, we describe new sample paths properties of a given white noise in terms of weighted Besov spaces. In particular, the smoothness and integrability properties of L\'evy white noises are characterized using the Blumenthal-Getoor indices. Our techniques rely on wavelet methods and generalized moments estimates for L\'evy white noises.
  • In analogy with steerable wavelets, we present a general construction of adaptable tight wavelet frames, with an emphasis on scaling operations. In particular, the derived wavelets can be "dilated" by a procedure comparable to the operation of steering steerable wavelets. The fundamental aspects of the construction are the same: an admissible collection of Fourier multipliers is used to extend a tight wavelet frame, and the "scale" of the wavelets is adapted by scaling the multipliers. As an application, the proposed wavelets can be used to improve the frequency localization. Importantly, the localized frequency bands specified by this construction can be scaled efficiently using matrix multiplication.
  • In this paper, we study the Besov regularity of L\'evy white noises on the $d$-dimensional torus. Due to their rough sample paths, the white noises that we consider are defined as generalized stochastic fields. We, initially, obtain regularity results for general L\'evy white noises. Then, we focus on two subclasses of noises: compound Poisson and symmetric-$\alpha$-stable (including Gaussian), for which we make more precise statements. Before measuring regularity, we show that the question is well-posed; we prove that Besov spaces are in the cylindrical $\sigma$-field of the space of generalized functions. These results pave the way to the characterization of the $n$-term wavelet approximation properties of stochastic processes.
  • In this paper, we propose a new technique for two-dimensional phase unwrapping. The unwrapped phase is found as the solution of an inverse problem that consists in the minimization of an energy functional. The latter includes a weighted data-fidelity term that favors sparsity in the error between the true and wrapped phase differences, as well as a regularizer based on higher-order total-variation. One desirable feature of our method is its rotation invariance, which allows it to unwrap a much larger class of images compared to the state of the art. We demonstrate the effectiveness of our method through several experiments on simulated and real data obtained through the tomographic phase microscope. The proposed method can enhance the applicability and outreach of techniques that rely on quantitative phase evaluation.
  • We describe a method for imaging 3D objects in a tomographic configuration implemented by training an artificial neural network to reproduce the complex amplitude of the experimentally measured scattered light. The network is designed such that the voxel values of the refractive index of the 3D object are the variables that are adapted during the training process. We demonstrate the method experimentally by forming images of the 3D refractive index distribution of cells.