• The modulation transfer function (MTF) is widely used to characterise the performance of optical systems. Measuring it is costly and it is thus rarely available for a given lens specimen. Instead, MTFs based on simulations or, at best, MTFs measured on other specimens of the same lens are used. Fortunately, images recorded through an optical system contain ample information about its MTF, only that it is confounded with the statistics of the images. This work presents a method to estimate the MTF of camera lens systems directly from photographs, without the need for expensive equipment. We use a custom grid display to accurately measure the point response of lenses to acquire ground truth training data. We then use the same lenses to record natural images and employ a data-driven supervised learning approach using a convolutional neural network to estimate the MTF on small image patches, aggregating the information into MTF charts over the entire field of view. It generalises to unseen lenses and can be applied for single photographs, with the performance improving if multiple photographs are available.
  • As handheld video cameras are now commonplace and available in every smartphone, images and videos can be recorded almost everywhere at anytime. However, taking a quick shot frequently yields a blurry result due to unwanted camera shake during recording or moving objects in the scene. Removing these artifacts from the blurry recordings is a highly ill-posed problem as neither the sharp image nor the motion blur kernel is known. Propagating information between multiple consecutive blurry observations can help restore the desired sharp image or video. Solutions for blind deconvolution based on neural networks rely on a massive amount of ground-truth data which is hard to acquire. In this work, we propose an efficient approach to produce a significant amount of realistic training data and introduce a novel recurrent network architecture to deblur frames taking temporal information into account, which can efficiently handle arbitrary spatial and temporal input sizes. We demonstrate the versatility of our approach in a comprehensive comparison on a number of challening real-world examples.
  • Single image super-resolution is the task of inferring a high-resolution image from a single low-resolution input. Traditionally, the performance of algorithms for this task is measured using pixel-wise reconstruction measures such as peak signal-to-noise ratio (PSNR) which have been shown to correlate poorly with the human perception of image quality. As a result, algorithms minimizing these metrics tend to produce over-smoothed images that lack high-frequency textures and do not look natural despite yielding high PSNR values. We propose a novel application of automated texture synthesis in combination with a perceptual loss focusing on creating realistic textures rather than optimizing for a pixel-accurate reproduction of ground truth images during training. By using feed-forward fully convolutional neural networks in an adversarial training setting, we achieve a significant boost in image quality at high magnification ratios. Extensive experiments on a number of datasets show the effectiveness of our approach, yielding state-of-the-art results in both quantitative and qualitative benchmarks.
  • Convolutional neural networks (CNNs) show impressive performance for image classification and detection, extending heavily to the medical image domain. Nevertheless, medical experts are sceptical in these predictions as the nonlinear multilayer structure resulting in a classification outcome is not directly graspable. Recently, approaches have been shown which help the user to understand the discriminative regions within an image which are decisive for the CNN to conclude to a certain class. Although these approaches could help to build trust in the CNNs predictions, they are only slightly shown to work with medical image data which often poses a challenge as the decision for a class relies on different lesion areas scattered around the entire image. Using the DiaretDB1 dataset, we show that on retina images different lesion areas fundamental for diabetic retinopathy are detected on an image level with high accuracy, comparable or exceeding supervised methods. On lesion level, we achieve few false positives with high sensitivity, though, the network is solely trained on image-level labels which do not include information about existing lesions. Classifying between diseased and healthy images, we achieve an AUC of 0.954 on the DiaretDB1.
  • State-of-the-art video deblurring methods are capable of removing non-uniform blur caused by unwanted camera shake and/or object motion in dynamic scenes. However, most existing methods are based on batch processing and thus need access to all recorded frames, rendering them computationally demanding and time consuming and thus limiting their practical use. In contrast, we propose an online (sequential) video deblurring method based on a spatio-temporal recurrent network that allows for real-time performance. In particular, we introduce a novel architecture which extends the receptive field while keeping the overall size of the network small to enable fast execution. In doing so, our network is able to remove even large blur caused by strong camera shake and/or fast moving objects. Furthermore, we propose a novel network layer that enforces temporal consistency between consecutive frames by dynamic temporal blending which compares and adaptively (at test time) shares features obtained at different time steps. We show the superiority of the proposed method in an extensive experimental evaluation.
  • Recently, several discriminative learning approaches have been proposed for effective image restoration, achieving convincing trade-off between image quality and computational efficiency. However, these methods require separate training for each restoration task (e.g., denoising, deblurring, demosaicing) and problem condition (e.g., noise level of input images). This makes it time-consuming and difficult to encompass all tasks and conditions during training. In this paper, we propose a discriminative transfer learning method that incorporates formal proximal optimization and discriminative learning for general image restoration. The method requires a single-pass training and allows for reuse across various problems and conditions while achieving an efficiency comparable to previous discriminative approaches. Furthermore, after being trained, our model can be easily transferred to new likelihood terms to solve untrained tasks, or be combined with existing priors to further improve image restoration quality.
  • Light field photography captures rich structural information that may facilitate a number of traditional image processing and computer vision tasks. A crucial ingredient in such endeavors is accurate depth recovery. We present a novel framework that allows the recovery of a high quality continuous depth map from light field data. To this end we propose a generative model of a light field that is fully parametrized by its corresponding depth map. The model allows for the integration of powerful regularization techniques such as a non-local means prior, facilitating accurate depth map estimation.
  • We present a neural network model approach for multi-frame blind deconvolution. The discriminative approach adopts and combines two recent techniques for image deblurring into a single neural network architecture. Our proposed hybrid-architecture combines the explicit prediction of a deconvolution filter and non-trivial averaging of Fourier coefficients in the frequency domain. In order to make full use of the information contained in all images in one burst, the proposed network embeds smaller networks, which explicitly allow the model to transfer information between images in early layers. Our system is trained end-to-end using standard backpropagation on a set of artificially generated training examples, enabling competitive performance in multi-frame blind deconvolution, both with respect to quality and runtime.
  • We present a feasibility study for a high frame rate, short baseline auroral tomographic imaging system useful for estimating parametric variations in the precipitating electron number flux spectrum of dynamic auroral events. Of particular interest are auroral substorms, characterized by spatial variations of order 100 m and temporal variations of order 10 ms. These scales are thought to be produced by dispersive Alfv\'en waves in the near-Earth magnetosphere. The auroral tomography system characterized in this paper reconstructs the auroral volume emission rate to estimate the characteristic energy and location in the direction perpendicular to the geomagnetic field of peak electron precipitation flux using a distributed network of precisely synchronized ground-based cameras. As the observing baseline decreases, the tomographic inverse problem becomes highly ill-conditioned; as the sampling rate increases, the signal-to-noise ratio degrades and synchronization requirements become increasingly critical. Our approach to these challenges uses a physics-based auroral model to regularize the poorly-observed vertical dimension. Specifically, the vertical dimension is expanded in a low-dimensional basis consisting of eigenprofiles computed over the range of expected energies in the precipitating electron flux, while the horizontal dimension retains a standard orthogonal pixel basis. Simulation results show typical characteristic energy estimation error less than 30% for a 3 km baseline achievable within the confines of the Poker Flat Research Range, using GPS-synchronized Electron Multiplying CCD cameras with broad-band BG3 optical filters that pass prompt auroral emissions.
  • In software and hardware development MATLAB and Simulink are used to model cyber physical systems for many years, , especially in automation technology and the automotive industry. Compliance with the required product quality and project efficiency is facilitated by analyzing and transforming Simulink models. The existing API, provided by MATLAB is only suitable for programmatic changing of Simulink models. We show using our own tool which is used in industry, how such as a Simulink model can be edited more easily. For this purpose the model, is converted to an object-oriented class structure that provides convenient access and editing APIs and allows applying well-known algorithms and analyses from graph theory directly. It is also designed as a bi-directional tool, so it transforms a Simulink model into a graph representation and vice versa. ----- In der Software- und Hardwareentwicklung wird seit Jahren verst\"arkt MATLAB und Simulink f\"ur die Modellierung von cyberphysikalischen Systemen, insbesondere in der Automatisierungstechnik und der Automobilindustrie eingesetzt. Die Einhaltung der notwendigen Produktqualit\"at und Projekteffizienz wird durch Analysen und Transformationen auf Simulink-Modellen erleichtert. Die bestehende, von MATLAB bereitgestellte, API ist f\"ur die programmatische Ver\"anderung von Simulink-Modellen nur bedingt geeignet. Wir zeigen deshalb anhand eines eigenen, im industriellen Einsatz befindlichen Werkzeugs, wie ein Simulink-Modell leichter bearbeitet werden kann. Dazu wird es in eine objektorientierte Klassenstruktur \"uberf\"uhrt, die einen komfortablen Zugang und Bearbeitungs-APIs bietet und es erlaubt bekannte Algorithmen und Analysen aus der Graphentheorie direkt anzuwenden. Das Werkzeug ist bidirektional entworfen, es transformiert also ein Simulink-Modell in eine Graphenrepresentation und umgekehrt.
  • We describe a learning-based approach to blind image deconvolution. It uses a deep layered architecture, parts of which are borrowed from recent work on neural network learning, and parts of which incorporate computations that are specific to image deconvolution. The system is trained end-to-end on a set of artificially generated training examples, enabling competitive performance in blind deconvolution, both with respect to quality and runtime.
  • In today's mailing, Hogg et al. propose image modeling techniques to maintain 10-ppm-level precision photometry in Kepler data with only two working reaction wheels. While these results are relevant to many scientific goals for the repurposed mission, all modeling efforts so far have used a toy model of the Kepler telescope. Because the two-wheel performance of Kepler remains to be determined, we advocate for the consideration of an alternate strategy for a >1 year program that maximizes the science return from the "low-torque" fields across the ecliptic plane. Assuming we can reach the precision of the original Kepler mission, we expect to detect 800 new planet candidates in the first year of such a mission. Our proposed strategy has benefits for transit timing variation and transit duration variation studies, especially when considered in concert with the future TESS mission. We also expect to help address the first key science goal of Kepler: the frequency of planets in the habitable zone as a function of spectral type.
  • Cosmic shear is a powerful probe of cosmological parameters, but its potential can be fully utilised only if galaxy shapes are measured with great accuracy. Two major effects have been identified which are likely to account for most of the bias for maximum likelihood methods in recent shear measurement challenges. Model bias occurs when the true galaxy shape is not well represented by the fitted model. Noise bias occurs due to the non-linear relationship between image pixels and galaxy shape. In this paper we investigate the potential interplay between these two effects when an imperfect model is used in the presence of high noise. We present analytical expressions for this bias, which depends on the residual difference between the model and real data. They can lead to biases not accounted for in previous calibration schemes. By measuring the model bias, noise bias and their interaction, we provide a complete statistical framework for measuring galaxy shapes with model fitting methods from GRavitational lEnsing Accuracy Testing (GREAT) like images. We demonstrate the noise and model interaction bias using a simple toy model, which indicates that this effect can potentially be significant. Using real galaxy images from the Cosmological Evolution Survey (COSMOS) we quantify the strength of the model bias, noise bias and their interaction. We find that the interaction term is often a similar size to the model bias term, and is smaller than the requirements of the current and shortly upcoming galaxy surveys.
  • We establish a link between Fourier optics and a recent construction from the machine learning community termed the kernel mean map. Using the Fraunhofer approximation, it identifies the kernel with the squared Fourier transform of the aperture. This allows us to use results about the invertibility of the kernel mean map to provide a statement about the invertibility of Fraunhofer diffraction, showing that imaging processes with arbitrarily small apertures can in principle be invertible, i.e., do not lose information, provided the objects to be imaged satisfy a generic condition. A real world experiment shows that we can super-resolve beyond the Rayleigh limit.
  • We present and describe im3shape, a new publicly available galaxy shape measurement code for weak gravitational lensing shear. im3shape performs a maximum likelihood fit of a bulge-plus-disc galaxy model to noisy images, incorporating an applied point spread function. We detail challenges faced and choices made in its design and implementation, and then discuss various limitations that affect this and other maximum likelihood methods. We assess the bias arising from fitting an incorrect galaxy model using simple noise-free images and find that it should not be a concern for current cosmic shear surveys. We test im3shape on the GREAT08 Challenge image simulations, and meet the requirements for upcoming cosmic shear surveys in the case that the simulations are encompassed by the fitted model, using a simple correction for image noise bias. For the fiducial branch of GREAT08 we obtain a negligible additive shear bias and sub-two percent level multiplicative bias, which is suitable for analysis of current surveys. We fall short of the sub-percent level requirement for upcoming surveys, which we attribute to a combination of noise bias and the mis-match between our galaxy model and the model used in the GREAT08 simulations. We meet the requirements for current surveys across all branches of GREAT08, except those with small or high noise galaxies, which we would cut from our analysis. Using the GREAT08 metric we we obtain a score of Q=717 for the usable branches, relative to the goal of Q=1000 for future experiments. The code is freely available from https://bitbucket.org/joezuntz/im3shape
  • Weak gravitational lensing has the potential to constrain cosmological parameters to high precision. However, as shown by the Shear TEsting Programmes (STEP) and GRavitational lEnsing Accuracy Testing (GREAT) Challenges, measuring galaxy shears is a nontrivial task: various methods introduce different systematic biases which have to be accounted for. We investigate how pixel noise on the image affects the bias on shear estimates from a Maximum-Likelihood forward model-fitting approach using a sum of co-elliptical S\'{e}rsic profiles, in complement to the theoretical approach of an an associated paper. We evaluate the bias using a simple but realistic galaxy model and find that the effects of noise alone can cause biases of order 1-10% on measured shears, which is significant for current and future lensing surveys. We evaluate a simulation-based calibration method to create a bias model as a function of galaxy properties and observing conditions. This model is then used to correct the simulated measurements. We demonstrate that this method can effectively reduce noise bias so that shear measurement reaches the level of accuracy required for estimating cosmic shear in upcoming lensing surveys.
  • GRavitational lEnsing Accuracy Testing 2010 (GREAT10) is a public image analysis challenge aimed at the development of algorithms to analyze astronomical images. Specifically, the challenge is to measure varying image distortions in the presence of a variable convolution kernel, pixelization and noise. This is the second in a series of challenges set to the astronomy, computer science and statistics communities, providing a structured environment in which methods can be improved and tested in preparation for planned astronomical surveys. GREAT10 extends upon previous work by introducing variable fields into the challenge. The "Galaxy Challenge" involves the precise measurement of galaxy shape distortions, quantified locally by two parameters called shear, in the presence of a known convolution kernel. Crucially, the convolution kernel and the simulated gravitational lensing shape distortion both now vary as a function of position within the images, as is the case for real data. In addition, we introduce the "Star Challenge" that concerns the reconstruction of a variable convolution kernel, similar to that in a typical astronomical observation. This document details the GREAT10 Challenge for potential participants. Continually updated information is also available from http://www.greatchallenges.info.
  • We present the results of the GREAT08 Challenge, a blind analysis challenge to infer weak gravitational lensing shear distortions from images. The primary goal was to stimulate new ideas by presenting the problem to researchers outside the shear measurement community. Six GREAT08 Team methods were presented at the launch of the Challenge and five additional groups submitted results during the 6 month competition. Participants analyzed 30 million simulated galaxies with a range in signal to noise ratio, point-spread function ellipticity, galaxy size, and galaxy type. The large quantity of simulations allowed shear measurement methods to be assessed at a level of accuracy suitable for currently planned future cosmic shear observations for the first time. Different methods perform well in different parts of simulation parameter space and come close to the target level of accuracy in several of these. A number of fresh ideas have emerged as a result of the Challenge including a re-examination of the process of combining information from different galaxies, which reduces the dependence on realistic galaxy modelling. The image simulations will become increasingly sophisticated in future GREAT challenges, meanwhile the GREAT08 simulations remain as a benchmark for additional developments in shear measurement algorithms.