Neural Style Transfer has shown very exciting results enabling new forms of
image manipulation. Here we extend the existing method to introduce control
over spatial location, colour information and across spatial scale. We
demonstrate how this enhances the method by allowing high-resolution controlled
stylisation and helps to alleviate common failure cases such as applying ground
textures to sky regions. Furthermore, by decomposing style into these
perceptual factors we enable the combination of style information from multiple
sources to generate new, perceptually appealing styles from existing ones. We
also describe how these methods can be used to more efficiently produce large
size, high-quality stylisation. Finally we show how the introduced control
measures can be applied in recent methods for Fast Neural Style Transfer.
Here we present a parametric model for dynamic textures. The model is based
on spatiotemporal summary statistics computed from the feature representations
of a Convolutional Neural Network (CNN) trained on object recognition. We
demonstrate how the model can be used to synthesise new samples of dynamic
textures and to predict motion in simple movies.
This note presents an extension to the neural artistic style transfer
algorithm (Gatys et al.). The original algorithm transforms an image to have
the style of another given image. For example, a photograph can be transformed
to have the style of a famous painting. Here we address a potential shortcoming
of the original method: the algorithm transfers the colors of the original
painting, which can alter the appearance of the scene in undesirable ways. We
describe simple linear methods for transferring style while preserving colors.
Here we demonstrate that the feature space of random shallow convolutional
neural networks (CNNs) can serve as a surprisingly good model of natural
textures. Patches from the same texture are consistently classified as being
more similar then patches from different textures. Samples synthesized from the
model capture spatial correlations on scales much larger then the receptive
field size, and sometimes even rival or surpass the perceptual quality of state
of the art texture models (but show less variability). The current state of the
art in parametric texture synthesis relies on the multi-layer feature space of
deep CNNs that were trained on natural images. Our finding suggests that such
optimized multi-layer feature spaces are not imperative for texture modeling.
Instead, much simpler shallow and convolutional networks can serve as the basis
for novel texture synthesis algorithms.
Here we introduce a new model of natural textures based on the feature spaces
of convolutional neural networks optimised for object recognition. Samples from
the model are of high perceptual quality demonstrating the generative power of
neural networks trained in a purely discriminative fashion. Within the model,
textures are represented by the correlations between feature maps in several
layers of the network. We show that across layers the texture representations
increasingly capture the statistical properties of natural images while making
object information more and more explicit. The model provides a new tool to
generate stimuli for neuroscience and might offer insights into the deep
representations learned by convolutional neural networks.
In fine art, especially painting, humans have mastered the skill to create
unique visual experiences through composing a complex interplay between the
content and style of an image. Thus far the algorithmic basis of this process
is unknown and there exists no artificial system with similar capabilities.
However, in other key areas of visual perception such as object and face
recognition near-human performance was recently demonstrated by a class of
biologically inspired vision models called Deep Neural Networks. Here we
introduce an artificial system based on a Deep Neural Network that creates
artistic images of high perceptual quality. The system uses neural
representations to separate and recombine content and style of arbitrary
images, providing a neural algorithm for the creation of artistic images.
Moreover, in light of the striking similarities between performance-optimised
artificial neural networks and biological vision, our work offers a path
forward to an algorithmic understanding of how humans create and perceive