
Humans possess the capability to reason at an abstract level and to structure
information into abstract categories, but the underlying neural processes have
remained unknown. Experimental evidence has recently emerged for the
organization of an important aspect of abstract reasoning: for assigning words
to semantic roles in a sentence, such as agent (or subject) and patient (or
object). Using minimal assumptions, we show how such a binding of words to
semantic roles emerges in a generic spiking neural network through Hebbian
plasticity. The resulting model is consistent with the experimental data and
enables new computational functionalities such as structured information
retrieval, copying data, and comparisons. It thus provides a basis for the
implementation of more demanding cognitive computations by networks of spiking
neurons.

Synaptic plasticity is implemented and controlled through over thousand
different types of molecules in the postsynaptic density and presynaptic
boutons that assume a staggering array of different states through
phosporylation and other mechanisms. One of the most prominent molecule in the
postsynaptic density is CaMKII, that is described in molecular biology as a
"memory molecule" that can integrate through autophosporylation Cainflux
signals on a relatively large time scale of dozens of seconds. The functional
impact of this memory mechanism is largely unknown. We show that the
experimental data on the specific role of CaMKII activation in dopaminegated
spine consolidation suggest a general functional role in speeding up
rewardguided search for network configurations that maximize reward
expectation. Our theoretical analysis shows that stochastic search could in
principle even attain optimal network configurations by emulating one of the
most wellknown nonlinear optimization methods, simulated annealing. But this
optimization is usually impeded by slowness of stochastic search at a given
temperature. We propose that CaMKII contributes a momentum term that
substantially speeds up this search. In particular, it allows the network to
overcome saddle points of the fitness function. The resulting improved
stochastic policy search can be understood on a more abstract level as
Hamiltonian sampling, which is known to be one of the most efficient stochastic
search methods.

Networks of spiking neurons (SNNs) are frequently studied as models for
networks of neurons in the brain, but also as paradigm for novel energy
efficient computing hardware. In principle they are especially suitable for
computations in the temporal domain, such as speech processing, because their
computations are carried out via events in time and space. But so far they have
been lacking the capability to preserve information for longer time spans
during a computation, until it is updated or needed  like a register of a
digital computer. This function is provided to artificial neural networks
through Long ShortTerm Memory (LSTM) units. We show here that SNNs attain
similar capabilities if one includes adapting neurons in the network.
Adaptation denotes an increase of the firing threshold of a neuron after
preceding firing. A substantial fraction of neurons in the neocortex of rodents
and humans has been found to be adapting. It turns out that if adapting neurons
are integrated in a suitable manner into the architecture of SNNs, the
performance of these enhanced SNNs, which we call LSNNs, for computation in the
temporal domain approaches that of artificial neural networks with LSTMunits.
In addition, the computing and learning capabilities of LSNNs can be
substantially enhanced through learningtolearn (L2L) methods from machine
learning, that have so far been applied primarily to LSTM networks and
apparently never to SSNs.
This preliminary report on arXiv will be replaced by a more detailed version
in about a month.

Neuromorphic hardware tends to pose limits on the connectivity of deep
networks that one can run on them. But also generic hardware and software
implementations of deep learning run more efficiently for sparse networks.
Several methods exist for pruning connections of a neural network after it was
trained without connectivity constraints. We present an algorithm, DEEP R, that
enables us to train directly a sparsely connected neural network. DEEP R
automatically rewires the network during supervised training so that
connections are there where they are most needed for the task, while its total
number is all the time strictly bounded. We demonstrate that DEEP R can be used
to train very sparse feedforward and recurrent neural networks on standard
benchmark tasks with just a minor loss in performance. DEEP R is based on a
rigorous theoretical foundation that views rewiring as stochastic sampling of
network configurations from a posterior.

Synaptic connections between neurons in the brain are dynamic because of
continuously ongoing spine dynamics, axonal sprouting, and other processes. In
fact, it was recently shown that the spontaneous synapseautonomous component
of spine dynamics is at least as large as the component that depends on the
history of pre and postsynaptic neural activity. These data are inconsistent
with common models for network plasticity, and raise the questions how neural
circuits can maintain a stable computational function in spite of these
continuously ongoing processes, and what functional uses these ongoing
processes might have. Here, we present a rigorous theoretical framework for
these seemingly stochastic spine dynamics and rewiring processes in the context
of rewardbased learning tasks. We show that spontaneous synapseautonomous
processes, in combination with reward signals such as dopamine, can explain the
capability of networks of neurons in the brain to configure themselves for
specific computational tasks, and to compensate automatically for later changes
in the network or task. Furthermore we show theoretically and through computer
simulations that stable computational performance is compatible with
continuously ongoing synapseautonomous changes. After reaching good
computational performance it causes primarily a slow drift of network
architecture and dynamics in taskirrelevant dimensions, as observed for neural
activity in motor cortex and other areas. On the more abstract level of
reinforcement learning the resulting model gives rise to an understanding of
rewarddriven network plasticity as continuous sampling of network
configurations.

Previous theoretical studies on the interaction of excitatory and inhibitory
neurons proposed to model this cortical microcircuit motif as a socalled
WinnerTakeAll (WTA) circuit. A recent modeling study however found that the
WTA model is not adequate for databased softer forms of divisive inhibition as
found in a microcircuit motif in cortical layer 2/3. We investigate here
through theoretical analysis the role of such softer divisive inhibition for
the emergence of computational operations and neural codes under spiketiming
dependent plasticity (STDP). We show that in contrast to WTA models  where the
network activity has been interpreted as probabilistic inference in a
generative mixture distribution  this network dynamics approximates inference
in a noisyORlike generative model that explains the network input based on
multiple hidden causes. Furthermore, we show that STDP optimizes the parameters
of this model by approximating online the expectation maximization (EM)
algorithm. This theoretical analysis corroborates a preceding modelling study
which suggested that the learning dynamics of this layer 2/3 microcircuit motif
extracts a specific modular representation of the input and thus performs blind
source separation on the input statistics.

Despite being originally inspired by the central nervous system, artificial
neural networks have diverged from their biological archetypes as they have
been remodeled to fit particular tasks. In this paper, we review several
possibilites to reverse map these architectures to biologically more realistic
spiking networks with the aim of emulating them on fast, lowpower neuromorphic
hardware. Since many of these devices employ analog components, which cannot be
perfectly controlled, finding ways to compensate for the resulting effects
represents a key challenge. Here, we discuss three different strategies to
address this problem: the addition of auxiliary network components for
stabilizing activity, the utilization of inherently robust architectures and a
training method for hardwareemulated networks that functions without perfect
knowledge of the system's dynamics and parameters. For all three scenarios, we
corroborate our theoretical considerations with experimental results on
accelerated analog neuromorphic platforms.

Cortical microcircuits are very complex networks, but they are composed of a
relatively small number of stereotypical motifs. Hence one strategy for
throwing light on the computational function of cortical microcircuits is to
analyze emergent computational properties of these stereotypical microcircuit
motifs. We are addressing here the question how spiketiming dependent
plasticity (STDP) shapes the computational properties of one motif that has
frequently been studied experimentally: interconnected populations of pyramidal
cells and parvalbuminpositive inhibitory cells in layer 2/3. Experimental
studies suggest that these inhibitory neurons exert some form of divisive
inhibition on the pyramidal cells. We show that this databased form of
feedback inhibition, which is softer than that of winnertakeall models that
are commonly considered in theoretical analyses, contributes to the emergence
of an important computational function through STDP: The capability to
disentangle superimposed firing patterns in upstream networks, and to represent
their information content through a sparse assembly code.

Emulating spiking neural networks on analog neuromorphic hardware offers
several advantages over simulating them on conventional computers, particularly
in terms of speed and energy consumption. However, this usually comes at the
cost of reduced control over the dynamics of the emulated networks. In this
paper, we demonstrate how iterative training of a hardwareemulated network can
compensate for anomalies induced by the analog substrate. We first convert a
deep neural network trained in software to a spiking network on the BrainScaleS
waferscale neuromorphic system, thereby enabling an acceleration factor of 10
000 compared to the biological time domain. This mapping is followed by the
intheloop training, where in each training step, the network activity is
first recorded in hardware and then used to compute the parameter updates in
software via backpropagation. An essential finding is that the parameter
updates do not have to be precise, but only need to approximately follow the
correct gradient, which simplifies the computation of updates. Using this
approach, after only several tens of iterations, the spiking network shows an
accuracy close to the ideal softwareemulated prototype. The presented
techniques show that deep spiking networks emulated on analog neuromorphic
devices can attain good computational performance despite the inherent
variations of the analog substrate.

General results from statistical learning theory suggest to understand not
only brain computations, but also brain plasticity as probabilistic inference.
But a model for that has been missing. We propose that inherently stochastic
features of synaptic plasticity and spine motility enable cortical networks of
neurons to carry out probabilistic inference by sampling from a posterior
distribution of network configurations. This model provides a viable
alternative to existing models that propose convergence of parameters to
maximum likelihood values. It explains how priors on weight distributions and
connection probabilities can be merged optimally with learned experience, how
cortical networks can generalize learned information so well to novel
experiences, and how they can compensate continuously for unforeseen
disturbances of the network. The resulting new theory of network plasticity
explains from a functional perspective a number of experimental data on
stochastic aspects of synaptic plasticity that previously appeared to be quite
puzzling.

Conventional neurocomputing architectures and artificial neural networks
have often been developed with no or loose connections to neuroscience. As a
consequence, they have largely ignored key features of biological neural
processing systems, such as their extremely lowpower consumption features or
their ability to carry out robust and efficient computation using massively
parallel arrays of limited precision, highly variable, and unreliable
components. Recent developments in nanotechnologies are making available
extremely compact and lowpower, but also variable and unreliable solidstate
devices that can potentially extend the offerings of availing CMOS
technologies. In particular, memristors are regarded as a promising solution
for modeling key features of biological synapses due to their nanoscale
dimensions, their capacity to store multiple bits of information per element
and the low energy required to write distinct states. In this paper, we first
review the neuro and neuromorphiccomputing approaches that can best exploit
the properties of memristor andscale devices, and then propose a novel hybrid
memristorCMOS neuromorphic circuit which represents a radical departure from
conventional neurocomputing approaches, as it uses memristors to directly
emulate the biophysics and temporal dynamics of real synapses. We point out the
differences between the use of memristors in conventional neurocomputing
architectures and the hybrid memristorCMOS circuit proposed, and argue how
this circuit represents an ideal building block for implementing braininspired
probabilistic computing paradigms that are robust to variability and
faulttolerant by design.