• Recent experimental advances are producing an avalanche of data on both neural connectivity and neural activity. To take full advantage of these two emerging datasets we need a framework that links them, revealing how collective neural activity arises from the structure of neural connectivity and intrinsic neural dynamics. This problem of {\it structure-driven activity} has drawn major interest in computational neuroscience. Existing methods for relating activity and architecture in spiking networks rely on linearizing activity around a central operating point and thus fail to capture the nonlinear responses of individual neurons that are the hallmark of neural information processing. Here, we overcome this limitation and present a new relationship between connectivity and activity in networks of nonlinear spiking neurons by developing a diagrammatic fluctuation expansion based on statistical field theory. We explicitly show how recurrent network structure produces pairwise and higher-order correlated activity, and how nonlinearities impact the networks' spiking activity. Our findings open new avenues to investigating how single-neuron nonlinearities---including those of different cell types---combine with connectivity to shape population activity and function.
  • An essential step toward understanding neural circuits is linking their structure and their dynamics. In general, this relationship can be almost arbitrarily complex. Recent theoretical work has, however, begun to identify some broad principles underlying collective spiking activity in neural circuits. The first is that local features of network connectivity can be surprisingly effective in predicting global statistics of activity across a network. The second is that, for the important case of large networks with excitatory-inhibitory balance, correlated spiking persists or vanishes depending on the spatial scales of recurrent and feedforward connectivity. We close by showing how these ideas, together with plasticity rules, can help to close the loop between network structure and activity statistics.
  • Advances in synthetic biology allow us to engineer bacterial collectives with pre-specified characteristics. However, the behavior of these collectives is difficult to understand, as cellular growth and division as well as extra-cellular fluid flow lead to complex, changing arrangements of cells within the population. To rationally engineer and control the behavior of cell collectives we need theoretical and computational tools to understand their emergent spatiotemporal dynamics. Here, we present an agent-based model that allows growing cells to detect and respond to mechanical interactions. Crucially, our model couples the dynamics of cell growth to the cell's environment: Mechanical constraints can affect cellular growth rate and a cell may alter its behavior in response to these constraints. This coupling links the mechanical forces that influence cell growth and emergent behaviors in cell assemblies. We illustrate our approach by showing how mechanical interactions can impact the dynamics of bacterial collectives growing in microfluidic traps.
  • In a constantly changing world, animals must account for environmental volatility when making decisions. To appropriately discount older, irrelevant information, they need to learn the rate at which the environment changes. We develop an ideal observer model capable of inferring the present state of the environment along with its rate of change. Key to this computation is an update of the posterior probability of all possible changepoint counts. This computation can be challenging, as the number of possibilities grows rapidly with time. However, we show how the computations can be simplified in the continuum limit by a moment closure approximation. The resulting low-dimensional system can be used to infer the environmental state and change rate with accuracy comparable to the ideal observer. The approximate computations can be performed by a neural network model via a rate-correlation based plasticity rule. We thus show how optimal observers accumulate evidence in changing environments, and map this computation to reduced models which perform inference using plausible neural mechanisms.
  • We consider model social networks in which information propagates directionally across layers of rational agents. Each agent makes a locally optimal estimate of the state of the world, and communicates this estimate to agents downstream. When agents receive information from the same source their estimates are correlated. We show that the resulting redundancy can lead to the loss of information about the state of the world across layers of the network, even when all agents have full knowledge of the network's structure. A simple algebraic condition identifies networks in which information loss occurs, and we show that all such networks must contain a particular network motif. We also study random networks asymptotically as the number of agents increases, and find a sharp transition in the probability of information loss at the point at which the number of agents in one layer exceeds the number in the previous layer.
  • We assess the impact of cell cycle noise on gene circuit dynamics. For bistable genetic switches and excitable circuits, we find that transitions between metastable states most likely occur just after cell division and that this concentration effect intensifies in the presence of transcriptional delay. We explain this concentration effect with a 3-states stochastic model. For genetic oscillators, we quantify the temporal correlations between daughter cells induced by cell division. Temporal correlations must be captured properly in order to accurately quantify noise sources within gene networks.
  • Organisms and ecological groups accumulate evidence to make decisions. Classic experiments and theoretical studies have explored this process when the correct choice is fixed during each trial. However, we live in a constantly changing world. What effect does such impermanence have on classical results about decision making? To address this question we use sequential analysis to derive a tractable model of evidence accumulation when the correct option changes in time. Our analysis shows that ideal observers discount prior evidence at a rate determined by the volatility of the environment, and the dynamics of evidence accumulation is governed by the information gained over an average environmental epoch. A plausible neural implementation of an optimal observer in a changing environment shows that, in contrast to previous models, neural populations representing alternate choices are coupled through excitation. Our work builds a bridge between statistical decision making in volatile environments and stochastic nonlinear dynamics.
  • Neuronal circuits can learn and replay firing patterns evoked by sequences of sensory stimuli. After training, a brief cue can trigger a spatiotemporal pattern of neural activity similar to that evoked by a learned stimulus sequence. Network models show that such sequence learning can occur through the shaping of feedforward excitatory connectivity via long term plasticity. Previous models describe how event order can be learned, but they typically do not explain how precise timing can be recalled. We propose a mechanism for learning both the order and precise timing of event sequences. In our recurrent network model, long term plasticity leads to the learning of the sequence, while short term facilitation enables temporally precise replay of events. Learned synaptic weights between populations determine the time necessary for one population to activate another. Long term plasticity adjusts these weights so that the trained event times are matched during playback. While we chose short term facilitation as a time-tracking process, we also demonstrate that other mechanisms, such as spike rate adaptation, can fulfill this role. We also analyze the impact of trial-to-trial variability, showing how observational errors as well as neuronal noise result in variability in learned event times. The dynamics of the playback process determine how stochasticity is inherited in learned sequence timings. Future experiments that characterize such variability can therefore shed light on the neural mechanisms of sequence learning.
  • Humans and other animals base their decisions on noisy sensory input. Much work has therefore been devoted to understanding the computations that underly such decisions. The problem has been studied in a variety of tasks and with stimuli of differing complexity. However, the impact of correlations in sensory noise on perceptual judgments is not well understood. Here we examine how stimulus correlations together with correlations in sensory noise impact decision making. As an example, we consider the task of detecting the presence of a single or multiple targets amongst distractors. We assume that both the distractors and the observer's measurements of the stimuli are correlated. The computations of an optimal observer in this task are nontrivial, yet can be analyzed and understood intuitively. We find that when distractors are strongly correlated, measurement correlations can have a strong impact on performance. When distractor correlations are weak, measurement correlations have little impact, unless the number of stimuli is large. Correlations in neural responses to structured stimuli can therefore strongly impact perceptual judgments.
  • In biochemical networks, reactions often occur on disparate timescales and can be characterized as either "fast" or "slow." The quasi-steady state approximation (QSSA) utilizes timescale separation to project models of biochemical networks onto lower-dimensional slow manifolds. As a result, fast elementary reactions are not modeled explicitly, and their effect is captured by non-elementary reaction rate functions (e.g. Hill functions). The accuracy of the QSSA applied to deterministic systems depends on how well timescales are separated. Recently, it has been proposed to use the non-elementary rate functions obtained via the deterministic QSSA to define propensity functions in stochastic simulations of biochemical networks. In this approach, termed the stochastic QSSA, fast reactions that are part of non-elementary reactions are not simulated, greatly reducing computation time. However, it is unclear when the stochastic QSSA provides an accurate approximation of the original stochastic simulation. We show that, unlike the deterministic QSSA, the validity of the stochastic QSSA does not follow from timescale separation alone, but also depends on the sensitivity of the non-elementary reaction rate functions to changes in the slow species. The stochastic QSSA becomes more accurate when this sensitivity is small. Different types of QSSAs result in non-elementary functions with different sensitivities, and the total QSSA results in less sensitive functions than the standard or the pre-factor QSSA. We prove that, as a result, the stochastic QSSA becomes more accurate when non-elementary reaction functions are obtained using the total QSSA. Our work provides a novel condition for the validity of the QSSA in stochastic simulations of biochemical reaction networks with disparate timescales.
  • In mammals, most cells in the brain and peripheral tissues generate circadian (~24hr) rhythms autonomously. These self-sustained rhythms are coordinated and entrained by a master circadian clock in the suprachiasmatic nucleus (SCN). Within the SCN, the individual rhythms of each neuron are synchronized through intercellular signaling. One important feature of SCN is that the synchronized period is close to the cell population mean of intrinsic periods. In this way, the synchronized period of the SCN stays close to the periods of cells in peripheral tissues. This is important for SCN to entrain cells throughout the body. However, the mechanism that drives the period of the coupled SCN cells to the population mean is not known. We use mathematical modeling and analysis to show that the mechanism of transcription repression plays a pivotal role in regulating the coupled period. Specifically, we use phase response curve analysis to show that the coupled period within the SCN stays near the population mean if transcriptional repression occurs via protein sequestration. In contrast, the coupled period is far from the mean if repression occurs through highly nonlinear Hill-type regulation (e.g. oligomer- or phosphorylation-based repression). Furthermore, we find that the timescale of intercellular coupling needs to be fast compared to that of intracellular feedback to maintain the mean period. These findings reveal the important relationship between the intracellular transcriptional feedback loop and intercellular coupling. This relationship explains why transcriptional repression appears to occur via protein sequestration in multicellular organisms, mammals and Drosophila, in contrast with the phosphorylation-based repression in unicellular organisms. That is, transition to protein sequestration is essential for synchronizing multiple cells with a period close to the population mean (~24hr).
  • Delay is an important and ubiquitous aspect of many biochemical processes. For example, delay plays a central role in the dynamics of genetic regulatory networks as it stems from the sequential assembly of first mRNA and then protein. Genetic regulatory networks are therefore frequently modeled as stochastic birth-death processes with delay. Here we examine the relationship between delay birth-death processes and their appropriate approximating delay chemical Langevin equations. We prove that the distance between these two descriptions, as measured by expectations of functionals of the processes, converges to zero with increasing system size. Further, we prove that the delay birth-death process converges to the thermodynamic limit as system size tends to infinity. Our results hold for both fixed delay and distributed delay. Simulations demonstrate that the delay chemical Langevin approximation is accurate even at moderate system sizes. It captures dynamical features such as the spatial and temporal distributions of transition pathways in metastable systems, oscillatory behavior in negative feedback circuits, and cross-correlations between nodes in a network. Overall, these results provide a foundation for using delay stochastic differential equations to approximate the dynamics of birth-death processes with delay.
  • How does connectivity impact network dynamics? We address this question by linking network characteristics on two scales. On the global scale we consider the coherence of overall network dynamics. We show that such \emph{global coherence} in activity can often be predicted from the \emph{local structure} of the network. To characterize local network structure we use "motif cumulants," a measure of the deviation of pathway counts from those expected in a minimal probabilistic network model. We extend previous results in three ways. First, we give a new combinatorial formulation of motif cumulants that relates to the allied concept in probability theory. Second, we show that the link between global network dynamics and local network architecture is strongly affected by heterogeneity in network connectivity. However, we introduce a network-partitioning method that recovers a tight relationship between architecture and dynamics. Third, for a particular set of models we generalize the underlying theory to treat dynamical coherence at arbitrary orders (i.e. triplet correlations, and beyond). We show that at any order only a highly restricted set of motifs impact dynamical correlations.
  • Persistent activity in neuronal populations has been shown to represent the spatial position of remembered stimuli. Networks that support bump attractors are often used to model such persistent activity. Such models usually exhibit translational symmetry. Thus activity bumps are neutrally stable, and perturbations in position do not decay away. We extend previous work on bump attractors by constructing model networks capable of encoding the certainty or salience of a stimulus stored in memory. Such networks support bumps that are not only neutrally stable to perturbations in position, but also perturbations in amplitude. Possible bump solutions then lie on a two-dimensional attractor, determined by a continuum of positions and amplitudes. Such an attractor requires precisely balancing the strength of recurrent synaptic connections. The amplitude of activity bumps represents certainty, and is determined by the initial input to the system. Moreover, bumps with larger amplitudes are more robust to noise, and over time provide a more faithful representation of the stored stimulus. In networks with separate excitatory and inhibitory populations, generating bumps with a continuum of possible amplitudes, requires tuning the strength of inhibition to precisely cancel background excitation.
  • Models of biochemical networks are frequently high-dimensional and complex. Reduction methods that preserve important dynamical properties are therefore essential in their study. Interactions between the nodes in such networks are frequently modeled using a Hill function, $x^n/(J^n+x^n)$. Reduced ODEs and Boolean networks have been studied extensively when the exponent $n$ is large. However, the case of small constant $J$ appears in practice, but is not well understood. In this paper we provide a mathematical analysis of this limit, and show that a reduction to a set of piecewise linear ODEs and Boolean networks can be mathematically justified. The piecewise linear systems have closed form solutions that closely track those of the fully nonlinear model. On the other hand, the simpler, Boolean network can be used to study the qualitative behavior of the original system. We justify the reduction using geometric singular perturbation theory and compact convergence, and illustrate the results in networks modeling a genetic switch and a genetic oscillator.
  • Transcriptional delay can significantly impact the dynamics of gene networks. Here we examine how such delay affects bistable systems. We investigate several stochastic models of bistable gene networks and find that increasing delay dramatically increases the mean residence times near stable states. To explain this, we introduce a non-Markovian, analytically tractable reduced model. The model shows that stabilization is the consequence of an increased number of failed transitions between stable states. Each of the bistable systems that we simulate behaves in this manner.
  • Emerging technologies are revealing the spiking activity in ever larger neural ensembles. Frequently, this spiking is far from independent, with correlations in the spike times of different cells. Understanding how such correlations impact the dynamics and function of neural ensembles remains an important open problem. Here we describe a new, generative model for correlated spike trains that can exhibit many of the features observed in data. Extending prior work in mathematical finance, this generalized thinning and shift (GTaS) model creates marginally Poisson spike trains with diverse temporal correlation structures. We give several examples which highlight the model's flexibility and utility. For instance, we use it to examine how a neural network responds to highly structured patterns of inputs. We then show that the GTaS model is analytically tractable, and derive cumulant densities of all orders in terms of model parameters. The GTaS framework can therefore be an important tool in the experimental and theoretical exploration of neural dynamics.
  • Motifs are patterns of subgraphs of complex networks. We studied the impact of such patterns of connectivity on the level of correlated, or synchronized, spiking activity among pairs of cells in a recurrent network model of integrate and fire neurons. For a range of network architectures, we find that the pairwise correlation coefficients, averaged across the network, can be closely approximated using only three statistics of network connectivity. These are the overall network connection probability and the frequencies of two second-order motifs: diverging motifs, in which one cell provides input to two others, and chain motifs, in which two cells are connected via a third intermediary cell. Specifically, the prevalence of diverging and chain motifs tends to increase correlation. Our method is based on linear response theory, which enables us to express spiking statistics using linear algebra, and a resumming technique, which extrapolates from second order motifs to predict the overall effect of coupling on network correlation. Our motif-based results seek to isolate the effect of network architecture perturbatively from a known network state.
  • We show that certain non-linear dynamical systems with non-linearities in the form of Hill functions, can be approximated by piecewise linear dynamical systems. The resulting piecewise systems have closed form solutions that can be used to understand the behavior of the fully nonlinear system. We justify the reduction using geometric singular perturbation theory, and illustrate the results in networks modeling a genetic switch and a genetic oscillator.
  • Novel experimental techniques reveal the simultaneous activity of larger and larger numbers of neurons. As a result there is increasing interest in the structure of cooperative -- or correlated -- activity in neural populations, and in the possible impact of such correlations on the neural code. A fundamental theoretical challenge is to understand how the architecture of network connectivity along with the dynamical properties of single cells shape the magnitude and timescale of correlations. We provide a general approach to this problem by extending prior techniques based on linear response theory. We consider networks of general integrate-and-fire cells with arbitrary architecture, and provide explicit expressions for the approximate cross-correlation between constituent cells. These correlations depend strongly on the operating point (input mean and variance) of the neurons, even when connectivity is fixed. Moreover, the approximations admit an expansion in powers of the matrices that describe the network architecture. This expansion can be readily interpreted in terms of paths between different cells. We apply our results to large excitatory-inhibitory networks, and demonstrate first how precise balance --- or lack thereof --- between the strengths and timescales of excitatory and inhibitory synapses is reflected in the overall correlation structure of the network. We then derive explicit expressions for the average correlation structure in randomly connected networks. These expressions help to identify the important factors that shape coordinated neural activity in such networks.
  • The Michaelis-Menten equation has played a central role in our understanding of biochemical processes. It has long been understood how this equation approximates the dynamics of irreversible enzymatic reactions. However, a similar approximation in the case of networks, where the product of one reaction can act as an enzyme in another, has not been fully developed. Here we rigorously derive such an approximation in a class of coupled enzymatic networks where the individual interactions are of Michaelis-Menten type. We show that the sufficient conditions for the validity of the total quasi steady state assumption (tQSSA), obtained in a single protein case by Borghans, de Boer and Segel can be extended to sufficient conditions for the validity of the tQSSA in a large class of enzymatic networks. Secondly, we derive reduced equations that approximate the network's dynamics and involve only protein concentrations. This significantly reduces the number of equations necessary to model such systems. We prove the validity of this approximation using geometric singular perturbation theory and results about matrix differentiation. The ideas used in deriving the approximating equations are quite general, and can be used to systematize other model reductions.
  • We consider a pair of stochastic integrate and fire neurons receiving correlated stochastic inputs. The evolution of this system can be described by the corresponding Fokker-Planck equation with non-trivial boundary conditions resulting from the refractory period and firing threshold. We propose a finite volume method that is orders of magnitude faster than the Monte Carlo methods traditionally used to model such systems. The resulting numerical approximations are proved to be accurate, nonnegative and integrate to 1. We also approximate the transient evolution of the system using an Ornstein--Uhlenbeck process, and use the result to examine the properties of the joint output of cell pairs. The results suggests that the joint output of a cell pair is most sensitive to changes in input variance, and less sensitive to changes in input mean and correlation.
  • The community structure of a complex network can be determined by finding the partitioning of its nodes that maximizes modularity. Many of the proposed algorithms for doing this work by recursively bisecting the network. We show that this unduely constrains their results, leading to a bias in the size of the communities they find and limiting their effectivness. To solve this problem, we propose adding a step to the existing algorithms that does not increase the order of their computational complexity. We show that, if this step is combined with a commonly used method, the identified constraint and resulting bias are removed, and its ability to find the optimal partitioning is improved. The effectiveness of this combined algorithm is also demonstrated by using it on real-world example networks. For a number of these examples, it achieves the best results of any known algorithm.
  • The magnitude of correlations between stimulus-driven responses of pairs of neurons can itself be stimulus-dependent. We examine how this dependence impacts the information carried by neural populations about the stimuli that drive them. Stimulus-dependent changes in correlations can both carry information directly and modulate the information separately carried by the firing rates and variances. We use Fisher information to quantify these effects and show that, although stimulus dependent correlations often carry little information directly, their modulatory effects on the overall information can be large. In particular, if the stimulus-dependence is such that correlations increase with stimulus-induced firing rates, this can significantly enhance the information of the population when the structure of correlations is determined solely by the stimulus. However, in the presence of additional strong spatial decay of correlations, such stimulus-dependence may have a negative impact. Opposite relationships hold when correlations decrease with firing rates.
  • One of the fundamental characteristics of a nonlinear system is how it transfers correlations in its inputs to correlations in its outputs. This is particularly important in the nervous system, where correlations between spiking neurons are prominent. Using linear response and asymptotic methods for pairs of unconnected integrate-and-fire (IF) neurons receiving white noise inputs, we show that this correlation transfer depends on the output spike firing rate in a strong, stereotyped manner, and is, surprisingly, almost independent of the interspike variance. For cells receiving heterogeneous inputs, we further show that correlation increases with the geometric mean spiking rate in the same stereotyped manner, greatly extending the generality of this relationship. We present an immediate consequence of this relationship for population coding via tuning curves.