• This paper investigates the synthesis of distributed economic control algorithms under which dynamically coupled physical systems are regulated to a variational equilibrium of a constrained convex game. We study two complementary cases: (i) each subsystem is linear and controllable; and (ii) each subsystem is nonlinear and in the strict-feedback form. The convergence of the proposed algorithms is guaranteed using Lyapunov analysis. Their performance is verified by two case studies on a multi-zone building temperature regulation problem and an optimal power flow problem, respectively.
  • This paper studies the cooperative training of two generative models for image modeling and synthesis. Both models are parametrized by convolutional neural networks (ConvNets). The first model is a deep energy-based model, whose energy function is defined by a bottom-up ConvNet, which maps the observed image to the energy. We call it the descriptor network. The second model is a generator network, which is a non-linear version of factor analysis. It is defined by a top-down ConvNet, which maps the latent factors to the observed image. The maximum likelihood learning algorithms of both models involve MCMC sampling such as Langevin dynamics. We observe that the two learning algorithms can be seamlessly interwoven into a cooperative learning algorithm that can train both models simultaneously. Specifically, within each iteration of the cooperative learning algorithm, the generator model generates initial synthesized examples to initialize a finite-step MCMC that samples and trains the energy-based descriptor model. After that, the generator model learns from how the MCMC changes its synthesized examples. That is, the descriptor model teaches the generator model by MCMC, so that the generator model accumulates the MCMC transitions and reproduces them by direct ancestral sampling. We call this scheme MCMC teaching. We show that the cooperative algorithm can learn highly realistic generative models.
  • This paper studies how a system operator and a set of agents securely execute a distributed projected gradient-based algorithm. In particular, each participant holds a set of problem coefficients and/or states whose values are private to the data owner. The concerned problem raises two questions: how to securely compute given functions; and which functions should be computed in the first place. For the first question, by using the techniques of homomorphic encryption, we propose novel algorithms which can achieve secure multiparty computation with perfect correctness. For the second question, we identify a class of functions which can be securely computed. The correctness and computational efficiency of the proposed algorithms are verified by two case studies of power systems, one on a demand response problem and the other on an optimal power flow problem.
  • This paper proposes a multi-grid method for learning energy-based generative ConvNet models of images. For each grid, we learn an energy-based probabilistic model where the energy function is defined by a bottom-up convolutional neural network (ConvNet or CNN). Learning such a model requires generating synthesized examples from the model. Within each iteration of our learning algorithm, for each observed training image, we generate synthesized images at multiple grids by initializing the finite-step MCMC sampling from a minimal 1 x 1 version of the training image. The synthesized image at each subsequent grid is obtained by a finite-step MCMC initialized from the synthesized image generated at the previous coarser grid. After obtaining the synthesized examples, the parameters of the models at multiple grids are updated separately and simultaneously based on the differences between synthesized and observed examples. We show that this multi-grid method can learn realistic energy-based generative ConvNet models, and it outperforms the original contrastive divergence (CD) and persistent CD.
  • Resistance switching random access memory (ReRAM), with the ability to repeatedly modulate electrical resistance, has been highlighted as a feasible high-density memory with the potential to replace negative-AND (NAND) flash memory. Such resistance modulation usually involves ion migration and filament formation, which usually lead to relatively low device reliability and yield. Resistance switching can also come from an entirely electronic origin, as in nanometallic memory, by electron trapping and detrapping. Recent research has revealed additional merits of its mechanism, which entails smart, atomic-sized floating gates that can be easily engineered in amorphous Si, oxides, and nitrides. This article addresses the basic ideas of nanometallic ReRAM, which may also be a contender for analogue computing and non-von Neumann-type computation.
  • To model impulsive noise in power line channels, both the Bernoulli-Gaussian model and the symmetric alpha-stable model are usually applied. Towards a merge of existing noise measurement databases and a simplification of communication system design, the compatibility between the two models is of interest. In this paper, we show that they can be approximately converted to each other under certain constrains, although never generally unified. Based on this, we propose a fast model conversion.
  • Much effort has been devoted to device and materials engineering to realize nanoscale resistance random access memory (RRAM) for practical applications, but there still lacks a rational physical basis to be relied on to design scalable devices spanning many length scales. In particular, the critical switching criterion is not clear for RRAM devices in which resistance changes are limited to localized nanoscale filaments that experience concentrated heat, electric current and field. Here, we demonstrate voltage-controlled resistance switching for macro and nano devices in both filamentary RRAM and nanometallic RRAM, the latter switches uniformly and does not require forming. As a result, using a constant current density as the compliance, we have achieved area-scalability for the low resistance state of the filamentary RRAM, and for both the low and high resistance states of the nanometallic RRAM. This finding will help design area-scalable RRAM at the nanoscale.
  • Without a periodic reference framework, local structures in noncrystalline solids are difficult to specify, but they still exert an enormous influence on materials properties. For example, thermomechanical responses of organic and inorganic glasses sensitively depend on the distribution of free volume or soft spots$^{1,2}$. Meanwhile, strong electron localization$^{3}$ that endows unparalleled electrical breakdown strengths to amorphous insulators is easily compromised by local defects that promote inelastic tunneling over a variable range$^{4,5}$. Here we report how metallic conduction can overcome strong localization in amorphous insulators of small dimensions, and how local structures can manifest their spectacular influence on such conduction. In amorphous Si, nanoscale electrons are so coherent that they exhibit robust quantum interferences reminiscent of the mesoscopic phenomena seen in weakly localized metal crystals$^{6}$. Yet ultrasoft Si bonds emerge as the key local structures whose extraordinarily strong electron-phonon interaction coerces itinerant electrons into moving slowly at low temperature, even becoming trapped at all temperature when Si-O/N sites are provided. The local structures can be manipulated by a voltage or pressure to regulate charge storage, charge flow and metal-insulator transition. Also made of Ge and oxides and nitrides, nanostructured amorphous conductors could offer opportunities for new applications.
  • Attractive negative-$U$ interactions between electrons facilitated by strong electron-phonon interaction are common in highly polarizable and disordered materials such as amorphous chalcogenides, but there is no direct evidence for them in large-band-gap insulators. Here we report how such negative-$U$ centers are responsible for widespread insulator-to-metal transitions in amorphous HfO$_2$ and Al$_2$O$_3$ thin films with a 10$^9$-fold resistance drop. Triggered by a static hydraulic pressure or a 0.1 ps impulse of magnetic pressure, the transition can proceed at such low pressure that there is very little overall deformation (strain~10$^{-5}$). Absent a significant energy change overall, the transition is attributed to the reversal of localized electron-phonon interaction: By reversing the sign of $U$, trapped electrons are destabilized and released, thus clearing conduction paths previously blocked by charged traps. The results also suggest that Mott insulators when disordered may become Anderson insulators with strong electron-phonon interactions regulating incipient conduction paths, a novel finding of technological significance for electronic devices.
  • Radio-loud high-redshift quasars (HRQs), although only a few of them are known to date, are crucial for the studies of the growth of supermassive black holes (SMBHs) and the evolution of active galactic nuclei (AGN) at early cosmological epochs. Radio jets offer direct evidence of SMBHs, and their radio structures can be studied with the highest angular resolution using Very Long Baseline Interferometry (VLBI). Here we report on the observations of three HRQs (J0131-0321, J0906+6930, J1026+2542) at z>5 using the Korean VLBI Network and VLBI Exploration of Radio Astrometry Arrays (together known as KaVA) with the purpose of studying their pc-scale jet properties. The observations were carried out at 22 and 43 GHz in 2016 January among the first-batch open-use experiments of KaVA. The quasar J0906+6930 was detected at 22 GHz but not at 43 GHz. The other two sources were not detected and upper limits to their compact radio emission are given. Archival VLBI imaging data and single-dish 15-GHz monitoring light curve of J0906+6930 were also acquired as complementary information. J0906+6930 shows a moderate-level variability at 15 GHz. The radio image is characterized by a core-jet structure with a total detectable size of ~5 pc in projection. The brightness temperature, 1.9x10^{11} K, indicates relativistic beaming of the jet. The radio properties of J0906+6930 are consistent with a blazar. Follow-up VLBI observations will be helpful for determining its structural variation.
  • This paper proposes an alternating back-propagation algorithm for learning the generator network model. The model is a non-linear generalization of factor analysis. In this model, the mapping from the continuous latent factors to the observed signal is parametrized by a convolutional neural network. The alternating back-propagation algorithm iterates the following two steps: (1) Inferential back-propagation, which infers the latent factors by Langevin dynamics or gradient descent. (2) Learning back-propagation, which updates the parameters given the inferred latent factors by gradient descent. The gradient computations in both steps are powered by back-propagation, and they share most of their code in common. We show that the alternating back-propagation algorithm can learn realistic generator models of natural images, video sequences, and sounds. Moreover, it can also be used to learn from incomplete or indirect training data.
  • It is generally impossible to separately measure the resistance of the functional component (i.e., the intrinsic device materials) and the parasitic component (i.e., terminals, interfaces and serial loads) in a two-terminal device. Yet such knowledge is important for understanding device physics and designing device systems. Here, we consider a case where an electric current, temperature, or magnetic field causes a small but identical relative conductivity change of the device materials. We find an exact solution to this relative change by a simple resistance-data analysis of similarly configured two-terminal devices. The solution is obtainable even if the change is quite small, say, less than 0.1%. In special cases of small relative changes in parasitic resistance, the absolute parasitic resistance is also obtainable. Our method is especially useful for studying the switching and transport characteristics of the emergent non-volatile resistance memory.
  • This paper presents a method, called AOGTracker, for simultaneously tracking, learning and parsing (TLP) of unknown objects in video sequences with a hierarchical and compositional And-Or graph (AOG) representation. %The AOG captures both structural and appearance variations of a target object in a principled way. The TLP method is formulated in the Bayesian framework with a spatial and a temporal dynamic programming (DP) algorithms inferring object bounding boxes on-the-fly. During online learning, the AOG is discriminatively learned using latent SVM to account for appearance (e.g., lighting and partial occlusion) and structural (e.g., different poses and viewpoints) variations of a tracked object, as well as distractors (e.g., similar objects) in background. Three key issues in online inference and learning are addressed: (i) maintaining purity of positive and negative examples collected online, (ii) controling model complexity in latent structure learning, and (iii) identifying critical moments to re-learn the structure of AOG based on its intrackability. The intrackability measures uncertainty of an AOG based on its score maps in a frame. In experiments, our AOGTracker is tested on two popular tracking benchmarks with the same parameter setting: the TB-100/50/CVPR2013 benchmarks, and the VOT benchmarks --- VOT 2013, 2014, 2015 and TIR2015 (thermal imagery tracking). In the former, our AOGTracker outperforms state-of-the-art tracking algorithms including two trackers based on deep convolutional network. In the latter, our AOGTracker outperforms all other trackers in VOT2013 and is comparable to the state-of-the-art methods in VOT2014, 2015 and TIR2015.
  • We show that a generative random field model, which we call generative ConvNet, can be derived from the commonly used discriminative ConvNet, by assuming a ConvNet for multi-category classification and assuming one of the categories is a base category generated by a reference distribution. If we further assume that the non-linearity in the ConvNet is Rectified Linear Unit (ReLU) and the reference distribution is Gaussian white noise, then we obtain a generative ConvNet model that is unique among energy-based models: The model is piecewise Gaussian, and the means of the Gaussian pieces are defined by an auto-encoder, where the filters in the bottom-up encoding become the basis functions in the top-down decoding, and the binary activation variables detected by the filters in the bottom-up convolution process become the coefficients of the basis functions in the top-down deconvolution process. The Langevin dynamics for sampling the generative ConvNet is driven by the reconstruction error of this auto-encoder. The contrastive divergence learning of the generative ConvNet reconstructs the training images by the auto-encoder. The maximum likelihood learning algorithm can synthesize realistic natural image patterns.
  • Traditional lens design is a numerical and forward process based on ray tracing and aberration theory. This method has limitations because the initial configuration of the lens has to be specified and the aberrations of the lenses have to considered. This paper is an initial attempt to investigate an analytic and inverse lens design method, called Lagrange, to overcome these barriers. Lagrange method tries to build differential equations in terms of the system parameters and the system input and output (object and image). The generalized Snell's law in three dimensional space and the normal of a surface in fundamental differential geometry are applied. Based on the Lagrange method equations for a single surface system are derived which can perfectly image a point object.
  • To attain seamless handover and reduce the han- dover failure probability for high-speed railway (HSR) com- munication systems, this paper proposes a remote antenna unit (RAU) selection assisted handover scheme where two antennas are installed on high speed train (HST) and distributed antenna system (DAS) cell architecture on ground is adopted. The RAU selection is used to provide high quality received signals for trains moving in DAS cells and the two HST antennas are employed on trains to realize seamless handover. Moreover, to efficiently evaluate the system performance, a new met- ric termed as handover occurrence probability is defined for describing the relation between handover occurrence position and handover failure probability. We then analyze the received signal strength, the handover trigger probability, the handover occurrence probability, the handover failure probability and the communication interruption probability. Numerical results are provided to compare our proposed scheme with the current existing ones. It is shown that our proposed scheme achieves better performances in terms of handover failure probability and communication interruption probability.
  • The convolutional neural network (ConvNet or CNN) has proven to be very successful in many tasks such as those in computer vision. In this conceptual paper, we study the generative perspective of the discriminative CNN. In particular, we propose to learn the generative FRAME (Filters, Random field, And Maximum Entropy) model using the highly expressive filters pre-learned by the CNN at the convolutional layers. We show that the learning algorithm can generate realistic and rich object and texture patterns in natural scenes. We explain that each learned model corresponds to a new CNN unit at a layer above the layer of filters employed by the model. We further show that it is possible to learn a new layer of CNN units using a generative CNN model, which is a product of experts model, and the learning algorithm admits an EM interpretation with binary latent variables.
  • Deploying multiple antennas on high speed trains is an effective way to enhance the information transmission performance for high speed railway (HSR) wireless communication systems. However, how to efficiently deploy multiple antennas on a train? This problem has not been studied yet. In this paper, we shall investigate efficient antenna deployment strategies for HSR communication systems where two multi-antenna deployment strategies, i.e., the equidistant strategy and the fixed-interval strategy, are considered. To evaluate the system performance, mobile service amount and outage time ratio are introduced. Theoretical analysis and numerical results show that, when the length of the train is not very large, for two-antenna case, by increasing the distance of neighboring antennas in a reasonable region, the system performance can be enhanced. It is also shown that the two strategies have much difference performance behavior in terms of instantaneous channel capacity, and the fixed-interval strategy may achieve much better performance than the equidistant one in terms of service amount and outage time ratio when the antenna number is much large.
  • This paper investigates the energy efficiency (EE) in multiple relay aided OFDM system, where decode-and-forward (DF) relay beamforming is employed to help the information transmission. In order to explore the EE performance with user fairness for such a system, we formulate an optimization problem to maximize the EE by jointly considering several factors, the transmission mode selection (DF relay beamforming or direct-link transmission), the helping relay set selection, the subcarrier assignment and the power allocation at the source and relays on subcarriers, under nonlinear proportional rate fairness constraints, where both transmit power consumption and linearly rate-dependent circuit power consumption are taken into account. To solve the non-convex optimization problem, we propose a low-complexity scheme to approximate it. Simulation results demonstrate its effectiveness. We also investigate the effects of the circuit power consumption on system performances and observe that with both the constant and the linearly rate-dependent circuit power consumption, system EE grows with the increment of system average channel-to noise ratio (CNR), but the growth rates show different behaviors. For the constant circuit power consumption, system EE increasing rate is an increasing function of the system average CNR, while for the linearly rate-dependent one, system EE increasing rate is a decreasing function of the system average CNR. This observation is very important which indicates that by deducing the circuit dynamic power consumption per unit data rate, system EE can be greatly enhanced. Besides, we also discuss the effects of the number of users and subcarriers on the system EE performance.
  • The convolutional neural networks (CNNs) have proven to be a powerful tool for discriminative learning. Recently researchers have also started to show interest in the generative aspects of CNNs in order to gain a deeper understanding of what they have learned and how to further improve them. This paper investigates generative modeling of CNNs. The main contributions include: (1) We construct a generative model for the CNN in the form of exponential tilting of a reference distribution. (2) We propose a generative gradient for pre-training CNNs by a non-parametric importance sampling scheme, which is fundamentally different from the commonly used discriminative gradient, and yet has the same computational architecture and cost as the latter. (3) We propose a generative visualization method for the CNNs by sampling from an explicit parametric image distribution. The proposed visualization method can directly draw synthetic samples for any given node in a trained CNN by the Hamiltonian Monte Carlo (HMC) algorithm, without resorting to any extra hold-out images. Experiments on the challenging ImageNet benchmark show that the proposed generative gradient pre-training consistently helps improve the performances of CNNs, and the proposed generative visualization method generates meaningful and varied samples of synthetic images from a large-scale deep CNN.
  • The dynamics of human mobility characterizes the trajectories humans follow during their daily activities and is the foundation of processes from epidemic spreading to traffic prediction and information recommendation. In this paper, we investigate a massive data set of human activity including both online behavior of browsing websites and offline one of visiting towers based mobile terminations. The non-Markovian character observed from both online and offline cases is suggested by the scaling law in the distribution of dwelling time at individual and collective levels, respectively. Furthermore, we argue that the lower entropy and higher predictability in human mobility for both online and offline cases may origin from this non-Markovian character. However, the distributions of individual entropy and predictability show the different degrees of non-Markovian character from online to offline cases. To accounting for non-Markovian character in human mobility, we introduce a protype model with three basic ingredients, \emph{preferential return, inertial effect, and exploration} to reproduce the dynamic process of online and offline human mobility. In comparison with standard and biased random walk models with assumption of Markov process, the proposed model is able to obtain characters much closer to these empirical observations.
  • LAGE is a systematic framework developed in Java. The motivation of LAGE is to provide a scalable and parallel solution to reconstruct Gene Regulatory Networks (GRNs) from continuous gene expression data for very large amount of genes. The basic idea of our framework is motivated by the philosophy of divideand-conquer. Specifically, LAGE recursively partitions genes into multiple overlapping communities with much smaller sizes, learns intra-community GRNs respectively before merge them altogether. Besides, the complete information of overlapping communities serves as the byproduct, which could be used to mine meaningful functional modules in biological networks.
  • The motivation for this paper is to apply Bayesian structure learning using Model Averaging in large-scale networks. Currently, Bayesian model averaging algorithm is applicable to networks with only tens of variables, restrained by its super-exponential complexity. We present a novel framework, called LSBN(Large-Scale Bayesian Network), making it possible to handle networks with infinite size by following the principle of divide-and-conquer. The method of LSBN comprises three steps. In general, LSBN first performs the partition by using a second-order partition strategy, which achieves more robust results. LSBN conducts sampling and structure learning within each overlapping community after the community is isolated from other variables by Markov Blanket. Finally LSBN employs an efficient algorithm, to merge structures of overlapping communities into a whole. In comparison with other four state-of-art large-scale network structure learning algorithms such as ARACNE, PC, Greedy Search and MMHC, LSBN shows comparable results in five common benchmark datasets, evaluated by precision, recall and f-score. What's more, LSBN makes it possible to learn large-scale Bayesian structure by Model Averaging which used to be intractable. In summary, LSBN provides an scalable and parallel framework for the reconstruction of network structures. Besides, the complete information of overlapping communities serves as the byproduct, which could be used to mine meaningful clusters in biological networks, such as protein-protein-interaction network or gene regulatory network, as well as in social network.
  • Strategies to optimally discriminate between quantum states are critical in quantum technologies. We present an experimental demonstration of minimum error discrimination between entangled states, encoded in the polarization of pairs of photons. Although the optimal measurement involves projecting onto entangled states, we use a result of Walgate et al. to design an optical implementation employing only local polarization measurements and feed-forward, which performs at the Helstrom bound. Our scheme can achieve perfect discrimination of orthogonal states and minimum error discrimination of non-orthogonal states. Our experimental results show a definite advantage over schemes not using feed-forward.
  • The band structures of strained graphene nanoribbons (GNRs) are examined by a tight binding Hamiltonian that is directly related to the type and strength of strains. Compared to the two-dimensional graphene whose band gap remains close to zero even if a large strain is applied, the band gap of graphene nanoribbon (GNR) is sensitive to both uniaxial and shears strains. The effect of strain on the electronic structure of a GNR strongly depends on its edge shape and structural indices. For an armchair GNR, uniaxial weak strain changes the band gap in a linear fashion, and for a large strain, it results in periodic oscillation of the band gap. On the other hand, shear strain always tend to reduce the band gap. For a zigzag GNR, the effect of strain is to change the spin polarization at the edges of GNR, thereby modulate the band gap. A simple analytical model is proposed to interpret the band gap responds to strain in armchair GNR, which agrees with the numerical results.