• We consider optimal designs for general multinomial logistic models, which cover baseline-category, cumulative, adjacent-categories, and continuation-ratio logit models, with proportional odds, non-proportional odds, or partial proportional odds assumption. We derive the corresponding Fisher information matrices in three different forms to facilitate their calculations, determine the conditions for their positive definiteness, and search for optimal designs. We conclude that, unlike the designs for binary responses, a feasible design for a multinomial logistic model may contain less experimental settings than parameters, which is of practical significance. We also conclude that even for a minimally supported design, a uniform allocation, which is typically used in practice, is not optimal in general for a multinomial logistic model. We develop efficient algorithms for searching D-optimal designs. Using examples based on real experiments, we show that the efficiency of an experiment can be significantly improved if our designs are adopted.
  • The interplay between superconductivity and structural phase transition has attracted enormous interests in recent years. For example, in Fe-pnictide high temperature superconductors, quantum fluctuations in association with structural phase transition have been proposed to lead to many novel physical properties and even the superconductivity itself. Here we report a finding that the quasi-skutterudite superconductors (Sr$_{1-x}$Ca$_{x}$)$_{3}$Ir$_{4}$Sn$_{13}$ ($x$ = 0, 0.5, 1) and Ca$_{3}$Rh$_{4}$Sn$_{13}$ show some unusual properties similar to the Fe-pnictides, through $^{119}$Sn nuclear magnetic resonance (NMR) measurements. In (Sr$_{1-x}$Ca$_{x}$)$_{3}$Ir$_{4}$Sn$_{13}$, the NMR linewidth increases below a temperature $T^*$ that is higher than the structural phase transition temperature $T_{\rm s}$. The spin-lattice relaxation rate ($1/T_1$) divided by temperature ($T$), $1/T_1T$, and the Knight shift $K$ increase with decreasing $T$ down to $T^*$, but start to decrease below $T^*$ and followed by more distinct changes at $T_{\rm s}$. In contrast, none of the anomalies was observed in Ca$_{3}$Rh$_{4}$Sn$_{13}$ that does not undergo a structural phase transition. The precursory phenomenon above structural phase transition resembles that occurs in Fe-pnictides. In the superconducting state of Ca$_{3}$Ir$_{4}$Sn$_{13}$, $1/T_{1}$ decays as ${\rm exp}(-\Delta/k_{\rm B}T)$ with a large gap $\Delta = 2.21 k_{\rm B}T_{\rm c}$, yet without a Hebel-Slichter coherence peak, which indicate strong-coupling superconductivity. Our results provide new insight into the relationship between superconductivity and the electronic-structure change associated with structural phase transition.
  • We investigate a lattice-structured LSTM model for Chinese NER, which encodes a sequence of input characters as well as all potential words that match a lexicon. Compared with character-based methods, our model explicitly leverages word and word sequence information. Compared with word-based methods, lattice LSTM does not suffer from segmentation errors. Gated recurrent cells allow our model to choose the most relevant characters and words from a sentence for better NER results. Experiments on various datasets show that lattice LSTM outperforms both word-based and character-based LSTM baselines, achieving the best results.
  • In this paper, we propose an active learning algorithm and models which can gradually learn individual's preference through pairwise comparisons. The active learning scheme aims at finding individual's most preferred choice with minimized number of pairwise comparisons. The pairwise comparisons are encoded into probabilistic models based on assumptions of choice models and deep Gaussian processes. The next-to-compare decision is determined by a novel acquisition function. We benchmark the proposed algorithm and models using functions with multiple local optima and one public airline itinerary dataset. The experiments indicate the effectiveness of our active learning algorithm and models.
  • We propose a low-resolution analog-to-digital converter (ADC) module assisted hybrid beamforming architecture for millimeter-wave (mmWave) communications. We prove that the proposed low-cost and flexible architecture can reduce the beam training time and complexity dramatically without degradation in the data transmission performance. In addition, we design a fast beam training method which is suitable for the proposed system architecture. The proposed beam training method requires only L + 1 (where L is the number of paths) time slots which is smaller compared to the state-of-the-art.
  • In this paper, we introduce YEDDA, a lightweight but efficient and comprehensive open-source tool for text span annotation. YEDDA provides a systematic solution for text span annotation, ranging from collaborative user annotation to administrator evaluation and analysis. It overcomes the low efficiency of traditional text annotation tools by annotating entities through both command line and shortcut keys, which are configurable with custom labels. YEDDA also gives intelligent recommendations by learning the up-to-date annotated text. An administrator client is developed to evaluate annotation quality of multiple annotators and generate detailed comparison report for each annotator pair. Experiments show that the proposed system can reduce the annotation time by half compared with existing annotation tools. And the annotation time can be further compressed by 16.47\% through intelligent recommendation.
  • This paper presents a generic Bayesian framework that enables any deep learning model to actively learn from targeted crowds. Our framework inherits from recent advances in Bayesian deep learning, and extends existing work by considering the targeted crowdsourcing approach, where multiple annotators with unknown expertise contribute an uncontrolled amount (often limited) of annotations. Our framework leverages the low-rank structure in annotations to learn individual annotator expertise, which then helps to infer the true labels from noisy and sparse annotations. It provides a unified Bayesian model to simultaneously infer the true labels and train the deep learning model in order to reach an optimal learning efficacy. Finally, our framework exploits the uncertainty of the deep learning model during prediction as well as the annotators' estimated expertise to minimize the number of required annotations and annotators for optimally training the deep learning model. We evaluate the effectiveness of our framework for intent classification in Alexa (Amazon's personal assistant), using both synthetic and real-world datasets. Experiments show that our framework can accurately learn annotator expertise, infer true labels, and effectively reduce the amount of annotations in model training as compared to state-of-the-art approaches. We further discuss the potential of our proposed framework in bridging machine learning and crowdsourcing towards improved human-in-the-loop systems.
  • Automated detection and segmentation of pulmonary nodules on lung computed tomography (CT) scans can facilitate early lung cancer diagnosis. Existing supervised approaches for automated nodule segmentation on CT scans require voxel-based annotations for training, which are labor- and time-consuming to obtain. In this work, we propose a weakly-supervised method that generates accurate voxel-level nodule segmentation trained with image-level labels only. By adapting a convolutional neural network (CNN) trained for image classification, our proposed method learns discriminative regions from the activation maps of convolution units at different scales, and identifies the true nodule location with a novel candidate-screening framework. Experimental results on the public LIDC-IDRI dataset demonstrate that, our weakly-supervised nodule segmentation framework achieves competitive performance compared to a fully-supervised CNN-based segmentation method.
  • It is well-known that the robustness of artificial neural networks (ANNs) is important for their wide ranges of applications. In this paper, we focus on the robustness of the classification ability of a spiking neural network which receives perturbed inputs. Actually, the perturbation is allowed to be arbitrary styles. However, Gaussian perturbation and other regular ones have been rarely investigated. For classification problems, the closer to the desired point, the more perturbed points there are in the input space. In addition, the perturbation may be periodic. Based on these facts, we only consider sinusoidal and Gaussian perturbations in this paper. With the SpikeProp algorithm, we perform extensive experiments on the classical XOR problem and other three benchmark datasets. The numerical results show that there is not significant reduction in the classification ability of the network if the input signals are subject to sinusoidal and Gaussian perturbations.
  • Most image smoothing filters in the literature assume a piecewise constant model of smoothed output images. However, the piecewise constant model assumption can cause artifacts such as gradient reversals in applications such as image detail enhancement, HDR tone mapping, etc. In these applications, a piecewise linear model assumption is more preferred. In this paper, we propose a simple yet very effective framework to smooth images of piecewise linear model assumption using classical filters with the piecewise constant model assumption. Our method is capable of handling with gradient reversal artifacts caused by the piecewise constant model assumption. In addition, our method can further help accelerated methods, which need to quantize image intensity values into different bins, to achieve similar results that need a large number of bins using a much smaller number of bins. This can greatly reduce the computational cost. We apply our method to various classical filters with the piecewise constant model assumption. Experimental results of several applications show the effectiveness of the proposed method.
  • A patchwork method is used to study the dynamics of loss and recovery of an initial configuration in spin glass models in dimensions d=1 and d=2. The patchwork heuristic is used to accelerate the dynamics to investigate how models might reproduce the remarkable memory effects seen in experiment. Starting from a ground state configuration computed for one choice of nearest neighbor spin couplings, the sample is aged up to a given scale under new random couplings, leading to the partial erasure of the original ground state. The couplings are then restored to the original choice and patchwork coarsening is again applied, in order to assess the recovery of the original state. Eventual recovery of the original ground state upon coarsening is seen in two-dimensional Ising spin glasses and one-dimensional Potts models, while one-dimensional Ising glasses neither lose nor gain overlap with the ground state during the recovery stage. The recovery for the two-dimensional Ising spin glasses suggests scaling relations that lead to a recovery length scale that grows as a power of the aging length scale.
  • An atomic-scale theory of the viscoelastic response of metallic glasses is derived from first principles, using a Zwanzig-Caldeira-Leggett system-bath Hamiltonian as a starting point within the framework of nonaffine linear response to mechanical deformation. This approach provides a Generalized-Langevin-Equation (GLE) as the average equation of motion for an atom or ion in the material, from which non-Markovian nonaffine viscoelastic moduli are extracted. These can be evaluated using the vibrational density of states (DOS) as input, where the boson peak plays a prominent role for the mechanics. To compare with experimental data of binary ZrCu alloys, numerical DOS was obtained from simulations of this system, which take also electronic degrees of freedom into account via the embedded atom method (EAM) for the interatomic potential. It is shown that the viscoelastic $\alpha$-relaxation, including the $\alpha$-wing asymmetry in the loss modulus, can be very well described by the theory if the memory kernel (the non-Markovian friction) in the GLE is taken to be a stretched-exponential decaying function of time. This finding directly implies strong memory effects in the atomic-scale dynamics, and suggests that the $\alpha$-relaxation time is related to the characteristic time-scale over which atoms retain memory of their previous collision history. This memory time grows dramatically below the glass transition.
  • Spatially localized deformation components are very useful for shape analysis and synthesis in 3D geometry processing. Several methods have recently been developed, with an aim to extract intuitive and interpretable deformation components. However, these techniques suffer from fundamental limitations especially for meshes with noise or large-scale deformations, and may not always be able to identify important deformation components. In this paper we propose a novel mesh-based autoencoder architecture that is able to cope with meshes with irregular topology. We introduce sparse regularization in this framework, which along with convolutional operations, helps localize deformations. Our framework is capable of extracting localized deformation components from mesh data sets with large-scale deformations and is robust to noise. It also provides a nonlinear approach to reconstruction of meshes using the extracted basis, which is more effective than the current linear combination approach. Extensive experiments show that our method outperforms state-of-the-art methods in both qualitative and quantitative evaluations.
  • Capturing the temporal dynamics of user preferences over items is important for recommendation. Existing methods mainly assume that all time steps in user-item interaction history are equally relevant to recommendation, which however does not apply in real-world scenarios where user-item interactions can often happen accidentally. More importantly, they learn user and item dynamics separately, thus failing to capture their joint effects on user-item interactions. To better model user and item dynamics, we present the Interacting Attention-gated Recurrent Network (IARN) which adopts the attention model to measure the relevance of each time step. In particular, we propose a novel attention scheme to learn the attention scores of user and item history in an interacting way, thus to account for the dependencies between user and item dynamics in shaping user-item interactions. By doing so, IARN can selectively memorize different time steps of a user's history when predicting her preferences over different items. Our model can therefore provide meaningful interpretations for recommendation results, which could be further enhanced by auxiliary features. Extensive validation on real-world datasets shows that IARN consistently outperforms state-of-the-art methods.
  • Example-based mesh deformation methods are powerful tools for realistic shape editing. However, existing techniques typically combine all the example deformation modes, which can lead to overfitting, i.e. using a overly complicated model to explain the user-specified deformation. This leads to implausible or unstable deformation results, including unexpected global changes outside the region of interest. To address this fundamental limitation, we propose a sparse blending method that automatically selects a smaller number of deformation modes to compactly describe the desired deformation. This along with a suitably chosen deformation basis including spatially localized deformation modes leads to significant advantages, including more meaningful, reliable, and efficient deformations because fewer and localized deformation modes are applied. To cope with large rotations, we develop a simple but effective representation based on polar decomposition of deformation gradients, which resolves the ambiguity of large global rotations using an as-consistent-as-possible global optimization. This simple representation has a closed form solution for derivatives, making it efficient for sparse localized representation and thus ensuring interactive performance. Experimental results show that our method outperforms state-of-the-art data-driven mesh deformation methods, for both quality of results and efficiency.
  • As a fundamental phenomenon in nature, randomness has a wide range of applications in the fields of science and engineering. Among different types of random number generators (RNG), quantum random number generator (QRNG) is a kind of promising RNG as it can provide provable true random numbers based on the inherent randomness of fundamental quantum processes. Nevertheless, the randomness from a QRNG can be diminished (or even destroyed) if the devices (especially the entropy source devices) are not perfect or ill-characterized. To eliminate the practical security loopholes from the source, source-independent QRNGs, which allow the source to have arbitrary and unknown dimensions, have been introduced and become one of the most important semi-device-independent QRNGs. Herein a method that enables ultra-fast unpredictable quantum random number generation from quadrature fluctuations of quantum optical field without any assumptions on the input states is proposed. Particularly, to estimate a lower bound on the extractable randomness that is independent from side information held by an eavesdropper, a new security analysis framework is established based on the extremality of Gaussian states, which can be easily extended to design and analyze new semi-device-independent continuous variable QRNG protocols. Moreover, the practical imperfections of the QRNG including the effects of excess noise, finite sampling range, finite resolution and asymmetric conjugate quadratures are taken into account and quantitatively analyzed. Finally, the proposed method is experimentally demonstrated to obtain high secure random number generation rates of 15.07 Gbits/s in off-line configuration and can potentially achieve 6 Gbits/s by real-time post-processing.
  • Neural network models have recently received heated research attention in the natural language processing community. Compared with traditional models with discrete features, neural models have two main advantages. First, they take low-dimensional, real-valued embedding vectors as inputs, which can be trained over large raw data, thereby addressing the issue of feature sparsity in discrete models. Second, deep neural networks can be used to automatically combine input features, and including non-local features that capture semantic patterns that cannot be expressed using discrete indicator features. As a result, neural network models have achieved competitive accuracies compared with the best discrete models for a range of NLP tasks. On the other hand, manual feature templates have been carefully investigated for most NLP tasks over decades and typically cover the most useful indicator pattern for solving the problems. Such information can be complementary the features automatically induced from neural networks, and therefore combining discrete and neural features can potentially lead to better accuracy compared with models that leverage discrete or neural features only. In this paper, we systematically investigate the effect of discrete and neural feature combination for a range of fundamental NLP tasks based on sequence labeling, including word segmentation, POS tagging and named entity recognition for Chinese and English, respectively. Our results on standard benchmarks show that state-of-the-art neural models can give accuracies comparable to the best discrete models in the literature for most tasks and combing discrete and neural features unanimously yield better results.
  • This paper considers the beamspace channel estimation problem in 3D lens antenna array under a millimeter-wave communication system. We analyze the focusing capability of the 3D lens antenna array and the sparsity of the beamspace channel response matrix. Considering the analysis, we observe that the channel matrix can be treated as a 2D natural image; that is, the channel is sparse, and the changes between adjacent elements are subtle. Thus, for the channel estimation, we incorporate an image reconstruction technique called sparse non-informative parameter estimator-based cosparse analysis AMP for imaging (SCAMPI) algorithm. The SCAMPI algorithm is faster and more accurate than earlier algorithms such as orthogonal matching pursuit and support detection algorithms. To further improve the SCAMPI algorithm, we model the channel distribution as a generic Gaussian mixture (GM) probability and embed the expectation maximization learning algorithm into the SCAMPI algorithm to learn the parameters in the GM probability. We show that the GM probability outperforms the common uniform distribution used in image reconstruction. We also propose a phase-shifter-reduced selection network structure to decrease the power consumption of the system and prove that the SCAMPI algorithm is robust even if the number of phase shifters is reduced by 10%.
  • Solving the global method of Weighted Least Squares (WLS) model in image filtering is both time- and memory-consuming. In this paper, we present an alternative approximation in a time- and memory- efficient manner which is denoted as Semi-Global Weighed Least Squares (SG-WLS). Instead of solving a large linear system, we propose to iteratively solve a sequence of subsystems which are one-dimensional WLS models. Although each subsystem is one-dimensional, it can take two-dimensional neighborhood information into account due to the proposed special neighborhood construction. We show such a desirable property makes our SG-WLS achieve close performance to the original two-dimensional WLS model but with much less time and memory cost. While previous related methods mainly focus on the 4-connected/8-connected neighborhood system, our SG-WLS can handle a more general and larger neighborhood system thanks to the proposed fast solution. We show such a generalization can achieve better performance than the 4-connected/8-connected neighborhood system in some applications. Our SG-WLS is $\sim20$ times faster than the WLS model. For an image of $M\times N$, the memory cost of SG-WLS is at most at the magnitude of $max\{\frac{1}{M}, \frac{1}{N}\}$ of that of the WLS model. We show the effectiveness and efficiency of our SG-WLS in a range of applications.
  • We propose a neural reranking system for named entity recognition (NER). The basic idea is to leverage recurrent neural network models to learn sentence-level patterns that involve named entity mentions. In particular, given an output sentence produced by a baseline NER model, we replace all entity mentions, such as \textit{Barack Obama}, into their entity types, such as \textit{PER}. The resulting sentence patterns contain direct output information, yet is less sparse without specific named entities. For example, "PER was born in LOC" can be such a pattern. LSTM and CNN structures are utilised for learning deep representations of such sentences for reranking. Results show that our system can significantly improve the NER accuracies over two different baselines, giving the best reported results on a standard benchmark.
  • Traditionally, kernel learning methods requires positive definitiveness on the kernel, which is too strict and excludes many sophisticated similarities, that are indefinite, in multimedia area. To utilize those indefinite kernels, indefinite learning methods are of great interests. This paper aims at the extension of the logistic regression from positive semi-definite kernels to indefinite kernels. The model, called indefinite kernel logistic regression (IKLR), keeps consistency to the regular KLR in formulation but it essentially becomes non-convex. Thanks to the positive decomposition of an indefinite matrix, IKLR can be transformed into a difference of two convex models, which follows the use of concave-convex procedure. Moreover, we employ an inexact solving scheme to speed up the sub-problem and develop a concave-inexact-convex procedure (CCICP) algorithm with theoretical convergence analysis. Systematical experiments on multi-modal datasets demonstrate the superiority of the proposed IKLR method over kernel logistic regression with positive definite kernels and other state-of-the-art indefinite learning based algorithms.
  • To pursuit high electrochemical performance of supercapacitors based on Faradaic charge-transfer with redox reaction or absorption/desorption effect, the intercalation efficiency of electrolyte ions into electrode materials is a crucial prerequisite to surpass the pure surface capacity with extra bulk contribution. Here we report layered barium transition metal fluorides, BaMF4 (M = Mn, Co, Ni) to be a series of new electrode materials applied in standard three-electrode configuration. Benefiting from the efficient immersing of electrolyte ions, these materials own prominent specific capacitance. Electrochemical characterizations demonstrate that all the BaMF4 electrodes show both capacitive behavior and Faradaic redox reactions in the cyclic voltammograms, and ability of charge storage by charging-discharging cycling with high cycling stability. Particularly, BaCoF4 shows the the highest specific capacitance of 360 F g-1 at current density of 0.6 A g-1, even the particle size is far beyond nanometer scale. In addition, first principles calculations reveal the possible underlying mechanisms.
  • Singlish can be interesting to the ACL community both linguistically as a major creole based on English, and computationally for information extraction and sentiment analysis of regional social media. We investigate dependency parsing of Singlish by constructing a dependency treebank under the Universal Dependencies scheme, and then training a neural network model by integrating English syntactic knowledge into a state-of-the-art parser trained on the Singlish treebank. Results show that English knowledge can lead to 25% relative error reduction, resulting in a parser of 84.47% accuracies. To the best of our knowledge, we are the first to use neural stacking to improve cross-lingual dependency parsing on low-resource languages. We make both our annotation and parser available for further research.
  • Neural word segmentation research has benefited from large-scale raw texts by leveraging them for pretraining character and word embeddings. On the other hand, statistical segmentation research has exploited richer sources of external information, such as punctuation, automatic segmentation and POS. We investigate the effectiveness of a range of external training sources for neural word segmentation by building a modular segmentation model, pretraining the most important submodule using rich external sources. Results show that such pretraining significantly improves the model, leading to accuracies competitive to the best methods on six benchmarks.
  • The process of using one image to guide the filtering process of another one is called Guided Image Filtering (GIF). The main challenge of GIF is the structure inconsistency between the guidance image and the target image. Besides, noise in the target image is also a challenging issue especially when it is heavy. In this paper, we propose a general framework for Robust Guided Image Filtering (RGIF), which contains a data term and a smoothness term, to solve the two issues mentioned above. The data term makes our model simultaneously denoise the target image and perform GIF which is robust against the heavy noise. The smoothness term is able to make use of the property of both the guidance image and the target image which is robust against the structure inconsistency. While the resulting model is highly non-convex, it can be solved through the proposed Iteratively Re-weighted Least Squares (IRLS) in an efficient manner. For challenging applications such as guided depth map upsampling, we further develop a data-driven parameter optimization scheme to properly determine the parameter in our model. This optimization scheme can help to preserve small structures and sharp depth edges even for a large upsampling factor (8x for example). Moreover, the specially designed structure of the data term and the smoothness term makes our model perform well in edge-preserving smoothing for single-image tasks (i.e., the guidance image is the target image itself). This paper is an extension of our previous work [1], [2].