• We propose a general and model-free approach for Reinforcement Learning (RL) on real robotics with sparse rewards. We build upon the Deep Deterministic Policy Gradient (DDPG) algorithm to use demonstrations. Both demonstrations and actual interactions are used to fill a replay buffer and the sampling ratio between demonstrations and transitions is automatically tuned via a prioritized replay mechanism. Typically, carefully engineered shaping rewards are required to enable the agents to efficiently explore on high dimensional control problems such as robotics. They are also required for model-based acceleration methods relying on local solvers such as iLQG (e.g. Guided Policy Search and Normalized Advantage Function). The demonstrations replace the need for carefully engineered rewards, and reduce the exploration problem encountered by classical RL approaches in these domains. Demonstrations are collected by a robot kinesthetically force-controlled by a human demonstrator. Results on four simulated insertion tasks show that DDPG from demonstrations out-performs DDPG, and does not require engineered rewards. Finally, we demonstrate the method on a real robotics task consisting of inserting a clip (flexible object) into a rigid object.
  • Quantum Key Distribution (QKD) guarantees the security of communication with quantum physics. Most of widely adopted QKD protocols currently encode the key information with binary signal format---qubit, such as the polarization states. Therefore the transmitted information efficiency of the quantum key is intrinsically upper bounded by 1 bit per photon. High dimensional quantum system is a potential candidate for increasing the capacity of single photon. However, due to the difficulty in manipulating and measuring high dimensional quantum systems, the experimental high dimensional QKD is still at its infancy. Here we propose a sort of practical high-speed high dimensional QKD using partial mutual unbiased basis (PMUB) of photon's orbital angular momentum (OAM). Different from the previous OAM encoding, the high dimensional Hilbert space we used is expanded by the OAM states with same mode order, which can be extended to considerably high dimensions and implemented under current state of the art. Because all the OAM states are in the same mode order, the coherence will be well kept after long-distance propagation, and the detection can be achieved by using passive linear optical elements with very high speed. We show that our protocol has high key generation rate and analyze the anti-noise ability under atmospheric turbulence. Furthermore, the security of our protocol based on PMUB is rigorously proved. Our protocol paves a brand new way for the application of photon's OAM in high dimensional QKD field, which can be a breakthrough for high efficiency quantum communications.
  • We are increasingly surrounded by artificially intelligent technology that takes decisions and executes actions on our behalf. This creates a pressing need for general means to communicate with, instruct and guide artificial agents, with human language the most compelling means for such communication. To achieve this in a scalable fashion, agents must be able to relate language to the world and to actions; that is, their understanding of language must be grounded and embodied. However, learning grounded language is a notoriously challenging problem in artificial intelligence research. Here we present an agent that learns to interpret language in a simulated 3D environment where it is rewarded for the successful execution of written instructions. Trained via a combination of reinforcement and unsupervised learning, and beginning with minimal prior knowledge, the agent learns to relate linguistic symbols to emergent perceptual representations of its physical surroundings and to pertinent sequences of actions. The agent's comprehension of language extends beyond its prior experience, enabling it to apply familiar language to unfamiliar situations and to interpret entirely novel instructions. Moreover, the speed with which this agent learns new words increases as its semantic knowledge grows. This facility for generalising and bootstrapping semantic knowledge indicates the potential of the present approach for reconciling ambiguous natural language with the complexity of the physical world.
  • Many language generation tasks require the production of text conditioned on both structured and unstructured inputs. We present a novel neural network architecture which generates an output sequence conditioned on an arbitrary number of input functions. Crucially, our approach allows both the choice of conditioning context and the granularity of generation, for example characters or tokens, to be marginalised, thus permitting scalable and effective training. Using this framework, we address the problem of generating programming code from a mixed natural language and structured specification. We create two new data sets for this paradigm derived from the collectible trading card games Magic the Gathering and Hearthstone. On these, and a third preexisting corpus, we demonstrate that marginalising multiple predictors allows our model to outperform strong benchmarks.