We propose a solution of the multiple target tracking (MTT) problem based on sets of trajectories and the random finite set framework. A full Bayesian approach to MTT should characterise the distribution of the trajectories given the measurements, as it contains all information about the trajectories. We attain this by considering multi-object density functions in which objects are trajectories. For the standard tracking models, we also describe a conjugate family of multitrajectory density functions.

In this paper, we provide for the first time an automated, correct-by-construction, controller synthesis scheme for a class of infinite dimensional stochastic systems, namely, retarded jump-diffusion systems. First, we construct finite abstractions approximately bisimilar to non-probabilistic retarded systems corresponding to the original systems having some stability property, namely, incremental input-to-state stability. Then, we provide a result on quantifying the distance between output trajectory of the obtained finite abstraction and that of the original retarded jump-diffusion system in a probabilistic setting. Using the proposed result, one can refine the control policy synthesized using finite abstractions to the original systems while providing guarantee on the probability of satisfaction of high-level requirements. Moreover, we provide sufficient conditions for the proposed notion of incremental stability in terms of the existence of incremental Lyapunov functions which reduce to some matrix inequalities for the linear systems. Finally, the effectiveness of the proposed results is illustrated by synthesizing a controller regulating the temperatures in a ten-room building modelled as a delayed jump-diffusion system.

The work done in this paper, proposes a complex Laplacian-based distributed control scheme for convergence in the multi-agent network. The proposed scheme has been designated as cascade formulation. The proposed technique exploits the traditional method of organizing large scattered networks into smaller interconnected clusters to optimize information flow within the network. The complex Laplacian-based approach results in a hierarchical structure, with formation of a meta-cluster leading other clusters in the network. The proposed formulation enables flexibility to constrain the eigen spectra of the overall closed-loop dynamics, ensuring desired convergence rate and control input intensity. The sufficient conditions ensuring globally stable formation for proposed formulation are also asserted. Robustness of the proposed formulation to uncertainties like loss in communication links and actuator failure has also been discussed. The effectiveness of the proposed approach is illustrated by simulating a finitely large network of thirty vehicles.

The Kalman filter and its extensions are used in a vast number of aerospace and navigation applications for nonlinear state estimation of time series. In the literature, different approaches have been proposed to exploit the structure of the state and measurement models to reduce the computational demand of the algorithms. In this tutorial, we survey existing code optimization methods and present them using unified notation that allows them to be used with various Kalman filter extensions. We develop the optimization methods to cover a wider range of models, show how different structural optimizations can be combined, and present new applications for the existing optimizations. Furthermore, we present an example that shows that the exploitation of the structure of the problem can lead to improved estimation accuracy while reducing the computational load. This tutorial is intended for persons who are familiar with Kalman filtering and want to get insights for reducing the computational demand of different Kalman filter extensions.

A compact version of the variation evolving method (VEM) is developed in the primal variable space for optimal control computation. Following the idea that originates from the Lyapunov continuous-time dynamics stability theory in the control field, the optimal solution is analogized to the stable equilibrium point of a dynamic system and obtained asymptotically through the variation motion. With the introduction of a virtual dimension, namely the variation time, the evolution partial differential equation (EPDE), which seeks the optimal solution with a theoretical guarantee, is developed for the optimal control problem (OCP) with free terminal states, and the equivalent optimality conditions with no employment of costates are established in the primal space. These conditions show that the optimal feedback control law is generally not analytically available because the optimal control is related to the future states. Since the derived EPDE is suitable to be computed with the semi-discrete method in the field of PDE numerical calculation, the optimal solution may be obtained by solving the resulting finite-dimensional initial-value problem (IVP).

Kalman filtering is a classic state estimation technique used in application areas such as signal processing and autonomous control of vehicles. It is now being used to solve problems in computer systems such as controlling the voltage and frequency of processors. Although there are many presentations of Kalman filtering in the literature, they usually deal with particular systems like autonomous robots or linear systems with Gaussian noise, which makes it difficult to understand the general principles behind Kalman filtering. In this paper, we first present the abstract ideas behind Kalman filtering at a level accessible to anyone with a basic knowledge of probability theory and calculus, and then show how these concepts can be applied to the particular problem of state estimation in linear systems. This separation of concepts from applications should make it easier to understand Kalman filtering and to apply it to other problems in computer systems.

We study the optimal control of an arbitrarily large constellation of small satellites operating in low Earth orbit. Simulating the lack of on-board propulsion, we limit our actuation to the use of differential drag maneuvers to make in-plane changes to the satellite orbits. We propose an efficient method to separate a cluster of satellites into a desired constellation shape while respecting actuation constraints and maximizing the operational lifetime of the constellation. By posing the problem as a linear program, we solve for the optimal drag commands for each of the satellites on a daily basis with a shrinking-horizon model predictive control approach. We then apply this control strategy in a nonlinear orbital dynamics simulation with a simple, varying atmospheric density model. We demonstrate the ability to control a cluster of 100+ satellites starting at the same initial conditions in a circular low Earth orbit to form an equally spaced constellation (with a relative angular separation error tolerance of one-tenth a degree). The constellation separation task can be executed in 71 days, a time frame that is competitive for the state-of-the-practice. This method allows us to trade the time required to converge to the desired constellation with a sacrifice in the overall constellation lifetime, measured as the maximum altitude loss experienced by one of the satellites in the group after the separation maneuvers.

This article presents the consensus of a saturated second order multi-agent system with non-switching dynamics that can be represented by a directed graph. The system is affected by data processing (input delay) and communication time-delays that are assumed to be asynchronous. The agents have saturation nonlinearities, each of them is approximated into separate linear and nonlinear elements. Nonlinear elements are represented by describing functions. Describing functions and stability of linear elements are used to estimate the existence of limit cycles in the system with multiple control laws. Stability analysis of the linear element is performed using Lyapunov-Krasovskii functions and frequency domain analysis. A comparison of pros and cons of both the analyses with respect to time-delay ranges, applicability and computation complexity is presented. Simulation and corresponding hardware implementation results are demonstrated to support theoretical results.

Linear-Quadratic-Gaussian (LQG) control is concerned with the design of an optimal controller and estimator for linear Gaussian systems with imperfect state information. Standard LQG assumes the set of sensor measurements, to be fed to the estimator, to be given. However, in many problems, arising in networked systems and robotics, one may not be able to use all the available sensors, due to power or payload constraints, or may be interested in using the smallest subset of sensors that guarantees the attainment of a desired control goal. In this paper, we introduce the sensing-constrained LQG control problem, in which one has to jointly design sensing, estimation, and control, under given constraints on the resources spent for sensing. We focus on the realistic case in which the sensing strategy has to be selected among a finite set of possible sensing modalities. While the computation of the optimal sensing strategy is intractable, we present the first scalable algorithm that computes a near-optimal sensing strategy with provable sub-optimality guarantees. To this end, we show that a separation principle holds, which allows the design of sensing, estimation, and control policies in isolation. We conclude the paper by discussing two applications of sensing-constrained LQG control, namely, sensing-constrained formation control and resource-constrained robot navigation.

One version of the concept of structural controllability defined for single-input systems by Lin and subsequently generalized to multi-input systems by others, states that a parameterized matrix pair $(A, B)$ whose nonzero entries are distinct parameters, is structurally controllable if values can be assigned to the parameters which cause the resulting matrix pair to be controllable. In this paper the concept of structural controllability is broadened to allow for the possibility that a parameter may appear in more than one location in the pair $(A, B)$. Subject to a certain condition on the parameterization called the "binary assumption", an explicit graph-theoretic characterization of such matrix pairs is derived.

We consider the problem of convergence to a saddle point of a concave-convex function via gradient dynamics. Since first introduced by Arrow, Hurwicz and Uzawa in [1] such dynamics have been extensively used in diverse areas, there are, however, features that render their analysis non trivial. These include the lack of convergence guarantees when the function considered is not strictly concave-convex and also the non-smoothness of subgradient dynamics. Our aim in this two part paper is to provide an explicit characterization to the asymptotic behaviour of general gradient and subgradient dynamics applied to a general concave-convex function. We show that despite the nonlinearity and non-smoothness of these dynamics their $\omega$-limit set is comprised of trajectories that solve only explicit linear ODEs that are characterized within the paper. More precisely, in Part I an exact characterization is provided to the asymptotic behaviour of unconstrained gradient dynamics. We also show that when convergence to a saddle point is not guaranteed then the system behaviour can be problematic, with arbitrarily small noise leading to an unbounded variance. In Part II we consider a general class of subgradient dynamics that restrict trajectories in an arbitrary convex domain, and show that when an equilibrium point exists their limiting trajectories are solutions of subgradient dynamics on only affine subspaces. The latter is a smooth class of dynamics with an asymptotic behaviour exactly characterized in Part I, as solutions to explicit linear ODEs. These results are used to formulate corresponding convergence criteria and are demonstrated with several examples and applications presented in Part II.

Policy iteration (PI) is a recursive process of policy evaluation and improvement for solving an optimal decision-making/control problem, or in other words, a reinforcement learning (RL) problem. PI has also served as the fundamental for developing RL methods. In this paper, we propose two PI methods, called differential PI (DPI) and integral PI (IPI), and their variants, for a general RL framework in continuous time and space (CTS), where the environment is modeled by a system of ordinary differential equations (ODEs). The proposed methods inherit the current ideas of PI in classical RL and optimal control and theoretically support the existing RL algorithms in CTS: TD-learning and value-gradient-based (VGB) greedy policy update. We also provide case studies including 1) discounted RL and 2) optimal control tasks. Fundamental mathematical properties -- admissibility, uniqueness of the solution to the Bellman equation (BE), monotone improvement, convergence, and optimality of the solution to the Hamilton-Jacobi-Bellman equation (HJBE) -- are all investigated in-depth and improved from the existing theory, along with the general and case studies. Finally, the proposed ones are simulated with an inverted-pendulum model and their model-based and partially model-free implementations to support the theory and further investigate them beyond.

Previous results reported in the robotics literature show the relationship between time-delay control (TDC) and proportional-integral-derivative control (PID). In this paper, we show that incremental nonlinear dynamic inversion (INDI) - more familiar in the aerospace community - are in fact equivalent to TDC. This leads to a meaningful and systematic method for PI(D)-control tuning of robust nonlinear flight control systems via INDI. We considered a reformulation of the plant dynamics inversion which removes effector blending models from the resulting control law, resulting in robust model-free control laws like PI(D)-control.

In this paper, a concurrent learning based adaptive observer is developed for a class of second-order nonlinear time-invariant systems with uncertain dynamics. The developed technique results in uniformly ultimately bounded state and parameter estimation errors. As opposed to persistent excitation which is required for parameter convergence in traditional adaptive control methods, the developed technique only requires excitation over a finite time interval to achieve parameter convergence. Simulation results in both noise-free and noisy environments are presented to validate the design.

This paper develops the theoretical framework and the equations of a new robust Generalized Maximum-likelihood-type Unscented Kalman Filter (GM-UKF) that is able to suppress observation and innovation outliers while filtering out non-Gaussian measurement noise. Because the errors of the real and reactive power measurements calculated using Phasor Measurement Units (PMUs) follow long-tailed probability distributions, the conventional UKF provides strongly biased state estimates since it relies on the weighted least squares estimator. By contrast, the state estimates and residuals of our GM-UKF are proved to be roughly Gaussian, allowing the sigma points to reliably approximate the mean and the covariance matrices of the predicted and corrected state vectors. To develop our GM-UKF, we first derive a batch-mode regression form by processing the predictions and observations simultaneously, where the statistical linearization approach is used. We show that the set of equations so derived are equivalent to those of the unscented transformation. Then, a robust GM-estimator that minimizes a convex Huber cost function while using weights calculated via Projection Statistics (PS's) is proposed. The PS's are applied to a two-dimensional matrix that consists of serially correlated predicted state and innovation vectors to detect observation and innovation outliers. These outliers are suppressed by the GM-estimator using the iteratively reweighted least squares algorithm. Finally, the asymptotic error covariance matrix of the GM-UKF state estimates is derived from the total influence function. In the companion paper, extensive simulation results will be shown to verify the effectiveness and robustness of the proposed method.

This paper is the second of a two-part series that discusses the implementation issues and test results of a robust Unscented Kalman Filter (UKF) for power system dynamic state estimation with non-Gaussian synchrophasor measurement noise. The tuning of the parameters of our Generalized Maximum-Likelihood-type robust UKF (GM-UKF) is presented and discussed in a systematic way. Using simulations carried out on the IEEE 39-bus system, its performance is evaluated under different scenarios, including i) the occurrence of two different types of noises following thick-tailed distributions, namely the Laplace or Cauchy probability distributions for real and reactive power measurements; ii) the occurrence of observation and innovation outliers; iii) the occurrence of PMU measurement losses due to communication failures; iv) cyber attacks; and v) strong system nonlinearities. It is also compared to the UKF and the Generalized Maximum-Likelihood-type robust iterated EKF (GM-IEKF). Simulation results reveal that the GM-UKF outperforms the GM-IEKF and the UKF in all scenarios considered. In particular, when the system is operating under stressed conditions, inducing system nonlinearities, the GM-IEKF and the UKF diverge while our GM-UKF does converge. In addition, when the power measurement noises obey a Cauchy distribution, our GM-UKF converges to a state estimate vector that exhibits a much higher statistical efficiency than that of the GM-IEKF; by contrast, the UKF fails to converge. Finally, potential applications and future work of the proposed GM-UKF are discussed in concluding remarks section.

To help mitigate road congestion caused by the unrelenting growth of traffic demand, many transit authorities have implemented managed lane policies. Managed lanes typically run parallel to a freeway's standard, general-purpose (GP) lanes, but are restricted to certain types of vehicles. It was originally thought that managed lanes would improve the use of existing infrastructure through incentivization of demand-management behaviors like carpooling, but implementations have often been characterized by unpredicted phenomena that is often to detrimental system performance. This paper presents several macroscopic traffic modeling tools we have used for study of freeways equipped with managed lanes, or "managed lane-freeway networks." The proposed framework is based on the widely-used first-order kinematic wave theory. In this model, the GP and the managed lanes are modeled as parallel links connected by nodes, where certain type of traffic may switch between GP and managed lane links. Two types of managed lane topologies are considered: full-access, where vehicles can switch between the GP and the managed lanes anywhere; and separated, where such switching is allowed only at certain locations called gates. We also describe methods to incorporate in three phenomena into our model that are particular to managed lane-freeway networks. The inertia effect reflects drivers' inclination to stay in their lane as long as possible and switch only if this would obviously improve their travel condition. The friction effect reflects the empirically-observed driver fear of moving fast in a managed lane while traffic in the adjacent GP lanes moves slowly due to congestion. The smoothing effect describes how managed lanes can increase throughput at bottlenecks by reducing lane changes. We present simple models for each of these phenomena that fit within the general macroscopic theory.

Biological and advanced cyberphysical control systems often have limited, sparse, uncertain, and distributed communication and computing in addition to sensing and actuation. Fortunately, the corresponding plants and performance requirements are also sparse and structured, and this must be exploited to make constrained controller design feasible and tractable. We introduce a new "system level" (SL) approach involving three complementary SL elements. System Level Parameterizations (SLPs) generalize state space and Youla parameterizations of all stabilizing controllers and the responses they achieve, and combine with System Level Constraints (SLCs) to parameterize the largest known class of constrained stabilizing controllers that admit a convex characterization, generalizing quadratic invariance (QI). SLPs also lead to a generalization of detectability and stabilizability, suggesting the existence of a rich separation structure, that when combined with SLCs, is naturally applicable to structurally constrained controllers and systems. We further provide a catalog of useful SLCs, most importantly including sparsity, delay, and locality constraints on both communication and computing internal to the controller, and external system performance. The resulting System Level Synthesis (SLS) problems that arise define the broadest known class of constrained optimal control problems that can be solved using convex programming. An example illustrates how this system level approach can systematically explore tradeoffs in controller performance, robustness, and synthesis/implementation complexity.

This paper presents a novel method for controlling teams of unmanned aerial vehicles using Stochastic Optimal Control (SOC) theory. The approach consists of a centralized high-level planner that computes optimal state trajectories as velocity sequences, and a platform-specific low-level controller which ensures that these velocity sequences are met. The planning task is expressed as a centralized path-integral control problem, for which optimal control computation corresponds to a probabilistic inference problem that can be solved by efficient sampling methods. Through simulation we show that our SOC approach (a) has significant benefits compared to deterministic control and other SOC methods in multimodal problems with noise-dependent optimal solutions, (b) is capable of controlling a large number of platforms in real-time, and (c) yields collective emergent behaviour in the form of flight formations. Finally, we show that our approach works for real platforms, by controlling a team of three quadrotors in outdoor conditions.

A standard operational requirement in power systems is that the voltage magnitudes lie within prespecified bounds. Conventional engineering wisdom suggests that such a tightly-regulated profile, imposed for system design purposes and good operation of the network, should also guarantee a secure system, operating far from static bifurcation instabilities such as voltage collapse. In general however, these two objectives are distinct and must be separately enforced. We formulate an optimization problem which maximizes the distance to voltage collapse through injections of reactive power, subject to power flow and operational voltage constraints. By exploiting a linear approximation of the power flow equations we arrive at a convex reformulation which can be efficiently solved for the optimal injections. We also address the planning problem of allocating the resources by recasting our problem in a sparsity-promoting framework that allows us to choose a desired trade-off between optimality of injections and the number of required actuators. Finally, we present a distributed algorithm to solve the optimization problem, showing that it can be implemented on-line as a feedback controller. We illustrate the performance of our results with the IEEE30 bus network.

The potential of demand side as a frequency reserve proposes interesting opportunity in handling imbalances due to intermittent renewable energy sources. This paper proposes a novel approach for computing the parameters of a stochastic battery model representing the aggregation of Thermostatically Controlled Loads (TCLs). A hysteresis based non-disruptive control is used using priority stack algorithm to track the reference regulation signal. The parameters of admissible ramp-rate and the charge limits of the battery are dynamically calculated using the information from TCLs that is the status (on/off), availability and relative temperature distance till the switching boundary. The approach builds on and improves on the existing research work by providing a straight-forward mechanism for calculation of stochastic parameters of equivalent battery model. The effectiveness of proposed approach is demonstrated by a test case having a large number of residential TCLs tracking a scaled down real frequency regulation signal.

We consider a network of evolving opinions. It includes multiple individuals with first-order opinion dynamics defined in continuous time and evolving based on a general exogenously defined time-varying underlying graph. In such a network, for an arbitrary fixed initial time, a subset of individuals forms an eminence grise coalition, abbreviated as EGC, if the individuals in that subset are capable of leading the entire network to agreeing on any desired opinion, through a cooperative choice of their own initial opinions. In this endeavor, the coalition members are assumed to have access to full profile of the underlying graph of the network as well as the initial opinions of all other individuals. While the complete coalition of individuals always qualifies as an EGC, we establish the existence of a minimum size EGC for an arbitrary time-varying network; also, we develop a non-trivial set of upper and lower bounds on that size. As a result, we show that, even when the underlying graph does not guarantee convergence to a global or multiple consensus, a generally restricted coalition of agents can steer public opinion towards a desired global consensus without affecting any of the predefined graph interactions, provided they can cooperatively adjust their own initial opinions. Geometric insights into the structure of EGC's are given. The results are also extended to the discrete time case where the relation with Decomposition-Separation Theorem is also made explicit.

Convergence properties of time inhomogeneous Markov chain based discrete and continuous time linear consensus algorithms are analyzed. Provided that a so-called infinite jet flow property is satisfied by the underlying chains, necessary conditions for both consensus and multiple consensus are established. A recenet extension by Sonin of the classical Kolmogorov-Doeblin decomposition-separation for homogeneous Markov chains to the inhomogeneous case is then employed to show that the obtained necessary conditions are also sufficient when the chain is of Class P*, as defined by Touri and Nedic. It is also shown that Sonin's theorem leads to a rediscovery and generalization of most of the existing related consensus results in the literature.

In a multi-agent system, unconditional (multiple) consensus is the property of reaching to (multiple) consensus irrespective of the instant and values at which states are initialized. For linear algorithms, occurrence of unconditional (multiple) consensus turns out to be equivalent to (class-) ergodicity of the transition chain (A_n). For a wide class of chains, chains with so-called balanced asymmetry property, necessary and sufficient conditions for ergodicity and class-ergodicity are derived. The results are employed to analyze the limiting behavior of agents' states in the JLM model, the Krause model, and the Cucker-Smale model. In particular, unconditional single or multiple consensus occurs in all three models. Moreover, a necessary and sufficient condition for unconditional consensus in the JLM model and a sufficient condition for consensus in the Cucker-Smale model are obtained.

Multi agent consensus algorithms with update steps based on so-called balanced asymmetric chains, are analyzed. For such algorithms it is shown that (i) the set of accumulation points of states is finite, (ii) the asymptotic unconditional occurrence of single consensus or multiple consensuses is directly related to the property of absolute infinite flow for the underlying update chain. The results are applied to well known consensus models.