
We propose a solution of the multiple target tracking (MTT) problem based on
sets of trajectories and the random finite set framework. A full Bayesian
approach to MTT should characterise the distribution of the trajectories given
the measurements, as it contains all information about the trajectories. We
attain this by considering multiobject density functions in which objects are
trajectories. For the standard tracking models, we also describe a conjugate
family of multitrajectory density functions.

In this paper, we provide for the first time an automated,
correctbyconstruction, controller synthesis scheme for a class of infinite
dimensional stochastic systems, namely, retarded jumpdiffusion systems. First,
we construct finite abstractions approximately bisimilar to nonprobabilistic
retarded systems corresponding to the original systems having some stability
property, namely, incremental inputtostate stability. Then, we provide a
result on quantifying the distance between output trajectory of the obtained
finite abstraction and that of the original retarded jumpdiffusion system in a
probabilistic setting. Using the proposed result, one can refine the control
policy synthesized using finite abstractions to the original systems while
providing guarantee on the probability of satisfaction of highlevel
requirements. Moreover, we provide sufficient conditions for the proposed
notion of incremental stability in terms of the existence of incremental
Lyapunov functions which reduce to some matrix inequalities for the linear
systems. Finally, the effectiveness of the proposed results is illustrated by
synthesizing a controller regulating the temperatures in a tenroom building
modelled as a delayed jumpdiffusion system.

The work done in this paper, proposes a complex Laplacianbased distributed
control scheme for convergence in the multiagent network. The proposed scheme
has been designated as cascade formulation. The proposed technique exploits the
traditional method of organizing large scattered networks into smaller
interconnected clusters to optimize information flow within the network. The
complex Laplacianbased approach results in a hierarchical structure, with
formation of a metacluster leading other clusters in the network. The proposed
formulation enables flexibility to constrain the eigen spectra of the overall
closedloop dynamics, ensuring desired convergence rate and control input
intensity. The sufficient conditions ensuring globally stable formation for
proposed formulation are also asserted. Robustness of the proposed formulation
to uncertainties like loss in communication links and actuator failure has also
been discussed. The effectiveness of the proposed approach is illustrated by
simulating a finitely large network of thirty vehicles.

The Kalman filter and its extensions are used in a vast number of aerospace
and navigation applications for nonlinear state estimation of time series. In
the literature, different approaches have been proposed to exploit the
structure of the state and measurement models to reduce the computational
demand of the algorithms. In this tutorial, we survey existing code
optimization methods and present them using unified notation that allows them
to be used with various Kalman filter extensions. We develop the optimization
methods to cover a wider range of models, show how different structural
optimizations can be combined, and present new applications for the existing
optimizations. Furthermore, we present an example that shows that the
exploitation of the structure of the problem can lead to improved estimation
accuracy while reducing the computational load. This tutorial is intended for
persons who are familiar with Kalman filtering and want to get insights for
reducing the computational demand of different Kalman filter extensions.

A compact version of the variation evolving method (VEM) is developed in the
primal variable space for optimal control computation. Following the idea that
originates from the Lyapunov continuoustime dynamics stability theory in the
control field, the optimal solution is analogized to the stable equilibrium
point of a dynamic system and obtained asymptotically through the variation
motion. With the introduction of a virtual dimension, namely the variation
time, the evolution partial differential equation (EPDE), which seeks the
optimal solution with a theoretical guarantee, is developed for the optimal
control problem (OCP) with free terminal states, and the equivalent optimality
conditions with no employment of costates are established in the primal space.
These conditions show that the optimal feedback control law is generally not
analytically available because the optimal control is related to the future
states. Since the derived EPDE is suitable to be computed with the
semidiscrete method in the field of PDE numerical calculation, the optimal
solution may be obtained by solving the resulting finitedimensional
initialvalue problem (IVP).

Kalman filtering is a classic state estimation technique used in application
areas such as signal processing and autonomous control of vehicles. It is now
being used to solve problems in computer systems such as controlling the
voltage and frequency of processors.
Although there are many presentations of Kalman filtering in the literature,
they usually deal with particular systems like autonomous robots or linear
systems with Gaussian noise, which makes it difficult to understand the general
principles behind Kalman filtering. In this paper, we first present the
abstract ideas behind Kalman filtering at a level accessible to anyone with a
basic knowledge of probability theory and calculus, and then show how these
concepts can be applied to the particular problem of state estimation in linear
systems. This separation of concepts from applications should make it easier to
understand Kalman filtering and to apply it to other problems in computer
systems.

We study the optimal control of an arbitrarily large constellation of small
satellites operating in low Earth orbit. Simulating the lack of onboard
propulsion, we limit our actuation to the use of differential drag maneuvers to
make inplane changes to the satellite orbits. We propose an efficient method
to separate a cluster of satellites into a desired constellation shape while
respecting actuation constraints and maximizing the operational lifetime of the
constellation. By posing the problem as a linear program, we solve for the
optimal drag commands for each of the satellites on a daily basis with a
shrinkinghorizon model predictive control approach. We then apply this control
strategy in a nonlinear orbital dynamics simulation with a simple, varying
atmospheric density model. We demonstrate the ability to control a cluster of
100+ satellites starting at the same initial conditions in a circular low Earth
orbit to form an equally spaced constellation (with a relative angular
separation error tolerance of onetenth a degree). The constellation separation
task can be executed in 71 days, a time frame that is competitive for the
stateofthepractice. This method allows us to trade the time required to
converge to the desired constellation with a sacrifice in the overall
constellation lifetime, measured as the maximum altitude loss experienced by
one of the satellites in the group after the separation maneuvers.

This article presents the consensus of a saturated second order multiagent
system with nonswitching dynamics that can be represented by a directed graph.
The system is affected by data processing (input delay) and communication
timedelays that are assumed to be asynchronous. The agents have saturation
nonlinearities, each of them is approximated into separate linear and nonlinear
elements. Nonlinear elements are represented by describing functions.
Describing functions and stability of linear elements are used to estimate the
existence of limit cycles in the system with multiple control laws. Stability
analysis of the linear element is performed using LyapunovKrasovskii functions
and frequency domain analysis. A comparison of pros and cons of both the
analyses with respect to timedelay ranges, applicability and computation
complexity is presented. Simulation and corresponding hardware implementation
results are demonstrated to support theoretical results.

LinearQuadraticGaussian (LQG) control is concerned with the design of an
optimal controller and estimator for linear Gaussian systems with imperfect
state information. Standard LQG assumes the set of sensor measurements, to be
fed to the estimator, to be given. However, in many problems, arising in
networked systems and robotics, one may not be able to use all the available
sensors, due to power or payload constraints, or may be interested in using the
smallest subset of sensors that guarantees the attainment of a desired control
goal. In this paper, we introduce the sensingconstrained LQG control problem,
in which one has to jointly design sensing, estimation, and control, under
given constraints on the resources spent for sensing. We focus on the realistic
case in which the sensing strategy has to be selected among a finite set of
possible sensing modalities. While the computation of the optimal sensing
strategy is intractable, we present the first scalable algorithm that computes
a nearoptimal sensing strategy with provable suboptimality guarantees. To
this end, we show that a separation principle holds, which allows the design of
sensing, estimation, and control policies in isolation. We conclude the paper
by discussing two applications of sensingconstrained LQG control, namely,
sensingconstrained formation control and resourceconstrained robot
navigation.

One version of the concept of structural controllability defined for
singleinput systems by Lin and subsequently generalized to multiinput systems
by others, states that a parameterized matrix pair $(A, B)$ whose nonzero
entries are distinct parameters, is structurally controllable if values can be
assigned to the parameters which cause the resulting matrix pair to be
controllable. In this paper the concept of structural controllability is
broadened to allow for the possibility that a parameter may appear in more than
one location in the pair $(A, B)$. Subject to a certain condition on the
parameterization called the "binary assumption", an explicit graphtheoretic
characterization of such matrix pairs is derived.

We consider the problem of convergence to a saddle point of a concaveconvex
function via gradient dynamics. Since first introduced by Arrow, Hurwicz and
Uzawa in [1] such dynamics have been extensively used in diverse areas, there
are, however, features that render their analysis non trivial. These include
the lack of convergence guarantees when the function considered is not strictly
concaveconvex and also the nonsmoothness of subgradient dynamics. Our aim in
this two part paper is to provide an explicit characterization to the
asymptotic behaviour of general gradient and subgradient dynamics applied to a
general concaveconvex function. We show that despite the nonlinearity and
nonsmoothness of these dynamics their $\omega$limit set is comprised of
trajectories that solve only explicit linear ODEs that are characterized within
the paper.
More precisely, in Part I an exact characterization is provided to the
asymptotic behaviour of unconstrained gradient dynamics. We also show that when
convergence to a saddle point is not guaranteed then the system behaviour can
be problematic, with arbitrarily small noise leading to an unbounded variance.
In Part II we consider a general class of subgradient dynamics that restrict
trajectories in an arbitrary convex domain, and show that when an equilibrium
point exists their limiting trajectories are solutions of subgradient dynamics
on only affine subspaces. The latter is a smooth class of dynamics with an
asymptotic behaviour exactly characterized in Part I, as solutions to explicit
linear ODEs. These results are used to formulate corresponding convergence
criteria and are demonstrated with several examples and applications presented
in Part II.

Policy iteration (PI) is a recursive process of policy evaluation and
improvement for solving an optimal decisionmaking/control problem, or in other
words, a reinforcement learning (RL) problem. PI has also served as the
fundamental for developing RL methods. In this paper, we propose two PI
methods, called differential PI (DPI) and integral PI (IPI), and their
variants, for a general RL framework in continuous time and space (CTS), where
the environment is modeled by a system of ordinary differential equations
(ODEs). The proposed methods inherit the current ideas of PI in classical RL
and optimal control and theoretically support the existing RL algorithms in
CTS: TDlearning and valuegradientbased (VGB) greedy policy update. We also
provide case studies including 1) discounted RL and 2) optimal control tasks.
Fundamental mathematical properties  admissibility, uniqueness of the
solution to the Bellman equation (BE), monotone improvement, convergence, and
optimality of the solution to the HamiltonJacobiBellman equation (HJBE) 
are all investigated indepth and improved from the existing theory, along with
the general and case studies. Finally, the proposed ones are simulated with an
invertedpendulum model and their modelbased and partially modelfree
implementations to support the theory and further investigate them beyond.

Previous results reported in the robotics literature show the relationship
between timedelay control (TDC) and proportionalintegralderivative control
(PID). In this paper, we show that incremental nonlinear dynamic inversion
(INDI)  more familiar in the aerospace community  are in fact equivalent to
TDC. This leads to a meaningful and systematic method for PI(D)control tuning
of robust nonlinear flight control systems via INDI. We considered a
reformulation of the plant dynamics inversion which removes effector blending
models from the resulting control law, resulting in robust modelfree control
laws like PI(D)control.

In this paper, a concurrent learning based adaptive observer is developed for
a class of secondorder nonlinear timeinvariant systems with uncertain
dynamics. The developed technique results in uniformly ultimately bounded state
and parameter estimation errors. As opposed to persistent excitation which is
required for parameter convergence in traditional adaptive control methods, the
developed technique only requires excitation over a finite time interval to
achieve parameter convergence. Simulation results in both noisefree and noisy
environments are presented to validate the design.

This paper develops the theoretical framework and the equations of a new
robust Generalized Maximumlikelihoodtype Unscented Kalman Filter (GMUKF)
that is able to suppress observation and innovation outliers while filtering
out nonGaussian measurement noise. Because the errors of the real and reactive
power measurements calculated using Phasor Measurement Units (PMUs) follow
longtailed probability distributions, the conventional UKF provides strongly
biased state estimates since it relies on the weighted least squares estimator.
By contrast, the state estimates and residuals of our GMUKF are proved to be
roughly Gaussian, allowing the sigma points to reliably approximate the mean
and the covariance matrices of the predicted and corrected state vectors. To
develop our GMUKF, we first derive a batchmode regression form by processing
the predictions and observations simultaneously, where the statistical
linearization approach is used. We show that the set of equations so derived
are equivalent to those of the unscented transformation. Then, a robust
GMestimator that minimizes a convex Huber cost function while using weights
calculated via Projection Statistics (PS's) is proposed. The PS's are applied
to a twodimensional matrix that consists of serially correlated predicted
state and innovation vectors to detect observation and innovation outliers.
These outliers are suppressed by the GMestimator using the iteratively
reweighted least squares algorithm. Finally, the asymptotic error covariance
matrix of the GMUKF state estimates is derived from the total influence
function. In the companion paper, extensive simulation results will be shown to
verify the effectiveness and robustness of the proposed method.

This paper is the second of a twopart series that discusses the
implementation issues and test results of a robust Unscented Kalman Filter
(UKF) for power system dynamic state estimation with nonGaussian synchrophasor
measurement noise. The tuning of the parameters of our Generalized
MaximumLikelihoodtype robust UKF (GMUKF) is presented and discussed in a
systematic way. Using simulations carried out on the IEEE 39bus system, its
performance is evaluated under different scenarios, including i) the occurrence
of two different types of noises following thicktailed distributions, namely
the Laplace or Cauchy probability distributions for real and reactive power
measurements; ii) the occurrence of observation and innovation outliers; iii)
the occurrence of PMU measurement losses due to communication failures; iv)
cyber attacks; and v) strong system nonlinearities. It is also compared to the
UKF and the Generalized MaximumLikelihoodtype robust iterated EKF (GMIEKF).
Simulation results reveal that the GMUKF outperforms the GMIEKF and the UKF
in all scenarios considered. In particular, when the system is operating under
stressed conditions, inducing system nonlinearities, the GMIEKF and the UKF
diverge while our GMUKF does converge. In addition, when the power measurement
noises obey a Cauchy distribution, our GMUKF converges to a state estimate
vector that exhibits a much higher statistical efficiency than that of the
GMIEKF; by contrast, the UKF fails to converge. Finally, potential
applications and future work of the proposed GMUKF are discussed in concluding
remarks section.

To help mitigate road congestion caused by the unrelenting growth of traffic
demand, many transit authorities have implemented managed lane policies.
Managed lanes typically run parallel to a freeway's standard, generalpurpose
(GP) lanes, but are restricted to certain types of vehicles. It was originally
thought that managed lanes would improve the use of existing infrastructure
through incentivization of demandmanagement behaviors like carpooling, but
implementations have often been characterized by unpredicted phenomena that is
often to detrimental system performance.
This paper presents several macroscopic traffic modeling tools we have used
for study of freeways equipped with managed lanes, or "managed lanefreeway
networks." The proposed framework is based on the widelyused firstorder
kinematic wave theory. In this model, the GP and the managed lanes are modeled
as parallel links connected by nodes, where certain type of traffic may switch
between GP and managed lane links. Two types of managed lane topologies are
considered: fullaccess, where vehicles can switch between the GP and the
managed lanes anywhere; and separated, where such switching is allowed only at
certain locations called gates.
We also describe methods to incorporate in three phenomena into our model
that are particular to managed lanefreeway networks. The inertia effect
reflects drivers' inclination to stay in their lane as long as possible and
switch only if this would obviously improve their travel condition. The
friction effect reflects the empiricallyobserved driver fear of moving fast in
a managed lane while traffic in the adjacent GP lanes moves slowly due to
congestion. The smoothing effect describes how managed lanes can increase
throughput at bottlenecks by reducing lane changes. We present simple models
for each of these phenomena that fit within the general macroscopic theory.

Biological and advanced cyberphysical control systems often have limited,
sparse, uncertain, and distributed communication and computing in addition to
sensing and actuation. Fortunately, the corresponding plants and performance
requirements are also sparse and structured, and this must be exploited to make
constrained controller design feasible and tractable. We introduce a new
"system level" (SL) approach involving three complementary SL elements. System
Level Parameterizations (SLPs) generalize state space and Youla
parameterizations of all stabilizing controllers and the responses they
achieve, and combine with System Level Constraints (SLCs) to parameterize the
largest known class of constrained stabilizing controllers that admit a convex
characterization, generalizing quadratic invariance (QI). SLPs also lead to a
generalization of detectability and stabilizability, suggesting the existence
of a rich separation structure, that when combined with SLCs, is naturally
applicable to structurally constrained controllers and systems. We further
provide a catalog of useful SLCs, most importantly including sparsity, delay,
and locality constraints on both communication and computing internal to the
controller, and external system performance. The resulting System Level
Synthesis (SLS) problems that arise define the broadest known class of
constrained optimal control problems that can be solved using convex
programming. An example illustrates how this system level approach can
systematically explore tradeoffs in controller performance, robustness, and
synthesis/implementation complexity.

This paper presents a novel method for controlling teams of unmanned aerial
vehicles using Stochastic Optimal Control (SOC) theory. The approach consists
of a centralized highlevel planner that computes optimal state trajectories as
velocity sequences, and a platformspecific lowlevel controller which ensures
that these velocity sequences are met. The planning task is expressed as a
centralized pathintegral control problem, for which optimal control
computation corresponds to a probabilistic inference problem that can be solved
by efficient sampling methods. Through simulation we show that our SOC approach
(a) has significant benefits compared to deterministic control and other SOC
methods in multimodal problems with noisedependent optimal solutions, (b) is
capable of controlling a large number of platforms in realtime, and (c) yields
collective emergent behaviour in the form of flight formations. Finally, we
show that our approach works for real platforms, by controlling a team of three
quadrotors in outdoor conditions.

A standard operational requirement in power systems is that the voltage
magnitudes lie within prespecified bounds. Conventional engineering wisdom
suggests that such a tightlyregulated profile, imposed for system design
purposes and good operation of the network, should also guarantee a secure
system, operating far from static bifurcation instabilities such as voltage
collapse. In general however, these two objectives are distinct and must be
separately enforced. We formulate an optimization problem which maximizes the
distance to voltage collapse through injections of reactive power, subject to
power flow and operational voltage constraints. By exploiting a linear
approximation of the power flow equations we arrive at a convex reformulation
which can be efficiently solved for the optimal injections. We also address the
planning problem of allocating the resources by recasting our problem in a
sparsitypromoting framework that allows us to choose a desired tradeoff
between optimality of injections and the number of required actuators. Finally,
we present a distributed algorithm to solve the optimization problem, showing
that it can be implemented online as a feedback controller. We illustrate the
performance of our results with the IEEE30 bus network.

The potential of demand side as a frequency reserve proposes interesting
opportunity in handling imbalances due to intermittent renewable energy
sources. This paper proposes a novel approach for computing the parameters of a
stochastic battery model representing the aggregation of Thermostatically
Controlled Loads (TCLs). A hysteresis based nondisruptive control is used
using priority stack algorithm to track the reference regulation signal. The
parameters of admissible ramprate and the charge limits of the battery are
dynamically calculated using the information from TCLs that is the status
(on/off), availability and relative temperature distance till the switching
boundary. The approach builds on and improves on the existing research work by
providing a straightforward mechanism for calculation of stochastic parameters
of equivalent battery model. The effectiveness of proposed approach is
demonstrated by a test case having a large number of residential TCLs tracking
a scaled down real frequency regulation signal.

We consider a network of evolving opinions. It includes multiple individuals
with firstorder opinion dynamics defined in continuous time and evolving based
on a general exogenously defined timevarying underlying graph. In such a
network, for an arbitrary fixed initial time, a subset of individuals forms an
eminence grise coalition, abbreviated as EGC, if the individuals in that subset
are capable of leading the entire network to agreeing on any desired opinion,
through a cooperative choice of their own initial opinions. In this endeavor,
the coalition members are assumed to have access to full profile of the
underlying graph of the network as well as the initial opinions of all other
individuals. While the complete coalition of individuals always qualifies as an
EGC, we establish the existence of a minimum size EGC for an arbitrary
timevarying network; also, we develop a nontrivial set of upper and lower
bounds on that size. As a result, we show that, even when the underlying graph
does not guarantee convergence to a global or multiple consensus, a generally
restricted coalition of agents can steer public opinion towards a desired
global consensus without affecting any of the predefined graph interactions,
provided they can cooperatively adjust their own initial opinions. Geometric
insights into the structure of EGC's are given. The results are also extended
to the discrete time case where the relation with DecompositionSeparation
Theorem is also made explicit.

Convergence properties of time inhomogeneous Markov chain based discrete and
continuous time linear consensus algorithms are analyzed. Provided that a
socalled infinite jet flow property is satisfied by the underlying chains,
necessary conditions for both consensus and multiple consensus are established.
A recenet extension by Sonin of the classical KolmogorovDoeblin
decompositionseparation for homogeneous Markov chains to the inhomogeneous
case is then employed to show that the obtained necessary conditions are also
sufficient when the chain is of Class P*, as defined by Touri and Nedic. It is
also shown that Sonin's theorem leads to a rediscovery and generalization of
most of the existing related consensus results in the literature.

In a multiagent system, unconditional (multiple) consensus is the property
of reaching to (multiple) consensus irrespective of the instant and values at
which states are initialized. For linear algorithms, occurrence of
unconditional (multiple) consensus turns out to be equivalent to (class)
ergodicity of the transition chain (A_n). For a wide class of chains, chains
with socalled balanced asymmetry property, necessary and sufficient conditions
for ergodicity and classergodicity are derived. The results are employed to
analyze the limiting behavior of agents' states in the JLM model, the Krause
model, and the CuckerSmale model. In particular, unconditional single or
multiple consensus occurs in all three models. Moreover, a necessary and
sufficient condition for unconditional consensus in the JLM model and a
sufficient condition for consensus in the CuckerSmale model are obtained.

Multi agent consensus algorithms with update steps based on socalled
balanced asymmetric chains, are analyzed. For such algorithms it is shown that
(i) the set of accumulation points of states is finite, (ii) the asymptotic
unconditional occurrence of single consensus or multiple consensuses is
directly related to the property of absolute infinite flow for the underlying
update chain. The results are applied to well known consensus models.