
We study the ergodic properties of a class of multidimensional piecewise
OrnsteinUhlenbeck processes with jumps, which contains the limit of the
queueing processes arising in multiclass manyserver queues with heavytailed
arrivals and/or asymptotically negligible service interruptions in the
HalfinWhitt regime as special cases. In these queueing models, the It\^o
equations have a piecewise linear drift, and are driven by either (1) a
Brownian motion and a purejump L\'evy process, or (2) an anisotropic L\'evy
process with independent onedimensional symmetric $\alpha$stable components,
or (3) an anisotropic L\'evy process as in (2) and a purejump L\'evy process.
We also study the class of models driven by a subordinate Brownian motion,
which contains an isotropic (or rotationally invariant) $\alpha$stable L\'evy
process as a special case. We identify conditions on the parameters in the
drift, the L\'evy measure and/or covariance function which result in
subexponential and/or exponential ergodicity. We show that these assumptions
are sharp, and we identify some key necessary conditions for the process to be
ergodic. In addition, we show that for the queueing models described above with
no abandonment, the rate of convergence is polynomial, and we provide a sharp
quantitative characterization of the rate via matching upper and lower bounds.

We study the ergodic control problem for a class of jump diffusions in
$\mathbb{R}^d$, which are controlled through the drift with bounded controls.
The Levy measure is finite, but has no particular structure; it can be
anisotropic and singular. Moreover, there is no blanket ergodicity assumption
for the controlled process. Unstable behavior is `discouraged' by the running
cost which satisfies a mild coercive hypothesis (i.e., is nearmonotone). We
first study the problem in its weak formulation as an optimization problem on
the space of infinitesimal ergodic occupation measures, and derive the
HamiltonJacobiBellman equation under minimal assumptions on the parameters,
including verification of optimality results, using only analytical arguments.
We also examine the regularity of invariant measures. Then, we address the jump
diffusion model, and obtain a complete characterization of optimality.

We study infinitehorizon asymptotic average optimality for parallel server
network with multiple classes of jobs and multiple server pools in the
HalfinWhitt regime. Three control formulations are considered: 1) minimizing
the queueing and idleness cost, 2) minimizing the queueing cost under a
constraints on idleness at each server pool, and 3) fairly allocating the idle
servers among different server pools. For the third problem, we consider a
class of boundedqueue, boundedstate (BQBS) stable networks, in which any
moment of the state is bounded by that of the queue only (for both the limiting
diffusion and diffusionscaled state processes). We show that the optimal
values for the diffusionscaled state processes converge to the corresponding
values of the ergodic control problems for the limiting diffusion. We present a
family of statedependent Markov balanced saturation policies (BSPs) that
stabilize the controlled diffusionscaled state processes. It is shown that
under these policies, the diffusionscaled state process is exponentially
ergodic, provided that at least one class of jobs has a positive abandonment
rate. We also establish useful moment bounds, and study the ergodic properties
of the diffusionscaled state processes, which play a crucial role in proving
the asymptotic optimality.

We study the infinite horizon optimal control problem for Nnetwork queueing
systems, which consist of two customer classes and two server pools, under
average (ergodic) criteria in the HalfinWhitt regime. We consider three
control objectives: 1) minimizing the queueing (and idleness) cost, 2)
minimizing the queueing cost while imposing a constraint on idleness at each
server pool, and 3) minimizing the queueing cost while requiring fairness on
idleness. The running costs can be any nonnegative convex functions having at
most polynomial growth.
For all three problems we establish asymptotic optimality, namely, the
convergence of the value functions of the diffusionscaled state process to the
corresponding values of the controlled diffusion limit. We also present a
simple statedependent priority scheduling policy under which the
diffusionscaled state process is geometrically ergodic in the HalfinWhitt
regime, and some results on convergence of mean empirical measures which
facilitate the proofs.

Four different fluid model formulations have been recently developed for
$G_t/GI/N+GI$ queues, including a twoparameter fluid model in Whitt (2006) by
tracking elapsed service and patience times of each customer, a measurevalued
fluid model in Kang and Ramanan (2010) and its extension in Zu{\~n}iga (2014)
by tracking elapsed service and patience times of each customer, and a
measurevalued fluid model in Zhang (2013) by tracking residual service and
patience times of each customer. We show that the two fluid models tracking
elapsed times (Whitt's and Kang and Ramanan's fluid models) are equivalent
formulations for the same $G_t/GI/N+GI$ queue, whereas Zu{\~n}iga's fluid model
and Zhang's fluid model are not entirely equivalent under general initial
conditions. We then identify necessary and sufficient conditions under which
Zu{\~n}iga's fluid model and Zhang's fluid model can be derived from each other
for the same system, in which certain measurevalued fluid processes tracking
residual service and patience times of each customer derived from KangRamanan
and Zu{\~n}iga's fluid models play an important role.The equivalence properties
discovered provide important implications for the understanding of the recent
development for nonMarkovian manyserver queues.

We study a dynamic scheduling problem for a multiclass queueing network with
a large pool of statistically identical servers. The arrival processes are
Poisson, and service times and patience times are assumed to be exponentially
distributed and class dependent. The optimization criterion is the expected
long time average (ergodic) of a general (nonlinear) running cost function of
the queue lengths. We consider this control problem in the HalfinWhitt (QED)
regime, that is, the number of servers $n$ and the total offered load
$\mathbf{r}$ scale like $n\approx\mathbf{r}+\hat{\rho}\sqrt{\mathbf{r}}$ for
some constant $\hat{\rho}$. This problem was proposed in [Ann. Appl. Probab. 14
(2004) 10841134, Section 5.2]. The optimal solution of this control problem
can be approximated by that of the corresponding ergodic diffusion control
problem in the limit. We introduce a broad class of ergodic control problems
for controlled diffusions, which includes a large class of queueing models in
the diffusion approximation, and establish a complete characterization of
optimality via the study of the associated HJB equation. We also prove the
asymptotic convergence of the values for the multiclass queueing control
problem to the value of the associated ergodic diffusion control problem. The
proof relies on an approximation method by spatial truncation for the ergodic
control of diffusion processes, where the Markov policies follow a fixed
priority policy outside a fixed compact set.

We consider a service system where agents are invited ondemand. Customers
arrive exogenously as a Poisson process and join a customer queue upon arrival
if no agent is available. Agents decide to accept or decline invitations after
some exponentially distributed random time, and join an agent queue upon
invitation acceptance if no customer is waiting. A customer and an agent are
matched in the order of customer arrival and agent invitation acceptance under
the nonidling condition, and will leave the system simultaneously once matched
(service times are irrelevant here).
We consider a feedbackbased adaptive agent invitation scheme, which controls
the number of pending agent invitations, depending on the customer and/or agent
queue lengths and their changes. The system process has two components  `the
difference between agent and customer queues' and `the number of pending
invitations', and is a countable continuoustime Markov chain.
For the case when the customer arrival rate is constant, we establish fluid
and diffusion limits, in the asymptotic regime where the customer arrival rate
goes to infinity, while the agent response rate is fixed. We prove the process
stability and fluidscale limit interchange, which in particular imply that
that both customer and agent waiting times in steadystate vanish in the
asymptotic limit. To do this we develop a novel (multiscale) Lyapunov drift
argument; it is required because the process has nontrivial behavior on the
state space boundary. When the customer arrival rate is timevarying, we
present a fluid limit for the processes in the same asymptotic regime.
Simulation experiments are conducted to show good performance of the invitation
scheme and accuracy of fluid limit approximations.

We consider Markovian multiclass multipool networks with heterogeneous
server pools, each consisting of many statistically identical parallel servers,
where the bipartite graph of customer classes and server pools forms a tree.
Customers form their own queue and are served in the firstcome firstserved
discipline, and can abandon while waiting in queue. Service rates are both
class and pool dependent. The objective is to study the limiting diffusion
control problems under the long run average (ergodic) cost criteria in the
HalfinWhitt regime. Two formulations of ergodic diffusion control problems
are considered: (i) both queueing and idleness costs are minimized, and (ii)
only the queueing cost is minimized while a constraint is imposed upon the
idleness of all server pools. We develop a recursive leaf elimination algorithm
that enables us to obtain an explicit representation of the drift for the
controlled diffusions. Consequently, we show that for the limiting controlled
diffusions, there always exists a stationary Markov control under which the
diffusion process is geometrically ergodic. The framework developed in our
earlier work is extended to address a broad class of ergodic diffusion control
problems with constraints. We show that that the unconstrained and constrained
problems are well posed, and we characterize the optimal stationary Markov
controls via HJB equations.

In this paper, we study the longtime behavior of a fluid particle immersed
in a turbulent fluid driven by a diffusion with jumps, that is, a Feller
process associated with a nonlocal operator. We derive the law of large
numbers and central limit theorem for the evolution process of the tracked
fluid particle in the cases when the driving process: (i) has periodic
coefficients, (ii) is ergodic or (iii) is a class of L\'evy processes. The
presented results generalize the classical and wellknown results for fluid
flows driven by elliptic diffusion processes.

We describe a distributed framework for resources management in peertopeer
networks leading to goldenrule reciprocity, a kind of oneversusrest
titfortat, so that the delays experienced by any given peer's messages in the
rest of the network are proportional to those experienced by others' messages
at that peer.

In order to obtain Markov heavytraffic approximations for infiniteserver
queues with general nonexponential servicetime distributions and general
arrival processes, possibly with timevarying arrival rates, we establish
heavytraffic limits for twoparameter stochastic processes. We consider the
random variables $Q^e(t,y)$ and $Q^r(t,y)$ representing the number of customers
in the system at time $t$ that have elapsed service times less than or equal to
time $y$, or residual service times strictly greater than $y$. We also consider
$W^r(t,y)$ representing the total amount of work in service time remaining to
be done at time $t+y$ for customers in the system at time $t$. The
twoparameter stochasticprocess limits in the space $D([0,\infty),D)$ of
$D$valued functions in $D$ draw on, and extend, previous heavytraffic limits
by Glynn and Whitt (1991), where the case of discrete servicetime
distributions was treated, and Krichagina and Puhalskii (1997), where it was
shown that the variability of service times is captured by the Kiefer process
with second argument set equal to the servicetime c.d.f.

We establish continuity of the integral representation
$y(t)=x(t)+\int_0^th(y(s)) ds$, $t\ge0$, mapping a function $x$ into a function
$y$ when the underlying function space $D$ is endowed with the Skorohod $M_1$
topology. We apply this integral representation with the continuous mapping
theorem to establish heavytraffic stochasticprocess limits for manyserver
queueing models when the limit process has jumps unmatched in the converging
processes as can occur with bursty arrival processes or service interruptions.
The proof of $M_1$continuity is based on a new characterization of the $M_1$
convergence, in which the time portions of the parametric representations are
absolutely continuous with respect to Lebesgue measure, and the derivatives are
uniformly bounded and converge in $L_1$.

This is an expository review paper illustrating the ``martingale method'' for
proving manyserver heavytraffic stochasticprocess limits for queueing
models, supporting diffusionprocess approximations. Careful treatment is given
to an elementary model  the classical infiniteserver model $M/M/\infty$, but
models with finitely many servers and customer abandonment are also treated.
The Markovian stochastic process representing the number of customers in the
system is constructed in terms of rate1 Poisson processes in two ways: (i)
through random time changes and (ii) through random thinnings. Associated
martingale representations are obtained for these constructions by applying,
respectively: (i) optional stopping theorems where the random time changes are
the stopping times and (ii) the integration theorem associated with random
thinning of a counting process. Convergence to the diffusion process limit for
the appropriate sequence of scaled queueing processes is obtained by applying
the continuous mapping theorem. A key FCLT and a key FWLLN in this framework
are established both with and without applying martingales.