Stability of preemptive EDF queueing networks

We show stability of preemptive, strictly subcritical EDF networks with Markovian routing. To this end, we prove that the associated fluid limits satisfy the first-in-system, first-out (FISFO) fluid model equations and thus, by an extension of a result of Bramson (2001), the corresponding fluid models are stable. We also demonstrate that in a preemptive multiclass EDF network, after a time large enough to process all the initial customers to completion, the maximal number of partially served customers in the system over a finite time horizon converges to zero in L under fluid scaling.


Introduction
A fundamental question in the theory of multiclass queueing networks is whether a given system is stable, i.e., the corresponding Markov process is positive Harris recurrent. The intuitive meaning of network stability is that the system performs well under reasonable workload: the queue lengths do not grow linearly with time and do not oscillate "wildly", there is no mutual blocking and forced idleness of the servers when work is present in the system. Apparently, there is no general criterion for this behavior; in particular the usual necessary traffic condition that ρ j < 1 at each station, called strict subcriticality of the underlying queueing system, is not sufficient, see, e.g., [15]. On the positive side, the condition ρ j < 1 for all j is sufficient for generalized Jackson networks [14] and multiclass networks with some disciplines, including first-in-first-out (FIFO) in networks of Kelly type [2], head-of-the-line proportional processor sharing [3], first-buffer-first-served and last-buffer-first-served [6,7].
Dai [6], generalizing and systematizing earlier work of Rybko and Stolyar [15], provided a general framework for proving such stability results. Its main idea is to reduce the problem to showing stability of the corresponding fluid model, a deterministic analog of the network under consideration. This approach has been applied to various queueing systems. The result most relevant to this paper is stability of multiclass earliest-deadline-first (EDF) networks with soft (i.e., permitting lateness) customer deadlines and no preemption. The EDF discipline, also called earliest-due-date-first-served (EDDFS), is the rule where each customer has a deadline, assigned upon arrival at the network and maintained until departure, and a customer with the earliest deadline is selected for service at each station of the network. Bramson [5] showed that the fluid limits of the performance processes for a non-preemptive, strictly subcritical EDF network satisfy the first-in-systemfirst-out (FISFO) fluid model equations. He then proved that a sufficiently rich class of FISFO fluid models is stable. This, by a variation of Theorem 4.2 of Dai [6], implies stability of the network under consideration.
It is natural to ask whether this stability result remains valid for preemptive EDF networks with soft deadlines. As it is observed in Bramson [5], this problem is more difficult and the analysis for the non-preemptive case does not generalize immediately to the preemptive setting. The main reason for this is that the number of partially served customers in a preemptive EDF system is unbounded, so it is not clear that the number of departed customers from a given class is asymptotically proportional to the service time devoted by the server to this class. Kruk [11] showed how to overcome this difficulty under the assumption that the customer routes in the network are fixed. The main idea of the stability proof from [11] is that since the initial lead time distributions disappear in the limit, the asymptotic behavior of a preemptive EDF system does not differ from the behavior of the corresponding FISFO system. More precisely, after a time large enough to process all the initial customers to completion at every station, the fluid limits for a preemptive EDF system satisfy the FISFO fluid model equations introduced in Bramson [5]. This is because under fluid scaling, the number of customers coming to the system in a small time interval is small, so the corresponding fluid limits are continuous. Also, since the order of service does not differ significantly from FISFO, the number of partially served customers at each station and the work associated with them are negligible in the limit. The latter finding is analogous to "crushing lemmas" from the papers on diffusion limits for EDF systems, see [9,13,17]. Once convergence to a FISFO fluid model is established, stability of the latter models proved in Bramson [5] and an argument similar to the proof of Theorem 4.2 in Dai [6] imply stability of preemptive EDF systems.
To our knowledge, the theorems presented in [11] were the first stability results for multiclass queueing networks with unbounded numbers of partially served customers and constituted the first application of the methodology of Dai [6] to such systems. Unfortunately, the arguments presented in [11] rely heavily on the assumption of fixed customer routes. Moreover, this assumption is not satisfied by a number of systems considered in the literature, for example by generalized Jackson networks.
The aim of this paper is to prove stability of general open, strictly subcritical preemptive multiclass EDF networks with soft deadlines and Markovian routing. The main idea of our argument is to divide customers into various types, according to the paths followed by them in the system. These types (paths) are counterparts of customer classes with fixed customer routes, considered in [11]. However, doing this does not immediately reduce the problem to the framework of [11], because, in general, the set of possible customer paths in the network is infinite. We deal with this problem by dividing this set of paths into two groups: one finite, but traversed with high probability, the other one infinite (cofinite), but very unlikely. Then, loosely speaking, we apply the methods developed in [11] (recalled above) to the first group and we show that the other one does not significantly alter the overall system performance. We hope that this proof strategy will turn out to be useful also for other queueing systems with infinite numbers of job types, in particular those, for which an initial assumption of fixed customer routes significantly simplifies the underlying analysis.
Along the way, we generalize Bramson's stability result from [5] for strictly subcritical, initially aging EDF fluid models satisfying an additional technical condition (see (5.1), to follow) to general (not necessarily initially aging) strictly subcritical EDF fluid models. It is noteworthy that Bramson conjectured the validity of such a generalization, see [5], pp. 88-89.
Our third contribution is to show that in a general preemptive multiclass EDF network, after a time large enough to process all the initial customers to completion, the maximal number of partially served customers in the system over a finite time horizon converges to zero in L 1 under fluid scaling. Although this fact is related to convergence of the fluid-scaled sample paths of the network performance processes to the FISFO fluid model solutions, it seems that none of these facts can be readily deduced from the other.
Together with [5] and [11], the results of this paper characterize asymptotic behavior of multiclass EDF networks with soft deadlines in the strictly subctricital case. It would be desirable to extend the analysis to the corresponding subcritical (in particular, critical) systems. Bramson ([5], p. 81) and Williams (private communication) posed a question whether the modular approach introduced by Bramson [4] and Williams [16] can be applied, at least in some situations, to subcritical EDF networks. The first step in this direction was made by Kruk [12], where the invariant manifold for the corresponding fluid models was characterized. Our paper also contributes to this project, because the issue of convergence of the fluid-scaled sample paths of the network performance processes to the FISFO fluid model solutions in the general preemptive EDF case is addressed here. (As in [5] and [11], this part of the analysis does not require the strict subcriticality assumption).
In spite of theoretical and practical importance of stochastic multiclass EDF queueing networks, there are still few mathematically rigorous results for such systems. Apart from the work recalled above, Yeung and Lehoczky [17] provided a diffusion approximation for measure-valued state descriptors of preemptive EDF feedforward networks. Their result has been generalized to the case of acyclic networks, with or without preemption, by Kruk, Lehoczky, Shreve, and Yeung [13]. However, the latter result rests on a strong assumption implying the existence of a heavy traffic limit for the corresponding real-valued workload process. Currently, we are able to verify this assumption only in a number of special cases. This amplifies the need for further research in this area.
The paper is organized as follows. Section 2 describes the model, provides background information on positive Harris recurrence of Markov processes and adjusts it to our setting. It also contains a formulation of Theorem 2.1, our main stability result. In Section 3, we present preemptive EDF queueing network equations, the corresponding FISFO fluid model equations and Theorem 3.1, assuring stability of an arbitrary strictly subcritical EDF fluid model. In Section 4 we formulate two important facts: Theorem 4.4, stating that the fluid limits of (properly shifted) performance processes describing a preemptive EDF network satisfy the FISFO fluid model equations, and Theorem 4.6, according to which the maximal number of partially served customers in the system over a finite time horizon converges to zero in L 1 under fluid scaling. We also derive Theorem 2.1 from Theorem 3.1 and the results presented in this section. Section 5 contains the proof of Theorem 3.1. In Section 6 we provide an auxiliary lemma, necessary for the last two sections. The proof of Theorem 4.4 is contained in Section 7. Finally, Section 8 contains proofs of Theorem 4.6 and an auxiliary state space collapse result from Section 4.

Terminology, background and the main result
2.1. Notation. The following notation will be used throughout the paper. Let N = {0, 1, 2, . . .}, let Q, R denote the set of rational and real numbers, respectively. Let R + = [0, ∞), and let R 2 + = (R + ) 2 be the nonnegative orthant. For a, b ∈ R, we write a ∨ b for the maximum of a and b, a ∧ b for the minimum of a and b and a + for a ∨ 0, respectively. For a vector a = (a 1 , . . . , a n ) ∈ R n , let |a| n i=1 |a i |. All vectors in the paper are to be interpreted as column vectors. For a matrix A, A denotes the transpose of A. For a finite set B, |B| denotes the cardinality of B. The Borel σ-field on a topological space Y will be denoted by B(Y ).

EDF networks.
We consider a network consisting of J single server stations, indexed by j = 1, . . . , J. The network is populated by K customer classes (or buffers), indexed by k = 1, . . . , K. There is a stationary external arrival process with rate α k associated with each class k. In particular, if α k = 0, there are no external arrivals to class k. We put α = (α 1 , . . . , α K ) and E = {k ∈ {1, . . . , K} : α k > 0}. A customer of class k receives service at a unique station j, written k ∈ C(j) or j = s(k). Let m k be the mean service time for the class k and let m = (m 1 , . . . , m K ). Upon being served at j, a customer of class k immediately becomes a customer of class l with probability p kl , independently of the network's past history. Thus, the probability that a customer of class k leaves the network after completion of service equals 1 − K l=1 p kl . The routing matrix P = (p kl ) is assumed to be transient, i.e., such that the matrix Θ = (q kl ) (I − P ) −1 = I + P + (P ) 2 + . . . exists. We define the total arrival rate vector λ = (λ 1 , . . . , λ K ) = Θα. Without loss of generality we assume that λ k > 0 for each k. Next, we define the traffic intensity at station j as When ρ j < 1 for each j, the network is called strictly subcritical.

Stochastic primitives.
We will now define the stochastic primitives for the model described in Section 2.2.1. The customer interarrival times are a sequence of strictly positive, i.i.d. random variables u k (i), i = 1, 2, . . . , where the subscript k ∈ E indicates the customer class. We assume that for k ∈ E, and for some n k > 0 and some nonnegative Borel function f k with In other words, the interarrival times are integrable, unbounded, and spread out. The residual interarrival times u k (0), k = 1, . . . , K, are assigned fixed nonnegative values. The arrival time of the n-th customer of class k to the system is given by U k (n) = n−1 i=0 u k (i), n = 1, 2, . . . . The service times of class k customers are a sequence of strictly positive, independent, and identically distributed random variables v k (i), i = 1, 2, . . . , where the index i denotes the order of arrival of customers to the buffer. We assume that for all k, The arrival rates α k , k ∈ {1, . . . , K}, are defined by α k 1/Eu k (1) if k ∈ E and α k 0 otherwise. Customers entering the network through the buffer k ∈ E at times U k (i) have initial lead times k (i), i = 1, 2, . . . , which are mutually independent nonnegative i.i.d. random variables. The deadline of such a customer is given by ∆ k (i) = U k (i) + k (i). We assume that for k ∈ E, We assume that the sequences u k (i) , k = 1, . . . , K, are mutually independent. We also assume that the sequences , k = 1, . . . , K, are mutually independent. For each k = 1, . . . , K, the initial condition specifies Q k (0), the number of initial customers present at the buffer k at time 0, as well as their residual service times and initial lead times, which are denoted byṽ k (i) and˜ k (i), i = 1, . . . , Q k (0), respectively. We assume that Q k (0) are fixed nonnegative integers,ṽ k (i) are fixed positive numbers and˜ k (i) are fixed real numbers. The deadlines of the initial customers are given by∆ k (i) =˜ k (i).

Lead times, service discipline.
To determine whether customers meet their timing requirements, one must keep track of each customer's lead time, where lead time = deadline -current time.
Customers are served at each station according to the preemptive EDF discipline. That is, the customer with the shortest remaining lead time, regardless of class, is selected for service at each station. Preemption occurs when a customer more urgent than the customer in service arrives (we assume preempt-resume). There is no set up, switch-over or other type of overhead. We assume that the customers are patient: they stay in the system until served to completion, even if they get late, i.e., their lead times become negative. The (natural) assumption that k (i) ≥ 0 was added only to simplify the exposition of the proofs. All our results are valid without this condition as long as k (i) are integrable.

Markov process background.
In EDF queueing systems, the individual customer lead times or some equivalent information must be kept to determine customer priorities. Since the number of customers present in the system at a given time is unbounded, it is necessary to model its evolution in an infinitely dimensional state space. In what follows, we use lists of infinite length to construct the state descriptor. An alternative approach utilizing finite Borel measures can be found, e.g., in [9,13,17]. Let d = |E| and let S = (R + × R) ∞ . Let be the state space. Under the product topology, Ω is a locally compact Polish space. The state of the process at any time is given by a point where for k = 1, . . . , K, q k is the queue length at buffer k, h k describes all customers present at buffer k so that each of them is listed in terms of his residual service time and lead time, and r k is the residual interarrival time for class k ∈ E. We assume that the customers in h k are listed in the order of their arrivals to the buffer and ties are broken in an arbitrary manner. Let 0 denote the element of Ω describing the empty system, i.e., with q k = 0, h k = ((0, 0), (0, 0), . . . ) and r k = 0 for all k. Let q = (q k ) k=1,...,K and w = (w k ) k=1,...,K , where w k is the sum of the residual service times of the customers listed in h k . Let r = (r k ) k∈E and let be the greatest lead time. For x ∈ Ω, let |x| = |q| + |w| + |r| + + be the "norm" of x.
The process describing the evolution of the EDF system is denoted by is the state of the system at time t. By definition, the process X has rightcontinuous sample paths. It is easy to see that X is a Markov process. The evolution of the process X between arrivals and departures is deterministic. Thus, X is a piecewise-deterministic Markov (PDM) process, so it is actually strong Markov (see [8]).
A Markov process X on the state space Ω is Harris recurrent if there exists a σ-finite measure ν on B(Ω) such that whenever It is known that Harris recurrence implies the existence of a unique (up to a multiplicative constant) invariant measure, see e.g., [10]. If this measure is finite, X is called positive Harris recurrent.

Main result.
Recall that a queueing network is stable when the underlying Markov process is positive Harris recurrent. The following theorem is the main result of this paper. Theorem 2.1. All strictly subcritical EDF queueing networks with preemption which satisfy (2.2)-(2.6) are stable.

Preemptive EDF network equations and fluid models
Let E(t, s) = (E k (t, s)) k=1,...,K , t ≥ 0, s ∈ R, denote the external arrival process defined as follows. If k ∈ E, then E k (t, s) is equal to the number of external arrivals by time t of type k customers with deadlines at time t less than or equal to s, otherwise E k (t, s) ≡ 0. For k = 1, . . . , K, t ≥ 0 and s ∈ R, let Z k (t, s) denote the number of class k customers who are visiting station j = s(k) at time t with deadlines at time t less than or equal to s. Let Z(t, s) = (Z k (t, s)) k=1,...,K . Similarly, the vectors A(t, s) = (A k (t, s)) k=1,...,K , D(t, s) = (D k (t, s)) k=1,...,K , T (t, s) = (T k (t, s)) k=1,...,K denote the number of arrivals and departures, and the cumulative service time by time t corresponding to each class k of customers with deadlines at time t less than or equal to s. Let Y j (t, s), j = 1, . . . , J, denote the cumulative idleness by time t at station j with regard to service of customers with deadlines at time t less than or equal to s and let Y (t, s) = (Y j (t, s)) j=1,...,J . For k = 1, . . . , K, t, t ≥ 0 and s ∈ R, let S k (t , t, s) denote the number of service completions of class k customers having deadlines at time t less than or equal to s, by the time the station j = s(k) has spent t units of time serving these customers. Finally, for k = 1, . . . , K, n ∈ N, t ≥ 0 and s ∈ R let the routing vector Φ k (n, t, s) = (Φ k,1 (n, t, s), . . . , Φ k,K (n, t, s)) be the number of the first n departures from class (buffer) k with deadlines at time t less than or equal to s that are routed to each class.
For t ≥ 0 and s ∈ R, let X(t, s) = (A(t, s), D(t, s), T (t, s), Y (t, s), Z(t, s)). Note that Q(t) = (Q k (t)) k=1,...,K = lim s→∞ Z(t, s) is the queue length vector. Let W (t) = (W k (t)) k=1,...,K denote the unfinished work in the system, i.e., W k (t) is the sum of the residual service times of customers in buffer k at time t. We will sometimes use superscript x ∈ Ω such as in X x (t, s) to indicate that the process starts at state x. For c > 0, cX(t, s) denotes componentwise multiplication.
The process X(t, s) satisfies the following network equations (compare [5]): valid for every for t ≥ 0 and s ∈ R. The equation (3.5 The equations (3.1)-(3.4) are general properties of queueing networks and they do not depend on the service discipline under consideration. The equation (3.5) is specific to preemptive EDF networks. Indeed, for any s, the server idleness with regard to customers with deadlines not greater than s cannot increase at time t in the presence of such customers if and only if the server is working under the preemptive EDF protocol.
It turns out that the deterministic analogs of the equations (3.1)-(3.5) are the FISFO fluid model equations (see [5]): where t, s ≥ 0. In analogy with the processes A, D, T , Y , Z, we assume that A(·, s), D(·, s), T (·, s), Y (·, s) are nondecreasing in each coordinate, Following [5], we additionally assume that We also define Q(t) = lim s→∞ Z(t, s) = Z(t, t), where the last equation follows from (3.11). As in the case of queueing networks, we say that a fluid model is strictly subcritical if ρ j < 1 for each j, where ρ j is defined by (2.1). We also say that a FISFO fluid model is stable if there exists c > 0 such that for all solutions of the equations (3.6)-(3.10), Q(t) = 0 for t ≥ c|Q(0)|.
The following result extends Theorem 2 of Bramson [5] to arbitrary strictly subcritical FISFO fluid models.

Fluid limits and network stability
Let k ∈ E, t ≥ 0 and let x ∈ Ω be the initial state of the network. Let By (2.2), (2.5), (2.6) and the strong law of large numbers, P(G) = 1.
We consider sequences of points x n = (q n , h n , r n ), q n ∈ N K , h n ∈ S K , r n ∈ R d + , such that Lemma 4.1 (Lemma 4.1 [11]). Let T 0 > 0. Let a sequence x n satisfy (4.4) and let Then lim n→∞ L n (ω)/|x n | = 0 for every ω ∈ G.
The proof of this lemma is the same as the proof of Lemma 5.1 in [11].
Let γ k be the expected number of visits to all buffers in the network by a customer entering the network at the class k ∈ E and let γ = max k∈E γ k .
For every sequence x n in (4.4), there exist a set G 1 ⊆ G with P(G 1 ) = 1 and a subsequence x η such that for ω ∈ G 1 and η sufficiently large (depending on ω), whereV xη is the departure time of the last initial customer from the network with initial state x η .
Proof. Let a sequence x n satisfy (4.4). In a preemptive EDF network with the initial state x n , the initial customers, together with customers arriving at the network after time zero with deadlines not greater than + n , form a priority class, i.e., as long as these customers are present at any station of the network, all the service capacity of this station is devoted to them. Since the initial lead times of the arriving customers are nonnegative, this priority class has at most |q n | + |N xn ( + n )| members. Using (4.5), we see that on the set G the number of these priority customers is bounded above by . Let I xn k be the set of indices n ∈ N corresponding to the class k service times v k (i) of the priority customers in the network with initial state x n . Proceeding as in the proof of (A.7) of [5], we can show that there exist an integer-valued random variable N and a setG ⊆ G with P(G) = 1 such that for ω ∈G and n ≥ N (ω) we have (4.10) |I xn k (ω)| ≤ 2γ(1 + |α|)|x n | + |x n |/|m|. Under the EDF service discipline, the index i of the arrival of a customer of class k is independent of v k (i). Thus, by (2.5), (4.4) and the weak law of large numbers, for k = 1, . . . , K we have By Theorem 20.5 in [1], there exist a set G 1 ⊆G with P(G 1 ) = 1 and a subsequence η such that for every ω ∈ G 1 and k = 1, . . . , K, we have Therefore, the sum of the service times of the priority customers in the network with initial state x η is bounded above on the set G 1 by where the second inequality follows from (4.10). This, together with (4.8), implies that for every ω ∈ G 1 there exists η 0 = η 0 (ω) such that Note that because all the priority customers arrive at the preemptive EDF system with initial state x η by time + η , V xη + + η is the upper bound for the time by which all the priority customers leave this system. Indeed, as long as the priority customers are present at the network, at least one server works on these customers. Consequently, by (4.11), for ω ∈ G 1 and η ≥ η 0 (ω), (4.9) holds.
For t 0 ≥ 0, we introduce the time shift operator ∆ t 0 acting on the coordinates of the process X as follows: for t ≥ 0, s ∈ R, we have for t ≥ 0. Intuitively, the processes ∆ t 0 X, ∆ t 0 Q describe the dynamics of the queueing system under consideration "restarted" at time t 0 .
The following theorem plays a crucial role in the proof of Theorem 2.1. Its intuitive meaning is that, after a time large enough to process all the initial customers to completion at every station, the fluid limits for a preemptive EDF system satisfy the FISFO fluid model equations.  The proof of Theorem 4.4 will be given in Section 7.
To show Theorem 2.1, we need the following proposition, which will be proved in Section 8.
Using the above results, we can prove Theorem 2.1 by repeating, with minor changes, the proof of Theorem 3.1 in [11]. In particular, our Theorem 4.4 and Proposition 4.5 should be quoted instead of Propositions 5.2, 6.1 of [11], respectively, and the process N 0 k from the proof of Theorem 3.1 in [11] should be replaced by the process N 0 0,k defined by (6.1), to follow. For t ≥ 0, let P(t) denote the number of partially served customers in the system at time t, i.e., those who have received some service in the time interval [0, t], but they have not been fully served by time t. Theorem 4.6. Let C be given by (4.8) and let x n be a sequence satisfying (4.4). Then for every T 0 > C, we have (4.14) lim Moreover, if = 0 in (4.4), then the constant C in (4.14) can be replaced by 0.
The proof of this result will be given in Section 8. Theorem 4.6 is closely related to Proposition 4.5 and to the assertion in Theorem 4.4 that the fluid limits (4.12) satisfy (3.8). This relation stands behind similarity of the proofs of these facts. However, it seems that none of them can be immediately deduced from another. For example, a longer service time increases the probability of the customer being preempted, so if P(t) is relatively small, it does not directly imply that the time devoted by the server to class k customers is roughly proportional to their mean service time multiplied by the number of departed customers from this class. Conversely, the latter relation does not rule out the possibility of P(t) being nonnegligible, since there may be a lot of customers preempted just after entering into service, before the server spends a lot of time working on them.

Proof of Theorem 3.1
In this section we prove Theorem 3.1. We will first introduce some additional notation and terminology. We introduce the set of multi-indices K = (k 1 , . . . , k n ) : n ≥ 1, k 1 , . . . , k n ∈ {1, . . . , K}, α k 1 p k 1 k 2 . . . p k n−1 kn > 0 , where p k 1 k 2 . . . p k n−1 kn should be interpreted as 1 for n = 1. The elements of K represent paths of finite length which are being followed with positive probability by customers since their arrival to the network. For k = (k 1 , . . . , k n ) ∈ K, let |k| = n be the length of the path k, let p k = p k 1 k 2 . . . p k n−1 kn , α k = α k 1 p k , m k = m kn and let b(k) = k 1 and e(k) = k n be the beginning and the end of the path k, respectively. For k ∈ K and k ∈ {1, . . . , K}, we write k ∈C(k) if e(k) = k. We will refer to customers following the path k ∈ K as type k customers.

Definition 5.1 ([5], p. 88). A queueing network (resp. its fluid model) is
initially aging if the set {1, . . . , K} of its customer classes can be divided into two disjoint subsets K 1 and K 2 such that (a) the classes from K 1 are not accessible from classes in K 2 , (b) for any given k ∈ K 1 , all k ∈ K with e(k) = k have the same length and this common length is not greater than |K 1 |.
The following result will be our starting point.

Theorem 5.2 (Theorem 2 [5]). Assume that an initially aging FISFO fluid model is strictly subcritical and satisfies
Then, it is stable.
Remark. The set of customer classes and the routing structure for the fluid modelX defined above are the same as for a network N used in [5], pp. 87-88, to reduce the question of stability of arbitrary strictly subcritical EDF queueing networks without preemption to stability of those of them which are initially aging and satisfy (5.1). Our approach, although clearly related to the one in [5], is different, since we use a similar method to obtain stability of fluid models rather than queueing networks. There are, of course, other possible extensions of the set of customer classes of X and its routing structure which can be used for a construction ofX. One such choice is K 1 = {k ∈ K : |k| < H} and K 2 = {k ∈ K : |k| = H} with nonzero transition probabilities p (k 1 ,...,k h ),(k 1 ,...,k h+1 ) = p k h ,k h+1 for h < H and p (k 1 ,...,k H ),(k 1 ,...,k H−1 ,l) = p k H ,l .

An auxiliary lemma
This section contains a technical Lemma 6.1, necessary for the proofs of Theorem 4.4 and 4.6. The following additional notation and terminology will be used in the sequel.
Recall the set of multi-indices K defined in Section 5. For m ∈ N and k, l = 1, . . . , K, let K m = {k ∈ K : |k| > m}, K m,l = K m ∩C(l) and let K m,k,l = {k ∈ K m,l : b(k) = k}. For x ∈ Ω, t ≥ 0 and k ∈ K, let N x k (t) be the number of type k customers who have arrived at the network with initial state x by time t. Also, for m ∈ N and k, l = 1, . . . , K, let α m,k,l = k∈K m,k,l α k = α k P m+1 + P m+2 + . . . k,l = (P ) m+1 + (P ) m+2 + . . . l,k α k = (P ) m+1 Θ l,k α k , Note that N x m,l (t) is, in general, not equal to the number of customers arriving at the network up to time t which will eventually visit class l by a path longer than m. Indeed, every such arriving customer increases the count in N x m,l (t) by the number of his visits to class l in more than m steps before his departure.
In analogy with the performance processes D k , T k , defined for k = 1, . . . , K, for k ∈ K, t ≥ 0 and s ∈ R, let D k (t, s) denote the number of departures from class k = e(k) of customers with deadlines at time t less than or equal to s which have arrived at class k following the path k, and let T k (t, s) denote the cumulative service time at station s(e(k)) by time t corresponding to such customers. Similarly, in analogy with the performance processes Q k , W k , defined for k = 1, . . . , K, for k = (k 1 , . . . , k n ) ∈ K and t ≥ 0, let Q k (t) denote the number of customers present in class k n at time t who have arrived at this class following the path k and let W k (t) denote the workload (i.e., the sum of the residual service times) for station s(k n ) corresponding to these customers. By service times of type k customers we shall mean the service times of class k = e(k) customers who have arrived at class k following the path k.
For k = 1, . . . , K and t 1 , t 2 ∈ R, let B n k (t 1 , t 2 ) denote the set of j = 1, 2, . . . , for which the customer corresponding to the service time v k (j) has entered the network with initial state x n in the time interval (t 1 |x n |, t 2 |x n |]∩ (0, ∞). In particular, B n k (t 1 , Similarly, for k = (k 1 , . . . , k l ) ∈ K and t 1 , t 2 ≥ 0, let B n k (t 1 , t 2 ) denote the set of j = 1, 2, . . . , corresponding to the class k l service times v k l (j) of those type k customers on the l-th step of their routes who have entered the network with initial state x n in the time interval (t 1 |x n |, t 2 |x n |]. Finally, for k = 1, . . . , K, m ∈ N and t ≥ 0, let B n m,k (t) = k∈K m,k B n k (0, t).
Lemma 6.1. Let a sequence x n satisfy (4.4) and let the set G 1 and the subsequence x η be as in Lemma 4.3. There exists a set G ⊆ G 1 with P(G ) = 1 and a subsequence x ξ of x η such that on G for every k ∈ K, m ∈ N and l = 1, . . . , K, we have u.o.c. in t as n → ∞ and for every r 0 > 0, k = 1, . . . , K and k ∈ K, as ξ → ∞, Moreover, on G for every k, k = 1, . . . , K and every t, s of the form t = C + t , s = C + s , where C was defined by (4.8), t ≥ 0, t , s ∈ Q, we have, as ξ → ∞, Proof. Proceeding as in the proof of (4.5) we can show that for every k ∈ K there exists a set G k with P(G k ) = 1 on which (6.2) holds u.o.c. in t. A similar reasoning, using strong laws of large numbers for partial sums and arrival processes, shows that for the sequence x n , m ∈ N and k, l = 1, . . . , K, there exists a set G m,k,l with P(G m,k,l ) = 1 on which u.o.c. in t, Put G m,l = K k=1 G m,k,l . Then P(G m,l ) = 1 and (6.3) holds on G m,l u.o.c. in t.
By the weak law of large numbers, together with the independence of the service times on the interarrival times, the initial lead times and the routing, However, (6.3) implies that on the set G 0,k , for , and hence (6.8) yields Using (6.9) and arguing as in the proof of (A.1) in [5] or in the proof of Proposition 3.4 in [9], we get, for every r 0 > 0, By Theorem 20.5 in [1], there exist a set G 2 with P(G 2 ) = 1 and a subsequence ξ of the sequence η such that on G 2 , we have pointwise convergence (6.4) for every r 0 > 0 and k = 1, . . . , K.
Arguing as in the proof of (6.10), but using (6.2) instead of (6.3), we can check that for every r 0 > 0, As in (6.4), we want to refine (6.11) to almost sure convergence for each k ∈ K. To this end, we use the fact that K is countable and enumerate its elements, getting K = {k 1 , k 2 , . . . }. By Theorem 20.5 in [1], there exist a setG 1 with P(G 1 ) = 1 and a subsequence ξ 1 of ξ such that onG 1 , we have pointwise convergence for j = 1, k = k 1 and every r 0 > 0. Using Theorem 20.5 in [1] again, we get a setG 2 ⊂G 1 with P(G 2 ) = 1 and a subsequence ξ 2 of ξ 1 such that onG 2 , we have pointwise convergence (6.12) for all r 0 > 0 and j = 2, k = k 1 , k 2 . Proceeding in this way, for every n ≥ 2 we construct a subsequence ξ n of ξ n−1 and a setG n with P(G n ) = 1 such that onG n (6.12) holds for all r 0 > 0, j = n and k = k 1 , . . . , k n . Using the Cantor diagonal procedure, we extract a subsequence (still denoted by ξ for convenience) of each sequence ξ n along which (6.5) holds on the set G 3 = ∞ n=1G n for all r 0 > 0 and every k ∈ K.
An argument similar to the one presented above shows that there exist a further subsequence (still denoted by ξ) and a set G 4 with P(G 4 ) = 1 on which for every k = 1, . . . , K, m ∈ N and r 0 > 0 we have (6.6).
Proceeding as in the proof of (A7) [5], we can show that for each t ≥ 0, This, together with (6.13) implies that for each fixed t ≥ 0 and s ∈ R, P [|D x ξ (t|x ξ |, s|x ξ |)| ≤ (4γ|α|t + 1)|x ξ | for ξ large enough] = 1. This, in turn, together with the independence of the i.i.d. routing vectors on other network primitives and the weak law of large numbers, yields for each k, k ∈ {1, . . . , K} and fixed t ≥ 0, s ∈ R. Using this fact and proceeding as in the next to last paragraph, we can extract from the sequence ξ a further subsequence (still denoted by ξ) and a set G 5 with P(G 5 ) = 1 on which for every k, k = 1, . . . , K and every t, s of the form t = C + t , s = C + s , where C was defined by (4.8), t ≥ 0, t , s ∈ Q, (6.7) holds.

Proof of Theorem 4.4
Let a sequence x n satisfy (4.4) and let the set G 1 and the subsequence x η be as in Lemma 4.3. Let the set G and the subsequence x ξ be as in Lemma 6.1. Fix ω ∈ G . Consider an arbitrary subsequence ϑ of the sequence ξ. For t ≥ 0 and s ∈ R, let The coordinate mappings of X (ϑ) (ω) inherit the monotonicity properties from the corresponding coordinate mappings of X x ϑ (ω). Thus, by Helley's selection theorem (see, e.g., [1], Theorem 25.9 and the remark in the proof of Theorem 29.3), there exists a subsequence ζ and a right-continuous function X 0 (t, s) = (A(t, s), D(t, s), T (t, s), Y (t, s), Z(0, s)), t ≥ 0, s ∈ R, (both depending on ω) such that each coordinate map of X (ζ) 0 (ω) converges to the corresponding coordinate map of X 0 at every point of continuity of the latter function. Define Z(t, s) for t > 0, s ∈ R, by (3.7) and let (The introduction of the auxiliary processes X (ϑ) 0 , X 0 in the above argument is necessary, because Z(t, s) is not necessarily monotone in t.) Since the coordinates of X   j (t, s)) are distribution functions of nonnegative measures, the mapping X 0 inherits this property, and hence its coordinate functions have at most countably many discontinuities. Consequently, the set I(X) of discontinuities of X 0 (and hence of X) in R + × R is contained in a countable union of vertical and horizontal lines. Since X (ζ) (ω)(t, s) → X(t, s) for every (t, s) ∈ (R + × R) \ I(X) and X is right-continuous, the equation (3.4) implies that X satisfies (3.9). In particular, because T k (·, s), Y j (·, s) are nondecreasing, (3.9) implies that the functions T (t, s) and Y (t, s) are Lipschitz in t. We will show that they are also continuous in s. Let T 0 > C +1. Suppose that for some k ∈ {1, . . . , K}, 0 ≤ t < T 0 − C − 1 and s ∈ R, Let s 1 , s 2 be such that s 1 < s < s 2 , and the function T k is continuous at the points (t, s 1 ), (t, s 2 ). By (7.1) and the monotonicity of T (t, s) in s, for ζ large enough we have . In other words, the cumulative work done by time (t + C)|x ζ | by server j = s(k) on class k customers with deadlines at time (t + C)|x ζ | belonging to the interval ((s 1 + C)|x ζ |, (s 2 + C|x ζ |] is at least |x ζ |. It is easy to check that these customers arrived at the network in the time interval ((s 1 + C)|x ζ | − L ζ , (s 2 + C)|x ζ |]. By (6.4), we have This, by (7.2) and Lemma 4.1, yields a contradiction for sufficiently large ζ.
We have proved continuity of T (t, s) in s (the argument actually shows that T k (t, s) is Lipschitz in s with the Lipschitz constant m k α 0,k ). By (3.9), Y is Lipschitz in both variables, so (T (ζ) , Y (ζ) )(ω)(t, s) → (T , Y )(t, s) for any t, s ≥ 0. As in the proof of Lemma 4.2, it is easy to see that this convergence is u.o.c. in t, s.
We will now show that X satisfies (3.8) Let k = (k 1 , . . . , k n ) ∈ K, T 0 > 0 and let 0 ≤ t, s < T 0 − C − 1. The first step in the justification of (3.8) is to show which is a pre-limit analog of (3.8), but for a fixed path k rather than a customer class k. This follows by an argument similar to the proof of (5.8) in [11], with (4.7) and (8.3) there replaced by our (6.2) and (6.5), respectively.
For 0 ≤ t, s < T 0 − C − 1 and k = 1, . . . , K, we have k (t, s), k (t, s). By (7.5), On the other hand, by (6.3), (6.6) and (7.6), for ζ large enough we have The relations (7.7)-(7.11) imply that at every point (t, s) ∈ R 2 + of continuity of the function D, |T k (t, s) − m k D k (t, s)| ≤ . Since > 0 is arbitrary, we actually have (3.8) at any such point and hence, by right-continuity of both T and D, at any t, s ≥ 0. We have proved that X satisfies (3.8). In particular, by Lipschitz continuity of T , the function D is Lipschitz in both variables. Thus, We will now show that X satisfies (3.6). By (6.7), for every t, s ≥ 0, t, s ∈ Q and l = 1, . . . , K, we have where the third equation follows from (3.1) and the fourth one is a consequence of Lemma 4.2, together with (6.7). Letting ζ → ∞ in (7.12), we get for every t, s ≥ 0, t, s ∈ Q. However, the functions A (ζ) l ,Ã l , l = 1, . . . , K, are nondecreasing in both variables andÃ is continuous, so it is not hard to check that (7.13) actually holds for every t, s ≥ 0. In particular,Ã = A and the proof of (3.6) is complete. Consequently, the function A is Lipschitz in both variables and thus (A We will now prove (3.11). To this end, by (3.6)-(3.9), it suffices to show T (t, s) = T (t, t), (7.14) Z(0, s) = Z(0, t), (7.15) for 0 ≤ t < s. The proof of (7.14) is similar to the argument showing continuity of T (t, s) in s. Suppose that (7.14) is false, i.e., for some k ∈ {1, . . . , K} and 0 ≤ t < s < T 0 − C − 1, By (7.16) and the monotonicity of T (t, s) in s, for ζ large enough, we have . In other words, the cumulative work done by time (t + C)|x ζ | by server j = s(k) on class k customers with deadlines at time (t + C)|x ζ | belonging to the interval ((t + C)|x ζ |, (s + C)|x ζ |] is at least |x ζ |. It is easy to check that these customers arrived at the network in the time interval Arguing as in (7.4), with s 1 = s 2 = t, and using Lemma 4.1, we obtain a contradiction with (7.16) for sufficiently large ζ, which proves (7.14).

Proofs of Proposition 4.5 and Theorem 4.6
Proof of Proposition 4.5. By Lemma 4.3, for ζ sufficiently large, all initial customers have left the system with initial state x ζ by time C|x ζ |. Let T 0 > C + 1 and let 0 ≤ t ≤ T 0 − C − 1. Arguing as in the proof of Proposition 6.1 in [11], one may check that for every k ∈ K, Fix > 0 and choose m so large that α m,k m k T 0 < /2, k = 1, . . . , K. We have As in (7.10)-(7.11), for ζ large enough, By (8.1)-(8.5), for ζ large enough, and (4.13) follows.
For the proof of Theorem 4.6, let P k (t), k = 1, . . . , K, and P k (t), k ∈ K, denote the number of partially served class k and type k customers, respectively, present at the system at time t. We further decompose P k (t) into P k,0 (t), counting those type k customers partially served at time t who were already present in the system at time 0, and P k,1 (t) = P k (t) − P k,0 (t), counting type k customers partially served at time t who arrived at the system after time 0. Note that at any time t there may be at most one type k partially served job which was already present in the system at time 0, because the corresponding customers move along the path k one by one in the order determined by their deadlines. Hence (8.6) P k (t) ≤ P k,1 (t) + 1.
Clearly, (8.7) P(t) = Proof of Theorem 4.6. We will prove only the second claim. The proof of the first one is similar, but simpler, since in this case, by Lemma 4.3, we do not have to take the initial customers into account. If the second claim is false, there exist T 0 > C, > 0 and a sequence x n ∈ Ω satisfying (4.4) such that (8.9) E max 0≤t≤T 0 P xn (t|x n |) ≥ |x n | for every n. Let the set G and the subsequence x ξ be as in Lemma 6.1. For k = 1, . . . , K and m ∈ N, let s m k be the probability that a customer entering the network at class k visits more than m (not necessarily distinct) classes before he exits the system and let s m = K k=1 s m k . We have lim m→∞ s m = 0, because the network is open. Let I x ξ m,k be the number of initial class k customers visiting more than m (not necessarily distinct) classes before exiting the network with initial state x ξ and let I x ξ k (0) ≤ |x ξ |, so (8.10) holds. In the second case (8.10) follows from (4.4) and the fact that I x ξ m,k ≤ Q x ξ k (0). We have proved (8.10). Using Theorem 20.5 in [1], together with the Cantor diagonal procedure, and arguing as in the proof of (6.5), we can construct a set G ⊆ G with P(G ) = 1 and a subsequence of x ξ (still denoted by x ξ ) such that on G we have pointwise convergence I x ξ m,k − 3s m k |x ξ |/2 + → 0 as ξ → ∞ for every m ∈ N, k = 1, . . . , K. In particular, for ω ∈ G , m ∈ N, ξ large enough and every t ≥ 0, We will show that on G , (8.12) lim ξ→∞ max 0≤t≤T 0 P x ξ (t|x ξ |)/|x ξ | = 0.
If this is not the case, there exist ω ∈ G , 1 > 0 and a subsequence x ϑ of the sequence x ξ such that for every ϑ, Choose m so large that s m ≤ 1 /6 and α m,k T 0 K < 1 /3 for k = 1, . . . , K.
Proceeding as in (7.10) or (8.4), on the set G for k = 1, . . . , K, ϑ large enough and all t ∈ [0, T 0 ], Let 2 = 1 /(3K|C(k) \ K m,k |). We will now show that for any k ∈C(k) \ K m,k and ϑ large enough, on the set G we have For k = (k 1 , . . . , k n ) ∈C(k) \ K m,k and t ≥ 0, let b (ϑ) k (t) be the arrival time at the network with initial state x ϑ of the type k customer who was the last one to receive service at station j = s(e(k)) by time t|x ϑ | among those routed to the class k n = e(k) along the path k. Each type k customer who arrived at the network before b  k (t) − (L ϑ ∨ l + ϑ ) − 1) has already received full service at k n , the n-th class on his route, by time t|x ϑ |. Type k customers who arrived at the network after b (ϑ) k (t) + (L ϑ ∨ l + ϑ ) cannot receive service before the type k customer who arrived at time b (ϑ) k (t). Consequently, none of these two groups of customers contributes to P x ϑ k,1 (t|x ϑ |) and hence, uniformly in 0 ≤ t ≤ T 0 , we have the bounds where the equalities follow from (6.2), Lemma 4.1 and the fact that = 0 in (4.4). This, together with (8.6), proves (8.15). By (8.7)-(8.8), (8.11) and (8.14)-(8.15), for ϑ large enough, we get max 0≤t≤T 0 P x ϑ (t|x ϑ |)(ω)| < 1 |x ϑ |, which contradicts (8.13). We have proved (8.12). Finally, and it was shown in the proof of Lemma 4.5 in [6] that for k = 1, . . . , K, the sequences {N 0 k (T 0 |x ξ |)/|x ξ |} are uniformly integrable. Hence the sequence {max 0≤t≤T 0 P x ξ (t|x ξ |)/|x ξ |} is also uniformly integrable, and thus, by (8.12), it converges to 0 in L 1 as ξ → ∞. This, however, contradicts (8.9).