The linear-quadratic stochastic optimal control problem with random horizon at the ﬁnite number of inﬁnitesimal events

– The aim of the article is to expand the results of the theory of Linear Quadratic Control (the state of the system is described with the help of stochastic linear equation while the quality coeﬃcient is of a quadratic form) in the case of random horizon independent of the states of the system. As for the question under consideration the control system horizon is an independent variable with a discreet decomposition and has got a limited number of possible accomplishments. The above mentioned situation takes places when the number of controls is brought out by the outside factor (generally independent of the system).


Introduction
The greatest part of the tasks of the theory of (adaptive, stochastic, linear quadratic etc.) control is defined for a finite or infinite horizon (see e.g. [1-3, 5, 6], [7,[11][12][13]15]). The tasks with a random horizon are less noticeable in the object literature and can be divided into two types. The first type is based on the classic optimal stopping of the decision process (see [8,9,14]) or the changing of stopping into control (see [4]). The second type of the task is based on the assumption that the finite or infinite horizon does not depend on the system (the state of the system does not influence the horizon). Such a situation ocurs in the case of when the definition of the control horizon e.g. the number of losses, the number of requests. In this case we must define the horizon using a random variable state independent (see [10]).
The paper presents the linear quadratic control problem with the random horizon independent in the system. Then decomposition of LQC with a random discreet time to LQC with the deterministic time and modified quality coefficient is shown. Also the optimal control laws are derived and the functional quality and quantity of controls for the systems with the determined time horizon and random horizon are compared.
The arrangement of the article is as follows. Section 2 introduces the task of linear quadratic control with the random horizon. Section 3 presents the modification of the task under consideration the one with the deterministic horizon. Section 4 gives the comparison of optimal controls with the classical deterministic horizon and random horizon for the simple linear system. Section 5 presents the numerical simulation of the above mentioned system.

Problem formulation
Let (Ω, F, P ) be a complete probability space. Suppose that w 1 ,..., w N are independent m-dimensional random vectors in this space, with the normal N (0, I m ) distribution, let ξ be a k-dimensional vector with a priori distribution P (dξ), and let y 0 be an initial state. We assume that all the above objects are stochastically independent and we define F k = σ (ξ) ∨ σ {w i : i = 1, 2, ..., k} and F = F N .
We will consider the control problem for a linear system with the state equation The matrix A is called the matrix of transformation of state. On (Ω, F, P ) we define the family of sub−σ−fields Y k = σ {y i : i = 0, 1, ..., k}. A Y j -measurable vector u j ∈ R l will be called a control action, and u = (u 0 , u 1 ,...,u N −1 ) an admissible control. The class of admissible controls is denoted by U . Let τ represent the random horizon of contol, which is state of system independent and has the density Let us consider the following task. We want to move the system from the state y 0 to the state a in τ steps and at the same time we would like to have the least energetic costs and have the least losses connected with a missing target. We assume that the energetic costs in the form u T i R i u i represent loss functions for every i = 0, 1, ..., τ − 1, however, the heredity function (y τ − a) T Q τ (y τ − a) represents losses connected with the missing state a. We introduce performance criteria, which represent the joint costs and final loss and are described by where R i ∈ R l×l for i = 0, ..., N − 1 and Q N ∈ R n×n . The task is to find and to determine an admissible control u * = u * 0 , ..., u * τ −1 for which inf is attained.
3 Decomposition LQC with the random horizon to deterministic horizon In the case, where the random variable τ , which presents the horizon of performance criterion, has distribution with the finite number of infinitesimal events i.e.
From the total probability formula we can present functional (2) in the form Finally where and Therefore, we substitute the task of optimal control with the random horizon of the finite number of events (3) for the task of optimal control with the finite horizon The technical results for arbitrary functionals (losses and heredity) and random horizon (finite and infinite number of events) can be seen in [10] . The theorem below represents both necessary and sufficient conditions of optimal control for the task (quadratic form) (7), which are obtained by equating the derivatives to zero.
An optimal control u * must be satisfied by the condition ∂ ∂u j J(u * ) = 0 thus substituting (15) to (14) and equating to zero we obtain which proves the assertion.

Comparison of optimal controls with deterministic and random horizons
We consider a linear system with the state equation The quadratic criterion with the random horizon is We can decompose the above task to the other form We have the optimal control of linear system (17), which contains the following: then the optimal control for the task with random horizon (17) is Pobrane z czasopisma Annales AI-Informatica http://ai.annales.umcs.pl Data: 05/11/2022 21:47:41 Proof. The value of Bellmann's function in the step N is We assume that equation (22) is true for i + 1 and

From (22) and the properties of condition expectation we have
Thus, the optimal control is The linear-quadratic stochastic optimal control problem. . . and finally tr σ T G j σ Remark 1. We use formulas (19)-(22) for the linear quadratic control with the deterministic horizon N It is sufficient to express, that the density of random horizon is P (τ = i) = 0 for i = 0, 1, 2, ..., N − 1 and P (τ = N ) = 1.

Corollary 1. For both random and detreministic horizons we have:
(1) the optimal controls of system (17) are where the matrices G i and R i are given: • in the case with the random horizon For i = 0, 1, 2, ..., N − 1.
Example 1. We consider the control problem for a linear system with state equation (17) Table 1 includes possible discrete densities of the random horizon P (τ = j). Case I presents the distribution of random horizon of control process, which is more concentrated at the beginning of time scale, in case II the random horizon is concentrated in the middle of time scale, in case III evenly distributed on the set of events {1, 2, ...10}, in case IV concentrated at the end of time scale. Case V presents the deterministic horizon of control P (τ = 10) = 1. Table 2 presents possible trajectories y j of states, optimal controls u * j and the values of Bellman's function W j (y j ) for each case. Fig. 1 and 2 present the values of Bellman's function W j (y j ) and energetic costs u T j R j u j on the generated trajectories included in Table 2 siutably.

Conclusion
In this article, the problem of control of a linear quadratic stochastic system with the random horizon and independent states system was introduced and the laws of control were worked out. The problem of random horizon with a finite number of events was reduced to the task with the deterministic horizon; at the same time the functions of losses and the functions of heredities were modified. In the above mentioned linear system the noises as well as parameters of system were modelled by the Gaussian distributions.
Then it was shown the comparison of optimal linear controls for the simple linear system (only control transformations and outer noises without transformation of the parameters of the system and drift) were added to the previous system state) with the random horizon of a finite number of events and the deterministic horizon. Similarities and differences of controls and values of quality functions were discussed. From the obtained results we can conclude that the knowledge about density of horizon has the essential influence on the value of losses and heredity sum.
Pobrane z czasopisma Annales AI-Informatica http://ai.annales.umcs.pl Data: 05/11/2022 21:47:41 U M C S Analyzing the behaviour of such a simple system we can conclude: • the controls for deterministic horizon systems in the random horizon systems must not be used directly; • the values of controls (control norms, value of costs, outlays) for the random horizon systems are definitely larger at the beginning whereas the values of controls for the deterministic horizon systems are evenly distributed until the end of the control horizon; • the values of controls for the random horizon systems are definitely larger up to the moment Eτ (expected value of control horizon) and are used for achieving of the main aim of control (hitting point a), however after the moment the values of the controls are smaller and the costs are reserved for leveling the results of outer disturbances in the systems (after the moment the system practically oscillates around point a).