Constrained stochastic predictive control of linear systems with uncertain communication

1 Introduction

This article combines principles of model predictive control (MPC) and networked control systems (NCS) to address timing effects that arise from wireless components within control loops [1]. As wireless sensors have become widely available and are simple to integrate [2], they also contribute to enabling concepts of Industry 4.0, e. g., the control of mobile systems with stationary motion capture systems [3]. On the downside, the drawback of electromagnetic susceptibility (like packet loss and/or latency) arises if used in control systems. As imperfect wireless communication can be interpreted as a stochastic process, the behavior, control performance, and stability of the NCS depend on stochastic information flow. In case the communication delays are in the order of the dominant time constants of the plant dynamics, the consideration of the stochastic process within the controller synthesis is mandatory, while being omitted in the vast majority of methods proposed for NCS. Some approaches model the imperfections by considering a worst case delay [4], [5], [6] – however, the use of constant worst case delays within controller synthesis typically leads to overly conservative results and degraded performance. Own work in this direction aimed at the reduction of conservatism by use of variable communication delays within the controller synthesis [7], [8]. The key point there is to predict and robustly bound the delay of the communication network, and to consider the time-varying bounds in distributed MPC.

However, as all these approaches aim at robust satisfaction of constraints (while the underlying communication process is of probabilistic nature), the reduction of conservatism is limited. Therefore, the objective of this contribution is to model the NCS completely stochastic, and to use stochastic MPC (SMPC). In SMPC the probabilistic system behavior is explicitly described by its underlying probability distribution to directly address the probability to satisfy or violate associated constraints. For a brief overview of SMPC and its application to control systems without network structures, see [9]. For more explicit literature on SMPC with state and input constraints, the interested reader is referred to [10], [11]. In the context of NCS, the satisfaction of state constraints – especially in the probabilistic setting – is still a largely open issue. Thus, [12] has further investigated the stochastic problem setup and provides a stochastic optimal control scheme, which is able to handle both, uncertain communication and additive disturbances affecting the plant as stochastic processes. The uncertainty of when future sensor information will be received by the controller are projected onto a tailored likelihood to satisfy state and input constraints. The promising result is that a poorly performing communication link will affect the uncertainty, but not necessarily the performance of controlled behavior. In this article, the findings of [12] are used to develop an MPC scheme, in order to cope with larger horizons and to further reduce the conservatism. As usual for MPC, this implies the formulation and use of a terminal control law and set for the stochastic system to ensure recursive feasibility and stability in a probabilistic sense.

The article is structured as follows: Section 2 introduces notation and clarifies different mathematical aspects used throughout the article. Section 3 specifies the networked control system and the control problem. Section 4, as main part of the paper, details aspects of the stochastic communication channel, the computation of the terminal control law, and the stochastic MPC scheme including properties. The scheme is then applied in Sec. 5 to the example of an aerial vehicle, and conclusions are contained in Sec. 6.

2 Preliminaries

The symbol s ( k ) denotes a discrete-time value of a signal s ( t ) ∈ R n s at time t = k · Δ t, with k ∈ N + , and constant time-step Δ t ∈ R + . The value of the signal s for a future time-step k + t , t ∈ N + as predicted in k is denoted by s t | k .

A convex polytope for s, parameterized by a pair ( C s , b s ) of matrix and vector of appropriate dimension, is referenced by:

An ellipsoidal set is defined by its center point and shape matrix ( s ¯ , S ):

A multivariate normal distribution of an n s -dimensional random vector s with covariance S ∈ R n s × n s and a mean-value s ¯ ∈ R n s is denoted by s ∼ N s ¯ , S . Correspondingly, the expectation of a value s is denoted by s ¯ = E s . An affine transformation of a normal distribution with matrix M and vector v is again a normal distribution according to:

and the sum of two normally distributed random vectors s 1 ∼ N s ¯ 1 , S 1 and s 2 ∼ N s ¯ 2 , S 2 is also again a normal distribution:

The level-curves of a Gaussian probability density function are ellipsoidal, such that the δ-confidence ellipsoid of a normal distribution with rank n and with probability δ ∈ 0 , 1 is defined by:

using a scaling factor c [ δ ] = F χ 2 − 1 δ , n , in which F χ 2 denotes the cumulative distribution function of a χ 2 -distribution. Throughout the paper, the mean value of a distribution coincides with the center point of the confidence ellipsoid, and the shape matrix is equal to the covariance matrix of the distribution. Thus, the same notation is used for these quantities.

The probability for an event A is referred to by P ( A ), and if this probability is specific for the time-step k, it is denoted by P k ( A ). Likewise, the conditioned expectation for a signal s t | k is denoted by E k s t | k = : s ¯ t | k , since prediction and expectation rely on the same time-step k throughout the paper.

Finally, the symbol s Q 2 = s ⊺ · Q · s denotes a weighted 2-norm of the vector s with weight Q.

3 Considered class of networked control systems

The class of systems under consideration in this paper represent single-loop feedback structures with wireless communication network in the signal link between plant and controller, see Fig. 1a. This setting can be motivated, e. g., in the context of controlling aerial vehicles, as explained in Sec. 5. The plant is modeled by discrete-time linear systems with probabilistic additive disturbances:

with state vector x ∈ R n x , input vector u ∈ R n u , and a vector of disturbances w ∈ R n w . The additive disturbances cannot be measured and are assumed to be normally distributed with zero-mean and covariance matrix W, and to be independent, identically distributed (i. i. d.).

Figure 1

Structure of the networked control system and the corresponding Markov chain modeling the communication network.

The states x ( k ) and inputs u ( k ) have to satisfy polytopic chance-constraints with likelihoods δ x , and δ u respectively (both typically chosen close to 1):

The controller and plant are statically connected in wired form, but sensor and controller are loosely connected by wireless communication. The communication network is established as a simple one-link channel and consists of a sender S, a receiver R, and the link-probability p ( k ), see Fig. 1a. The link probability can be understood as an abstract representation of the network characteristics (or protocol), as, e. g., resending information in case of failed transmission, or activation/deactivation of the channel by a network controller, but it could also be interpreted as the probability that measured sensor data is available. However, for assuming a time-varying link-probability, the communication network is modeled by a Markov chain [13] in each time-step k, as shown in Fig. 1b.

In time-step k, the current state x ( k ) is available to S. The information is broadcast to R, and thus available to the controller with probability p ( k ). If information is not received, the information remains with S. Then, the last information received by the controller is outdated, such that the age of the currently available state information increments. Similar to the previous work in [8], [12], the following definition is proposed:

If x ( k + 1 ) is received, the AoI is reset to zero, otherwise it is incremented again.

The main control objective for this networked control system is to steer the states to the origin with acceptable costs for the input, i. e., to find a control sequence { u ( k ) } k = 0 ∞ which minimizes the infinite horizon cost function:

subject to the state and input constraints, and the uncertain AoI respectively. The stage costs are formulated in the following common form:

with symmetric and positive semi-definite weighting matrix Q x = Q x ⊺ ≥ 0 for the states, and definite weighting matrix Q u = Q u ⊺ > 0 for the inputs.

4 SMPC based on stochastic optimization

To cope with the challenges of control under uncertainty of the reception of data and the satisfaction of chance constraints, the following control scheme is proposed: In each time-step k in which information is received (i. e., a ( k ) = 0), the controller solves a finite-time stochastic optimal control problem with horizon H. The solution of the optimal control problem is a control sequence { u t | k } t = 0 H − 1 , which has to guarantee the satisfaction of the constraints (2) with respect to the upcoming communication link probabilities { p ( k + t ) } t = 1 H − 1 . Then, as usual for MPC, the first control input is applied to the system, i. e., u ( k ) = u 0 | k . In case no information is received, i. e., a ( k ) > 0, the controller solves no optimization problem, but applies the subsequent control input of the optimal control law computed in the previous optimization. This was accomplished at time k − a ( k ), thus the input is chosen according to u ( k ) = u a ( k ) | k − a ( k ) . Note that the use of a previously computed control input is relatively unusual in predictive control, but here it is used to avoid accounting for uncertain initial states within the optimization problem, discussed later on.

This scheme is repeated recursively, thus the case of an infinite horizon control, as mentioned in Sec. 3, would be approximated. However, due to the properties of the probabilistic communication link, the possibility exists that the communication link fails for more than H subsequent time-steps, i. e., P a ( k ) ≥ H > 0. In this case, no control law computed before is available. This problem is immanent to the considered setting, and cannot be healed by any closed-loop control strategy (while its probability of occurrence can, of course, be reduced by increasing H within finite-horizon MPC). Consequently, closed loop control may not be possible, if the lack of measured information persists permanently, while open-loop control remains to be an option (obvious from Fig. 1a). Thus, the following assumption needs to be stated:

Obviously, the age of information (AoI) is a crucial value in operating the NCS and for the success of the predictive control policy. Hence, this quantity is detailed in the following subsection.

4.1 Modelling the age of information

According to the one-link communication network in Fig. 1, the AoI depends on the behavior of the communication link, and its transmission probability. In each time-step, the transmission succeeds if x ( k ) is made available to the controller, implying according to Def. 1 that a ( k ) = 0. If the transmission fails, the newest information previously sent to the controller remains the same as before. But since this information is older by one time-step, a ( k ) : = a ( k − 1 ) + 1 results. According to this scheme, the behavior of the AoI is modeled (and becomes therefore predictable) according to a discrete-time Markov chain with states σ j ( k + t ) implying a ( k + t ) = j, and with probabilities q ( k + t ) = 1 − p ( k + t ), as shown in Fig. 2.

Figure 2

Example of a Markov chain to model the AoI for 3 subsequent time-steps with k = 0 and a ( 0 ) = 0.

With an initial distribution μ ( k ) (determined by the current AoI), the distribution for future values of the AoI results to:

(4) μ t | k = ∏ r = 1 t P ( r ) · μ ( k )

with time-varying transition probability matrix P ( k ). For the example of Fig. 2, P ( k ) is given by:

P ( k ) = p ( k ) p ( k ) p ( k ) 1 q ( k ) 0 0 0 0 q ( k ) 0 0 0 0 q ( k ) 0 .

The probability for a specific value of the AoI in time-step k + t predicted in k is given by:

(5) P k ( a ( k + t ) = j ) = μ t | k [ j ] ,

where μ [ j ] denotes the ( j + 1 )-th entry of the vector μ.

5 Numerical simulation

This section describes the application of the proposed method to the example of an unmanned aerial vehicle (UAV) moving in a constrained area. Usually, the control of UAVs is twofold: i) an inner control loop stabilizes the UAV in hovering mode, and ii) an outer control loop determines the path to follow. While the inner loop controls states which are locally measurable, as rotor speeds (in the case of drones) or the yaw, roll, and pitch angles, the outer loop controls the position and height in the 3-dimensional air-space. Therefore, the outer control loop is responsible for safe operation with respect to the UAV’s environment, i. e., physical and virtual obstacles like buildings or no flight zones in urban areas. In contrast to the inner loop, the states of the outer loop, i. e., the exact position in 3D, is often not locally measurable because of an unmeasurable drift of the gyros used for stabilization, uneven terrain (e. g., if cameras or infrared sensors are used), or varying air pressure. Hence, the position is often measured by GPS for outdoor motion, and via motion tracking systems in case of indoor motion. Both methods have the drawback that the control loop relies on these external signals. In motion tracking systems, the position of the UAV is pre-processed by a stationary unit, and afterwards transmitted to the UVA via a communication link (commonly via wireless LAN). In contrast, GPS is based on the run-time measurement of electromagnetic satellite-signals and relies therefore on possibly faulty connectivity to satellites. However, in both cases the availability of the exact position measurements for the outer UAV control loop can be modeled by the communication link in Fig. 1b, and its transmission probability. Here, the UAV motion is modeled in a two-dimensional space with integrator dynamics, and with an underlying inner control loop, as proposed in [11]. The dynamics (1) is specified by:

with state vector x ⊺ = p x ⊺ , v x ⊺ , p y ⊺ , v y ⊺ , input vector u ⊺ = a x ⊺ , a y ⊺ , and disturbances w ⊺ = w x ⊺ , w y ⊺ . Here, { p x , p y } are the positions, { v x , v y } the velocities, { a x , a y } the acceleration, and { w x , w y } the disturbances in x- and y-direction. In contrast to [11], only one disturbance per direction is considered, modeling a wind force, and affecting both the position and the velocity of the UAV. The disturbance is modeled as a Gaussian random variable, and the communication link is considered to transmit with constant probability:

Position and velocity are constrained according to (2a) with likelihood δ x = 0.95 and the admissible set:

Similarly, the input is constrained according to (2b) with likelihood δ u = 0.9 and the admissible set U = u u ∞ ≤ 7.5 . Caused by the uncertain AoI, the threshold δ τ is selected to 99 %. With respect to the constant link probability and according to (17), the threshold results in δ τ ≈ 99.19 % with delay-length τ = 4 for the terminal system and the control law. Note, that τ = 4 implies H ≥ 4, and that the expected value of the last 4 predicted states have to lie in the origin (cf. (19c)).

The feedback matrix of the terminal control law K τ is chosen to:

satisfying the assumptions 4. 3 and 4. 4.

The prediction horizon is chosen to H = 12, and the stage cost function from [12] is used with:

Starting from an initial state x 0 = 35 , − 2 , − 20 , − 4 ⊺ (blue square), Fig. 3 shows 200 Monte Carlo simulations with 50 time-steps each in both directions of the state space, where each color denotes a different simulation. Note, that in none of these simulations the use slack variables was needed. For simulation, the covariance of the disturbance is chosen to W ( k ) = W ˆ. All simulations show that system trajectories converge into a neighborhood of the origin. The neighborhood is significantly smaller than the 95 %-confidence ellipsoid of Σ ξ , which is identical to the region for which the control law (11) is defined.

Figure 3

Simulation results for 200 Monte-Carlo simulations with the initial state marked as blue square, the admissible state space as light gray box, and the projection of the terminal set onto the corresponding direction as red ellipsoid.

Figure 4 shows the distribution of the AoI values for each time-step k of the simulations. Even if the probability is small, the maximum value for the AoI is 8, thus larger than τ = 4. Note, that the control strategy only guarantees probabilistic constraint satisfaction and mean-square stability as long as a ( k ) ≤ τ. By a solution of (20) without the use of slack variables, it provides a suitable control input as long as a ( k ) < H. Here, suitability means that a possible violation of the state and/or input constraints with an aged control input u ( k ) = u k | k − t , τ ≤ t < H is considered in the optimal control problem, together with the likelihoods { γ k ( u ) , γ k ( x ) }. Since aged control inputs are rarely used (according to Fig. 4 only with probability around 40 % in each time-step), the satisfaction of constraints is much higher than { δ u , δ x }. Here, the admissible state space is never left over all simulations.

Figure 4

Distribution of the Age of Information for the Monte-Carlo simulations and each time-step of the simulation time.

6 Conclusion

This article has introduced a model predictive control scheme for constrained linear systems with uncertain availability of state information. The proposed method is based on a recently suggested optimal control scheme for linear systems with uncertain communication, which projects the uncertainty of available information onto a tailored likelihood to satisfy state and input constraints. Here, solving such a stochastic optimal control problem in receding horizon fashion implies the use of terminal constraints to guarantee recursive feasibility and stability of the control scheme. Due to the uncertain communication, the use of a state-augmented terminal system with delayed control law is proposed. In contrast to existing methods, which use reachable sets as terminal sets, the terminal set here is derived with respect to the confidence ellipsoid of an invariant covariance. This approach scales better with the dimension of the system and the communication link probability, compared to the use of polytopic sets.

The simulation results show that the proposed predictive control scheme is able to steer a networked control system with high (and specified probability) into the target, while the AoI changes probabilistically. Although the control scheme is based on probabilistic constraint satisfaction, the system state remains in the admissible state space. At the same time, the state trajectory does not require considerable safety margins to the state constraints, thus introducing only little conservatism. When approaching the target state, the proposed control scheme keeps the system state in a much tighter neighborhood than approaches which are based solely on a worst case delay of the communication link.

It is worth noting, that the computational complexity of the predictive control does not significantly grow with the complexity of the communication network between sensor and controller, or if the threshold δ τ (and therefore the dimension of the terminal system) is increased.

This control scheme opens the field to distributed predictive control systems using collectively a single communication network – this is matter of future work of the authors.