Ning Wang and Xiaojian Li, Member, IEEE

Abstract—This paper investigates the secure synchronization control problem for a class of cyber-physical systems (CPSs) with unknown system matrices and intermittent denial-of-service(DoS) attacks. For the attack free case, an optimal control law consisting of a feedback control and a compensated feedforward control is proposed to achieve the synchronization, and the feedback control gain matrix is learned by iteratively solving an algebraic Riccati equation (ARE). For considering the attack cases, it is difficult to perform the stability analysis of the synchronization errors by using the existing Lyapunov function method due to the presence of unknown system matrices. In order to overcome this difficulty, a matrix polynomial replacement method is given and it is shown that, the proposed optimal control law can still guarantee the asymptotical convergence of synchronization errors if two inequality conditions related with the DoS attacks hold. Finally, two examples are given to illustrate the effectiveness of the proposed approaches.

I. Introduction

CYBER-PHYSICAL systems (CPSs) combine physical processes, computational resources, with communication capability [1], which have been studied intensively for their various fields of applications, such as power grid systems in[2], multi-agent systems in [3]–[6] and smart networked systems in [7], [8]. However, CPSs embedded networked control technique are more vulnerable on account of cyber attacks. Therefore, the security of CPSs has recently attracted extensive concern in control field, such as secure state estimation under sparse attacks in [9], [10], secure control and distributed filtering under deception attacks in [11], [12], the resilient control for systems under replay attacks in [13] and the effects of denial-of-service (DoS) attacks on systems in[14]–[19].

DoS attacks, a kind of common attacks in CPSs, attempt to render some or all components of a control system inaccessible by preventing the information transmission.Some existing studies referred to DoS attacks have been published. For example, the stability analysis for a networked system in the presence of DoS attacks, which is on assumptions about the limited DoS attacks frequency and duration, is given in [14]. In [15]–[19], some resilient control methods for CPSs under DoS attacks, which is used to maximize frequency and duration of DoS attacks under the condition that the close-loop stability is not destroyed [15], are introduced. In order to improve the resilience ability of a system under DoS attacks, the state estimation problem is proposed in [20]–[22]. While, the above results are all for a single system.

Recently, based on the studies for a single system, the secure synchronization problem of complex networked systems, which results from that the synchronization behavior relying on the information of other nodes will be affected when DoS attacks successfully break down the information transmission channels, has triggered considerable attentions[23]–[26]. For example, for complex dynamical networks(CDNs) under recoverable attacks destroying the network topology, the authors in [24] introduced the attack frequency and the average recovering time to study the secure synchronization problem. In the presence of DoS attacks interrupting the communication channels of each agent in[25], [26], some sufficient conditions about achieving secure consensus of multi-agent systems are derived by using the Lyapunov function approach. However, it should be pointed that the exact system dynamic matrices are required to be known to analyze the stability of a networked system in[14]–[22] or to solve the secure synchronization control problem in [23]–[26].

In fact, in the absence of DoS attacks, the synchronization problem for the complex networked systems with unknown dynamics has been investigated. To solve optimal synchronization problems of multi-agent systems, a Qlearning method and a model-free off-policy reinforcement learning approach are developed in [27], [28], respectively.Besides, for CDNs of which the couplings are involved in node dynamic equations, the optimal synchronization control problem has been explored in our previous works [29], [30].Nevertheless, the above results [27]–[30] about the complex networked systems with unknown dynamics were all based on the assumption that the networks were attack free. For the case that the CDNs are with unknown dynamics, how to design the secure synchronization control strategy and further to characterize the frequency and duration of DoS attacks under which the stability of synchronization errors is not destroyed, are challenging. Therefore, it is highly desired to study those problems, which motivates the current research.

In this paper, the secure synchronization control problem is considered for a class of CDNs with unknown dynamics in the presence of intermittent DoS attacks. The main contributions of this paper are summarized as follows.

1) For the attack cases, the stability analysis of synchronization errors cannot be directly carried out by using the existing Lyapunov function approach due to the presence of unknown system matrices. To overcome this difficulty, a matrix polynomial replacement method is proposed in the paper. More specifically, the unknown system matrices involved in the Lyapunov function are replaced with the known control gain matrices derived by iteratively solving an algebraic Riccati equation (ARE). And based on the replacement method, the decay rates about sleeping and active intervals of DoS attacks are further computed by solving a set of linear matrix inequalities.

2) According to the obtained decay rates and using the switching technique, the upper bounds of the frequency and duration of DoS attacks, under which the CDNs with unknown dynamics achieve secure synchronization, are given in terms of inequalities.

This paper is organized as follows. In Section II, some necessary concepts about the graph theory and the attack model are presented. An optimal synchronization control protocol is proposed in Section III. Stability analysis of secure synchronization is given in Section IV. In Section V, two examples of the CDNs are provided. Conclusion is given in Section VI.

II. Preliminaries and Problem Formulation

A. Graph Theory

A directed graph (digraph) G=(V,E) contains a set V=1,2,...,N of vertices and a set E=1,2,...,M of arcs (i,j)leading from the initial vertex i to the terminal vertex j. Each arc (i,j)∈E is associated with a real-valued weight with lij>0, while the ith node and the jth node have no connection with lij=0. Assumed that there is no self loop in the graph.Furthermore, define the Laplacian matrix of a digraph as

More detailed concepts on the digraph were shown in [31].

B. System Description

Consider the CDNs model with N linear nodes given as

where xi∈Rnand ui∈Rmare the state and the controlled input, respectively. A ∈Rn×nis an unknown matrix and Bi∈Rn×nis the constant invertible matrix. lijare known coupling weights, and Γi∈Rmis the known inner connecting matrix of the ith node.

Assume the dynamic equation of the leader has the following form

where z0∈Rnis the leader’s state. The leader can be regarded as a command generator, which generates the desired target trajectory.

Assumption 1: (A,Bi) is controllable.

C. Denial-of-Service Attack

In practice, the state information ximeasured by using sensors is sent to controllers via networked channels. As a kind of common attacks in CPSs, DoS attacks compromise certain or all transmission channels of a control system (see Fig.1), which leads to the unavailability of transmitted information.

Fig.1. Framework of the CDNs under DoS attacks.

Due to the energy constraint of adversaries [32], [33], DoS attacks need to terminate attack activities and shift to a sleep period to supply its energy for next attack. That is to say, one adversary launches DoS attacks discontinuously. As mentioned in [14] and [19], {zs}s∈N+ represents the time instant when the sth DoS attack is active. Then

means the time interval of the sth DoS attack with a length ds∈R+. Thus, for given t>τ, define that

which denote, during each interval [τ,t], the sets of time instants when the communication is denied or allowed,respectively. Moreover, |Ξ(τ,t)| and |Λ(τ,t)| represent the total lengths of DoS attacks being active and sleeping during the interval [τ,t].

for k ∈N. Thus, the state information received by controllers is considered as

with i=0,1,2,...,N. And the information received by actuators is given as

Similar to [14], there are two assumptions for DoS attacks.Assumption 2 (DoS frequency): There exist constants η ≥0 and τD>0 such that for all t,τ ∈R+

Assumption 3 (DoS Duration): There exists constant T ≥1 such that for all t,τ ∈R+

Assumption 2 is inspired by the concept of average dwell time [14] to specify the number of DoS attacks occurring on the interval [τ,t] and Assumption 3 is used to describe the length of the interval over which communication is interrupted. From [14] and [16], it is known that those two assumptions are common for reflecting the constraint energy of DoS attacks.

Now, the considered problem in this paper is presented as follows.

Problem 1: The main objective of this paper is to design a secure synchronization control law for the system (1) with unknown system matrices, such that the synchronization errors are asymptotically convergent for both attack-free and attack cases.

III. The Optimal Synchronization Controller Design Without Attacks

A. The Optimal Synchronization Control Law

By using (1) and (2), define the synchronization error eifor the ith node as

Then the synchronization problem is described as

The dynamic equation of the synchronization error eihas the following form

Besides, similar to [30], define the performance function for the ith node (11), which involves the couplings information

where Qi∈Rn×nand Ri∈Rn×nare symmetric positive weight matrices.

Theorem 1: For the error dynamics of CDNs (11) with the quadratic performance index (12), a control protocol is an optimal one if and only if it has the following form

where Piis a symmetric positive matrix and satisfies the following ARE

Proof: First, one gives the necessity proof of Theorem 1.Based on (12), define the Hamiltonian function for (11) as

where Piis obtained by solving the ARE (14). According to(17) and the canonical equation shown in [29], one gets the ARE

Using (16) and (17), it yields the control law (13).Therefore, the necessity is proven.

The proof process of sufficiency for Theorem 1, which is divided into two steps, is given as follows.

Step 1: It is necessary to prove that the control law (13)ensures the asymptotical stability of the synchronization errors. That is to say

According to ARE (14), (19) is rewritten as

Step 2: Consider the following equation:

Based on the ARE (14) and the condition (18), the performance index Jiis rewritten as

B. The Optimal Synchronization Controller Design With Unknown Dynamics

Consider the optimal controller with the following form

where

When all system dynamics are accurately known, the feedback gain Kiis determined by solving the ARE (14).Since (14) is nonlinear in Pi, the solution Picannot be obtained from (14) directly. To overcome this difficulty, an algorithm has been developed in [35] by solving (25)iteratively

According to (26)–(28), one has the following equation:

Remark 1: The feedback gain Kiis obtained under the attack free case, and the proposed iterative policy method requires that all the information of states xi, control inputs uiand couplings ωiare available.

IV. Synchronization Errors of Stability Analysis With Intermittent DoS Attacks

As shown in [14], there may be a time interval elapsing from the time zs+dsto the time zs+ds+vsat which the state information is successfully sampled and transmitted. And the time interval vs(shown in Fig.2) satisfies

Fig.2. An example of DoS attacks.

with ??representing the upper bound on the inter-sampling rate in [14]. Thus the time interval [τ,t) is composed of the following two sub-intervals:

where Vs=[zs,zs+ds+vs). Specially, for all τ,t ∈R≥0,

Next, by using the Lyapunov function method and switching technique, the stability of the error systems (11)under DoS attacks will be discussed.

Note that, to analyze the stability of synchronization errors,the system dynamic matrices are required to be known in[23]–[26]. However, such a requirement cannot be satisfied in this paper, and then these existing methods are no longer applicable. To overcome this difficulty, a matrix polynomial replacement method is proposed based on the ARE (37) and the iterative learning results Piaand Ki. The details are shown as follows.

Divide the process of stability analysis into two steps.

By resorting to ARE (37) and the iterative learning results Piaand Kiin (37), one has

with αi>0. That is to say,

Therefore, the convergence rate αiof the system (11) can be computed by (41). Before applying the optimal control gain Kito system (1), the matrix Piaand the feedback gain matrix Kiin ARE (37) have been obtained by iteratively solving (30).

where its derivative satisfies

with the exponential divergence rates βi>0. That is

Based on Assumption 4, one has

which yields the following inequality

Combining the above analysis results (41) and (47) of the piecewise Lyapunov functions (36) and (42), respectively, the stability of closed-loop system, which switches between stable and unstable subsystems, is established by using the switching technique. The conditions on the frequency and duration of DoS attacks, under which the secure synchronization can be guaranteed, are shown in Theorem 2.

Theorem 2: For system (11), suppose Assumptions 1–4 hold. Given any positive definite symmetric matrices Riand Qiwith i=1,2,...,N, if there exist positive constants θisuch that the following two inequalities hold

where αiand βisatisfy (41) and (47), respectively, andμi=max{λmax(Pia)/λmin(Pib),λmax(Pib)/λmin(Pia)}. Then, the optimal control laws (13) ensure that the synchronization errors are convergent under DoS attacks.

Proof: Consider any time interval [zs,zs+1) that Vibis activated in [zs,zs+ds+vs) and Viais activated in[zs+ds+vs,zs+1). By comparison lemma in [37], one gives

Similar to [26], there are the following two cases:

Case 1: If t ∈[zs,zs+ds), n(t0,t)=s ,

Thus, for ?t ≥t0,

According to Assumptions 2 and 3, and inequalities (34)and (35), it yields that

From (48) and (49), it is known that δi>0. Thus the synchronization errors ei, i=1,2,...,N, are convergent exponentially, which implies that limt→∞ei(t)=0.■

Remark 2: DoS attacks compromise certain or all transmission channels, which is also introduced in [15], [16],[19] and [26]. The case that all transmission channels are interrupted when DoS attacks occur is considered in Theorem 2. This case can also be found in [23] and [26].

V. Simulation Example

In this section, two complex dynamical network examples are taken to verify the effectiveness of above methods.

Example 1: Similar to [7], consider a spring-connected multi-vehicle system (shown in Fig.3) composed of an isolated leader and four slave vehicles. The springs are regarded as the interconnection between two vehicles. In fact,the ith vehicle is modeled as

Fig.3. Spring-connected multi-vehicle system with four slave vehicles.

with B=I2, Γ1=0.9I2, Γ2=0.7I2, Γ3=0.9I2, Γ4=I2, Γ5=2I2. Suppose the initial values e1=[?6,?6]T, e2=[?5,?5]T,e3=[?4,?4]T, e4=[?2,?6]Tand e5=[?5,?3]T. Then choose the weighting matrices Q5=12I2.

Fig.9. Graph structure for the CDNs.

Similar to Example 1, the simulation process is divided into two steps.

Step 1: Use the policy iteration method to obtain the solutions Piaand Ki. The initial controller gain is chosen as

Besides, the other parameters are assumed to be the same as those in Example 1. By utilizing the PI method, the optimal solutions Piaand Kiwith i=1,2,3,4,5 are obtained after 4 iterations. They are given as

Step 2: Stability analysis of secure synchronization

According to the analysis results (41) and (47), suppose that the DoS attacks with τD=2 and T =1.2 occur during time interval (0,20), which satisfying the conditions in (48) and(49) in Theorem 2.

VI. Conclusion

This paper investigates the secure synchronization control problem for CPSs subject to intermittent DoS attacks. The

Fig.10. Synchronization errors of nodes 1–5 with unknown system matrix A for the DoS attacks case.

Fig.11. Control laws of nodes 1–5 with unknown system matrix A for the DoS attacks case.

considered CPSs are modeled as a class of CDNs with unknown dynamics. First, to deal with the state couplings, a distributed optimal controller is proposed based on our previous works [29]. And the optimal feedback gain matrix is learned by iteratively solving the ARE. Especially, based on the ARE and the iteratively learning results, the decay rates for each node about sleeping and active intervals of DoS attacks, are determined by solving a set of linear matrix inequalities. Moreover, by using the switching technique and the obtained decay rates, the upper bounds of the DoS attacks frequency and duration, under which the synchronization for all nodes is still achieved, have been proposed. Finally, the simulations of two examples verify the effectiveness of the proposed methods.