Yun Wang, Yuchen He*, De Gu

1 Mechanical and Electrical Engineering Department, Zhejiang Tongji Vocational College of Science and Technology, Hangzhou 311231, China

2 College of Mechanical & Electrical Engineering, China Jiliang University, Hangzhou 310018, China

3 School of Internet of Things Engineering, Jiangnan University, Wuxi 214122, China

Keywords:Multimode processes monitoring Dual iterations Double layer information extraction High order expansion Quality related

A B S T R A C T Due to higher demands on product diversity, flexible shift between productions of different products in one equipment becomes a popular solution,resulting in existence of multiple operation modes in a single process.In order to handle such multi-mode process,a novel double-layer structure is proposed and the original data are decomposed into common and specific characteristics according to the relationship between variables among each mode. In addition, both low and high order information are considered in each layer. The common and specific information within each mode can be captured and separated into several subspaces according to the different order information. The performance of the proposed method is further validated through a numerical example and the Tennessee Eastman (TE) benchmark.Compared with previous methods, superiority of the proposed method is validated by the better monitoring results.

1. Introduction

Nowadays,process monitoring has become one of the essential techniques to ensure the reliability and safety of modern industry.In the past decades, significant research efforts have been carried out to contribute to the development of process modeling and monitoring[1-4].A number of successful methods with good accuracy and stability have been developed[5-7].In order to meet the diversity of production demands,a production line may be used to make products with different specifications, resulting in the existence of multiple operation modes [8,9]. Hence, multimode production has become a popular solution for manufacturers to meet different requirements. As a result, increasing attention has been paid to multimode process monitoring to enhance the effectiveness and stability of the processes.

In order to cope with multimode processes, several methods have been proposed [8,10-13]. In addition, recursive principal component analysis(PCA)/partial least squares(PLS)and localized discriminant analysis have also been considered as effective solutions for multimode processes monitoring[14-16].In these methods, different kinds of similarity measures have been applied for model self-adaption. However, faults may deteriorate the performance of these models since a faulty sample in one mode may be misclassified as a normal sample in another mode. As a result,separate modeling seems to be a reasonable solution for fine description of multimode process behavior. A common solution for multimode modeling is to set up a series of models separately where the uniqueness of each mode can be maintained.Zhaoet al.proposed a two-step multi-set analysis algorithm, called MsPCA,which is able to obtain the correlation between variables in each data space [17]. Such structure usually captures individual information within each mode.However,it should be noted that correlation may also exist between different modes in multimode process. Consequently, between-mode relationship may deteriorate the power of separate modeling and the uniqueness of each mode cannot be clearly revealed. Furthermore, repeated modeling for common information will also increase the modeling complexity. Therefore, it is quite reasonable to provide a different pointview of modeling for multimode processes where the common and specific information of each mode can be well reserved simultaneously.In recent years,several relevant modeling methods have been proposed. For example, Zhao designed a projection method for measurement data to solve the cross-mode information separation problem[18].In the common subspace,all modes share the same common basis vector which can be obtained by the optimization of the designed cost function. The angle between common basis vector and specific basis vector could be as close as possible so that the geometrical meaning can be explained. Furthermore, a between mode subspace decomposition was carried out to analyze the relative changes between two adjacent modes including increased changes, decreased changes and unchanged part [19]. These variations are evaluated and separated based on a predetermined reference PCA model. On this basis, betweenmode model can be developed and applied in the multimode monitoring.Zhanget al.divided the whole space into several subspaces where the between-mode correlation and mode uniqueness within different modes are identified as common and specific subspaces,respectively[20].A weight matrix is designed to balance the correlation between each pair of adjacent modes. The main problem of this method is that the common space is established by minimizing the residual of reconstruction derived by second-order information. Different subspaces share the same weight vector ω in the process data space. However, it is unreasonable to only consider weight vector in process data while neglecting the weight vector in quality related data. Moreover, a single weight vector cannot guarantee the full extraction of common information throughout all modes. To overcome the above problems, Zhanget al. improved the common and specific subspace structure and proposed a between-mode transition monitoring method [21-23].The common subspace is established by a series of single models designed for each mode.Although such models make it possible to handle the monitoring of between-mode transitions, there still remains some serious restrictions such as difficulties in monitoring for processes with three or more modes. Recently, Zhaoet al. proposed a full condition monitoring method which cares about nonstationary dynamic process analysis [24]. Cointegration analysis(CA) and slow feature analysis (SFA) are employed for the extraction of long-term equilibrium relation and slow/fast feature.A series of statistics are then established for the monitoring of static and dynamic information as well as slow and fast feature. In order to enhance capability of quality-related common and specific subspace theories, a common and individual (CnI) algorithm was designed for hot strip mill plants where the common score vector is constructed by the combination of all possible sub-scores derived by subspace projection [25]. Then corresponding projection parameters can be obtained through maximizing the variance or covariance of the common score vectors. It provides a comprehensive method to common and specific subspaces construction.The common information represented by latent variables is composed of the relevant common information of each mode. Nevertheless, in spite of the great improvements of the CnI algorithm,there still remains some problems. Firstly, in the above methods,the basic assumption is that data should obey the Gaussian distribution which cannot accord with the real situations. As a result,the common subspace based on the maximization of second order statistics, such as variance or covariance, may lead to imperfect information extraction in non-Gaussian distributed situations.Actually, the concept of non-Gaussian data description has been explained by the classical independent component regression(ICR) [5,26]. Contrary to principal component (PC), the independent component (IC) defined by the negentropy is usually derived based on high order moments, which makes it possible for non-Gaussian information extraction. Consequently, instead of low order information, the high order common information should be emphasized simultaneously. Secondly, although CnI models were involved to cope with the quality-oriented common information,the latent information of quality-related variables is not well captured. In CnI based algorithm, only original quality information is introduced in the covariance maximization, which is incomplete since only the common information shared by process variables and quality variables should be extracted. Alternatively, reference[27] provides a suitable approach for high-order quality-related information extraction based on mutual information (MI). The established latent space balances between negentropy and mutual information. However, a large amount of low-order information still needs to be extracted. In order to take a full consideration for quality-related information, a two layer latent subspaces were proposed where both low and high order information were described in our previous research[28].The performance of quality related information extraction can be improved by using this two layer latent subspaces structure.

On the basis of our previous research,a hierarchical monitoring strategy is proposed for multimode process monitoring where a novel hierarchical common and specific structure (HCnS) is designed. First, high order common information is extracted from the original data space, with emphasis on quality-related high order common information(HC).It should be noted that even after the extraction,there is still much common information retained in the residuals of each mode. Therefore, according to the residual information, a low-order common feature subspace is established where quality-related low order common information (LC) is derived.Finally, there will be no common information in the individual subspace after two extraction steps, but it still contains quality unrelated information of different orders. Consequently,the data is further divided into high order and low order specific information(HS and LS).Therefore,the original multimode process data is separated into four subspaces(HC,LC,HS,LS)where different information can be monitored accurately.

The rest of this article is organized as follows. In Section 2, the quality-related methods Double layer non Gaussian model(DLNGM) technology is briefly introduced and the four subspaces will be given, respectively. The concept of HCnS will be explained in details in Section 3. The online fault detection method will be introduced for multimode processes in Section 4. The feasibility and superiority of the proposed method will be demonstrated through a numerical process and the TE benchmark in Section 5.Finally,in Section 6,some conclusions will be given to summarize the main contribution of this paper.

2. Preliminaries

In this section, DLNGM is briefly introduced to better understand the novel algorithm proposed in this article.

In our previous research, the DLNGM is proposed to cope with quality-related information with different orders where NGM and PLS are employed for high order and low order information [28].This method includes a supervised fault detection algorithm,where a double-layer structure is adopted. The part with highorder information is explained by the non-Gaussian regression method(NGM),while the rest part,indicating the low-order information, is described by PLS. SupposeXandYare the process variables and quality variables, respectively. According to previous work,the latent spaces of NGM can be obtained by solving the following optimization issue [29].

where wi,cianddiare the corresponding weight vectors and num-ber of latent variables, respectively.Irepresents the mutual information between two high-order latent variables where Ξxand Ξyare the corresponding whitening matrices, respectively.J（·） is the negentropy operator. After the high-order information is extracted, the low-order information is applied by using the traditional PLS model. In this way, the components with different order information is monitored by the two-layer structure shown as follows [28].

where TNGand UNGare the corresponding score matrices representing high-order information. PNGand QNGare the corresponding loading matrices. TGand UGare the score matrices of low-order information and PG, QGare the corresponding loading matrices.

3. HCnS Structure for Multimode Processes

The quality-related information with different orders are mainly considered in the multimode processes monitoring. Both high order information and low order information are involved in the HCnS construction. According to previous research, the NGM and PLS are adopted for quality-oriented high order information and low order information extraction [28]. That is the reason why this method is called as hierarchical CnS. Inspired by Zhang’s CnI construction, the common subspace is expected to be composed of the latent information from different modes. Therefore,different order information should be considered in the common and mode-specific information extraction. Basically, the subspace construction starts with common information. Then the residual is accepted as specific information within each single mode. In order to tackle quality-related information, the DLNGM is performed to draw out parts with different orders and put them into corresponding subspaces.The extraction order of each subspaces is shown in the schematic of the HCnS in Fig. 1. The original data space is separated into four different subspaces (HC, HS, LC, LS),indicating different order information in common and specific subspace, respectively. In the quality-related high order common space, the latent variables of quality should be involved. Hence,the corresponding loading vectors and weight vectors are designed to obtain the quality-related latent variables which will be introduced in the objective function of common information extraction.

Fig. 1. The schematic of HCnS.

Fig. 2. Monitoring results of fault 1 in the numerical example. (a)-(b) High & low order common information of HCnS, (c)-(d) High & low order specific information of HCnS,(e)-(f) Common & individual information of CnI-PLS.

First,assume that there areNmodes,the input variable data can be expressed asX（1）∈Rn×m,X（2）∈Rn×m, ...,X（N）∈Rn×mwherenandmare the number of samples and input variables,respectively.Accordingly, output variable data can be expressed asY（1）∈Rn×l,Y（2）∈Rn×l, ...,Y（N）∈Rn×l. Then the common score variablestcanduccan be defined as follows:

wherepcandqcare the common weight vectors of input and output respectively;α ＝ [α（1） α（2） ... α（N）]T, β ＝ [β（1） β（2） ... β（N）]Tare the corresponding input and output weight vector for all possible modei＝ 1，2，...，N, respectively.

Compared with low order statistics such as variance and covariance,the mutual information between the latent scores of process variables and quality variables are introduced as the objective function to extract the quality-related common information:

In order to obtain corresponding parameters,the following double iterations are carried out as follows:

(1) Random initialization α and β;

(3) Use particle swarm optimization (PSO) to obtain corresponding loading vectors pcand qc.

(4) Substitute pcand qcinto the objective function;

(5) Use particle swarm optimization to get α and β;

(6) Repeat step (2) - (6) until all parameters converge.

The parameters of objective function in Eq. (3) represent the high order information. According to our previous discussion, the rest part of the original data space still contains a large amount of low order common components which should be analyzed simultaneously. As a result, once the high-order common part is removed from the multimode process data,the remaining information can be written as follows:

Similarly,the common score vectorstc，Ganduc，Gcan be defined as follows:

whereXG（i）∈Rn×mindicates remaining information of each mode after high order common information extraction. pc，Gand qc，Gare the loading vectors of low order common information.αG＝ [αG（1） αG（2） ... αG（N）]Tand βG＝[βG（1） βG（2） ... βG（N）]Trepresent corresponding weight vectors. The PLS method is used to extract low-order common information and the objective function can be defined as:

Similarly, the double iteration algorithm is adopted to obtain the above parameters.

(1) Random initialize αGand βG;

(2) Calculate the weighted input dataand output data

(3) Use partial least squares method to obtain pc，G, qc，G;

(4) Substitute pc，G, qc，Ginto Eq. (7) and calculate αG, βGusing PSO;

(5) Repeat steps (2)-(6) until all parameters converge.

The common information of different orders is fully extracted through Eqs. (4) and (7), which means the remaining information mainly represents the specific information within each mode. In addition, the specific information of different orders still needs to be separated before it can be adopted for multimode processes monitoring. As a result, the remaining data of each mode can be defined as:

whereXI（i）andYI（i）represent the specific information of process data and quality data in theith mode. Suppose thatPI,PI，GandRI，Grepresent the loading vectors of high order information, low order information and residues in the specific subspace. These parameters are designed for each single mode and can be obtained using the DLNGM method.

The main steps of HCnS can be summarized as follows:

1. Normalize multimode training data;

2. Construct objective function in Eq. (4) and set up high order common subspace;

3.Calculate the remaining information in step 2)and extract the low order information through Eq. (7).

4. The high and low order information of the remaining part in step 3),can be derived using DLNGM.The specific part and the residues can be obtained.

In next section,online identification and fault detection of multimode processes will be introduced in details.

4. Multi-mode Online Identification and Fault Detection

In the HCnS structure, a series of common and specific models are designed for multimode processes monitoring. These models should be carefully selected according to current process status.Therefore, the online identification should be discussed so that appropriate models are adopted for fault detection. Fortunately,the common subspace of HCnS is shared by all possible modes.That means the common information of the multimode processes can be monitored using one uniformed model, which significantly reduces the computation load of online identification and fault detection. Assume that the current online test sample is defined as (xnew,ynew), two common score vectors of high and low order can be defined as follows:

wherexnew，Grepresents the remaining part of the test sample after the high order common information is extracted.In order to monitor the common subspace, the following two statistics are defined:

where Tc（i）and Tc，G（i）are the common weight vector matrices of high-order and low-order information of theith mode training data,respectively. On the basis of KDE method, the control limit ofstatistic can be estimated as follows:

whereKis the kernel function andˉh represents a smoothing parameter or bandwidth.indicates the corresponding scaled kernel function.As for the low order information,the Hotelling’sT2statistic is introduced and the corresponding control limit ofstatistics can be estimated as follows:

whereF（A，n-A）andArepresent the corresponding critical value ofFdistribution and the number of principal components,respectively.

The fault detection of HCnS structure is extended to assist fault detection,either in common part or specific part.In order to establish the monitoring statistics for specific information,the following score variables are defined:

Accordingly, statistics in each subspace can be defined as:

Similarly, theandstatistics represent corresponding high and low order statistics, and corresponding control limitsfs（x）andT2Ican be obtained as follows:

whereF（AI，n-AI）andAIrepresent the corresponding critical value ofFdistribution and the number of principal components, respectively.The online identification and fault detection can be achieved based on the above statistics and control limits.Suppose there areNoperating modes and the label of the test samples is unknown.It is noted that mode switching order is relatively stable in the real industry.On this basis,it is assumed that the online mode switching order is consistent with training data and the first test sample is normal.In the beginning,all possible models are employed to monitor the first test sample. The current system condition belongs to the mode that all relevant statistics are less than their corresponding control limits.After the initial mode is determined, subsequent online sample will be monitored. It should be mentioned that one test sample is normal only when its corresponding statistics are below all control limits. This test sample can be judged as a faulty one when the statistic with respect to two adjacent modes both exceed their corresponding control limits. The explicit procedure of online identification and monitoring can be summarized as follows:

Calculate the statistics of theith modeand corresponding control limitsfc（i）,fs（i）,

The current system condition is normal and no faults occurs only whenIf any statistic exceeds the corresponding control limit, calculate the statisticsand control limits of the next mode

Judge whether the statistics still exceed the control limits. If, the current system condition is normal and switched to the next mode.

A fault occurs in the system if step 4 is not satisfied.

If the system condition is normal,monitor the next test sample and repeat step 1-5.

The iteration ends when all available test samples have been tested.

5. Case Studies

In order to demonstrate the performance of the proposed method, two case studies including a numerical example and the TE benchmark are applied in the paper. For comparison, the CnIPLS method is considered.

5.1. Case study on a numerical system

In order to simulate the complex multi-modal processes, a numerical system with four modes based on the classical ‘‘Swiss roll”model is adopted and served as the training data.The mathematical model is as follows:

In order to simulate different order information,the initial input variable set is reconstructed as follows:

In the numerical system, 500 samples are generated for each distribution in Eq.(16), and a total of 1500 training samples have been collected. In the meantime, three faulty data sets, each containing 1500 samples are applied as test samples. The fault is introduced as follows:

Fault 1: In mode 1, a 0.4 step fault is added onz2from sample 201 to 500;

Fault 2: In mode 2, a ramp fault with a slope of 0.008 is added onz2from sample 701 to 1000, whereiis the sample number;

Fault 3:In mode 3,a Gaussian white noise with zero mean and unit variance is added onz2from sample 1201 to 1500.

The proposed method and CnI-PLS are both adopted to monitor fault 1,respectively.The monitoring results of the two methods are shown in Fig. 2.

In Fig.2,fault 1 can be clearly detected by using the corresponding common and specific statistics. The control limits of three modes can be adjusted by the confidence level according to the real situation of the process.It should be noted that fault 1 is not shown in the low order common subspace since the common information of fault 1 is totally extracted in the high order information.On the contrary, no common information is considered in CnIPLS, which leads to missing alarms in the common information subspace. That may bring trouble for subsequent fault localization and isolation.

In order to demonstrate the performance of the proposed method on drift detection, fault 2 is adopted where the ramp change can be seen as a slowly-changing fault. The monitoring results of two methods are shown in Fig. 3.

Fig.3. Monitoring results of fault 2 in the numerical example.(a)-(b)High&low order common information of HCnS,(c)-(d)High&low order specific information of HCnS,(e)-(f) Common & individual information of CnI-PLS.

In Fig.3,the ramp fault can be detected immediately when fault 2 is introduced using the specific information of HCnS.In contrast,fault 2 cannot be detected in either common or individual subspace of the CnI-PLS. Compared with the CnI-PLS, the detection performance of specific information is improved using HCnS since the efficient extraction of common information.The better capability of HCnS in extracting different order information leads to the better ability in detecting slowly changing fault.

Another fault is introduced in this case to demonstrate the monitoring performance of fault detection method under zero mean noise fluctuation. The monitoring results of two methods are shown in Fig. 4.

Fig.4. Monitoring results of fault 3 in the numerical example.(a)-(b)High&low order common information of HCnS,(c)-(d)High&low order specific information of HCnS,(e)-(f) Common & individual information of CnI-PLS.

Fig. 6. Monitoring results of fault 2 in the TE process. (a)-(b) High & low order common information of HCnS, (c)-(d) High & low order specific information of HCnS, (e)-(f)Common & individual information of CnI-PLS.

When fault 3 occurs, the sudden changes can be found in the statistics of two specific subspaces of HCnS. Similarly, the CnIPLS based monitoring algorithm seems not very sensitive to this fault and cannot detect corresponding faulty samples immediately.

The false alarm rates and detection rates of the two methods are listed in Table 1 and Table 2, respectively. Compared to CnI-PLS,the performance of HCnS is better since different order information is emphasized in both common and specific information.

Table 1 False alarm rates of HCnS and CnI-PLS in the numerical example

Table 2 Detection rates of HCnS and CnI-PLS in the numerical example

5.2. Case study of the TE process

The Tennessee Eastman process (TEP) is a simulated chemical process developed by Eastman chemicals Inc. The simulation is often used to test the performance of process control methods.For more details about the TEP,interested readers can refer to reference [30].

According to our previous work[31],28 typical process variables and 6 quality variables are selected for performance validation.Besides, a multi-mode process with three modes is designed as training dataset, with each mode containing 1000 samples. Hence a training dataset with the dimension 3000×34 is obtained.Similarly, another three datasets are generated as test samples, each containing 3000 faulty samples.The types of faults are as follows.

Fault 1: a step change in the D feed flow (stream 2) of the first mode from the 301st to 1000th online sample.

Fault 2:a ramp change in the D feed flow(stream 2)of the second mode from the 1501st to 2000th online sample.

Fault 3:a white noise added in the D feed flow(stream 2)of the third mode from the 2501st to 3000th online sample.

The monitoring results of HCnS and CnI-PLS for fault 1 are shown in Fig. 5. It can be seen that the step fault in mode 1 can be detected using both methods. However, due to the mixing of different orders, the common part of fault 1 cannot be well reflected in subplot (Fig. 5(e)) using CnI-PLS. Furthermore, several false alarms are found around the 2000th sample, which compromises the monitoring performance of CnI-PLS.

Fig.5. Monitoring results of fault 1 in the TE process.(a)-(b)High&low order common information of HCnS,(c)-(d)High&low order specific information of HCnS,(e)-(f)Common & individual information of CnI-PLS.

Fig. 6 shows the monitoring results of the two methods when monitoring the faults occurring in mode 2. In the result of HCnS,both common and specific can detect the early drifting of fault 2.The drifting trends can be detected without much delay.However,the monitoring statistics of CnI-PLS are not sensitive to the ramp fault and several false alarms can be found around the 1000th sample. Similar to the situation in the numerical example, different order statistics can capture more useful information for online monitoring.

In Fig. 7, the comparison of both methods are presented where the white noise is introduced to mode 3. The different order common and specific subspace in HCnS can detect more faulty samples than the two-layer structure in CnI-PLS.

Fig. 7. Monitoring results of fault 3 in the TE process. (a)-(b) High & low order common information of HCnS, (c)-(d) High & low order specific information of HCnS, (e)-(f)Common & individual information of CnI-PLS.

The false alarm rates and detection rates of two methods are listed in Table 3 and Table 4, respectively. Superiority of HCnS can be found in these tables, which validates the effectiveness of monitoring algorithm.

Table 3 False alarm rates of HCnS and CnI-PLS in the TEP

Table 4 Detection rates of HCnS and CnI-PLS in the TEP

6. Conclusions

For multi-modal processes, this paper proposes a hierarchical process monitoring method based on common and individual feature extraction method. The proposed method extracts common and individual features of multi-modes. Meanwhile, the highorder and low-order information of multi-modal dataset is obtained by combining the double-layer fault monitoring algorithm. The effectiveness of this method is verified by a numerical study and the TE process. Compared with previous research, the method performs well in fault detection of multimodal processes.The highly complex coupling relationship between variables can be fully considered, and high-order information can be extracted from data with unknown distribution characteristics. At the same time, the common and specific features of multimodal data can be extracted effectively, which makes fault monitoring of multimodal process more efficient and accurate.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgement

This work is supported by the National Natural Science Foundation of China (61903352), China Postdoctoral Science Foundation(2020M671721), Zhejiang Province Natural Science Foundation of China (LQ19F030007), Natural Science Foundation of Jiangsu Province (BK20180594), Project of department of education of Zhejiang province (Y202044960) ,Project of Zhejiang Tongji Vocational College of Science and Technology (TRC1904) and Foundation of Key Laboratory of Advanced Process Control for Light Industry (Jiangnan University), Ministry of Education, P.R. China,APCLI1803.