Congli Mei*,Yong Su,Guohai Liu,Yuhan Ding,Zhiling Liao

School of Electrical and Information Engineering,Jiangsu University,Zhenjiang 212013,China

1.Introduction

In fermentation processes,some important quality variables,e.g.,biomass concentration,are difficult to measure online due to measurement limitations such as cost,reliability,and long dead time.From the viewpoint of control and optimization,these measurement limitations may cause important problems such as product loss,energy loss,and undesired byproduct generation.Over the past decades,soft sensors have been widely used to tackle these problems,which provide frequent estimations of key process variables through those that are easy to be measured on line[1–4].

The most populars of tsenso methods are parti alleastsquares(PLS)[5,6],artificial neural networks(ANN)[7,8],and support vector machines(SVM)[2,9].Recent reviews of soft sensor methods can be found in[4,1,10].Usually,soft sensors are constructed based on process measurements easy to measure online.From the viewpoint of measuring,one of the main disadvantages of those traditional soft sensors is lacking information of precision.That restricts above-mentioned soft sensors in practical cases.Another important problem which cannot be ignored is that dynamic multiphase/multimode processes are wide in fermentation processes and cannot be modeled effectively by single data driven regression models[11],e.g.,PLS,ANN and SVM.Generally speaking,these processes may result in complexity and poor performance of single models.

Recently,arelative new machine learning method,i.e.,Gaussian process regression(GPR),has been developed,and began to be applied in soft sensor modeling[12,13].GPR is usually trained by optimizing the hyper parameters using the expectation maximization(EM)algorithm with the squared exponential covariance function which is commonly employed[14].This regression method has many useful features that distinguish it from other machine learning techniques,particularly in the field of nonlinear modeling,such as ability to measure prediction confidence,few training hyper-parameters and possibility to include prior knowledge into the model.It should be noticed that GPR soft sensors are constructed based on the assumption that process data are generated from a single operating region and followa unimodal Gaussiandistribution. However, for complexmultimode/multiphase processes, thebasic assumption of multivariate Gaussian distribution may not be metbecause of the mean shifts or covariance changes. Then Gaussian mixtureregression (GMR)was introduced to construct soft sensors for those complexprocesses [11]. Besides the process characteristics of multimode/multiphase, it cannot be neglected that fermentation processes are dynamicsystems. Conventional static soft sensors commonly rely on the assumptionthat processes are operating at steady states. It was pointed outthat based on static regression models applied in a dynamic process mayresult in complexity of modeling and large errors of estimates [15,16]. Recently,considering the merits of GPR and dynamics in processes, a dynamicGPR soft sensor was proposed to estimate biomass concentrationin a fermentation process [13]. In themodel, besides input measurements,delayed outputs are also fed back and used as regressors. However,regressors of the GPR soft sensor were selected heuristically from numerous alternatives[13].A systematic method of selecting regressors needs to be studied further.

For dynamic multiphase/multimode fermentation processes, thispaper presented a systematic method of constructing a dynamic GMRsoft sensor to overcome the abovementioned problems. How tooptimize the number of Gaussian components and select regressors arecrucial to the dynamic GMR soft sensor. The former is related to the numberof process phases/modes [11], and the latter can be represented usingthe order of the model. In the work, for simplicity and effectiveness ofmodeling the soft sensor, an iterative strategy was presented to optimizethe two parameters simultaneously. In the strategy, the Bayesianinformation criterion (BIC) commonly used in the field of modeling isemployed as an evaluation criterion to determine the structure of thesoft sensor model.

The remainder of this paper is organized as follows.The GMR and the expectation maximization(EM)estimation are introduced in Section 2.Then,in Section 3,we introduce the proposed dynamic GMR soft sensor and the iterative strategy.A numerical example and an application example are investigated together to verify the effectiveness of the proposed GMR soft sensor in Section 4.Finally,in Section 5,conclusions are made.

2.Introduction to GMR and Expectation Maximization Estimation

2.1.GMR

Assume X represents the space of the explanatory variables and Y is the space of the response variables.xis the input of training data(x∈X)andyis the ideal output data(y∈Y).For the givenxandy,the joint probability density is given as[17]

where,subsequently,the mean μjand covariance ∑jcan be divided into the input and output parts like the following

Eq.(1)shows that the relationship between the explanatory variables and the prediction value can be described by Gaussian mixture models(GMMs)where ?(x,y;μj,∑j)denotes the probability density function of the multivariate GMM. The parameters of this model include the number of the mixture componentsK,the prior πj,the mean value μj,and the variance of each Gaussian component∑j,which are represented as θ=(θ1,θ2,…,θk)with θj= （πj，μj，∑j）and the constraint

Similarly,the marginal probability density can be given as[17]

The global GMR function can be deduced by combining Eqs.(1)and(2)

with the mixing weight

The mean and the variance of the conditional distribution can be acquired in closed form by

The prediction given a new input can be obtained by computing the expectation over the conditional distributionfY/X(y/x)[17]

It can be seen that the weight functionwj(x)is not determined by the local structure of the data but the components of a global GMM.Therefore,the GMR model is a global parametric model with nonparametric flexibility.

2.2.EM algorithm for GMMs

To use a GMM,the unknown parameter set θ of probabilistic weights and model parameters of each Gaussian component should be estimated first.Commonmethods for this probleminclude themaximum likelihoodestimation (MLE) and EM algorithm. With a set of given data(X,Y)which s realized by estimating model parameters θ in Eq.(1).This process can e realized by maximizing the log-likelihood functionL(θk)which can e expressed as[18]

For the given training data,θ is calculated by maximizing this function with the EM algorithm in the iterative means.It includes two steps[19].

(1)E-step(expectation step)

wherep(s)(lk/xi)denotes the posterior probability of theith training sample within thekth Gaussian component at thesth iteration.

(2)M-step(maximum step)

whereare the mean,covariance,and prior probamponent at the(s+1)th iteration,respectively.

The detailed description of the EMal gorithmcanbefound in[18,19].Usually,the parameters of the GMMs should be initialized before using the EM algorithm.In this work,thek-means clustering method was employed to initialize the GMMs,which is commonly used to automatically partition a data set intokgroups.

3.Proposed Dynamic GMR Soft Sensor

3.1.Analysis of soft sensor modeling in dynamic fermentation processes

As known,for a complex biochemical process,it is very difficult to establish its accurate mathematical model.As a compromise,a soc alled gray box model based on its partial knowledge has been used in many cases[13,20].Also,it is difficult to derive a dynamic soft sensor model for a specified variable from the gray box model[20].

To overcome the problem,it is natural to use data-driven regression models in the field of dynamic soft sensors.For a dynamic model,regress or selection is a crucial problem related to the choice of states in a state space representation of the system,and they are chosen as finite-dimensional projections of past data[21].To simplify dynamic soft sensors,existing knowledge on bioprocess models can be used in modeling.In fact,most developed bioprocess models are represented as first order differential equations and discretized as a state space representation with current inputs,current states,and predicted states[13,20].Then,a specified state can be represented by a high-order deferentialequation with inputs,i.e.currentinputvectoru(k)and current measurement vector Ym(k).So a dynamic soft sensor can be written by a nonlinear difference equation as follows

where,y*(k)is the primary variable,pdenotes the order of the model.From the viewpoint of soft sensors,the dimensions of u(k)and Ym(k)have to be reduced by variable selection methods for simplification,e.g.principal component analysis(PCA)[22,23].

3.2.Basic structure of the proposed dynamic GMR soft sensor

Even though different types of soft sensors have been developed,most of them are based on the assumption that process data are generated from a single operating region and follow a unimodal Gaussian distribution.However,for complex processeswithmultiple operating conditions,the basic assumption was not met [11]. Compared with GPR, GMR performsbetter in soft sensor modeling for multiphase/multimode processesfor its nature of multiple models [11]. Therefore, in this work, we used aGMR model to construct a dynamic soft sensor. Considering Eq. (13),the structure of the dynamic GMR soft sensor was designed as Fig. 1. In the structure,two parameters have to be optimized for the best performance,the number of Gaussian components and the order of the model.

Fig.1.The structure of the dynamic GMR soft senor.

3.3.Iterative strategy for parameter optimization

In the field of modeling,BIC is widely used to evaluate models.The formula of BIC can be written as[24]

whereNandprepresent the number of observations and the number of parameters respectivelydenotes the residual variance,whichcanbe calculated as

whereyi*andyirepresent the computed and measured values of the primary variable respectively.

In this work,BIC was also used to optimize parameters of the proposed soft sensor.To optimize the number of Gaussian components and the order simultaneously,an iterative optimization strategy was designed.Two relative increment BIC criterions were used to stop iterative procedures,which can be written as follows

where,pdenotes the order,andlrepresents the number of Gaussian components.During iterations, the optimal number of Gaussian componentsand the optimal order are corresponding to the minimumvalue ofrelative increment BIC criterion. The step-by-step procedures of theiterative strategy are given as follows.

Step 1.Initializei=1,l=1 andp=0;calculateCi=BICl(p).

Step 2.CalculateClandCprespectively.

Step 3.IfCp≤Cl,letCi=Cpandp=p+1;otherwise,letCi=Clandl=l+1;leti=i+1.

Step 4.If|Ci-Ci-1|≤0.05,output the value ofpandk;otherwise

turn to Step 2.

4.Case Studies

4.1.Simulation case—Penicillin fermentation process

The Penicillin fermentation process is a common biochemical batch benchmark widely used to verify soft sensor and fault diagnosis algorithms.It is a typical nonlinear,multiphase,dynamic and non-unimodal process[25].A simulator of the process can be found at the website http://simulator.iit.edu/web/software.html.Fig.2 shows the flow sheet of the process.Detailed descriptions of the process can be found in[26].For different demands,the simulator provides several settings including the controller,process duration,sampling rates,etc.

In the fermentation process, 16 variables can be measured online.11 highly related variables selected as input variables are tabulated in Table 1[11].Biomass concentration is unmeasured online and chosen as the primary variable needed to be estimated. In this study, simulationtime was set to 400 h. To simulate a multimode process, substrate feedrate was coupled with a step signal starting at the 200th hour. A totalof 5 batches were simulated under normal condition for training softFig. 1. The structure of the dynamic GMR soft senor. sensors and one was simulated for testing. To evaluate the performance of soft sensors,the root mean square error(RMSE)[27]was common used(seeEq.(17)),which reflects the prediction accuracy of a soft sensor.

Fig.2.The flow sheet of the Penicillin fermentation process[26].

Table 1Input variables in the Penicillin fermentation process

By the proposed iterative optimization strategy, the optimal number of Gaussian components and the optimal order of the model can be achieved on training set.Table 2 gives optimized results of each iteration on training set.For comparisons,the indexes of RMSE of dynamic GPR soft sensors and dynamic GMR soft sensors with different orders are both given in Table 3.

In Table 2,the bestBICvalue of the dynamic GMR soft sensor is achieved at iteration 5 withBIC=-6.5413.So the optimal structure parameters of the dynamic GMR soft sensor were chosen as the corresponding parametersk=3andp=3.Ontestset,Fig.3givese stimated dynamics of the optimal dynamic GMR soft sensor and original measurements,where the blue line denotes the biomass concentration measurements,the red line represents the prediction results of the optimal dynamic GMR soft sensor and the black dotted line depicts the 95%confidence interval of estimates.From Fig.3,it can be observed that predicted values of the GMR soft sensor can track the measurements closely with small confidence intervals. For comparisons in detail,Table3 gives test results of the GMR dynamic soft sensors with different structure parameters in Table 2.It can be observed that the best value of RMSE is achieved with the optimal model(k=3,p=3).

For comparisons,the dynamic GPR soft sensors with the different order values which range from 1 to 5 were also trained on training set.Table 3 also gives the results of those dynamic GPR soft sensors on test set.From the table,it can be seen that the optimal dynamic GPR soft sensor appears withk=1 andp=3.It means that the optimal dynamic GPR soft sensor has the best ability of generation in this simulation case.Also,it can be observed that the RMSE value of the optimal dynamic GMR soft sensor is smaller than that of the optimal dynamic GPR soft sensor.Generally,it can be concluded that the dynamic GMR soft sensor has higher predicted accuracy and is more suitable to model multiphase/multimode fermentation.

To show the identification ability of the presented GMR soft sensor approach for phases/modes,Fig.4 provides the posteriors to Gaussian components.It can be observed that different local models switch at different phases/modes automatically.This means that the dynamic GMR soft sensor can treat process changes of operating condition;thus,its automation level is higher than the dynamic GPR soft sensor.

4.2.Industrial case—Erythromycin fermentation process

In this work,the industrial data were collected from a practical Erythromycin fermentation process in Zhenjiang medicine company,P.R.China[2].In the process,15 process variables can be measured online:time,pH,dissolved oxygen,dextrin flow,propanol flow,soybeanoil flow,water flow,air flow,soybean oil volume,propanol volume,dextrin volume,water volume,temperature,speed,and relative pressure.In each batch,180 groups of data points can be collected by automatic instruments.Process data were collected from 8 fermentation batches.Runs 1–7 are used as training set,and the remaining one batch is used as test set.Before constructing soft sensors,experimental data have tobe normalized.Moreover, their dimensions should be reduced by variableselectionmethods for simplification. In this research, the PCAmethodwasused to select variables [23]. Finally, five variables, temperature, pH,speed, relative pressure and dissolved oxygen, were chosen as secondaryvariables in soft sensormodeling. Biomass concentration, difficult to measureonline, was chosen as the primary variable in soft sensingmodeling.

Table 2BIC values of dynamic GMR soft sensors on training set in the Penicillin fermentation process

Table 3Comparisons of different dynamic soft sensors on test set in the Penicillin fermentation process

Fig.3.Biomass concentration of the Penicillin fermentation process.

Fig.4.Posteriors to modes 1–3 on test set in the Penicillin fermentation process.

By the proposed iterative strategy,the optimal number of Gaussian components and the optimal order of the presented dynamic GMR soft sensor can be achieved on training set.Table 4 gives optimized parameters and its corresponding BIC value of the dynamic GMR soft sensor of each iteration on training set.For comparisons,the RSME indexes of dynamic GPR soft sensors and dynamic GMR soft sensors with different para meters arebo thgiven in Table5.Fig.5gives predicted values of the dynamic GMR soft sensor and original measurements,where the blue line denotes the fitted biomass concentration measurements[20],the red line represents the prediction results of the dynamic GMR soft sensor and the black dotted line depicts the 95%confidence interval of estimates.Fig.6 gives the posteriors to Gaussian components.

From Table 4,it can be observed that the optimal dynamic GMR soft sensor achievesBIC=-3.5012 at iteration 6.Therefore,the optimal structure parameters of the dynamic GMR soft sensor were chosen ask=3 andp=4.From Table 5,it can be observed that the RMSEvalue of the optimal dynamic GMR soft sensor is the smallest one of theresults of dynamic GMR soft sensors,which ismuch smaller than that ofthe optimal dynamic GPR soft sensor. Thismeans the proposed dynamicGMR soft sensor has better ability of generalization. Also, it can be observedthat the RMSE index of the dynamic GPR soft sensor doesn't decreasesignificantly as that of the presented dynamic GMR soft sensorwhen the order of the model increases. As we know, a fermentation processis a multiphase process, which can be divided into three parts: lagphase, exponential phase and stationary phase. This process characteristicresults in the limitation of the dynamic GPR soft sensor with asingle regression model. Generally speaking, the presented GMR softsensor has advantage in tackling dynamic multiphase processes forthe nature of the multiple model structure of GMR.

Table 4BIC values of dynamic GMR soft sensors on training set in the Erythromycin fermentation process

Table 5Comparisons of different dynamic soft sensors on test set in the Erythromycin fermentation process

Fig.5.Biomass concentration of the Erythromycin fermentation process.

5.Conclusions

In this paper,a dynamic GMR soft sensor was proposed for biomass concentration prediction in fermentation processes.The basic structure of the dynamic soft sensor was determined after fermentation process model analysis.Two parameters,the number of Gaussian components and the order of the model,are crucial to the dynamic model.Therefore,an iterative strategy was developed,which can optimize the two parameters simultaneously.TheBIC,widely used in modeling,was integrated in the strategy to evaluate iterations.Due to the probabilistic mixture framework,the presented dynamic GMR soft sensor can run under different operating phases/modes with automatic switching between different local models.Both feasibility and effectiveness of the dynamic GMR soft sensor have been con firmed through a simulation benchmark and an industrial case.

Fig.6.Posteriors to modes 1–3 on test set in the Erythromycin fermentation process.

[1]P.Kadlec,R.Grbi?,B.Gabrys,Review of adaptation mechanisms for data-driven soft sensors,Comput.Chem.Eng.35(2011)1–24.

[2]G.Liu,D.Zhou,H.Xu,C.Mei,Model optimization of SVM for a fermentation soft sensor,Expert Syst.Appl.37(2010)2708–2713.

[3]H.Jin,X.Chen,J.Yang,L.Wang,L.Wu,Online local learning based adaptive soft sensor and its application to an industrial fed-batch chlortetracycline fermentation process,Chemom.Intell.Lab.Syst.143(2015)58–78.

[4]R.Luttmann,D.G.Bracewell,G.Cornelissen,K.V.Gernaey,J.Glassey,V.C.Hass,et al.,Soft sensors in bioprocessing:A status report and recommendations,Biotechnol.J.7(2012)1040–1048.

[5]R.Sharmin,U.Sundararaj,S.Shah,L.V.Griend,Y.-J.Sun,Inferential sensors for estimation of polymer quality parameters:Industrial application of a PLS-based soft sensor for a LDPE plant,Chem.Eng.Sci.61(2006)6372–6384.

[6]Z.X.Wang,Q.P.He,J.Wang,Comparison of variable selection methods for PLS-based soft sensor modeling,J.Process Control26(2015)56–72.

[7]L.Cui,P.Xie,J.Sun,T.Yu,J.Yuan,Data-driven prediction of the product formation in industrial 2-keto-L-gulonic acid fermentation,Comput.Chem.Eng.36(2012)386–391.

[8]K.Sun,J.Liu,J.-L.Kang,S.-S.Jang,D.S.-H.Wong,D.-S.Chen,Development of a variable selection method for soft sensor using artificial neural network and nonnegative garrote,J.Process Control24(2014)1068–1075.

[9]H.Kaneko,K.Funatsu,Adaptive soft sensor based on online support vector regression and Bayesian ensemble learning for various states in chemical plants,Chemom.Intell.Lab.Syst.137(2014)57–66.

[10]P.Kadlec,B.Gabrys,S.Strandt,Data-driven Soft Sensors in the process industry,Comput.Chem.Eng.33(2009)795–814.

[11]X.Yuan,Z.Ge,Z.Song,Soft sensor model development in multiphase/multimode processes based on Gaussian mixture regression,Chemom.Intell.Lab.Syst.138(2014)97–109.

[12]H.Wang,Gaussian process and its application to soft-sensor modeling,J.Chem.Ind.Eng.58(2007)2840–2845.

[13]F.D.Sciascio,A.N.Amicarelli,Biomass estimation inbatch biotechnological processes by Bayesian Gaussian process regression,Comput.Chem.Eng.32(2008)3264–3273.

[14]G.McLachlan,D.Peel,Finite Mixture Models,John Wiley&Sons,2004.

[15]Z.Fan,J.Cao,Y.Wei,Dynamic soft sensor modeling based on state detection and impulses response template,Chinese Control and Decision Conference 2014,pp.4031–4037.

[16]A.Bernstein,C.Stein,Enhancing dynamic soft sensors based on DPLS:a temporal smoothness regularization approach,J.Process Control28(2015)17–26.

[17]H.G.Sung,Gaussian Mixture Regression and Classification,Rice University,2004.

[18]P.Guo,M.R.Lyu,Software quality prediction using mixture model with EM algorithm,Proceedings of the First Asia-Pacific Conference on Quality Software,(APAQS 2000),Hong Kong 2000,pp.69–78.

[19]J.Yu,S.J.Qin,Multimode process monitoring with Bayesian inference-based finite Gaussian mixture models,AICHE J.54(2008)1811–1829.

[20]X.Dai,W.Wang,Y.Ding,Z.Sun,“Assumed inherent sensor”inversion based ANN dynamic soft-sensingmethod andits application inerythromycin fermentation process,Comput.Chem.Eng.30(2006)1203–1225.

[21]L.Ljung,Some Aspects on Nonlinear System Identification,2007.

[22]E.Zamprogna,M.Barolo,D.E.Seborg,Optimal selection of soft sensor inputs for batch distillation columns using principal component analysis,J.Process Control15(2005)39–52.

[23]J.R.King,D.A.Jackson,Variable selection in large environmental data sets using principal components analysis,Environmetrics10(1999)67–77.

[24]K.P.Burnham,D.R.Anderson,Multimodel inference:understanding AIC and BIC in model selection,Sociol.Methods Res.33(2004)261–304.

[25]X.Zhang,Y.Li,M.Kano,Quality prediction in complex batch processes with just-intime learning model based on non-Gaussian dissimilarity measure,Ind.Eng.Chem.Res.54(31)(2015)7694–7705.

[26]G.Birol,C.ündey,A.Cinar,Amodularsimulation packagefor fed-batch fermentation:penicillin production,Comput.Chem.Eng.26(2002)1553–1565.

[27]S.Calinon,F.D'Halluin,E.L.Sauser,D.G.Caldwell,A.G.Billard,Learning and reproduction of gestures by imitation,IEEE Rob.Autom.Mag.17(2010)44–54.