，，

College of Civil Aviation，Nanjing University of Aeronautics and Astronautics，Nanjing 211106，P.R.China

Abstract:Performance parameter prediction technology is the core research content of aeroengine health management，and more and more machine learning algorithms have been applied in the field.Regularized extreme learning machine（RELM）is one of them.However，the regularization parameter determination of RELM consumes computational resources，which makes it unsuitable in the field of aeroengine performance parameter prediction with a large amount of data.This paper uses the forward and backward segmentation（FBS）algorithms to improve the RELM performance，and introduces an adaptive step size determination method and an improved solution mechanism to obtain a new machine learning algorithm.While maintaining good generalization，the new algorithm is not sensitive to regularization parameters，which greatly saves computing resources.The experimental results on the public data sets prove the above conclusions.Finally，the new algorithm is applied to the prediction of aero-engine performance parameters，and the excellent prediction performance is achieved.

Key words：extreme learning machine；aeroengine；performance parameter prediction；forward and backward segmentation algorithms

0 Introduction

Due to the harsh working environment，large number of parts and complicated internal structure of aeroengines，performance degradation inevitably occurs during operation.Therefore，it is important to take some measures to get these signs of degradation in advance to get the rest of the engine’s life，hence the engine performance parameter prediction has been the research highlights.The physical model of engines and the correlation between several parts are complicated，so it is very difficult to establish the model to predict the parameters.In recent years，with the development of machine learning methods，data-driven prediction methods have attracted more and more researchers’attention.Databased methods do not require complex research models，and the accuracy of their predictions depends heavily on historical data.In the field of engine health detection，more and more data-based methods have been applied［1-3］，and these studies also prove that the correct use of data-based methods can effectively improve the accuracy of diagnosis.

Artificial neural network（ANN）［4］has excellent nonlinear mapping function and is intensely suitable for complex fault diagnosis，so it has been widely studied.Yuan et al.［5］used the long shortterm memory（LSTM）network for remaining useful life（RUL）prediction.Janssenset al.［6］adopt convolutional neural networks for fault diagnosis of rotating machinery.Qu et al.［7］used a stacked denoising auto-encoder（SDA）to solve aero-engine sensor fault diagnosis.

The extreme learning machine（ELM）is a new neural network training approach represented by Huang et al.［8］Due to its rapid learning speedand better performance，it has been widely utilized in classification，regression and other fields.Ye et al.［9］proposed to combine QR decomposition into an incremental ELM（IELM）to obtain QR-IELM in 2015.In 2017，the Gram-Schmidt process was introduced into IELM to obtain GSI-ELM［10］.Cao et al.［11］applied the entropy theory into ELM to obtain ATELM.Zhao et al.［12］also provided to suggest householder transformation and gave spins into ELM to accelerate its solution process.In addition to the improvement for the solution process，some other scholars combined ELM with other models to make improved methods suitable for certain problems.Nobrega et al.［13］combined the Kalman filter with ELM for regression problems and achieved positive results.Pacheco et al.［14］made certain breakthroughs by combining the restricted Boltzmann machine with ELM for classification problems.Anwesha et al.［15］combined the autoencoder（AE）with ELM to get a network with excellent performance.

Another group of scholars utilized the improved ELM in the field of engine fault diagnosis and achieved favorable effect.Jiang et al.［16］adopt multi-class Bayesian ELM（BELM）for engine gaspath fault diagnosis.Feng et al.［17］used multi-layer kernel ELM（KELM）for the aero-engine fault diagnosis.Lu et al.［18］employed the distributed ELM for engine fault diagnosis and achieved positive outcome.Similarly，Zhao et al.［19］employed a soft ELM for engine fault diagnosis.And more applications for ELM are utilized to estimate the RUL of complex machines like aeroengines［20-22］.

Among these improved methods，the regularized ELM（RELM），provided by Deng et al.［23］in 2009，performs superior to ELM in many problems.But RELM has poor prediction performance on the time series prediction problem of SINC function in experiment，which makes RELM not applicable in some practical scenarios.To address this issue，many scholars have studied in the construction of regularization terms and training algorithms.In construction of regularization terms，Luo et al.［24］proposed a L1-L2mixed regularization in 2016.Later，Li et al.［25］provided the Laplacian twin ELM，Yi et al.［26］represented a linear combination of several regularization terms and obtained an adaptive regularization term，and Inaba et al.［27］constructed a distributed regularized ELM in 2018.These RELM improvements have achieved excellent performance.In terms of training approaches，Ma et al.［28］utilized the Lagrangian algorithm to train ELM in 2019.Mahmooda et al.［29］used forward-backward splitting algorithm for training L1-RELM for the first time.In 2019，Song et al.［30］utilized the alternating direction method of multipliers（ADMMs）for L1-RELM training，and obtained the online ELM.

This paper contributes the forward-backward segmentation（FBS）algorithm［31］for the training of ELM with L2regularization（L2-RELM），adopts a new calculation method to determine the step size，and obtains an algorithm with fewer number of training iterations.On this basis，this paper further improves the solution mechanism so that the number of training iterations is reduced again and the accuracy is reduced in an acceptable range.This study successfully overcomes the shortcomings of RELM in SINC data set and other time series prediction problems.The two represented algorithms are used for engine performance parameter prediction.Both algorithms have achieved better prediction performance compared with RELM.

1 Related Work

1.1 ELM

Assume the existing data set to be trained is(x，t)with the dimension ofn×（m+N），wherexrepresents the sample input data，tthe sample output data，nthe sample number，mthe number of input features，Nthe dimension of the output data，andMthe number of neurons in the hidden layer.Besides，the connection weight of the input layer to the hidden layer of the ELM representsWwith the dimension ofM×m.The threshold representsBwith the dimension ofM×1.The hidden layer activation function is expressed asg(x)，and the hidden layer output can be expressed as

Next，the connection weight β of the hiddenlayer to the output layer is calculated.In the network training phase，the network output is known，that means，Tis known，βis to be calculated，and solvingβis equivalent to solving the optimal loss function as follows

The final solution ofβis not difficult to obtain by least squares method，shown as

whereH?represents the Moore-Penrose generalized inverse ofH.

1.2 L 2-RELM

From Eq（.2），the solution of ELM is a process of minimizing empirical risk，but this solution may report over-fitting problems in some cases.Deng et al.［23］combined structural risk terms to avoid over-fitting problems，and updated the loss function of ELM as follows

whereCis a regularization parameter，and the final solution of Eq.（4）is obtained according to the least squares method，shown as

whereIis an identity matrix whose dimension is the number of neurons in the hidden layer.Generally，the solution obtained by Eq（.5）is sparse and has better performance than the solution obtained by Eq（.4）.L2-RELM does not increase too much in computational complexity，so it is a perfect choice in practical applications.

2 Improvement of ELM

2.1 ELM combined with FBS

The FBS algorithm was developed by in 2016，which is applicable for the extremum problem of separable convex functions.FBS is generally adopted to calculate the following problems

wheref(x)，g(x)are all convex functions.It is not difficult to get Algorithm 1 to calculate Eq（.6）.

Algorithm 1Forward-backward splitting

While not converged do

End

In Algorithm 1，xis the solution to be calculated andτthe step factor.In general，in order to ensure the final convergence，the value of the step factor needs to meet the following condition［31］

whereL(?f)is the spectral radius ofATA，such as when

This study adopts the FBS algorithm to solve L2-RELM.At this time，the problem that FBS needs to solve is transformed to Eq（.4），which is split into the form suitable for FBS solution，shown as

Thus，the forward step in Algorithm 1 is calculated as follows

The backward step is calculated as follows

Eq（.11）can be calculated using the least squares method，shown as

This study solves L2-RELM using FBS（called FELM），which is summarized in Algorithm 2.

Algorithm 2FELM

InitializeC，β，τ，k=1，and set the maximum number of iterationsK

Whilek＜Kor not converged do

End

FELM performs better on some problems than L2-RELM.But on some time series problems，such as the shared bicycle time series data set in the UCImachine learning library［32］，the number of iterations of FELM is close to 5×103，and the time spent is close to 60 s.This is obviously not conducive to practical use.

2.2 FELM combined with adaptive step size determination method

In order to reduce the number of iterations of the FELM algorithm，this section introduces an adaptive step size determination method［33］in FELM（called AFELM）.The step size can be determined according to Eq.（7）in the first iteration，from the second iteration，thekth step size is calculated as follows

Through the above calculation process，two steps of Eqs（.15，16）are obtained，and the adaptive step size can be determined by

AFELM is summarized in Algorithm 3.

Algorithm 3AFELM

InitializeC，β，τ，k=1，and set the maximum number of iterationsK

Whilek＜Kor not converged do

End

Compared with the FBS algorithm，the adaptive FBS algorithm reduces the number of iterations significantly，and the solution accuracy can remain unchanged or superior.The performance comparison between the adaptive FBS algorithm and the FBS algorithm is extremely detailed in Ref［.31］.

2.3 Improved AFELM

It is found that the core guaranteeing the convergence of the FBS algorithm is the forward and backward steps，which mainly play animportant role in solving the current optimal solution.However，in FBS or the adaptive FBS，the forward step is just to provide a reference point for the backward step，and the backward step finds a closest advantage solution to the reference point.

This study considers more about the effect of the forward step on the final solution in each iteration，assuming that the solution obtained in the forward step isand the solution obtained in the backward step is.Combine a parameterαwith the solution of the forward step and the backward step，shown as

AFELM is improved by adding Eq（.18）to Algorithm 3.The improved one is called IAFELM，and is summarized in Algorithm 4.

Algorithm 4IAFELM

InitializeC，β，τ，k=1，and set the maximum number of iterationsK

Whilek＜Kor not converged do

End

The above omitted steps are the same as steps in Algorithm 3.

Whenαis taken as 1，Algorithm 3 is obtained.And the smaller the value ofαis，the faster the convergence speed is.But the accuracy of the experiment is found to decrease，so the value ofαis important.

In fact，the improvement made the ELM algorithm similar to using the gradient descent method to replace the least square method.This paper usesan improved FBS algorithm to solve the ELM problem.The FBS algorithm is essentially a type of gradient descent algorithm.The loss function of ELM is a typical convex function，so when the step size is selected appropriately，the gradient descent algorithm can guarantee the convergency.The selection of the step size has been given in Eq.（7），and the selection basis is from Ref.［31］，which has a detailed derivation process on convergence，so this paper does not repeat it.

Similarly，the adaptive step size used by AFELM is essentially two mature step sizes：The steepest descent and the minimum residual，which are used more in various gradient algorithms.But this paper introduces them to the ELM solution for the first time in the process，so the convergence of AFELM can also be guaranteed.The question about the convergence of the adaptive step size is also very detailed in Ref［.33］.

As for the IAFELM algorithm proposed in this paper，it is not difficult to see that there are not many changes in the selection of gradients，and the focus is on improving the solution of the weights of the actual output layer.AFELM and FELM can be regarded as the two limits of IAFELM，so their convergence problems can be guaranteed as IAFELM is a form between the two learning machines.

2.4 Determination of relevant parameters

The determination of the convergence condition also has a great influence on the performance of the algorithm.The convergence conditions of Algorithms 2—4 determine whether the difference between the solutions obtained by the two iterations is less than a preset threshold.The conditions are judged as follows

whereMrepresents the dimension of the solution andεa preset threshold.

Regarding the value of the parameterα，it is found from many experiments thatαtakes the step factor of each step to obtain superior performance and can also reduce the iteration time to some extent，shown as

2.5 Time ser ies pr ediction problem

The engine performance parameter prediction problem is actually a time series prediction problem.The time series prediction problem can be summarized as a given time seriesS={s1，s2，…，sn}.If we only consider single-step prediction，it is equivalent to constructe anm-dimensional vectorx i={si+1，si+2，…，si+m}and adopts this vector as the input to predict the next data，wheremrepresents the size of the time window.Multi-step prediction means that the outputting more than one datum is the extrapolation of single-step prediction.It can be seen that the time series prediction problem is a special kind of regression problem，or it can be considered as a function fitting problem.

3 Numerical Experiments on Public Data Sets

In order to verify the performance of the proposed algorithm，this section conducts comparative experiments on several time series data sets.The experimental objects include ELM，L2-RELM，AFELM and IAFELM.Since the training time of the algorithm FELM is too long，it is not considered for performance on the dataset.The performance comparison between the adaptive FBS algorithm and the FBS algorithm is described in great detail in Ref.［31］.The relevant information of several selected data sets is illustrated in Table 1.

Table 1 Data set settings

The data setAis a set of data generated by the SINC function，data setsB—Eare classic stock time series data sets selected from Yahoo Finance［34］，andFis a commonly used data set in the field of time series prediction.All experiments were performed on computer configuration information as follows：Intel（R）Core（TM）i5-4210U CPU@1.70 GHz/2.40 GHz；RAM 4.00 GB；Windows 10 64-bit，and the platform adopt for the experiment is MATLAB R2014a.

The number of neurons in the input layer of the neural network is set to the number of time windowsm（m=5），and the number of neurons in the output layer is set to the number of prediction steps.If single-step prediction is performed，it is set to 1，and set to the number of steps when performing multi-step prediction.The evaluation indicators are root mean squares error（RMSE）and mean absolute error（MAE），shown as

wherenis the number of test samples，tthe actual output value，andthe model prediction value.

The stop thresholdεis taken as 1×10-10.Equal considering structural risks and empirical risks，the regularization parameterCis taken as 1，the activation function takes the Sigmoid function，the input weight and the threshold are randomly determined，and all the data are normalized to［-1，1］in experiments.The number of hidden layer neurons is taken as 50 and the results are shown in Tables 2，3.

Table 2 RMSE and MAE for all experiments

Table 3 Training time and number of iterations for all exper iments

From Table 2，it can be concluded that the RMSEs obtained by AFELM and IAFELM are always better than those by ELM and L2-RELM on all datasets.And from Table 3，it can be gotten that although AFELM has obtained more predictive effects，the number of iterations and training time increase.While IAFELM is not as positive at predicting performance as AFELM，but it performs better in the number of iterations and training time，and the accuracy is reduced in an acceptable range.

First，the influence of the regularization parameterCon the prediction effect is studied，here the range ofCis［2-20，220］，and the varying diagrams of RMSE with the range ofC（2x）are illustrated in Fig.1.From Fig.1 we can see that，with the increase of the regularization parameterC，the performance of L2-RELM is firstly improved and then deteriorated，while AFELM and IAFELM can always maintain excellent performance and not sensitive to parameterC.

Fig.1 Influence of regularization parameter C on RMSE

Next，the influence of the number of neurons in the hidden layer on the prediction effect is studied.The number of nodes in the hidden layer is increased from 10 to 150，and experiments on data setsA—Fare performed.The varying diagrams of RMSE with the number of layer nodes are illustrated in Fig.2.It can be seen from Fig.2 that as the number of hidden layer nodes increases，the RMSEs of ELM and L2-RELM increase and generally decrease respectively，while the RMSEs of AFELM and IAFELM have a certain increasing trend.On some data sets，when the number of hidden layer nodes is low，ELM reports the optimal prediction effect，but the stability isextremely poor.When the number of neurons is increasing，the prediction effect of L2-RELM closes to that of AFELM and IAFELM.On some datasets，when the number of hidden layer nodes is large，the prediction effect of L2-RELM is optimal，but its performance is still not as positive as that of AFELM with fewer nodes.The prediction effect of AFELM and IAFELM is always better than that of L2-RELM when the number of neurons is small.

Fig.2 Influence of the number of layer nodes on RMSE

The above experiments consider single-step prediction.Next experiment studies the multi-step prediction，and prediction results with different numbers of steps are illustrated in Table 4.

From Table 4，we can see that with the increase of the number of prediction steps，the prediction effect is getting worse，but the prediction effects of AFELM and IAFELM are always not weaker than those of ELM and L2-RELM.

Table 4 RMSE under multi-step prediction

Finally，the influence of the number of training samples on the prediction effect is studied.The training samples are increased from 50%to 80%of the total，and the varying diagrams of RMSE with the percentageof training samplesareillustrated in Fig.3.

From Fig.3，it can be concluded that AFELM and IAFELM indicate superior predictive performance and outstanding stability when other conditions are the same，and ELM and L2-RELM sometimes illustrate better or comparable performance，but not stable enough for practical applications.

Fig.3 Influence of percentage of training samples on RMSE

4 Aeroengine Performance Parameter Prediction

4.1 Pr ediction on simulation data

Aeroengines have a harsh operating environment and complex components，so performance degradation is inevitable after an increase in their service life.This degradation is irreversible and will become a fault to a certain extent，seriously jeopardizing the operational safety of the engine.And there will often be some signs on the relevant performance parameters of the engine in the early stage of performance degradation.If engineers can get this information in advance，it is intensely effective to avoid engine failure.Aeroengines typically consist of an air inlet，a fan，a low pressure compressor（LPC），a high pressure compressor（HPC），a combustor，a high pressure turbine（HPT），a low pressure turbine（LPT），bypasses，and a nozzle.The simplified diagram of the engine is shown in Fig.4，and the performance parameters are collected by sensors attached to these components.NASS’Commercial Modular Aeronautical Propulsion System Simulation（C-MAPSS）［35］generates a set of turbofan engine performance degradation data sets，each of which is a set of multivariate time series，including engine unit ID，operation cycle index，three values indicating the operational settings and 21 sensor measurements contaminated by unknown noises.

Fig.4 Simplified diagram of aircraft engine

T he data-driven aeroengine performance parameter prediction method is mainly divided into the following steps：（1）Preprocess the historical data of the obtained performance parameters，including normalization and smoothing，（2）use the pre-processed data to train the model and obtain a mature model，（3）input the pre-predicted parameter data into the model obtained by the last step，and the model outputs the predicted valueof theperformanceparameter.

This section uses the algorithm contributed above to predict five important performance parameters of aeroengines.This study compares the prediction results with L2-RELM to illustrate the effect of the improved algorithm.The important parameters in the C-MAPSS data set selected for prediction are listed in Table 5［36］.

Table 5 Specification of the selected measurement sensor signals based on C-MAPSS

FD001 is the data under a class of working conditions in the C-MAPSS data set，including the“train_FD001”data set.This data set is the time series data of performance parameters of the engine from normal to degraded under a single operating condition，and is often used for the verification of aeroengine performance prediction algorithms.Performance parameter prediction is performed using“train_FD001.txt”，in which engines#1—#10 are selected as test sets，and engines#11—#100 are selected as training sets.Before the training，the data are denoised.The twenty-point moving average method［36］is used to smooth the time series raw data.

Based on the previous analysis，we can draw a conclusion that AFELM and IAFELM can have fewer hidden layers with better prediction effect，which is meaningful for saving space in practical applications.Therefore，when predicting engine performance parameters，the number of hidden layer neurons is reduced to 20，the time windowmis set to 5，and the stop thresholdεis set to 1×10-10.Because of the experimental results on the insensitivity of the new algorithm to regularization parameterC，we setCto be 1，take Sigmoid as the activation function，randomly determine the input weight and the threshold，and normalize all the data to［-1，1］in experiments.Resultsareillustrated in Table6.

Table 6 RMSE and M AE of par ameter pr ediction of engines#1—#10

It can be concluded from Table 6 that the two new algorithms perform better than L2-RELM on engines#1—#10.

This paper also provides a prediction of five parameters of engine#1，shown in Fig.5.It is not difficult to find that in the first 50 cycles of performance parameters，the fluctuations are relatively large，and the irregular fluctuation is a difficult problem in the time series forecasting problem，so the algorithm will perform poorly in the first 50 cycles.

Fig.5 Varying diagrams of different parameters and errors of engine#1 with cycle

Next，a simple experiment on the training and test time is carried out.The engine#8 with the smallest amount of data is selected in the test sample and the engine#2 with the largest data volume is selected to conduct the experiment.The training sample selection is unchanged，still using engines#11—#100，then test experiments are carried out on engines#8 and#2，respectively.The results on training time and test time are obtained，shown in Tables 7—9.

Table 7 Tr aining time s

Table 8 Test time of engine#8 s

Table 9 Test time of engine#2 s

Through Table 7 we can see that：Although AFELM can obtain the higher prediction accuracy，it often consumes more computing resources during training；IAFELM has a certain reduction in training time compared with AFELM，and the accuracy is reduced within an acceptable range.From Tables 8，9，we find out that the test time of the three algorithms is not much different，which means the three algorithms all have a faster calculation speed.It is in-tensely instructive to predict engine performance parameters with certain requirements on prediction accuracy and training time.

4.2 Pr ediction on real data

The above experiments are all performed on the simulation data.Although the algorithm performs well，the degradation of the predicted performance parameters is a exceedingly ideal state.Therefore，the performance parameters are predicted on the real data.The operating conditions in the real situation are overwhelmingly complicated，and the varieties in engine performance parameters are more complicated.Two typical performance parameters from the same model of Honeywell collected from an airline are selected，which are turbo turbine total temperature（EGT）and low pressure rotor speedN1，and their variation with cycle is shown in Fig.6.

Fig.6 Performance parameter variation with cycle

A total of 15 648 flight cycle parameters are recorded，but it is not difficult to find that the parameters of the first 5 000 flight cycles are relatively stable.The parameters of the 10 648 flight cycles fluctuate greatly.This paper selects the performance parameters after 5 000 flight cycles to predict.The data of the first 7 986 flight cycles are selected as the training samples，and the parameters of the 2 662 flight cycles are predicted.After normalizing and smoothing，the experimental results are shown in Tables 10，11 and Figs.7，8.

Table 10 RM SE of two parameters by three algor ithms

Table 11 MAE of two par ameter s by thr ee algor ithms

It can be seen from Fig.7 that within the prediction range of the test set of the entire EGT parameter，the prediction performance of the two new algorithms is better than that of L2-RELM at all peaks.At some troughs，the prediction performance of L2-RELM is better，but the two new algorithms can always perform similar to L2-RELM，which means AFELM and IAFELM even perform better in troughs.

Fig.7 EGT prediction results

It can be seen from Fig.8 that within the prediction range of theN1parameter test set，the two new algorithms are better than L2-RELM in all peaks and troughs.It is proved that AFELM and IAFELM always obtain the higher prediction accuracy.

Fig.8 N 1 prediction results

5 Conclusions

In order to solve the poor performance of L2-RELM in time series prediction under the condition that the regular parameters are not easy to be determined，this paper uses the FBS algorithm to solve L2-RELM.But it is found that the new algorithm costs more training time and needs multiple iterations through experiments.Therefore，this paper introduces an adaptive algorithmic iterative step size determination method to obtain the AFELM algorithm.On the basis of AFELM，an improved method for the solution mechanism of the forward and backward steps is proposed to further reduce the number of iterations.During the training time，experiments are carried out on several commonly used time series prediction data sets.Results show that the newly proposed algorithm is not sensitive to regularization parameters，and can obtain better prediction performance than L2-RELM at a faster training speed.The proposed algorithm is applied to the engine public data set for performance parameter prediction，and achieves better prediction performance than L2-RELM.In order to further compare the performance of the algorithm，the actual engine performance parameters of an airline are predicted，and results prove that the proposed algorithm still obtains good prediction performance.

AuthorMr.CAO Yuyuan received the B.S.degree from the College of Civil Aviation，Nanjing University of Aeronautics and Astronautics（NUAA），Nanjing，China.In 2009，he obtained the M.S.degree in aviation safety management jointly issued by the French National University of Civil Aviation and the French University of Aviation.He is currently a researcher in the College of Civil Aviation，NUAA.His current research interests include aeroengine fault diagnosis and machine learning.

Author contributionsMr.CAO Yuyuan proposed the idea and designed the experiment.Mr.ZHANG Bowen wrote the manuscript.Dr.WANG Huawei reviewed previous research.All authors commented on the manuscript draft and approved the submission.

Competing interestsThe authors declare no competing interests.