Jie Cai,Xiaoli Jiang,Yazhou Yang,Gabriel Lodewijksand Minchang Wang

Abstract A corrosion defect is recognized as one of the most severe phenomena for high-pressure pipelines,especially those served for a long time. Finite-element method and empirical formulas are thereby used for the strength prediction of such pipes with corrosion. However, it is time-consuming for finite-element method and there is a limited application range by using empirical formulas. In order to improve the prediction of strength, this paper investigates the burst pressure of line pipelines with a single corrosion defect subjected to internal pressure based on data-driven methods.Three supervised ML(machine learning) algorithms, including the ANN (artificial neural network), the SVM (support vector machine) and the LR (linear regression), are deployed to train models based on experimental data. Data analysis is first conducted to determine proper pipe features for training. Hyperparameter tuning to control the learning process is then performed to fit the best strength models for corroded pipelines.Among all the proposed data-driven models, the ANN model with three neural layers has the highest training accuracy, but also presents the largest variance.The SVM model provides both high training accuracy and high validation accuracy.The LR model has the best performance in terms of generalization ability.These models can be served as surrogate models by transfer learning with new coming data in future research,facilitating a sustainable and intelligent decision-making of corroded pipelines.

Keywords Pipelines;Corrosion;Burst strength;Internal pressure;Data-driven method;Machine learning

1 Introduction

Due to the advantages of high safety, high efficiency and low cost, pipelines are widely applied in the transpor‐tation of large quantities of oil and gas over long distances(Xie and Tian,2018). However, pipelines also face big challenges with the exploitation of oil and gas into deep and/or ultra-deep water(Cai et al.,2017;Chin et al.,2020).One of the biggest challenges is the corrosion defect,which may significantly compromise structural integrity and there‐by bring severe consequences to the safety of structures and environment(Macdonald and Cosham,2005;Mohd et al., 2015;Cai et al., 2018b, 2019). Therefore, it is neces‐sary to find effective ways to evaluate the integrity of the pipeline so that the risk of pipe leaking or rupture may be reduced.

There have been large interests in the effect of corro‐sion defects on line pipes subjected to internal pressure since the late 1960s(ASME B31G,2012).Researchers are trying to find an exact relation between corrosion defects and internal pressures. Thus, a large number of experi‐ments on the burst pressure of corroded pipelines were conducted. Cronin et al. (1996) evaluated assessment pro‐cedures of pipeline corrosion defects from pipe experi‐ments. Benjamin et al. (2000) carried out a burst test (9 specimens together) to investigate the behavior of a pipe‐line with long corrosion defects. Choi et al. (2003) pro‐posed limit load solutions for corroded pipelines of API 5L X65 material (Specification, 2004) through burst tests with machined pits.Astanin et al.(2009)conducted a pipe test to verify an empirical method for the burst pressure of corroded pipelines.Cai et al.(2018b,2019,2018a)studied the effect of a single corrosion defect on pipe strength based on both experimental tests and numerical simula‐tions.

As a result,a considerable number of empirical and semiempirical formulas(Cai et al.,2018b;ASME B31G,2012;DNV, 2017; Chen et al., 2015b;Amaya-Gómez et al., 2019)are well-established. Choi et al. (2003) proposed a simple formula for the burst prediction of X65 pipelines assuming that the pipe burst strength only relies on factors such as diameter to thickness ratio(D/t),corrosion depth to thickness ratio (dn/t) and the normalized corrosion length (ln/Rt).Chen et al. (2015a) proposed a formula for the burst pre‐diction of X80 and X90 pipelines through a small-size dataset. An extra parameter about the corrosion width to diameter (wn/D) has been considered. Cai et al. (2018b)constructed a similar formula for corroded pipelines under bending moments based on physical models and experi‐mental data.The benefit of such empirical formulas is ob‐vious, for instance, a rapid evaluation of corroded pipes by hands.

Recently,the low price environment of oil and gas moti‐vates industry to extend the useful productive life of cor‐roded pipelines (Rosen et al., 2016). Therefore, improved models and alternative methods for the prediction of resid‐ual burst strength of pipes are needed. The past few years have witnessed the booming of artificial intelligence (Xie and Tian, 2018; Bishop, 2006; LeCun et al., 2015). The successful applications of machine learning in different fields are revolutionary (Ghaboussi and Sidarta, 1998;Ling and Templeton, 2015; Fukami et al., 2020; Rafiei and Adeli,2017;Sen et al.,2019;De Masi et al.,2015;Gholami et al., 2020; Cai et al., 2021). Tremendous improvements in terms of training efficiency,learning accuracy and model performance are made thanks to the increase of computa‐tional capacity (LeCun et al., 2015). In the field of pipe‐line, De Masi et al. (2015) proposed a neural network model to assess the corrosion of subsea pipelines in terms of corrosion rate,metal loss,the area of loss and the corro‐sion numbers. Ossai (2020) built a corrosion growth model based on the historical operation parameters through a feedforward multilayer neural network. Abbas and Shafiee(2020) discussed the prediction method by using artificial intelligence for corrosion models. Chin et al. (2020) ini‐tially studied the burst pressure of corroded pipes based on a two-layer neural network.Gholami et al.(2020)predicted the burst pressure of high-strength carbon steel pipelines with gouge flaws using artificial neural network based on a dataset from finite-element analysis.Amaya-Gómez et al.(2016) generated equivalent dimensions of grouped corro‐sion defects accounting for defect interactions, using the SVM and K-mean methods. Liu et al. (2019) adopted su‐pervised machine leaning algorithms such as SVM and Random Forest to match corrosion features. Mattioli et al.(2019) stated that building pipeline digital twins with ma‐chine learning algorithms is necessary for the future safety of pipeline operations.

However, the application range of existing empirical models are limited, and the performance is not high due to a lack of generalization ability. For instance, a fitted model for pipelines with X65 material and a random shape corrosion can be hardly applied to X80 pipelines with a rectangular shape corrosion. In a foreseeable fu‐ture of digitalization for pipeline service, such empirical formulas from traditional methods may not serve this pur‐pose well.Alternatively, the finite-element method based on traditional physical mechanism is normally adopted for prediction, but it is too time-consuming. Therefore,the objective of this paper is to explore a novel data-driv‐en methodology for the prediction of the burst strength of corroded pipelines without too much domain knowl‐edge needed, which can overcome such drawbacks. In this research, only an external corrosion defect is consid‐ered. The supervised machine learning algorithms, in‐cluding ANN, SVM and LR, are deployed. All the data are found from literature and are properly arranged for utilisation. Note that one big problem for data-driven methods is the limited size of dataset in pipeline engi‐neering, which affects the generalization ability of pro‐posed models. In other words, the proposed models may still not be accurately used for unseen case predictions,which is an unavoidable drawback and will introduce bi‐as. Thanks to the transfer learning (Weiss et al., 2016),which is able to account for previous learned knowledge,the latest development and the successful results in the field of artificial intelligence are made. Thus, the current research work is meaningful and can be further served as surrogate models by transfer learning. Further work on the application of transfer learning on pipelines will be done based on the results of current research.

The structure of this paper is arranged as follows. In Section 2, we introduce the internal mechanism of differ‐ent data-driven algorithms. How to properly evaluate the trained models is described as well.In Section 3,the exist‐ing experimental data of corroded pipes found from litera‐ture are described. The characteristics of pipe corrosion and pipe experiments are illustrated. In Section 4, empiri‐cal formulas for corroded pipelines from DNV and ASME B31G are discussed. Afterwards, pipe data exploration is conducted in Section 5 so as to properly choose pipe fea‐tures for model training. In Section 6, prediction models are proposed through typical machine learning algorithms and comparison results are discussed. Possible intelligent application of the improved models in digital operation/monitoring of pipelines is analysed. Conclusions are drawn in Section 7.

2 Data-driven methodologies

Data-driven methodologies in terms of machine learn‐ing algorithms are generally used to construct prediction models.The typical algorithms,including ANN,SVM and LR, are deployed. These three models are chosen because of the popularity and widely used for regression problems in machine learning. Their strengths and weaknesses are introduced for a better application in practice. Evaluation strategies and how to improve trained models are discussed in order to obtain better performance.

2.1 Artificial neural network

ANN belongs to the category of DL(deep learning). Its successful application in practical engineering field has been revolutionary over the past years,which is motivated by the structure of biological neural circuits of human brains.This concept is commonly referred to MNN (multi-layer neural network), MLP (multi-layer perceptron) or DNN(deep neural network).ANN is constructed through connect‐ing of perceptrons in layers.As can be seen in Figure 1, a perceptron (Rumelhart et al., 1986) is defined as the mini‐mum unit of an entire neural network. Each perceptron is made up of many neurons, receiving and activating signal among different neurons.The weaknesses of applying ANN include complex training models, careful preprocessing of data and low-efficiency of training, whereas the strengths are high accuracy under limited dataset and demonstrated successful application in various engineering fields.

Given a set of training dataXand labelled targety:X∈R(nx,m),y∈R(m,ny), the output prediction (?) can be trained into a L-layer neural network based on their inner relation.In ANN,the values of units(Z)and their activation values(A) are computed based on a feed-FP (forward propaga‐tion),as expressed by Eq.(1).TheWis the training weight and thebis the training bias.The superscriptldenotes the layers of neural network (excluding the input layer, which is considered as the layer zero).

Figure 1 Basic architectures of neural networks

Note that the using of an activation function in hidden layers of neural network is crucial,which brings nonlinear effects for model training. Otherwise, the architecture of deep learning would be meaningless, making no differ‐ence with training by algorithms like linear regression.As seen in Figure 2, three basic types of activation functions are adopted in ANN(Haykin,2007),including Sigmoid(g(Z)=(1+e?Z)?1), ReLU (Rectified Linear Unit,g(Z)=max(0,Z)),and Tanh(hyperbolic tangent,g(Z)=(eZ?e?Z)(eZ+e?Z)?1).eis the Euler’s number.In general,the ReLU works better than the Sigmoid function for deep neural network (Nair and Hinton, 2010). However, it sometimes produces poor performance due to the discontinuous derivatives(Haghighat et al.,2020).The Tanh activation function can be then used accordingly. In this paper, only ReLU function is adopted in pipeline models.

Figure 2 Three typical activation functions (Haykin, 2007) including ReLU,Tanh,and Sigmoid

In order to evaluate the trained model with respect to target (burst pressure in this paper), a loss function (J) is customized for objective optimization, as seen in Eq. (2).The BP (backward propagation) (Nair and Hinton, 2010)is deployed to calculate the derivatives in Eq.(3).The pro‐cess procedure is denoted in Eq. (4), which provides de‐tailed expressions of all necessary derivatives. In order to break the symmetry of neural network architecture during training,the Xavier initialization(Glorot and Bengio,2010)strategy is deployed.

2.2 Support vector regression

Figure 3 Flowchart of the basic computation procedure of ANN using FP and BP for the training of pipe burst strength

SVM is considered to be one of the most powerful“black box” learning algorithms in conventional ML field(Bennett and Mangasarian, 1992; Cortes and Vapnik, 1995).By utilizing a cleverly-chosen optimization objective, it’s one of the most widely used machine learning methods nowadays (Fukami et al., 2020; Smola and Sch?lkopf,2004).The algorithm works well with sparse data but may provide a lower generalization performance for low-dimen‐sional data. SVR (Support Vector Regression) is an exten‐sion of SVM for regression problems with the same inter‐nal mechanism (Bishop, 2006). The mathematical theory of SVM is first described,which will be used for the train‐ing of pipe models in Section 6.

In order to demonstrate how SVM works, a linear bina‐ry classification problem is considered here. As seen in Figure 4 (a), it illustrates a training set with two classes,indicated by circles and crosses. The target of this prob‐lem is to find a boundary line(or a hyperplane for high di‐mensional problems)to properly separate all samples.The decision boundary should be evenly distributed between the two classes of data with the maximum margin, ex‐pressed asF(X)=WTX+b. Likewise, the learning process is to establish the functionF(X) between input data and out‐put target through mapping the training data into a high-di‐mensional feature space.The margin is defined as the dis‐tance between the decision boundary and the closest sam‐ples, expressed as (WTxi+b)/‖W‖ based on the distance formula from a point to a plane in the Euclidean space.All thexiare called support vectors which are samples to determine margins. Note that the numerator (WTxi+b) is always a constant which does not affect optimization re‐sults. Hence, a constant value of 1 could be used for sim‐plification in the distance formula.

Therefore,the task is to learn the bestWandbin a deci‐sion boundary so that the margin in Figure 4 (a) is maxi‐mum when subject toyi(WTXi+b)≥margin, ?i, as shown in Eq.(5).Xihere indicates the matrix of training samplei,yiis the signed values of +1 or ?1, representing different classes. Due to a reciprocal relationship, such optimiza‐tion problem is further simplified as a minimum problem when subject toyi(WTXi+b)≥1, ?i, as seen in Eq. (6). The concept of soft margin (Bennett and Mangasarian, 1992;Cortes and Vapnik, 1995) is used so that an extra term is added into the objective function of Eq. (6) for penalty, as shown in Eq.(7).

As an extension of the SVM algorithm, SVR has been introduced to handle the regression problems. Instead of predicting the category label of an input as the SVM does,SVR is used to predict some discrete values. Following the same idea of the SVM, SVR introduces anε-insensi‐tive region around the function, called theε-tube. Then,the regression problem is reformulated to find suchε-tube that it best approximates the continuous-valued function.More specifically,SVR first defines a convexε-insensitive loss function (subject to the constrainth(yi?WTf(xi))=max(0,|yi?WTf(xi)|?ε)) to be minimized and then searches for the flattest tube that contains most of the training instances,as shown in Figure 4(b).Thef(x)is the mapping function,h() is theε-insensitive hypothesis function (or extra loss)of predicted values and true values. Similar to the SVM,SVR can be used to regress a non-linear function, or a curve, through the use of kernel functions, the so-called“kernel trick” (Boser et al., 1992). Three kernels are used for the pipe model training in Section 6, including Linear,RBF, and Polynomial. SVR has great generalization capa‐bility and excellent prediction accuracy (Awad and Khanna,2015).

2.3 Linear regression

Figure 4 SVM illustration

In statistics(Downey,2011),linear regression is a linear approach to describe the relationship between one set of variables(or target)and one or more explanatory variables(or features).As seen in Eq. (8), it indicates a linear rela‐tionship between true value y and featurexi,where?is the residual,w0...wnare the weights of features. The objective function is,therefore,to minimize the loss function(MSE(mean square error)) in the form of the least squares so as to find the best weightWand biasb, as seen in Eq. (9).One of the biggest weaknesses to apply regression is that a strong assumption is needed in advance based on data structure and physical theory, largely reducing the training accuracy.For instance,the corrosion width along pipe length is often assumed to have little effect on burst strength.The major factors,including corrosion size and material rough‐ness, determine the resistance to material fracture failure and thereby the bust strength of pipes. With the increase of feature numbers, it is getting harder and harder to con‐struct a model in advance with reasonable assumptions.Note that non-linear approaches (e.g., polynomial regres‐sion) can be also used for the fitting of models. However,more features are needed when no reasonable assumptions can be made.As a consequence, more data are needed for better accuracy, which is not practical to use in this paper.Therefore, only the linear regression models by using least-squares approach are adopted.

In order to avoid overfitting,regularization penalty should be introduced with extra penality. For the linear regres‐sion with the penalized version of least-squares (L2-norm penalty),it is called Ridge regression(Hoerl and Kennard,1970), as seen in Eq. (10).With the increase of tuning pa‐rameterλ,the overfitting of model decreases. When using the Ridge algorithm,λis not sensitive and may be set to large value between 1 and 100.For Lasso regression(Tib‐shirani, 1996), it uses theL1-norm penalty, as seen in Eq.(11).WithL1penalty, the weight inWis possible set to be zero. Hence, Lasso takes care of model selection by re‐moving the least influential features.Theλin Lasso is sen‐sitive so that it normally set to be small, ranging between 0 and 1. Besides, the ElasticNet algorithm (Zou and Has‐tie, 2005) is used, which is a convex combination of the Ridge and the Lasso. It can reduce the error over Lasso method.

2.4 Evaluation and improvement strategies

The training models by supervised machine learning al‐gorithms generally face the bias-variance trade-off (De Masi et al., 2015; Haykin, 2007). The definition of vari‐ance is the variability of model prediction for a given data‐set, indicating the spreading of data. Bias is the difference between the predictions of trained models and the true val‐ues.In general,models with high variance pay much atten‐tion to training data.The generalization ability on the test/validation data which it has not seen before is not suffi‐cient. In other words, such models may perform very well on a training set but have high errors on a test set(overfit‐ting).For models with high bias,instead,little attention to the training set is paid. High errors occur on both training set and test/validation set.As a result,high bias causes the miss of relevant relations between features and target out‐puts,which is called underfitting.This evaluation criterion will be utilized for discussions among the trained pipe models in Section 6.

For regression problems, the coefficient of determina‐tion (R2) (Tabachnick et al., 2007) is used for the evalua‐tion of model accuracy,expressed asR2=1-SSreg/SStotal.The ratio of the sum squared regression error (SSreg) against the sum squared total error (SStotal) tells us how much of the total error remains in trained models. As a result, aR2of 1.0 indicates no error in the model, while aR2of 0 means that the model is no better than taking the mean value of data.A negativeR2is also possible, indicating that theSSregis greater thanSStotaland the prediction performance is even worse than using the mean value. The MSE can also be used as an indication of model accuracy.

In order to improve prediction models, strategies such as ensemble strategy (Naftaly et al.,1997) and transfer learning(Taylor and Stone,2009)can be used.The ensem‐ble strategy is to combine several improved models togeth‐er so as to produce a better prediction performance. The framework of transfer learning will accelerate the conver‐gence of machine learning using the current models as sur‐rogate models, especially for problems with large-size da‐taset.The utilization of transfer learning is crucial in engi‐neering fields where new samples are constantly added into.

3 Experiments data of corroded pipes

The pipeline corrosion is a natural process that happens when pipe materials interact with the working environ‐ment such as water and carbon dioxide. In this section,typical experimental tests of pipe burst strength are intro‐duced. The definitions of corrosion characteristics in terms of shape,orientation and fabrication workmanship are given.Recorded pipe data from tests are categorized and discussed.

Figure 5 shows a typical failure mode of corroded pipes in experiments (Astanin et al., 2009). The failure in terms of a crack-like rupture occurs along the pipe longitudinal direction in the corroded region. This is due to the plastic collapse that occurred locally in the corrosion ligament when the von Mises stress exceeds the ultimate tensile strength of the material through this region. For pipelines without significant defects, plastic collapse may occur by a global geometric instability of the specimen.As a result of the decreasing wall thickness and the increasing of pipe ovalization, the structure suffers an increasing of stress (Cai et al., 2017). For pipe test in laboratories, the length of each specimen is varied, generally ranging be‐tween 1.0 m to 2.3 m as observed from existing data. The specimens ends are sealed with very thick steel plate so that the boundary effects could be eliminated. Nozzles are deployed on end plates which are used for pressure pump‐ing.

Figure 5 The failure mode of the burst experiment of a corroded pipe specimen from Astanin et al.(2009)

Corrosion defects are usually fabricated by two ways in a pipe test. One way is the machining method (such as spark erosion(Freire et al.,2006)),the other way is purely natural processes. Figure 6 shows the definition of corro‐sion geometry. The maximum corrosion depth through a real nature process is simplified as the depth of defect(dn),while the maximum length of corrosion is simplified as the length of defect (ln). The corrosion angle (θn) is de‐fined as the angle between the pipe longitudinal axis and the corrosion length, ranging from 0° to 90°. Forθnin 0 deg, the defect is considered as longitudinal corrosion,while forθin 90 deg,the defect is considered as hoop cor‐rosion. For other angles between 0 deg and 90 deg, the defects are all defined as angled corrosion in this study.Multiple corrosions normally appear on the pipe surface.Under this circumstance, scanning is conducted and then combining them into one patch automatically with soft‐ware unless the corrosions are separated by a distance greater than six time of full pipe thickness as described by Cronin and Pick(2000).L1andL2in Figure 6 are parame‐ters that would be only used for the burst pressure estima‐tion by DNV-RP-F101(DNV,2017)in Section 4.

Tables 1 and 2 present the two types of features record‐ed in a pipe burst experiment including numeric feature and categorical feature from literature(Cronin et al.,1996;Benjamin et al., 2000; Choi et al., 2003; Mok et al., 1991;Freire et al., 2006; Cronin and Pick, 2000). Not all origi‐nal data and their features are listed here for the sake of clarity. Overall, 115 tests are obtained. Nine different line pipe materials are adopted based on API 5L (Specifica‐tion, 2004), including the grades of A25, B, X42, X46,X52, X56, X60, X65, and X80. The numeric feature con‐sists of the geometry of specimens,the geometry of corro‐sion defects and the material properties of specimen.Based on physical theory, failure stress is normally ex‐pressed as the function of such numeric features. Besides,the categorical feature includes corrosion shape, orienta‐tion, fabrication method, location, the grade of material and pipe type.For the corrosion shape which is not rectan‐gular,it is defined as“Irregular”.Only two types of corro‐sion defect method are defined: “Machined” and “Nature”.Properly selection and investigation of all these pipeline features are very important in developing a reliable, riskfree and cost-effective prediction method, which will be discussed in Section 5.

Figure 6 Geometry of the simplified corrosion defect in the shape of rectangular profile in rotational angle(θ)in this paper

4 Corrosion assessment in engineering standards

In this section, the estimation methods from engineer‐ing standards (DNVGL-RP-F101 (DNV, 2017), ASME B31G (ASME B31G, 1991) and the Modified B31G(ASME B31G, 2012)) for corroded pipelines are intro‐duced.Note that there are some limitations of using these empirical formulas. For instance, ASME B31G does not apply to the crack-like corrosion defects or mechanical surface damage with a coarse surface. The corrosion damage should be located at the pipe center,not affecting by the pipe seams or girth welds. These limitations may introduce possible discrepancies of the prediction in Sec‐tion 6.

4.1 Barlow’s formula for intact pipes

Barlow’s formula is widely used in the piping industry to compute the burst pressure (Pf) of intact pipelines with‐out corrosion subjected to internal pressure, as seen in Eq.(12).

Table 1 Numeric data recorded from pipe burst strength experiments

Table 2 Categorical data recorded from pipe burst strength experiments

4.2 Empirical formulas in DNV

In DNVGL-RP-F101 (DNV, 2017), the formula for the failure pressure of pipes with a single rectangular longitu‐dinally oriented defect subjected to internal pressure is proposed based on a large number of FE analyses, as seen in Eq. (13). The formulas may be applied to pipes with diameters ranging from 291.1 mm to 914.4 mm and the grades of pipeline material ranging from API 5L X42 to X65.For pipes with grades up to X80,however,these for‐mulas may not be used due to model limitation. Note that the formula cannot be applied to corrosion shapes other than a single rectangular shape in pipe longitudinal direc‐tion.The measured defect depths should be less than 85%of the wall thickness.

4.3 Empirical formulas in ASME B31G

Empirical formulas are also proposed in ASME B31G standard and its further modified version. ThePfof line pipes with medium strength and high material toughness can be expressed as Eqs.(15)and(16).The corrosion fea‐tures in terms of longitudinal length (ln) and depth (dn) are accounted for. Note that when the rotational angle is less than 45 deg, as seen in Figure 6, the corresponding corro‐sion length (ln) is changed toL1for calculation, otherwise,the most severe longitudinal section(L2)should be consid‐ered. As we have noticed, an extra relationln= 20Dtis roughly deployed to imply the severity of corrosion.Mis the bulging stress magnification factor, as seen in Eq. (17).For plain carbon steel operated at temperatures below 120 ℃,the flow stress(Pflow)may be defined as Eq.(18).

5 Data exploration for model training

In this section, the pipe experimental data described in Section 3 are statistically analyzed. All the data are split,cleaned and aggregated after careful investigation so as to choose proper pipe features for model training. A crossvalidation generator (Bishop, 2006) is used to randomly shuffle the pipe data,and to split the data into independent subsets for model training. Thus, the data are categorized into three different sets, including a training set (80% of all data), a validation set (10% of all data) and an extra test set(10%of all data).Note that,due to the limited size of dataset, only training dataset and validation dataset will be used for model performance evaluations in Section 6 to avoid bias.The accuracy from test dataset will be still pre‐sented as a reference.

5.1 Data exploration for the training by ANN

Data exploration is essential for the proper training of a machine learning model. As seen in Tables 1 and 2, the numeric features, such as pipe thickness and pipe outer diameter,can be directly applied on training after data clean‐ing.However,the categorical features such as pipe materi‐al grade need to be transformed before applying. Strate‐gies including label encoding and one-hot encoding(Bish‐op,2006)are used for the categorical features processing.

Feature importance should be determined so that we can properly keep and/or discard existing features for models.One of the strategies is to check the correlation coeffi‐cients between independent variables and the output target(pipe burst pressure(Pf)).It is defined as the covariance of two variables divided by the product of their standard deviations. The highly correlated features should be first selected.As seen from the heatmap in Figure 7,all the cor‐relations among features are visualized.The legend of col‐or denotes the level of correlation.The value close to 1 im‐plies a strong positive correlation between two features,while the value close to ?1 implies a strong negative corre‐lation.The value of 0 indicates no correlation relation.We found that the parameters such as corrosion length (ln),corrosion depth (dn) and pipe outer diameter (D) have a large negative correlation to the burst pressure, while the pipe thickness has a large positive correlation to the burst pressure.The specimen length (Lp) and the corrosion width(wn) have a very small correlation value to the burst pres‐sure.These findings coincide with the former assumptions about the insignificant effect of corrosion width on the burst pressure of pipelines (Cai et al., 2018b; Choi et al.,2003).Therefore, these two features (Lpandwn) are inten‐tionally removed, and are not used for model training in Section 6 due to the limitation of data size.

Figure 7 Correlation coefficients among pipe burst pressure and pipe test features

The one-hot transformation is performed to convert the categorical variables into numeric indicators. Either 1 or 0 is used to specify the presence of the variable attribute.For instance,the material type attribute of a pipe specimen with X60 material is designated as 1,whereas all the other material type attributes are assigned 0. Three categorical features, namely, corrosion defect method, material type and corrosion orientation, are selected for utilizing.Among these features, the corrosion defect method is categorized as machined corrosion,nature corrosion and intact without corrosion. The corrosion orientation is categorized as four types, including long type, hoop type, angled type and intact type.Other categorical features such as shape,notch location,pipe type are not considered due to the limitation of data size. Moreover, we noticed that there is a single sample with the material grade of A25, which has been intentionally removed. Note that the intact specimens are also added for training.

For the model training by ANN in this paper, we have obtained 21 features after a meticulous data analysis, in‐cluding material yield stressYS(σy), material ultimate ten‐sile stressUTS(σu), outer diameter (D), pipe thickness (t),corrosion length (ln), corrosion depth (ld), corrosion defect method (intact type, machined type, and nature type), cor‐rosion orientation (angled type, hoop type, intact type and long type), and material categories (B, X42, X46, X52,X56,X60,X65 and X80).

5.2 Data exploration for the training by SVM and linear regression

In order to improve the accuracy of learned models by SVM and linear regression algorithms, a data transforma‐tion is conducted in this section.Figure 8 shows the densi‐ty distribution of burst pressure and its KDE(kernel densi‐ty estimation). In this way, we can check the skewness of data, which is a measurement of the asymmetry of the probability distribution of features/target about their mean values.A right-skewed distribution which is concentrated on the left side of the histogram is observed, as seen in Figure 8(a).A skewness of 0.543 is computed for the pipe pressure. Hence, a logarithm transformation is performed to transform the data distribution into a normalized distri‐bution.Note that the corresponding inverse transformation is needed for the predicted burst pressure when using the learned model.

With the same data analysis and visualization strategies,other features of pipe tests are explored. It is found that the features such as pipe thickness (t), outer diameter (D),corrosion depth (dn) and corrosion length (ln) have large skewness (larger than 1.0). Thus, the Box-Cox (Box and Cox, 1964) power transformation is applied such that the non-normalized distributed data are all transformed into a normalized-like distribution. Figure 9 shows an example of the feature distribution under the situation before and after transformation, respectively. For clarity reason, only one example of the corrosion depth is illustrated.

6 Results and discussions

In this section, different models for the prediction of burst pressure of corroded pipelines are trained by the three algorithms, as discussed in Section 2. Hyperparam‐eter tuning is first conducted to find the best learning pa‐rameters. Meanwhile, a parametric study is performed to study the effect of critical learning parameters on model accuracy. Prediction results of all the experimental speci‐mens are computed and then compared through both standards and proposed numerical models. The accuracy of each learned model is discussed based on bias-vari‐ance trade-off. Note that all the numerical calculations,data analyses and model training in this paper are per‐formed based on open-source Python libraries, including NumPy, Pandas, Matplotlib, SciPy, Scikit-learn, Keras and Tensorflow modules.

6.1 Prediction results through engineering standards

The engineering standards, as discussed in Section 4,are first adopted for the prediction of test specimens.Only the data in training and validation set are deployed.Eleven intact specimens without corrosion are removed for results comparison. Note that there is no need to differentiate the prediction results in training set and validation set since the formulas in standards are not intentionally fitted by any of these dataset.

Figure 8 The distribution of burst pressure

Figure 9 The distribution of the corrosion depth before and after transformation,respectively

Figure 10 (a) shows the comparison results of pipe burst pressure between predictions and experiments. It shows that there are 37 out of 90 non-conservative predictions based on DNVGL-RP-F101 (DNV, 2017). Here, the nonconservative means that the prediction values by the em‐pirical formulas are larger than the real values from experi‐mental tests,which may cause unsafe consequence in prac‐tice. The largest discrepancy reaches as much as 33.09%for the specimen S.N.42(X60)in Mok et al.(1991)(among all specimens with longitudinal corrosion).The estimation of specimens with angled corrosion has a larger discrepancy,as shown from S.N.34 in Mok et al. (1991) with an error of 72.22%. For the predictions in ASME B31G (ASME B31G, 2012), only 5 out 90 specimens are non-conserva‐tive. The largest discrepancy reaches 39.1% for the speci‐men S.N.109 (X80) in Chauhan et al. (2009) (among all specimens with longitudinal corrosion). In the modified ASME B31G, 19 out of 90 specimens are non-conserva‐tive.The largest discrepancy reaches 30.01%for the speci‐men S.N. 8 (X60) in Benjamin et al. (2000) (among all specimens with longitudinal corrosion).

The empirical formulas also present different perfor‐mance in terms of different material grades. As seen in Figure 11,it graphically compares the distributions of pre‐dicted burst pressure of pipes with different material grades using box plots. Each category of pipes is displayed in a statistical way, including the minimum, the first quartile,the median, the third quartile and the maximum. Outliers are displayed. We found that the prediction discrepancies are relative large for pipes with material grades in lower strength (e.g., B and X42) when comparing with the stan‐dards results (P_DNV, P_B31G, and P_B31G_M) and the test results(P_EXP).

Figure 10 Comparison of pipe burst pressure between predictions and experimental tests

Figure 11 Distribution of predicted dataset of pipe burst pressure with respect to different material type(the labels of X ticks are shared)

A more refined comparison in terms ofR2and MSE is conducted to quantify the estimation accuracy of empiri‐cal formulas in standards. The corresponding prediction accuracy through empirical formulas is presented in Tables 3 and 4, respectively. As we can see, the overallR2are 0.765, 0.46, and 0.612 for DNV, B31G and B31GM, re‐spectively. The corresponding MSEs are 5.62, 12.92 and 9.28, respectively. The errors vary when it comes to indi‐vidual sets with different material grades.NegativeR2val‐ue occurs for the predictions of pipes with material grades of B and X42. Such poor performance implies the limited generalization ability of empirical formulas to new cases.The reason is mainly due to the small number of features when the formulas were proposed for practical use. Al‐though conservatism rather than accuracy is more con‐cerned in engineering practice for safety reason, these re‐sults imply that there is a space for existing engineering formulas to be improved.

6.2 Prediction results through ANN

In this section, the best prediction model is learned through an exhaustive grid search based on hyperparame‐ter tuning.A cross-validation generator is used to randomly shuffle the pipe dataset, splitting the data into 10 indepen‐dent subsets, including 9 sets for model training and 1 set for model validation.As a result,for every set of hyperpa‐rameter, it produces 10 training models with 10 different validation results. In order to improve the accuracy, the robust scaling method to normalize feature values is used before model training. Note that the test/validation dataset must use the identical scaling as the training set for predic‐tions.A parametric study about the effect of neural layers on model accuracy is conducted for the burst pressure pre‐diction with ANN.

Table 3 Coefficient of determination(R2)of prediction results through standards

Table 4 Mean square error(MSE)of prediction results through standards

The hyperparameter tuning is conducted on the model parameters in terms of optimizer types, initialization strat‐egies for weigh matrix,the number of epochs and the num‐ber of mini-batches. An overall of 960 fitting models are simulated in order to obtain the best tuned hyperparame‐ter. In addition, 50 models are trained based on a grid search in terms of the number of neural layers when fixing other hyperparameters. It is found that the best hyperpa‐rameters consist of three neural layers, 450 epochs (an epoch represents one iteration over the entire dataset), 2 mini-batches (the number of batches the dataset splitted),“normal” strategy for the initialization of kernel weights and “Adam” optimization algorithm. The corresponding best model for pipe burst strength prediction by ANN is trained accordingly. As shown in Table 5, theR2of three layers NN is 0.969, with the highest validation accuracy of 0.545.

For the parametric study on neural layers, five models are considered here,including a two-layer model(16,1),a three-layer model(16,8,1),a four-layer model(16,12,8,1),a five-layer model(16,12,8,4,1)and a six-layer model(16,12,8,4,2,1).The numbers inside parentheses are the corresponding number of neurons in each layer.As a result,each model contains the number of model features of 369,497, 669, 701 and 709, respectively. Figure 121Note that the legend Val_loss_L3 denotes the loss of validation set with 3 layers,while the Loss_L2 denotes the loss of training set.Others are similar.presents the diagram of loss values in terms of MSE with respect to the number of epochs during training, which indicates a good convergence for all the trained models by ANN.All the trained accuracy in terms ofR2and MSE is listed in Table 5 for comparison.With the increase of neural layers(from L2 to L6), the accuracy in training set (R2_train)increases as expected. However, the validation accuracy first increases, and then decreases with the increase of model complexity. It implies a high variance occurs. The best learned model with three neural layers (L3) for the prediction of pipe burst pressure has the best general per‐formance.

Table 5 Accuracy of learned models through ANN with different layers

Figure 12 The diagram of loss value in terms of MSE with respect to the number of epochs during ANN traning

6.3 Prediction results through linear regression

In this section, three types of regression methods ac‐counting for various regularization strategies including Ridge,Lasso and ElasticNet are utilized for the model training.

The learned linear regression model by Ridge algorithm is showed in Eq. (22), where the parameters with “Id”symbol are the categorical features after one-hot transfor‐mation (details are described in Section 5.1). When using this equation to predict a corroded pipe withX42 material type, for instance, the one-hot parameterIdX42will be set to 1 and all the other parameters within the material cate‐gorical feature (e.g.,IdX46) are set to 0. Other categorical features are similar.The best hyperparameterλforL2regu‐larization in the training model is 1.0. For the Lasso algo‐rithm,the corresponding learned weights are ?0.050,0.324,?0.459,0.469,?0.136,?0.339,?0.349,0,0,0.308,?0.032,?0.061, 0.001, 0, 0.012, ?0.066, 0, ?0.043, ?0.212, 0.009 and 0.002 with the bias of 2.745. The parameter sequence is the same as Eq. (22). The best hyperparameterλforL1regularization is 0.001 6. Likewise, for the ElasticNet algorithm, the corresponding learned weights are ?0.069,0.341,?0.462,0.480,?0.149,?0.357,?0.403,0,0,0.332,?0.077,?0.056,0.001 4,0,0.006,?0.086,0,?0.050,?0.206,0.016 and 0.021 with the bias of 2.747.The best hyperpa‐rameters of regularization are 1.0 and 0.000 9.As we have noticed, some features are automatically set to zero when adopting theL1regularization strategy in both Lasso and ElasticNet. Table 6 lists the different accuracy of learned models by applying different regularization penalty. The learned model by ElasticNet algorithm has the highest train‐ing accuracy with a high validation accuracy as well. The comparison results indicate a good performance of the pipe burst prediction models by just using a linear regression.

Table 6 Accuracy of learned models through linear regression with different regularization strategies

6.4 Prediction results through SVR

Three tuning hyperparameters in SVR, including regu‐larization coefficient (C), kernel width coefficient (γ) and the types of kernels, are used for grid search. Similar with the way in Section 6.2, a cross-validation generator is used to randomly shuffle and split data. As a result, an overall of 1 470 training models are trained in order to learn the best hyperparameters.The simulation results show that the best combined hyperparameters consist ofC=50,γ=0.01 and the RBF kernel (Gaussian transformation).The estima‐tion accuracy of models with different kernels is listed in Table 7.It is observed that the best model with RBF kernel has a good general performance accounting for both the validation and test performance.

Table 7 Accuracy of learned models through SVR with different kernels

6.5 Model comparisons and discussions

In this section, a comprehensive comparison of each learned model for the pipe burst strength prediction is per‐formed. The possible intelligent application of the improved models is explained.Discussions are presented in terms of model limitations.

Figure 10 (b) shows the comparison results of pipe burst pressure between the predictions and the experiments.The corresponding predicted strength is listed in Table 8. Only nine specimens are presented for clarity reason. It is obvi‐ous that the prediction scatter has been largely improved compared with the results from empirical formulas. For instance,B31GM provides a prediction with ?68.67%dis‐crepancy for sample S.N.9,whereas the SVR method gives a more accurate prediction with discrepancy of ?10.54%.Figure 13 explicitly illustrates the comparison results be‐tween predicted values through all the best learned models and the true values of experimental test.All the test speci‐mens are sorted with respect to their series numbers(S.N.).The experimental values of all specimens are connected by a black line so as to have a clear visualization. Predic‐tion results match very well with the experimental tests in general.As observed, there are outliers on specimen S.N.61 with material grade X65, with the relative prediction errors (|Predicted value-True value|/True value) of 26.1%,16.35% and 7.22% for the trained models by ElasticNet,SVR and ANN, respectively. The largest relative error occurs on the specimen of S.N. 11 with material grade B for all the learned models. The errors are 65.7% for the learned model by ElasticNet, 76.2% by SVR and 43.4%by ANN, respectively. The reason is due to the limited number of training pipes with grade B, which will be im‐proved in further research by transfer learning.

Table 8 Predicted values of pipe burst pressure from learned models

Figure 13 The comparison diagram between predicted values through learned models by supervised machine learning (three-layer ANN,SVR with RBF kernel and ElasticNet)and the experimental results

Figure 14 The diagram of learning accuracy for learned models with different machine learning algorithms

Figure 142Note that L6 denotes the ANN model with six neural layers, while It should be Li_R represents the Ridge model.Besides,it should be SVR_P denotes the SVR model using a polynomial kernel.Other signs are similar.shows the comparison diagram of learning accuracy in terms ofR2and MSE for both the learned models and the empirical formulas in standards. It is found that the learned models by the ANN algorithm have the highest training accuracy, but with a relative low valida‐tion accuracy. This phenomenon indicates that overfitting appears. Compared with the ANN models, the training accuracy of models by SVR and LR slightly decreases but with a relative high validation accuracy. As a result, the generalization of these training models has been improved.In addition,accounting for the accuracy results from test set,as seen in Tables 5, 6 and 7, the trained models by linear regression provide the best prediction performance among all the proposed models. Note that, due to limited data size, the test set has not been used in this work for model performance evaluation.The generalization ability of models should be further improved.

Compared with the improved models by machine learn‐ing algorithms, it cannot be denied that these empirical formulas from standards are easier to apply by hand due to less features and simpler formulas structures. However,these empirical models present a relative low performance in terms of accuracy and generalization. The DNV stan‐dard for corroded pipelines has a better performance than the ASME B31G. Meanwhile, the Modified B31G formu‐las have further improved the prediction ability of burst strength of corroded pipelines compared with the formulas in B31G.The developed models by machine learning algo‐rithms imply a promising utilization of intelligent predic‐tion in future, which provides a solid foundation for our further work.

The improved prediction models by data-driven method in this paper can facilitate an intelligent decision-making of corroded pipelines accounting for all the measured da‐ta.In the foreseeable future of digital operation in pipeline domain, these models can be easily integrated into soft‐ware for the burst strength prediction and real-time moni‐toring. A brief procedure of such potential applications is listed as follows.

? Integrate the prediction models into pipe monitoring software(ensemble strategy is used when necessary);

? Collect all the static pipeline features such as pipe di‐ameter,thickness and material level;

? Collect all the real-time corrosion features (angle and length,etc.)on pipelines by sensors or cameras;

? Automatically calculate the burst pressure and set an alarm when violating the prescribed safety factors.

The models in this paper have been initially demonstrat‐ed to be effective with good generalization ability.Howev‐er, it should be noted that model limitations still exist due to the limited number of training data. More cases are needed to further validate the accuracy of these models.In the future research, the transfer learning should be applied to improve current models when more data are found.Note that the yield stress and UTS of pipes in reality are not easy to obtain rapidly. Thus, these features could be re‐placed by parameters such as the SMYS (Specified Mini‐mum Yield Strength)from pipe material standards instead.Further research is needed to be done in order to introduce proper correction coefficients for such replacement.

7 Conclusions and future work

In this paper,a novel research on the burst strength pre‐diction of corroded pipelines has been conducted based on data-driven methodologies including three typical machine learning algorithms.Compared with the conventional meth‐ods, the work has provided an alternative research fashion on pipeline strength investigation. A single rectangular corrosion defect located on the specimen center was con‐sidered.Pipe data were obtained from existing experimen‐tal test of corroded pipe specimens and were explored care‐fully based on various data analysis techniques. Improved prediction data-driven models with high performance were proposed. Parametric studies on ML training parameters were performed.The prediction results from existing engi‐neering standards were compared to the proposed models.The conclusions of this paper are drawn as follows:

1) The ANN with three neural layers has the highest training accuracy for pipe burst strength based on limited dataset,but also with the highest variance.

2) The SVR provides a slightly lower training accuracy than the ANN with limited dataset.However,the high vari‐ance is largely improved.

3) The proposed pipe models based on limited dataset by LR present the best performance in terms of generaliza‐tion ability among the three algorithms.

4)Compared to the trained models by machine learning algorithms, empirical models from engineering standards show a relative low performance. Despite the fact that the empirical formulas are simple and easy to use by hands,there is still some space to be improved with respect to ac‐curacy and generalization ability.

Due to the limited number of data in engineering prac‐tice, the performance of trained models by data-driven methods will be inevitably affected. One way is to further collect effective data either from literature or conducting more pipe tests. Then, a re-training process is needed but is cumbersome and impractical.The other option is,instead of doing model re-training, to apply transfer learning for the improvement of the model performance. The improved models in this research can be therefore used as the surro‐gate models for transfer learning. Further research in this aspect is needed to be done.

Data availability statement

The original training data of corroded pipe experimental test and all the trained models are available from https://github.com/jiejie168/pipeBurstPressure_prediction_dataDriven