，，

College of Aeronautical Engineering，Civil Aviation University of China，Tianjin 300300，P.R.China

Abstract:In order to obtain the image of airframe damage region and provide the input data for aircraft intelligent maintenance，a multi-dimensional and multi-threshold airframe damage region division method based on correlation optimization is proposed.On the basis of airframe damage feature analysis，the multi-dimensional feature entropy is defined to realize the full fusion of multiple feature information of the image，and the division method is extended to multi-threshold to refine the damage division and reduce the impact of the damage adjacent region’s morphological changes on the division.Through the correlation parameter optimization algorithm，the problem of low efficiency of multi-dimensional multi-threshold division method is solved.Finally，the proposed method is compared and verified by instances of airframe damage image.The results show that compared with the traditional threshold division method，the damage region divided by the proposed method is complete and accurate，and the boundary is clear and coherent，which can effectively reduce the interference of many factors such as uneven luminance，chromaticity deviation，dirt attachment，image compression，and so on.The correlation optimization algorithm has high efficiency and stable convergence，and can meet the requirements of aircraft intelligent maintenance.

Key words：airframe damage region division；multi-dimensional feature entropy；multi-threshold；correlation optimization；aircraft intelligent maintenance

0 Introduction

With the development of aircraft intelligent maintenance technology，the image processing and modeling of airframe damage has increasingly become an important means of maintenance support.The segmentation of the damage region in the airframe damage image is the basis of this process.

Through analyzing a large number of common local damage of aircraft，such as pit，crack，corrosion，wear and so on，it can be seen that the pixels in the damaged region and the non-damaged region have obvious differences in color，gray level，brightness and other features，and these features are basically similar in the same region.Besides，most of the airframe damage has a certain affected range，which makes the morphology of the damage adjacent region change in different degrees.The existence of the damage transition region has a significant adverse effect on the accurate division of the damage region.

Multi-threshold segmentation method is a typical method to implement the damage region segmentation involving transition region.Numerous theoretical approaches have been proposed by scholars，and solutions have been given for applications in various fields.A multi-threshold image segmentation algorithm was proposed using the Bernstein polynomial to uniformly approximate histogram curve in Ref.［1］.A multilevel image segmentation based on the Tsallis entropy was proposed in Ref.［2］.Ref.［3］introduced multi-threshold image processing into the adaptive object selection.A fuzzy divergence multi-threshold image segmentation based on the standard deviation was proposed in Ref.［4］.A multi-threshold segmentation based on background subtraction was proposed in Ref.［5］and applied to animal tracking.A gradient and multi-threshold optimization was used to defect detection in Ref.［6］.A multi-threshold automatic segmentation method was proposed in Ref.［7］for field straw mulching detection.The image multi-threshold method was used to characterize the mechanical stability of latex concentrate in Ref.［8］.Ref.［9］presented a stochastic fractal search（SFS）with fuzzy entropy-based multilevel thresholding model for the proper segmentation of color satellite images.These methods represent image features by histogram，information entropy，gradient，divergence and so on，and realize target region division by the multi-threshold method.Most of these methods only consider single-image features such as gray or RGB value，while the appearance of aircraft damage often changes in color，shadow，texture and other aspects at the same time.Because of the single feature representation，some information is lost during damage region segmentation，which affects the segmentation and applicability.

Because of the shortcomings of the threshold segmentation method，such as high computation cost and long computation time，many researches combine the swarm intelligence optimization algorithm to achieve fast threshold optimization.A hybrid bio-inspired learning algorithm proposed in Ref.［10］has an acceptable compromise between its convergence time and its computational cost.Improved artificial bee colony algorithms were proposed in Refs.［11-12］to optimize multi-level threshold image segmentation with lower computational complexity.The modified flower pollination algorithm proposed in Ref.［13］uses a random position vector to improve exploration.The Cuckoo search algorithm in Ref.［14］is improved to search more efficiently in the global range with no gradient information.The modified moth-flame optimization used for multilevel threshold color image segmentation in Ref.［15］has strong exploration ability at the end of iteration.The modified grasshopper algorithm in Ref.［16］can balance the global and local search process through adaptive mechanism.The levy flight trajectory-based salp swarm algorithm was designed for multilevel thresholding image segmentation in Ref.［17］，which can maximize the efficiency of resource searches in uncertain environments.In Ref.［18］，local search operators were defined to improve the performance of hybrid salp swarm algorithm，combining with fuzzy entropy to realize image multilevel threshold.However，in dealing with multi-dimensional and multi-objective optimization problems，the swarm intelligence algorithm still has some shortcomings，such as low convergence accuracy，poor distribution of solution set，insufficient stability and long calculation time，which cannot meet the requirements of rapid and efficient aircraft intelligent maintenance.

The remainder of the paper is organized as follows.In Section 1 the multi-dimensional and multithreshold division method based on feature fusion is proposed.The correlation parameter optimization algorithm is designed for the airframe damage region division method in Section 2.Section 3 presents experimental results and analysis.Finally，the conclusions are drawn in Section 4.

1 M ulti-dimensional and Multithreshold Division Method Based on Feature Fusion

1.1 Multi-dimensional feature entropy division method

For a damage image with the size ofM×N，multiple feature information functions of pixel(x，y)are defined asf1(x，y)，f2(x，y)，…，fn(x，y)，all of which satisfyfi(x，y)∈[0，L-1]，where 1≤x≤M，1≤y≤N，andLis the feature level of damage image.

There is an-dimensional feature threshold vectorT=(T1，T2，…，T n)divided the pixels of damage image into two categories.

（1）Target class

（2）Background class

Lethi1i2…indenote the frequency that the feature function corresponding to any dimensionj(j=1，2，…，n)satisfiesfj(x，y)=ij，which is abbreviated ashi1n.Then the joint probability density isp i1n=.

The target class feature entropy of thekth-dimensional andT kth-feature level is defined as

The background class feature entropy of thekth-dimensional andT kth-feature level is defined as

Thus the feature entropy sum of thekth-dimensional andT kth-feature level is

The feature entropy sum of target class and background class is

where

For multi-dimensional feature entropy division method，when the feature entropy sumΦ(T)is taken as the maximum value，the optimal thresholdT*of overall image segmentation can be obtained.

1.2 Multi-dimensional and multi-threshold division method

Due to the existence of damage transition region，the damage region can be accurately divided by the multi-dimensional and multi-threshold division method.

For the damage image with the size ofM×N，assuming that there aremn-dimensional threshold vectorsT r=(T r1，T r2，…，T rn)， wherer=1，2，…，m，the pixels of the damage image are divided intom+1 categories.

Therth class is Classr={(x，y)|T r1＜f1≤T(r+1)1，…，T rn＜f n≤T(r+1)n}.

Therth class feature entropy of thekth-dimension is defined as

Thus the feature entropy sum of therth class is

The characteristic entropy sum of theR+1 classes is

For multi-dimensional and multi-threshold division method，the optimal threshold(T*1，T*2，…，T*m)should meet

For the multi-dimensional and multi-threshold division method based on feature fusion，the color，brightness，texture and other feature information of the damage image are fused to be the judgment basis for damage region division by the definition of multi-dimensional feature entropy.And combined with multi-threshold，it is possible to finely distinguish the complex morphological changes of the damage adjacent regions，reduce the confusion introduced by the damage adjacent regions，and improve the accuracy of damaged region division.

2 Correlation Parameter Optimization Algorithm

In the practical application of damage region segmentation，the definition of multi-dimensional features and the determination of the number of thresholds mostly depend on professional knowledge，which affects the performance of the division method.When the threshold dimension and number increase，the time complexity of the multi-dimensional and multi-threshold division method increases exponentially，which is difficult to meet the rapid and efficient demand of aircraft intelligent maintenance.

In order to solve the computational efficiency problem of the multi-dimensional and multi-threshold division method，the optimization algorithm can be introduced to select the optimal threshold efficiently.

Through the analysis of the method，it can be seen that there are correlations among the multi-dimensional entropy thresholds，which need to be optimized in turn.

For themn-dimensional feature entropy thresholds，the optimization process is to determine points inmn-dimensional space to satisfy Eq.（8）.The process of the correlation parameter optimization algorithm is shown in Fig.1，and the steps are as follows：

Fig.1 Process of the proposed method

Step 1Initializing the algorithm parameters.

The parameters of the correlation parameter optimization algorithm include the grid sizes，the number of the initial threshold（except the first dimension）c0，the number of the first dimensional initial thresholdc1，the optimization stept，the discrete ratiod，and the cycle timesNc.

Step 2Gridding the optimization space.

Then-dimensional optimization space is discretized according to the grid sizes.

Step 3Generating the coordinates of initial thresholds.

The initial threshold coordinates are generated randomly according to the association logic：Classr={(x，y)|T r1＜f1≤T(r+1)1，…，T rn＜fn≤T(r+1)n}.

In the first dimension，a certain number（c1）of valuesv1j(j=1，…，c1)are determined randomly within the value range.In the second dimension，for eachv1j，the number（c0）of valuesv2jare determined randomly betweenv1jand its maximum，and a total ofc1×c0values are obtained.Similarly，in theith dimension，for eachv(i-1)j，the number（c0）of valuesv ijare determined randomly betweenv(i-1)jand its maximum，and a total ofc1×values are obtained.This process continues to thenth dimension so as to generate the number（c1×ofn-dimensional initial thresholds’coordinates(v1j，v2j，…，v nj)，wherej=1，2，…，c1×

Step 4Calculating the sum of initial feature entropy.

Taking the coordinates of the thresholds as input，the feature entropyΦjand the maximum valueΦmaxare calculated according to Eq.（7）.

Step 5Local optimization.

For each threshold coordinate，starting from the first dimension，the feature gradients of 2ndirections are calculated for the current dimensionvij.The direction with the largest feature gradient is selected as the optimization direction ofv ij，and taking a unit steptforward to generate a new threshold coordinate.Calculate the feature entropy sumΦ'jof the new coordinate according to Eq.（7）.

Compare the feature entropy sumΦ'jof the current coordinate with the maximum valueΦmax.IfΦ'jis greater thanΦmax，then continues to take a unit steptforward along the optimization direction，generate a new threshold coordinate and calculate the sum feature entropy.The cycle stops untilΦ'j≤Φmax，and the next dimension value of the current threshold coordinate is processed.

Step 6Local discretization.

Select several threshold coordinates with the larger feature entropy sum randomly，then select several dimensions of each threshold coordinate randomly.Discrete the selected dimension value of the selected coordinate according to random directions and random distances，then generate new threshold coordinates.

Repeat Step 5.

Step 7Global discretization.

Calculate global discrete quantityc1×d%.

According to Step 3，generated%new threshold coordinates by global discretization.

Repeat Steps 4，5 to calculate the feature entropy sum，the maximum valueΦ'maxand the corresponding threshold coordinates of global discrete coordinates.

Compare the global discrete maximumΦ'maxwith the maximum valueΦmax.If the global discrete maximumΦ'maxis greater than the maximum valueΦmax，Φmax=Φ'max.Repeat Steps 6，7 until the number of cyclesNcis reached or there is no new maximum inNc/2 cycles.

Step 8Damage region division.

The damage region image is divided by the threshold coordinate corresponding to the maximum valueΦmax.

The proposed algorithm can be applied to the optimization problem of multiple variables with association relationship.Compared with the traditional optimization algorithm，the association judgment is mostly carried out in the optimization process or after optimization，which will lead to unstable optimization time，ignoring the optimal solution and so on.By defining association rules，the correlation parameter optimization algorithm delimits the effective solution space.On the basis of reducing the necessary optimization space，it avoids the invalid operation that does not conform to the association logic rules，and improves the calculation efficiency.Therefore，the algorithm takes association rules as the premise of random location，which can accelerate the optimization process and solve the complex problem of multivariable association optimization withinO（nlogn）time complexity.Meanwhile，the local discrete process and global discrete process are set to improve the global search ability of the algorithm and avoid the sensitivity of the initial value.

3 Experimental Results and Analysis

Six Instances 1—6 are given to verify the multidimensional and multi-threshold airframe damage region division method based on correlation optimization.The images of Instances 1—6，i.e.，cracks in airframe surface，bird strike damage at wing leading edge，skin surface patch cracks，lightning ablation，exfoliation of skin paint layer，and pit at vertical tail leading edge，are shown in Fig.2，and the image features of each instance are shown in Table 1.

Fig.2 Airframe damage image instances

3.1 Division effect verification

In order to verify the division quality of the proposed method，the 1-D single-threshold division method is selected for comparison.In the selected method，the gray entropy is used to divide the damage region of instances.In the method，two feature information functionsf1(x，y)，f2(x，y)are used.f1(x，y)is the chromaticity value at the pixel(x，y)andf2(x，y)is the maximum luminance difference in the neighborhood.The calculation results are shown in Tables 2，3，and the result images of each instance are shown in Fig.3.

Fig.3 Damage region division images

Table 1 Instance image features

Table2 Calculation resultsbased on 1-D singlethreshold division method

Table 3 Calculation results based on the proposed method

Through the verification results，it can be seen that the operation speed of the 1-D single-threshold division method is faster.The computation of both methods can be completed in milliseconds.The operation speed of the proposed method is acceptable.However，the division effect of the 1-D singlethreshold division method is not ideal.In the original image of Instance 1，the boundary of damage region is clear，and the chromaticity contrast between the damage region and non-damage region is obvious.There are several obvious cracks，and the damage adjacent region is small.After the division by the 1-D single-threshold method，lots of non-damage regions still exist，and crack damage could not be divided accurately.The division result of the proposed method shows that the damage region is accurately divided with intact boundary，clear divided crack，good consistency and less noise.The nondamage region introduced by 1-D single-threshold method is excluded.

In the original image of Instance 2，the damage is composed of two parts，namely the skin fragmentation of wing leading edge and the peeling off of skin due to bird strike.The damage form is complicated with hollow damage structure.Adjacent region usually does not exist in the foreign object damage，but some bird strike stains and a few feathers around the damage region of the image have some influence on the damage region division.After the division by the 1-D single-threshold method，the information loss of damage region is serious，and lots of non-damage regions are introduced.The division result of the proposed method shows that the damage region is completely divided with few noise.The boundary of the two damage parts is clear.Obviously，the bird strike dirt and the complex structure have little influence on damage region division.

In the original image of Instance 3，the boundary of patch cracks is obvious，but a large range of pitting corrosion appears on the skin of damage adjacent region，which greatly interferes the damage region division.The division based on the 1-D singlethreshold method could not distinguish the noise of adjacent regions，and is poor in division effect.The division result of the proposed method shows that the damage region is clear，the details of the rivets and fractures in the damage region are obvious and the boundary is complete.A little noise is introduced by the adjacent region.

In the original image of Instance 4，the ablation region is concentrated，the adjacent region is small，and the chromaticity contrast of non-damaged region is obvious.However，the indiscernible luminance change introduced in the image acquisition process and the information compression in the image transmission process make adverse effect on division，so that a large number of non-damage regions are introduced into the division results of the 1-D singlethreshold method.However，the divided ablation region of the proposed method is clear and its boundary is complete.The luminance change and information compression of original image do not interfere with the division.The non-damage region introduced by 1-D single-threshold method is excluded.

Instance 5 is the image of exfoliation of skin paint layer，which contains many damage regions with clear boundary and slight morphology change around damage.But dust adhering to the lower part of the skin overlapping structure greatly influences the division.The damage region divided by the 1-D single-threshold method could not be divided completely and lots of non-damage regions are introduced.Through the proposed method，many damage regions in the image are divided clearly，and the boundary is coherent.It avoids the interference of dust adhering to the lower part of skin overlapping structure，and eliminates the non-damage region existed in the 1-D single-threshold division result.

The original image of Instance 6 has certain luminance changes at the pit，the damage region has no obvious boundary，and the interference of manual marking is existed.The damage region divided by the 1-D single-threshold method is not accurate，and lots of non-damage regions with similar color are introduced.Through the proposed method，the complete pit damage region is obtained，and the interference of influence change and the color similarity to the division is reduced.However，due to the obvious manual marking in the original image，there is still a small amount of marking information in the divided image of the damage region.

It can be seen that in the actual aircraft inspection and maintenance，the acquired damage image may be affected by the external factors such as uneven luminance，chromaticity deviation，dust adhesion，image compression and so on.For the morphological diversity of damage area and adjacent area，the proposed method eliminates these interferences，and the division effect is better.

3.2 Optimization effect verification

In order to verify the effect of the correlation parameter optimization algorithm，the bacterial foraging optimization（BFO）algorithm is selected for comparative analysis.As shown in Table 4，in order to ensure the fairness of algorithm comparison，the corresponding parameter settings of the two algorithms are consistent.The results of BFO algorithm are shown in Table 5.

Table 4 Parameter setting of optimization algorithm

Table 5 Calculation results based on the BFO algor ithm

It can be seen that a large number of optimization points generated by the BFO algorithm do not conform to the association logic.In the BFO optimization process of each instance，the proportion of invalid optimization points reaches more than 75%.And there are also a certain number of edge overflow points.These invalid points cause a waste of computing resources and time.

From Tables 3，5，it can be seen that the proposed algorithm has certain advantages in computing time compared with the BFO algorithm.The proposed algorithm takes the defined association rules as the basis for the generation of optimization points，which avoids the occurrence of logical inefficiency and improves the calculation efficiency.

In the following，ten groups of operations are performed on each instance to further verify the correlation parameter optimization algorithm.The convergence processes of feature entropy sum are shown in Fig.4.The deviation of feature entropy sum of each group is shown in Table 6.

Fig.4 Convergence process of ten groups

As can be seen from Fig.4 and Table 6，for Instances 1—3，the information quantity of the image is rich，which can provide sufficient data support for the optimization algorithm.It can ensure the continuous optimization of the feature entropy of each link of the algorithm.The convergence effects of ten groups are stable，and the group deviation of feature entropy is small.The image of Instance 4 has low information quantity and large background area，which affects the discretization in the optimization process and increases the probability that the points generated in the local discretization and global discretization process fall into the background area.However，the background area with uniform features and large region cannot effectively support the local optimization of points，and it is difficult to further optimize the points after falling into the background area，which may affect the optimization results.From the convergence effect of ten groups of operations，it can be seen that there is a certain deviation between the feature entropy groups.The image information quantity of Instance 5 is rich，but the damage adjacent region is not significant.The verification of the proposed method uses double thresholds，which can avoid the influence of the damage adjacent region while dividing the damage region.But there is no obvious adjacent region in Instance 5，the double thresholds subdivide the damage area，which makes the local optimization continue and makes the optimization range be limit in the damage area.It will affect the convergence result and cause some fluctuation.The Instance 6 is pit damage，the image chroma is single，but there is a certain brightness change.In the proposed method，the 2nd dimension feature entropy is the maximum luminance difference in the neighborhood.The luminance change can be perceived in the optimization process，which can provide information support for optimization.The convergence effects of ten groups of operations are relatively stable，and the group deviation of characteristic entropy is relatively small.

In Table 6，the calculation results of each instance are statistically tested.Under the significance levelα=0.05，supposeH0：μ≤μ0，the total number of samples isn=10，the critical point ist0.05(9)=1.833 1，and calculate the test statistict=，whereis the arithmetic mean of the calculation results andSthe standard deviation of the calculation results.It can be seen that the test statistics of each instance are less than the critical point，andH0is accepted.That is to say，for each instance，the average error of the proposed method is at a low level.

Table 6 Calculation r esults based on the pr oposed method

4 Conclusions

Based on the definition of multi-dimensional feature entropy，introducing the idea of multi-threshold segmentation and designing the optimization algorithm for correlation parameters，the multi-dimensional and multi-threshold airframe damage region division method based on correlation optimization is proposed.The method is verified by various airframe structure damage images.The verification results show that：

（1）From the perspective of the operation speed，the operation process of the proposed method can be completed in the method within millisecond scope，and with low time complexity.

（2）From the perspective of division quality，compared with the traditional 1-D single threshold division method，the damage region obtained by this method has high accuracy，consistent and complete boundary.It can effectively divide cracks，pits，foreign object damage and other common structural damage forms of the airframe with less noise.For the interference factor introduced in the image acquisition process，it has good anti-interference effect，and the overall division result is better than the traditional method.

（3）From the perspective of optimization effect，the rapid convergence could be achieved and the optimal solution of convergence is stable.

（4）From the perspective of applicability，the proposed method is suitable for the processing of complex damage images with adjacent damage regions or high information quantity.For the simple damage image with large range of low information quantity area，the proposed method may subdivide the region and introduce errors.

Therefore，the proposed method can rapidly and effectively achieve the division of airframe damage region，and provide data foundation and technical support for the airplane structure damage model reconstruction，airplane structure damage analysis and repair design.