,

Research Institute,Baoshan Iron & Steel Co.,Ltd.,Shanghai 201999,China

Abstract: This paper reports the quantitative study of martensite/austenite (M/A) constituents in high-grade heavy steel plates using the gradient-based watershed algorithm.Compared with several other image segmentation algorithms,the gradient-based watershed algorithm can effectively remove noises in images with various qualities and identify M/A constituents.The results prove that the gradient-based watershed algorithm is an effective method that can be used to digitize microstructures and integrate the data into Process Intelligent Data Application System (PIDAS).

Key words: M/A constituent; microstructure digitizing; gradient; watershed; image segmentation

1 Introduction

Metallographic observation is the most basic and conventional method in the field of material research.A martensite/austenite (M/A) constituent is a com-mon microstructure in advanced high-strength structural steels.Because M/A constituents have the martensite hard phase (M) and retained austenite soft phase (RA),they possess strength and tough-ness.Furthermore,the instability of the RA in M/A might lead to a complicated phase transformation,causing property uncertainties.This condition im-plies that the M/A constituent has an important in-fluence on the strength and toughness of materials.Nonetheless,its effect on properties is mainly related to its size,content,distribution,and mor-phology in the microstructure.Studies have shown that with the increase in the size and content of the M/A constituent,the strength of pipeline steels increases and its toughness decreases.Therefore,quantitative characterization of the M/A constituent is highly important.The traditional observation of the M/A constituent usually uses the Lepera reagent for coloring and etching.Statistical analysis of the M/A constituent’s size and content after coloring is complex[1-2].

Image segmentation is an essential part of image processing and computer vision.In recent years,it has also become a research hotspot,widely used in computer graphics,computer vision,and robotics[3].Its main goal is to segment and extract the components with a strong correlation with real things or regions from the image as eigenvalues using a certain mathematical theory algorithm for the study of laws[4].Traditional image segmentation methods can be divided into threshold-based,region-based,boundary-based,and morphological and specific theoretical segmentation algorithms.The region-based watershed algorithm is widely used because of its simplicity,fast calculation speed,and accurate positioning[5-6].

The watershed algorithm is an algorithm based on the gray gradient of image,which is very sensitive to noise.Oversegmentation can be easily produced by directly using the watershed algorithm to seg-ment the image.To restrain oversegmentation,this study attempts to segment and identify the M/A constituent structure in advanced high-strength steel using the improved gradient-based watershed algo-rithm.It is combined with expert knowledge through OpenCV Python programming to realize the digiti-zation of M/A constituent-related statistics and then perform a technical screening for Process Intelligent Data Application System (PIDAS) integrated micro-structure data[7].

2 Principle of the gradient-based watershed segmentation

The watershed segmentation algorithm is a morphological segmentation method based on topological theory.Its basic idea is to take an image as the topological landform of geodesy.The gray value of each pixel in the image represents the altitude of the point,and each local minimum and its affected area are used as the basin.The boundary of the basin forms a watershed.The calculation process of the watershed algorithm is a continuous iterative labeling process.The classic watershed cal-culation method was proposed by L.Vincent.In this algorithm,the watershed calculation is mainly divid-ed into two steps,the sorting process and the inunda-tion process.First,the gray value of each pixel is sorted from low to high.Then,in the process of flooding from low to high,the first-in-first-out structural unit is used to judge and label the influence domain of each local minimum at theh-order height.The output is shown as the catchment basin image of the input image,and the boundary point between the catchment basins and the image maximum point is the watershed.Therefore,to obtain the boundary information of the image,the gradient image is usually used as the input image,that is,

(1)

where,g(x,y) is the output image;f(x,y) is the original image;and grad{.} is the gradient operation.

The watershed algorithm has a good response to the weak boundary and can basically ensure the continuous closed boundary.Therefore,closure is an important feature of the watershed algorithm.Moreover,the watershed algorithm is vulnerable to image noise and subtle gray changes on the object surface,resulting in excessive segmentation.

To eliminate the oversegmentation caused by the watershed algorithm,this paper adopted the im-proved watershed algorithm of image preprocessing before segmentation.

3 Improved watershed algorithm

The input image was preprocessed.First,the median filter was used for denoising.Second,mor-phological reconstruction was performed,and the gradient image was obtained again.Finally,the foreground and background were marked to reduce the small basin and the number of oversegmented areas.

3.1 Median filtering

The median filter is the most commonly used and well-known statistical sorting nonlinear filter.The algorithm uses the median gray value in the neigh-borhood of the central pixel to replace the central pixel value,which can particularly effectively re-move the salt-and-pepper noise in the image.The output image of the median filter is as follows:

f1(x,y)=med{f(x-i,y-j),(i,j)∈C}

(2)

where,f1(x,y) is the denoised image;andCis a two-dimensional template.Commonly used tem-plates include 3·3,5·5,7·7,and odd templates.In this study,the 5·5 median filter is used to pro-cess noisy images.

3.2 Morphological reconstruction

Corrosion and expansion are the most basic mor-phological operations.A morphological reconstruction involves two pictures.First,a gray image is etched as the marker,and the original image is used as the mask.Then,the marker is expanded until it appro-aches the mask to a certain extent,which is called an open reconstruction.If the original image is complemented first,then it is reconstructed through an open operation.The result is obtained and then complemented again.This process is called a closed reconstruction.A morphological open/close opera-tion is used to reconstruct the image.This operation can further eliminate the local extremum caused by irregular gray disturbances and noises in the gradient image while retaining the important boundary contour extremum information[8].

3.3 Marker-based watershed segmentation

We set the point with a gradient less than 10 as the marking starting point to mark the image.The segmentation is guided by the marker graph,which is represented as the foreground.The watershed algorithm performs an accurate segmentation near it.The watershed algorithm ignores the part marked as the background.This process is classified as internal marking and external marking to separate each target area from the background.The marking can be realized through the opening and closing morphology operations.The specific implementation steps are as follows:(1) the pixel value at the local maximum of the marked image is set to 255;(2) a closed operation is used to close the local maximum;(3) corrosion is used to corrode the image;(4) the open operation is used to open the image;and (5) the foreground is set to 255.

Generally,any part of the foreground image is set to 255,and the background image is set to a value between 0 and 255.

4 Experimental materials and methods

4.1 Experimental materials

The experimental material is an advanced high-strength structural steel produced by a heavy-plate production line.The metallographic specimen was taken from the steel plate.After mechanical grinding and polishing,it was etched with 4% nital and observed with a German Zeiss microscope Imager M2m.The field of view was randomly selected to take photos under the 50× objective lens to obtain the microstructure of materials as the input image of subsequent image segmentation.

4.2 Experimental method

On a desktop computer with an Intel CoreTM2 Duo T7700 processor,16 GB memory,and 64-bit operating system,the M/A constituent segmentation and recognition from the microstructure of the advan-ced high-strength structural steel were conducted using OpenCV Python and the gradient-based water-shed algorithm.The segmentation results obtained by the gradient-based watershed algorithm were com-pared with those obtained by using the global threshold Otsu binarization algorithm,K-means clus-tering algorithm,and random walking image seg-mentation algorithm from the aspects of the M/A constituent content and boundary-shape segmenta-tion results.

5 Comparative analysis of experimental results

The microstructure of the investigated steel is shown in Fig.1.In the figure,the dispersed,dark-color,and island-like structure is an M/A con-stituent,and the large area in gray white is the bainite ferrite matrix.

Fig.1 Microstructure of advanced high-strength structural steel

To increase the operation speed,the color image was cropped with OpenCV Python to the size of 250·1 000 px.After the gray processing of the crop-ped image,the 5·5 median filter was used for denoising.The results are shown in Fig.2.Con-sequently,the noise was effectively removed,and the fine areas became smooth and fuzzy.

Fig.2 Comparison between the filtered image and original image

In this study,the morphology module in skimage was used to further perform the morphological processing on the median filtered image and reconstruct the gradient image.The results are shown in Fig.3.Compared with the original image,the gradient reconstructed the image clearly and completely extracted the boundary of the M/A constituent in the metallographic image.

Fig.3 Gradient image after morphological processing

The results of the gradient-based watershed image segmentation were compared with those of the global binarization,global threshold Otsu binariza-tion algorithm,K-means clustering algorithm,and random walking image segmentation algorithm,as shown in Table 1,enlarged figure of area 2 is shown in Fig.4.The M/A constituent content obtained by various algorithms is shown in Fig.5.On the whole,the content of the M/A constituent obtained by all the algorithms is similar,which is consistent with the empirical value.The comparison results show that the result obtained by the random walking algorithm is the smallest,which is 27.278%.The result is mainly related to the filtering of small particles and excessive segmentation.With theK-means clustering algorithm,the result is the highest,which is 30.436%.The result of the gradient-based watershed algorithm is close to that of theK-means clustering algorithm.

The comparison of the segmentation results with the original image shows that the global threshold binarization Otsu algorithm has high requirements for image quality,such as area 1 identified in Table 1.The algorithm marks the central part of the M/A constituent as the matrix but fails to completely identify this area.Hence,the identification rate is lower than those of the other two algorithms.Concerning area 2 in Table 1,the local enlarged images are shown in Fig.4.The random walking image segmentation algorithm fails to completely separate the boundary of the M/A constituent effectively.It regards many independent M/A constituents as a whole.The boundary contour is not clear,which increases the average size of the M/A constituent and affects the quantitative results of its shape.This result will be misleading in the material analysis process.As far as the boundary contour is concerned,the global threshold binarization Otsu algorithm andK-means algorithm are better than the other algorithms,but these algorithms tend to coarsen the contour boundary and are influenced by some noises.In the gradient-based watershed algorithm,after median filtering and gradient-based morphological processing,the influence of noise becomes very small.Then,the boundary can be completely segmented,which provides a basis for the quantification of subsequent M/A constituent morphological data.

Table 1 Image segmentation results by various algorithms

Fig.4 Corresponding enlarged view of area 2 in Table 1

Fig.5 Volume fraction of the martensite/austenite constituent obtained by different image segmentation algorithms

6 Conclusions

In this study,the gradient-based watershed al-gorithm was used to segment and identify the M/A constituent in the microstructure of advanced high-strength structural steel.The comparison results show the algorithm can denoise and smoothen the metal-lographic images with noise through median filtering and gradient reconstruction after morpho-logical processing while keeping the boundary infor-mation of the original image in the denoising process.In addition,it can effectively avoid overseg-mentation.It can correctly segment and identify the M/A constituent in the microstructure and realize the volume and morphology quantification of M/A constituents.These functions enable the integration of microstructure data into PIDAS.