Yanpeng Wu,Xiaoqi Peng, ,Mohammad Nur and Hengfu Yang

Abstract:Foam image segmentation,represented by watershed algorithm,is wildly used in the extraction of bubble morphology features.H-minima transformation was proved to be effective in locating the catchment basins in the traditional watershed segmentation method.To further improve the accuracy of watershed segmentation,method of topbottom-cap filters and method of morphological reconstruction were implied to marking the catchment basins.In this paper,instead of H-minima transformation,a method of contour lines is specially proposed to obtain the catchment basins for foam image segmentation by using top-bottom-cap filters and less morphological reconstruction.Experimental results in foam segmentation show that the proposed method is equally accurate but more efficient than the method of H-minima plus morphological reconstruction,and equally efficient but more accurate than the method of H-minima plus top-bottom-cap filters.

Keywords:Foam image,segmentation,watershed algorithm,catchment basin,contour line.

1 Introduction

Image segmentation methods are widely applied in such areas as smart transportation system,medical image analysis,remote probing,floatation process,etc.In the process of mineral flotation,the bubble shape can directly reflect the production conditions and production indicators.Foam image segmentation is a key step in the extraction of bubble morphology features [Lezoray and Christophe (2009);Sadr-Kazemi and Gilliers (1997);Zhou,Yang,Gui et al.(2010)].

With the introduction of many new theories and new methods in various disciplines,many image segmentation methods have been developed,in terms of segmentation methods based on threshold [Wu (2014)],region [Girshick,Donahue,Darrell et al.(2016);Zhou,Qiu,Li et al.(2018)],edge [Al-Am ri,Kalyankar and Khamitkar (2010)],cluster analysis [Arifin and Asano (2006);He,Dantong,Wang et al.(2018)],fuzzy set theory [Tobias and Seara (2002)],wavelet transform [Unser (1995)],neural network[Kuntimad and Ranganath (1999)],etc.

The watershed algorithm is the most commonly used foam image segmentation algorithm.However,because the flotation bubbles are stuck,affected by the ambient light,and unobvious at the boundary,therefore,it is necessary to improve foam image segmentation.

2 Related works

2.1 Standard watershed algorithm for foam segmentation

The foam image segmentation method based on watershed transform is a mathematical morphology segmentation method based on topological theory,in which the image is regarded as an inverted geodesic topological feature.Taking the gray value of each pixel in the gradient image,the connected areas of the e image are found.The watershed transformation method assumes that each local minimum value and its surrounding area form a catchment basin.As the water potential increases,the catchment basin gradually expands,and the confluence of the catchment basin is the watershed.

The classical watershed transformation was proposed by Vincent Luc et al.[Vincent and Soille (1991)],including the sorting process and the flooding process.The watershed represents the maximum point of the input image.

The watershed calculation process is an iterative labeling process.First,each pixel is sorted from low to high gray level,and then in the process of submerging from low to high,each local minimum value in H-minimum area uses a first-in,first-out structure to judge and mark the catchment basin.

In formula (1),function Hmin represents an H-minima transformation,andHwrepresents a H-minima transformation threshold.By setting the depth thresholdHw,the local minimum of the basin below a given threshold is eliminated.

After inverting the foam image,each bubble can be regarded as a catchment basin.The watershed transformation can be used for edge segmentation.The basic steps are as follows:First,search the catchment basin in the foam image,using H-minima transform.

Second,reconstruct the shape of the catchment basin.

Finally,perform edge segmentation to the foam image with watershed transform method.The key to determining the effect of the watershed transformation segmentation is to accurately locate the catchment basins [Cai,Li and Guo (2014)].It is inevitable to find low qualities or faults in the foam image,such as uneven illumination,noise,quantization error,texture,etc.,which result in too many local minima and a so-called oversegmentation that the target image be segmented into a large number of small areas.

Fig.1 shows a foam imageF(size:800*600),and Fig.2 shows the result of edge segmentation directly using the watershed transform ofF.Obviously,the oversegmentation phenomenon is very serious.

Figure1:Foam image F

Figure2:Watershed segmentation of foam image F

2.2 Enhancement of the catchment basins with top-bottom-cap filters

According to the research of Yingying Gu et al.[Gu,Lin,Li et al (2007)],in the foam image,the gray value less than 20 is the shadow of bubbles,and the gray value greater than 225 is the reflection of the illumination light on the top of the bubble.Therefore,the foam image can be transformed with top-bottom-cap filters to improve the effect of watershed transformation [Liu,Yuan and Guo (2011)].Assuming the gray value of top part of bubbles is exceededb1,and the gray value of bottom part of bubbles is lowerb2,then the target of top-cap filter is to make the data in top part distributed at the highest intensity ofc1,and the target of bottom-cap filter is to make the data in bottom part distributed at the lowest intensity ofc2.The transformation formula of top-bottom-cap filters is:

For example,through the transformation of top-bottom-cap filters,the gray values the foam image are linearly mapped to [0,255] range,with top 1% data assigned to 255 and bottom 1% data assigned to 0.

The enhancement effect to the foam imageFis shown in Fig.3 and Fig.4 in a threedimensional view.The processed image has an obviously higher recognition degree for bubble object.

Figure3:3D view of the original foam image F

Figure4:3D view of foam image F after top-bottom-cap transformation

2.3 Enhancement of the catchment basins with morphological reconstruction

Morphological reconstruction of the catchment basins with prior know ledge is an effective way to enhance the catchment basin markers [Hu and Cai (2011)].

The result of the expansion operation is to merge the background pixels that are in contact with the object boundary into the object,filling the small holes in the image and the small concave portions at the edges of the e image.The structural elementQis recorded for the expansion of the imageF,which is defined as:

The etching operation is the inverse of the expansion operation.The corrosion of the structural elementQon the imageFis described as:

The filling operation can be achieved by using multiple iterations of combination of expansion operations,complement operations,and intersection operation.

In whichBis a cross-shaped structural element.

Commonly,the morphological expansion operations are used to merge the closed marks to reduce the over-segmentation of the image,and morphological corrosion operations are used to eliminate the blanks inside the mark and reduce the under-segmentation of the image.

Fig.5 shows the catchment basins of foam imageFunder the condition ofHw=50.

Figure5:Catchment basins of foam image F

Fig.6 shows the catchment basins ofFafter morphological operations are performed,including ‘expansion 2’,‘corrosion 3’,‘expansion 3’.

Figure6:Catchment basins of F with morphological reconstruction

Figure7:Merged catchment basins of foam image F

Using multiple image enhancement techniques,an improved image segmentation method based on watershed transform is proposed by Yanpeng Wu et al.[Wu,Peng,Ruan et al.(2016)],with steps as follows:

First,using H-minima transforming algorithm,large,medium and small catchment basins were searched and saved separately.

Second,invalid catchment basins were checked and deleted to ensure each catchment basin has area in a proper range.

Third,the shape of catchment basins was separately fixed with a series of morphological operations,mainly including corrosion operation and expansion operation.Then,all catchment basins were merged into a unique catchment basin map.

Finally,the foam image was reconstructed with the unique catchment basin map as shown in Fig.8,and segmented with watershed algorithm.

Figure8:Reconstruction of foam image F

As shown in Fig.8,almost every bubble has a corresponding high-lighted block at central-top area,which will be treated as catchment basin for watershed transformation.

3 Proposed method

3.1 The relationship between height of contour line and bubble size

By setting the bubble height to its gray value,the gray space can be converted into a three-dimensional space.According to surface area formula of ball crown,the distribution function for the count of pixels in a bubble above a certain gray value is:

In whichsiis the count of pixels in a bubble with radius ofriand height ofh,andnis the number of bubbles.

All bubbles contained in a foam image are semi-sphere with same height and different radius,as can be seen from Fig.3 and Fig.4.Then the equation of total surface area of bubbles should be:

In whichSmeans total surface area of all bubbles with average radius ofrand height ofh,and eis a constant coefficient.

Considering the case that bubbles in a specific foam image are separated by several contour lines,it is easy to set up the relationship between height of contour line and area or radius of bubbles section crossed by them.

Assumingkcontour lines are performed,the corresponding section area is:

According to formula (8),the bubble section between contour linehjandhj+1should have a valid size range of [Sj,Sj+1] and a valid radius range of [rj,rj+1].

3.2 Foam image segmentation method based on gray contour lines

The steps of the e image segmentation method based on gray contour lines are as follows:

(1) Enhance the foam image with top-bottom-cap filters.

(2) Set up the model about the relationship between height of contour lines,area and radius of bubbles by using curve fitting method.

(3) Determine the number and height of the contour lines used to cut the bubbles,including at least three contour lines:the top contour,the middle contour,and bottom contour.

(4) Expand each bubble section separated by contours lines and determine whether they have a valid size.

(5) Merge foam images with all bubble sections and count how many repeated parts existing in a bubble.A morphological filling operation is recommended to eliminate blanks inside the compound bubble sections.

(6) Taking the compound bubble sections as the catchment basins,the original foam image after the inversion is segmented by watershed method to curve accurate bubble edge.

4 Experiments and discussion

4.1 Obtaining catchment basins of foam image F with proposed method

First,the foam imageFis enhanced with top-bottom-cap filters as shown in Fig.9.

Figure9:Enhanced foam image F with top-bottom-cap filters

Then pixels with different gray value are summarized as shown in Fig.10.

Figure10:Distribution of gray value for foam image F

Through comparing Fig.10 to Fig.4,the valid distribution of gray value for foam imageFis in range of [40,225],which is summed cumulatively as shown in Fig.11.

Figure11:Summation of gray value distribution of foam image F

The fittedScurve is

The sample date is changed by a square root operation and its fitted linear curve as shown in Fig.12 is

Then,the grow th rate of radius is

Figure12:Relationship between radius and height of foam image F

According to formula (9) and (10),the range ofSis [0,356896] and the range ofris [0,591.7].Assuming the contour line interval for foam imageFbe 40 pixels,there are 6 contour lines as shown in Fig.13.Respectively,there are 6 bubble layers,whose height set is {40,80,120,160,200,225},and area set is {258000,159854,94000,49009,16784,-24.02},and radius set is {498.0,404.2,310.5,216.7,123.0,64.4}.

Figure13:Foam image F divided into 6 sections

If 100 bubbles contained in foam image F,according to formula (11),then the average radius increment to bubble sections is -9.37 pixels,which means the area of each bubble section will be same after a series expanding operation of {0,9.37,9.37*2,9.37*3,9.37*4,9.37*5}.However,for large bubbles,the radius increment may be more than 100 pixels.For small bubbles,the radius increment may be less than 1 pixel.Let the identifiable bubble has a radius of from 6 pixels to 100 pixels,then radius of the e smallest bubble at top layer is 1 and 100 for the largest bubble at bottom layer.To make sure all sections belong to a bubble,instead of sections belong to different bubbles,are connected to each other,the expansion operation to bubble section is suggested to be {0,1,2,3,4,5}.Certainly,there are many proposals for other categories of bubbles if every category of bubbles can be dealt separately.The processes of expanding 6 layers of F are shown in Fig.14.

Figure14:Expanding processes of bubble sections for foam image F

The valid range of radius of bubble in each layer is [6,100],which means the valid range of area of bubble in each layer is [113,31415].Base on the above filter condition,catchment basins of foam imageFare obtained as shown in Fig.15.

Figure15:Catchment basins of foam image F

4.2 Accuracy of segmentation of foam image by using watershed transformation

Three methods,including H-m inima plus top-bottom-cap filters,H-minima plus morphological reconstruction,the proposed method mixed with top-bottom-cap filters,less morphological reconstruction and contour lines,are used to enhance the catchment basins and segment a series of foam images by using watershed transformation.

The segment results of foam imageFare shown from Fig.16 to Fig.18,which indicate that for bubble segmentation the proposed method is as accurate as the method of H-minima plus morphological reconstruction,while the method of H-minima plus topbottom-cap filters is a bit less accurate.

Figure16:Segmentation of foam image F with H-minima plus top-bottom-cap filters

Figure17:Segmentation of foam image F with H-minima plus morphological reconstruction

Figure18:Segmentation of foam image F with proposed method

4.3 Efficacy of segmentation of a series of foam images by using watershed transformation

The time consumption of three segmentation methods based on watershed algorithm to foam images is shown in Fig.19,which indicates that for bubble segmentation the proposed method is as efficient as the method of H-minima plus top-bottom-cap filters,and 5 times faster than the H-minima plus morphological reconstruction method.

Figure19:Compared segmentation methods based on watershed algorithm on time

5 Conclusions

H-minima transformation,top-bottom-cap filters and method of morphological reconstruction were proved to be effective for the traditional segmentation method in locating the catchment basins.However,instead of H-minima transformation,method of contour lines proposed in this paper with top-bottom-cap filters and less morphological reconstruction is also suitable to accurately and efficiently obtain the catchment basins.Experimental results in foam segmentation show that the proposed method is as highly accurate as the method of H-minima plus morphological reconstruction,and as efficient as the method of H-minima plus top-bottom-cap filters.Meanwhile,the proposed method is more accurate than the method of H-minima plus top-bottom-cap filters,and more efficient than the method of H-minima plus morphological reconstruction.

Acknowledgement:This work is supported by the Natural Science Foundation of China with Nos.61621062,61773407 and 61872408,and Hunan Province Science Foundation of China with No.2019JJ40050.