Feng Zhao,Yue Dong,Jian-lei Zhang

Xi’an University of Posts and Telecommunications,School of Electronic Engineering,Xi’an,710121,China

Keywords:Polarization visualization Perceptually uniform space Lightness prediction Low irradiance

ABSTRACT Currently,polarization visualization strategies are accomplished by mapping polarization information into a perceptually uniform color appearance model CAM02-UCS.However,the deviation of the CAM02-UCS space from the lightness prediction results in an inaccurate match between the polarization information and the perceptual information.In this paper,we propose a novel polarization visualization strategy based on the perceptual uniform space Jzazbz.The polarization visualization be completed by placing the polarization information into the lightness Jz,colorfulness Cz and hue angle hz channels of the Jzazbz space.The experimental results show that the proposed method can significantly improve the lightness of the low irradiance and high polarization region,hence more polarization information can be sensed by human visual system.

1.Introduction

Polarization is a fundamental property of light waves,containing wealthy characteristic information of the measured object[1].However,the human eye is insensitive to polarization information[2],hence mapping polarization parameters to color parameters becomes an important strategy for visualizing polarization information[3-6].

Initially,Bernard and Wehner[7]discussed the structural similarities between invertebrate polarization vision and human color vision,providing a theoretical basis for the visualization of polarization imaging data.Based on this similarity,the polarization information can be visualized by mapping into the HSV(Hue,Saturation,and Value)color space[8,9].In 2016,Tyo et al.built a confidence map by using the statistics of the angle of polarization,and magnified the irradiance in the high confidence region,while the low confidence region remained unchanged[10].This method effectively solves the problem that polarization information in regions with low irradiance but high degree of polarization tends to be invisible.However,HSV is not a perceptually uniform color space.When the polarization information is mapped into HSV space,the inherent non-uniformity means perceived differences in HSV parameters do not reflect the same difference in polarization parameters.To eliminate the non-uniformity,Kruse et al.[11]proposed a mapping strategy using the perceptually uniform color space CAM02-UCS.The normalized intensity(s0),degree of linear polarization(DoLP)and angle of polarization(AoP)are linearly mapped to the lightness J′,colorfulness M′and hue angle h of the CAM02-UCS[12].After that,Kruse et al.revealed out the channel mixing phenomenon in this mapping strategy and proposed an improvement method.In CAM02-UCS space,channel mixing can be relieved by tuning the mapping function[13].

The key to polarization visualization strategies is accurately mapping the polarized image data into the perceived color channel.However,the CAM02-UCS color space does not have a good prediction of SL2 data in terms of lightness,which results in a certain difference between perceived lightness and actual lightness.This means that the visualization strategy based on the CAM02-UCS color space cannot accurately match the polarization information with the perceptual information.At present,a new perceptual uniform color space Jzazbzis proposed[14].Compared with CAM02-UCS,the lightness channel Jzof Jzazbzis more accurate for the prediction of lightness.In this paper,we propose a novel polarization visualization strategy based on Jzazbzspace.The visualization is performed by placing the polarization information s0,DoLP andψinto the Jz,Czand hzchannels of the Jzazbzspace,respectively.This method can effectively improve the lightness of the low irradiance and high polarization region.

2.Mapping methodology

In polarization imaging,the expressions for the s0,DoLP andψ are[15].

where I0,I45,I90and I135are the intensity of the light wave filtered with a linear polarization filter in the 0°,45°,90°and 135°orientation[16].s0represents the total light intensity of the light wave,s1represents the difference in light intensity between the 0°direction and the 90°direction,and s2represents the difference in light intensity between the+45°direction and the-45°direction.

Eq.(4)-Eq.(7)is the mapping method based on CAM02-UCS color space[13],

where J′,M′and h represent the lightness,colorfulness and hue angle of the CAM02-UCS color space,respectively.

2.1.Lightness prediction analysis of Jzazbz space

Jzazbzis a newly proposed perceptually uniform color space with good performance in lightness prediction.Fig.1 shows the comparison results of SL2 data predicted by CAM02-UCS and Jzazbz[17].The SL2 data was obtained by Fairchild et al.through psychophysical experiments,which can effectively predict human perceived lightness values.The accurate prediction ability of the color space can reduce the deviation generated in the polarization visualization process,and make the polarization information reflected by the mapping result more objective.

Fig.1.Color space prediction SL2 data results.

Fig.2.Gamut curves y formed by the inscribed circle of Cz at each Jz slice in Jzazbz.

From Fig.1,we can see that the CAM02-UCS lightness prediction result is obviously different from the experimental data.The lightness prediction result of CAM02-UCS is lightly larger than the experimental data at the low luminance value(0-1000 cd/m2)and smaller than the experimental data at the high luminance value(1000-2000 cd/m2),which leads the mapping method based on CAM02-UCS space unable to accurately match the polarization information with the perceptual color visual information.In contrast,we can see the prediction of lightness in the Jzazbzspace shows a high similarity to the SL2 data compared with CAM02-UCS.

When the lightness value of the color space is constant,the luminance value of the Jzazbzspace below 100 lightness is larger than that of CAM02-UCS.This clarifies that when the same polarization information irradiance s0is mapped to the lightness channels,J′and Jz,of the CAM02-UCS and Jzazbzspaces,the latter will have a value closer to the SL2 data.As can be seen from the curve,when the luminance value is less than 1000 cd/m2,the perceived lightness of CAM02-UCS is higher than that of Jzazbz.However,in polarization visualization strategies,this prediction is the opposite.Usually through the mapping,the polarization information is placed into the sensing channel lightness,colorfulness,and hue angle.Then the polarization information is converted from the sensing channel to the CIE XYZ tristimulus values,and finally converted to the RGB space to display the results.Therefore,when the perceived lightness value is less than 100,the luminance prediction result of the Jzazbzspace is higher than the CAM02-UCS,and is closer to the SL2 data.Compared with CAM02-UCS,the overall lightness of the polarization visualization results using the Jzazbzspace can be improved.In particular,the visualization of low irradiance and high polarization regions is improved.

2.2.Jzazbz-Based polarization mapping

We use Jzazbzas the target space for polarization mapping.Jzrepresents the achromatic component(lightness,Jz)of the Jzazbzcolor space.azand bzrepresent two opponent color components(redness-greenness,azand yellowness-blueness,bz).The three parameters of a common mapping target space are lightness,colorfulness,and hue angle.Similarly,the three parameters in the Jzazbzspace are as follows[14]:

where Iz,azand bzare the outputs of the CIEXYZ tristimulus values after passing through the Jzazbzmodel.d and d0are the internal parameters of the Jzazbzmodel.d=-0.56,d0=1.6295499532821566×10-11.

The Jzazbzcolor space includes all the color gamut visible to the human eye.Due to the limitation of the display technology,a part of colors cannot be reproduced on the display device at the current stage.This causes the polarization information mapped to this part of the color to be lost.In order to preserve more polarization information,the mapping should be limited to the gamut of the display device.Meanwhile,to relieve channel mixing,it is necessary to perform a certain cutting process on the gamut before the mapping.Here,we cut the Jzazbzin the cylindrical coordinates to make it a rotationally symmetric color gamut GJab,which ensures the existence of the corresponding Jzfor any different Czof hz.We use the method similar to Kruse[13]along the Jzaxis,to calculate the inscribed circle radius of Cz.Then generate a curve of the relationship between Jzand Cz,called y.GJabis the solid of revolution of y curve about the Jzaxis.The gamut curves of sRGB and AdobeRGB are shown in Fig.2.

Similar to the previous mapping method,the polarization information s0,DoLP is normalized before mapping,whileψremains the same[10],

where s0min,s0maxand are DoLPmaxare the maximum and minimum values in the corresponding data.

Then,the polarization information s0,DoLP andψare converted to Jz,Czand hz:

Jzazbzwhere c=0.0105 is the maximum value of the sRGB color gamut curve y in Fig.2.Jz0and Jz1are the two intersection points of Czwith curve y after the Czvalue is determined.Jz0corresponds to the smaller of the two intersection points.is linearly stretched into the Jz0～Jz1range to ensure that polarization information is not lost[13].

In order to make the low irradiance and high polarization region visible,Kruse finds the high polarization region to amplify it by evaluating the variability of the angle of polarization.This may have a similar effect to the method of this article.It is worth noting,however,that the method of amplifying the regions of high polarization and low irradiance to achieve visibility does not take into account the effect of such treatment on the uniformity of the results.The method in this paper is based on the characteristics of the color space to achieve the effect of improving the lightness,there is no additional selection process to destroy the uniformity.

3.Experiment and discussion

This experiment uses the division of focal plane polarization imaging camera[18].The camera′s sensor consists of 2×2 pixels,which consists of four different directions:0°,45°,90°and 135°orientation.When the focal plane polarization imaging camera acquires polarization information,per pixel records only one of the four necessary light intensity measurements,resulting in loss of spatial resolution[19].The loss of spatial resolution causes noise in the results of the mapping,which can be improved by superresolution reconstruction of the polarized image.In this experiment,the super-resolution reconstruction processing is not involved,but the result process a simple median filtering.When acquiring polarization information,the camera′s gain is set to 1.The resolution is 2448×2048.Due to the loss of spatial resolution,the final resolution is 1224×1024.

We select two different scenes,the laboratory under fluorescent light and the car under natural light,to visualize polarization information.Fig.3 shows the mapping results of laboratory scene.The mark in the figure is the path for analysis.The mapping result of the proposed method has a higher lightness value than CAM02-UCS.Due to the increase in lightness,some‘invisible’areas with low irradiance and high polarization information have significant improvements.For example,the experimental table on the left and the chair on the right.

To compare the effects of the proposed method under different irradiances,we choose the two irradiances for car scene.Fig.4 shows the original polarization image of the car scene.Among them,the license plate part in Figs.4-6 was occluded.

Fig.5 shows the mapping results of low irradiance for car scene.Similar to the results of the laboratory,the processing results of this method have a higher lightness value than the CAM02-UCS,as shown in Fig.5,in the bottom of the car and the car door.Fig.6 shows the results of processing in a high irradiance environment in the same scenes.Since the scene is in a high irradiance environment,the overall lightness improvement has no effect on the details.The advantages of this method are not obvious in the case of processing in a high irradiance in the same scenes.

Since the default range of lightness J in CAM02-UCS and Jzazbzcolor spaces is different,and the two methods cannot be directly compared.Here,we convert the two results into HSV space and compare their V value.Fig.7 shows the V value of laboratory scene for CAM02-UCS and Jzazbz-based mapping results in the particular path.The path includes a high polarization region A(left computer display)and a non-polarization region B(right background window)as shown in Fig.3(a).We can see that the V of the Jzazbzresult is generally higher than that of CAM02-UCS.Although the overall V is improved from the curve,the region A is more noticeable than the non-polarization region B.

Fig.8 shows the polarization parameters in the particular path in laboratory scene.In low irradiance region A,more polarization information is usually retained.The trend of V is closer tothanin this region.Conversely,the trend of V is closer toin region B.

The polarization parameters of different irradiance in the particular path in car scenes as shown in Fig.9.In region A,regardless of whether the irradiance is low or high,the value ofhas a rising trend,and the value of Itends to be stable.

Fig.3.Mapping results of laboratory scene.(a)CAM02-UCS,and(b)Jzazbz.

Fig.4.Original polarization image of the car scene.(a)low irradiance environment,and(b)high irradiance environment.

Fig.5.Mapping results of low irradiance for car scene.(a)CAM02-UCS,and(b)Jzazbz.

Fig.10 shows the V value of car scenes for different results in the same particular path in different irradiance.The red path includes a high polarization region A(upper car glass)and a non-polarization region B(underside car hood)as shown in Fig.6(a).In the region A,the trend of the V value is rising at low irradiance and is stable at high irradiance.That is the V value is more biased towardin the low irradiance environment,while at high irradiance,the V value is more biased toward.In the region B,the V value is more biased towarddue to the low,whether in low irradiance or high irradiance.Although the increase in the V value of the Jzazbzresult is substantially similar in the polarization region A,the rising is slightly larger than that of the high irradiance environment in the low irradiance environment.From Fig.10 we can see that the method of this paper works best for the region A at low irradiance.

Fig.6.Mapping results of high irradiance for car scene.(a)CAM02-UCS,and(b)Jzazbz.

Fig.7.The V value of different results in the particular path in laboratory scene.

Fig.8.The polarization information in the particular path in laboratory scene.(a),and(b).

In order to quantify the effectiveness of the proposed method,Fig.11 shows the V value increase percentage of a particular path in car scene using the proposed method based on Jzazbzspace under different irradiances,compare with CAM02-UCS space.Regardless of whether the irradiance is low or high,the lower the V value,the greater the increase percentage.In low irradiance environment,the increase percentage is greater than in high irradiance environment.Table 1 shows statistics of the percentage increase of V value in car scene under different irradiances.The data shows that when the V value is lower than 0.4,the percentage increase in the low irradiance environment is obviously greater than that in the high irradiance environment.When the V value is higher than 0.4,the percentage improvement is almost the same in both environments.

The proposed method in this paper improves the lightness,and’magnifies’the low irradiance part to make it visible.However,as the polarization information is amplified,the noise is also amplified.As can be seen from Figs.3 and 5,compared with(a),more information can be seen in(b),but there is more noise,and the quality of the results is reduced.

4.Conclusion

In this paper,we propose a novel polarization visualization strategy by placing the polarization information into the lightness Jz,colorfulness Czand hue angle hzchannels of the perceptual uniform color space Jzazbz.Compared with the CAM02-UCS space,the prediction of the lightness for the Jzazbzspace enables the polarization information and the perceptual information to be more accurately matched.The experimental results show that the lightness of the mapping result based on Jzazbzspace is significantly improved compared with CAM02-UCS.In particular,for regions with low irradiance and high polarization,the increase in lightness is more pronounced,thereby reflecting more polarization information.By visualizing the polarization information of low irradiance,the detection and recognition ability of targets at night can be effectively improved.Future work will aim at improving the quality of the results by optimizing the reliability of the acquired polarization information.

Fig.9.The polarization parameters of different irradiance in the particular path in car scenes.(a)-(c)low irradiance car,and(d)-(f)high irradiance car.

Fig.10.The V value of different results in the particular path in car scene.(a)low irradiance car,and(b)high irradiance car.

Fig.11.The V value increase percentage of a particular path in car scene under different irradiances.

Table 1Statistical table of percentage increase of V value in car scene under different irradiances.

Disclosures

School of Electronic Engineering,Xi’an University of Posts and Telecommunications,Chang’an West St.Chang’an District,China.

Declaration of competing interest

The authors declare no conflicts of interest.

Acknowledgements

This work was supported by the Key Research and Development Program of Shaanxi(2018ZDXM-GY-091),the National Key Research and Development Project of China(2018YFB1309403),the Natural National Science Foundation of China(61805199),and Natural Science Basic Research Plan in Shaanxi Province of China(2018JQ6065).We would like to sincerely thank all reviewers for their helpful comments and suggestions.