Yun-Fang Cai , Xu Yang , Yong-Yuang Xiang, Xiao-Li Yan, Zhen-Yu Jin, Hui Liu, and Kai-Fan Ji

1 Yunnan Observatories, Chinese Academy of Sciences, Kunming 650011, China; jkf@ynao.ac.cn

2 Big Bear Solar Observatory, New Jersey Institute of Technology, 40386 North Shore Lane, Big Bear City, CA, 92314, USA

Received 2022 January 23; revised 2022 April 18; accepted 2022 April 19; published 2022 May 20

Abstract The New Vacuum Solar Telescope(NVST)has been releasing its novel winged Hα data(WHD)since 2021 April,namely the Hα imaging spectroscopic data.Compared with the prior released version,the new data are further coaligned among the off-band images and packaged into a standard solar physics community format. In this study,we illustrate the alignment algorithm used by the novel WHD,which is mainly based on the optical flow method to obtain the translation offset between the winged images. To quantitatively evaluate the alignment results of two images with different similarities, we calculate the alignment accuracies between the images of different off-band and line center,respectively.The result shows that our alignment algorithm could reach up to the accuracy of about 0 1 when the off-band of winged image is lower than 0.6 ?. In addition, we introduce the final product of the WHD in detail, which can provide convenience for the solar physicists to use high-resolution Hα imaging spectroscopic data of NVST.

Key words: Sun: atmosphere – methods: data analysis – techniques: imaging spectroscopy – techniques: image processing

1. Introduction

The two-dimensional (2D) imaging spectroscopy observations always play an important role in solar physics studies,which can obtain not only the multi-wavelength spatial images but also the spectral information of the observation target(Huang et al.1995).There are two classical methods to acquire 2D imaging spectroscopy: one is based on spatial scanning with a slit spectrograph to take a series of spectrograms(Hanaoka 2003; Chae et al. 2013; Li et al. 2019). The advantage of this method is its high spectral resolution and free spectral range,which is capable of investigating the high-speed and multi-atmospheric layer solar features. The 2D spectroscopic images of different wavelengths,which are subsequently synthesized by the scanning spectrograms, are strict matching within a single scan in principle, because each spectrum is the performance of a same position on the solar surface within the wavelength range of a spectrogram (Cai et al. 2018b). The other method takes monochrome images in different wavelengths based on the narrow-band filtergrams (Bonaccini &Stauffer 1990; W?hl et al. 2002; Liu et al. 2014; Deng et al.2019). It directly delivers high spatial resolution monochromatic images with different off-bands. However, considering the temporal resolution cannot be too low, there are only several off-bands selected for observation generally. Because of the influence of the near-ground atmospheric turbulence and telescope mechanics, the monochromatic images of off-band observed by this method are nonalignment usually for groundbased solar telescopes, which requires subsequent processing of alignment.

The Hα channel imaging system of the New Vacuum Solar Telescope (NVST) is based on the mode of filtergrams, which can realize the off-band observation in the range of±4 ? by a tunable Lyot filter with a bandwidth of 0.25 ?. To achieve a high temporal resolution,the most off-band observations of Hα is the three bands mode (i.e., two symmetrical off-bands and one line center).The off-band wavelength is usually selected in the range of 0.3–0.6 ? during observation.There always exists a translation transformation between different off-band monochromatic images, and this translation offset can sometime reach 5–10 pixels(0 8–2″).The image scale of the Hα winged images is 0 164,so the alignment accuracy is always required to reach sub-pixel level of about 0 1. The old version of winged Hα data (WHD) released only did the preprocessing and high-resolution image reconstruction, and it was no coalignment among the off-band images. For further scientific research,the winged images whose wavelength close to the line center are aligned roughly by the cross-correlation alignment,which method always failed for the far winged data because of the low similarity between the images of winged and line center. In the Figure 1, we show the local high resolution monochromatic images of nine different off-bands. It can be seen that the greater wavelength of off-band, the lower similarity between the images of winged and line center.Therefore, we urgently need to study an effective algorithm to high-precision align the off-band images. For the three-band observations, we should calculate the alignment accuracy of different off-band image relative to line center one,and find out which off-bands can meet the accuracy requirements of subpixel. All those works have important guiding significance for the NVST routine off-band observations.

Figure 1.The local high resolution monochromatic images of nine off-bands(B080 corresponds to the blue wing 0.8 ? and R080 corresponds to the red wing 0.8 ? and so on).

The multi-band observation and research of the solar atmosphere fine structure and the small-scale magnetic field have become an important topic in solar physics, which requires a high alignment accuracy among multi-band images.There are two registration methods of the solar multi-band images used commonly: one is based on cross-correlation alignment (Kuehner et al. 2010) using a similar feature structure between images, and the alignment accuracy (about 0 1–0 5)(Berkebile-Stoiser et al.2009)is severely limited by the image structure. For the images with large difference, the other alignment of feature point matching is the most commonly used (Guglielmino et al. 2010). In this method,the more feature points, the higher the alignment accuracy.However, the feature points of off-band image are artificially selected with a certain physical model at present, so there are often too few feature points and too much manual intervention to improve accuracy.The alignment accuracy of this method is about 1″–2″ and completely depends on the characteristics of the images(Smith et al.2006).In this paper,we propose a new co-alignment method of winged images based on the optical flow, which is very popular algorithm for motion image analysis. The method has higher accuracy and lower dependence on similar structures compared with cross-correlation alignment, which have successfully applied to the alignment among high-resolution monochromatic images of the NVST Hα channel.

The paper is organized as follows: In Section 2, the instrumentations and observations of winged images are described, and then the alignment method of optical flow is exhibited in Section 3. In Section 4, we assess the alignment accuracies of our algorithm for the different off-band image relative to the line center one. The novel WHD product is introduced in Section 5 and the conclusion and discussion are given in Section 6.

2. Instrumentations and Observations

The NVST is a vacuum solar telescope equipped with a 985 mm clear aperture primary mirror, operating by Fuxian Solar Observatory (FSO)/ Yunnan Observatories (Liu et al. 2014).Its scientific goal is to obtain the solar information by the high resolution imaging (Xiang et al. 2016) and spectral observations(Xu et al.2014;Cai et al.2017)in the wavelength ranges from 0.3 to 2.5 μm. There are two terminal instruments: the Multi-band High Resolution Imaging System(MHRIS)and the grating spectrometer system. The MHRIS includes two broadband interference filter channels (TiO-band and G-band) and three narrow-band Lyot filter channels (Hα, Ca II 3933 ? and He I 10830 ?). The grating spectrometer system (Cai et al.2018a,2018b)consists of the Multi-band Spectrograph(MBS)in visible lines and the High Dispersion Spectrometer(HDS)in near-infrared lines. In this paper, we focus on aligning the Hα winged data of MHRIS.

The NVST is a ground-based solar telescope, and the adaptive optics(AO)system(Rao et al.2016)is not yet widely used in daily observations. In order to overcome the influence of seeing, the speckle masking technique (Liu et al. 1998;Xiang et al. 2016) is used to obtain the high resolution monochromatic images of each off-band.Here,we take a set of three off-band images as an example for alignment. The data were observed on 2021 February 26 over a quiet Sun region.The offset wavelengths are±0.4 ? and Hα line center,respectively. The size of an image is 1024×1024 pixels2and the scale of each pixel is 0 164,so the image covers a field of view(FOV)of 168″×168″.A high resolution image of each off-band is reconstructed by 100 frames short exposure images which is a single frame image taken at an exposure time of 20 ms.The spatial resolution of a high resolution image can reach about 0 1–0 3.The cadence of one winged and Hα line center images is about 14 s,and it takes about 40 s in total for a single off-band scanning in the routine mode of WHD.

3. Alignment Method of the Winged Images

The alignment method proposed in this paper includes three processes: First, we apply the optical flow method to calculate the relative motion field between the imaging spectroscopic data set. Then, to improve the alignment accuracy, the method of histogram statistics is used to obtain the system translation between the winged images. Furthermore, we introduce the weighted least-squares to achieve a final high-precise offset.The specific methods and steps are as follows.

3.1. Calculate the Relative Movement Field

The optical flow, as an important research direction of computer vision, describes the instantaneous apparent motion of the object (Horn & Schunck 1981). This method is mainly used in object recognition and object tracking to obtain the object motion information from the corresponding relationship between two neighboring image frames in the time domain.Compared with general natural images, the observed solar image is more complex,which always contains many different activity phenomena. The motions of these phenomena are not only nonrigid and random, but also often have a large range.There are many kinds of theoretical bases and mathematical methods for calculating optical flow. The Gunnar farneback’s algorithm (Farneb?ck 2003) is a kind of dense optical flow,which can calculate the motion information of pixels one by one and generate a Gaussian Pyramid of images with different resolutions to perform the multi-resolution image search.Although this algorithm is more time-consuming than other optical flow methods (such as the sparse optical flow), it can obtain a higher calculation accuracy for the images with complex structure and motion characteristics, so we chose it to calculate the motion information of each pixel for the solar images. The Gunnar farneback’s algorithm is complex in calculation and requires huge computation power. However,with the development of computer application ability in recent years, many computer languages have encapsulated the algorithm into a standard code base. In this work, we apply the standard OpenCV calcOpticalFlowFarneback processing package of the python to realize the operation above. The left and middle panels of Figure 2 present the local monochromatic images of Hα blue wing 0.4 ? and line center, respectively,and the right panel shows their relative movement field (offset vectorgraph)generated by the above algorithm of optical flow.From the figure we can clearly see the motion information of pixel.

3.2. Obtain the System Translation of Winged Images

The images of Hα winged and line center are observed in a coaxial focal plane system. Therefore, the distortion between them is primarily caused by the translation transformation.Theoretically,the movement field calculated by the optical flow reflects all the motion of each pixel, including not only the system translation of the image, but also the evolution of solar structure.However,because the time delay of the adjacent offband and line center image is too short for major solar feature evolution,that is the evolution of solar structure only occurs in a small number of pixels, but the system translation of image exists in all the pixels. Therefore, we can retrieve the system translation of the image by the statistical method.

Figure 2.The local monochromatic images of Hα blue wing 0.4 ?,line center and their offset vectorgraph(B040 relative to line center),15 pixel samples are used to display the offset distribution conveniently.

Figure 3. The histogram statistics for the movement field from Figure 2 (right) in the X and Y directions, respectively.

Figure 3 presents histogram statistics for the movement field from Figure 2 (right), in the X and Y directions, respectively.To reduce random errors, we use the following steps to obtain the system translation of winged images:First,we calculate the position of the center of gravity within a fixed range near the peak of histograms as the reference position. Then, we define the points within five pixels from the reference position as the effective points and discard points out of this range. It is necessary to ensure that the pixel is an effective point in both the X and Y directions. Finally, we measure the median value of all effective points as the system translation in X and Y directions, respectively.

3.3. Achieve the Final Offset of Winged Images

As is well known, the higher the similarity of the two images, the higher the calculation accuracy of their movement field (Smith et al. 2006), and the more effective points in thefixed range. Therefore, we can make full use of the number of effective points to achieve a high-precision final offset of winged images. That is, we can get three groups system translations between the three off-band images with the above method (B040 to center, R040 to center, and B040 to R040),and make the system translation which has more effective points to play a dominant role in determining the final offset.To achieve this purpose, solving the weighted least-squares is an excellent way.

Table 1 The Proportion of Effective Points, System Translation and Final Offset between the Images of Hα blue/red Wings and Line Center

Besides the system translations obtained with the above method, we also calculate the corresponding effective point proportions between two of the three spectroscopy images,respectively. The effective point proportion is the ratio of the number of effective points to the total number of pixels. Then,take the effective point proportions as the weighting coefficient of the least-squares method to obtain the final offsets of each off-band relative to each other. Table 1 lists the proportion of effective points,system translation and final offset between the images of Hα blue/red wings and line center. The two values in the square brackets of the system translation and final offset represent the offset in the X and Y axes,respectively.It shows that there is a higher proportion of effective points between the winged and line center image (0.891 and 0.856) than between two winged images (0.6). The system translation and the final offset are close but not identical, which plays a certain role in improving the alignment accuracy.

4. Alignment Accuracy

The asynchronous data sets from filter-based imaging spectroscopy are insufficient to assess the algorithm’s alignment accuracy.However,taking advantage of the grating-based spectrograph with a slit, we can create a test data set with its synchronized 2D spectroscopic images and customized offsets.In this way, the alignment accuracy can be evaluated by the pre-set translation displacements and the measured final offsets.In this research, we applied our algorithm to the 2D spectroscopic images observed by the Fast Imaging Solar Spectrograph (FISS, Chae et al. 2013) onboard the Goode Solar Telescope (GST, Cao et al. 2010). To verify the universality of the algorithm, we select the Hα spectrum over a quiet Sun region (the alignment of a quiet region is more difficult than that of region with obvious solar activities). The FISS observation was taken on 2021 June 8. The spectral resolution near the Hα line is 0.02 ? pixel?1, and the wavelength range is about 10 ?. The spatial resolution of FISS is 0 16 in the slit direction and in the scan direction.

The specific steps to evaluate the alignment accuracy are as follows: First, the two FISS images of winged and line center are composed of the bandwidth of 0.25 ? (same as NVST) as the original data set. Second, to reduce the randomness of the results, we randomly generate 100 groups of 2D floating-point numbers (x and y-axis) as the known translation offsets, and shift the winged image 100 times relative to line center one with the known translation offsets in x and y-axis respectively,to obtain a test set.Then,two alignment methods of the optical flow and the cross-correlation are used to calculate translation offsets for comparison with each other. In Figure 4, we show the 100 groups residual error of two alignment methods in the X-axis of the off-band 0.2 ? image. This residual error is the difference between the measured and known translation offset.From the figure, we can clearly see that the residual error of optical flow method is much smaller than that of the crosscorrelation.Finally,the root-mean-square error(RMSE)of 100 groups known and measured translation offset are calculated as the alignment accuracy of the two images. According to the above steps,we compose 10 winged images with different offband and evaluate their alignment accuracy relative to the image of line center,respectively.As the alignment accuracy is closely related to the image structures, we also calculate the similarity which is expressed by the correlation coefficient of two images with the maximum value of 1.The larger the value,the higher the similarity between two images. In Table 2, we list the similarity and corresponding RMSE of 10 off-bands,the RMSEoptand RMSEccrepresent the accuracies of the optical flow and cross-correlation method,respectively.Here,we only list the results of Hα blue wings, because the measured results of red and blue symmetric off-bands relative to the line center are very similar. It can be seen from Table 2 that the image similarity is larger and the RMSE of both methods are all relatively small at the wavelength near from the line center.For example,the similarity is about 0.788 at the off-band of 0.2 ?,and the alignment accuracy of optical flow and crosscorrelation can reach up to 0 004 and 0 068 respectively.However,the similarity decreases with the increase of off-band,and the accuracy of both methods also gradually decreases,especially the cross-correlation method. As for the off-band 0.6 ?, the winged image has a low similarity (0.217) with the line center one. The alignment error of the cross-correlation is about 6″ but it still can reach about 0 3 with the optical flow algorithm, which indicates the cross-correlation method has failed at this off-band, and the dependence of optical flow alignment on the image similarity is not as strong as that of cross-correlation method.In general,our optical flow algorithm works more accurately than the cross-correlation method. It could reach up to the accuracy of within 0 1 when the Figure 4.The residual error of the calculated and the known translation offset of Hα blue wing 0.2 ? relative to line center,the red and blue lines represent the results of cross-correlation and optical flow, respectively.similarity is more than 0.3 which corresponds to the maximum off-band is about 0.5 ?.If the off-band is more than this value,the alignment of optical flow does not meet the accuracy requirements of sub-pixel, so it is necessary to increase some intermediate off-band images as reference images for transition alignment.It is worth mentioning that the original data set used for accuracy calculation are real observations in the solar quiet region, which barely contains obvious solar structures. With stronger solar feature (sunspot or filament), we can expect a higher off-band of about 0.6 ? or higher with the alignment accuracy of 1 pixel.

?

Table 2 The Similarity and Alignment Accuracy of 10 Hα Off-band Images Relative to Line Center

5. The Products of NVST Winged Hα Data

The winged Hα data of NVST is processed with the flatfield, dark correction, high resolution reconstruction with the speckle masking technique. Alignment between off-band images with the method of optical flow described in the paper is now applied as well.In addition,the sequence images of single band are also be aligned. As the image structures of two adjacent frames in the same wavelength sequence are similar, their alignment can be well achieved by either the cross-correlation or the optical flow method.To ensure higher accuracy,we also use the optical flow alignment for sequence images. After the above processing, the well-aligned and scientifically ready winged Hα images are packaged in three standard forms: datacube, picture, and movie.

The scientific data is a FITS file of three-dimensional datacube,and each FITS is composed of pseudo-color off-band images. The detailed information of observation data can be obtained from a standard FITS header, including observation time, image size, exposure time, spatial resolution, bandwidth,and the order of off-band images, etc. In addition, in order to facilitate scientists to quickly acquire the off-band information,we synthesize the images and their off-band information into a JPEG picture, as shown in Figure 5, in which the composite pseudo-color image of three off-bands is shown in the left panel, and the Line-of-sight (LOS) velocity map and monochromatic images are in the right respectively.We can see that the composite pseudo-color image visually presents a wellaligned Hα data set.The fine-scale filamentary solar structures are sharp and clear without being smoothed or blurred by nonalignment. It is worth mentioning that the LOS velocity is calculated by the center of gravity method from the information of off-band images (Cai et al. 2017), which may exist certain errors due to too few scanning wave bands.We list here just for the convenience of scientists’reference and quick browsing.To avoid misleading the data users,we would not add the color bar to show the maximum and minimum value of the LOS velocity. If it needs to be used in scientific research, the algorithm for calculating LOS velocity of three off-band images would be further studied. Furthermore, we make the sequence JPEG pictures into a movie for scientists to quickly browse the evolution of the observation target over time.

Figure 5. The winged Hα data of the NVST.

6. Conclusion and Discussion

The high-precision alignment between the winged images has always been an important problem restricting multi-band images joint research. Based on the current situation of NVST Hα imaging spectroscopic observation, we introduce a highprecision alignment algorithm for the off-band images based on the optical flow, and evaluate the alignment accuracy and criterion reliability under the different similarity (off-band)with the 2D spectral data of FISS/GST. The research shows that the optical flow method can reach up to a high alignment accuracy in the near wing from line center, such as 0 004 and 0 095 in the off-band of 0.2 ? and 0.4 ?, respectively, which correspond the similarity of about 0.8 and 0.4. It can meet the alignment accuracy requirements of 0 1(within 1 pixel)under the similarity ≥0.3, which corresponds to the maximum offset wavelength of about 0.6 ?. Otherwise, it is necessary to increase more intermediate off-band images for transition alignment.This result is helpful to select the appropriate offset wavelength for the routine three-bands observations of MHRIS/NVST.After the fine alignment,we pack all off-band and line center images into three kinds of standard formats for the convenience of NVST data users. In the paper, we mainly describe the application of the optical flow method in three offband images. Actually, it is also suitable for the alignment of more off-bands images, and the alignment accuracy would be higher with the transition alignment of more off-band images.

In order to improve the accuracy of optical flow alignment algorithm as mentioned in Section 3.2, we make histogram statistics of the movement field on each pixel, and select a certain range near the peak of the histograms to calculate the reference position. Technically, the histogram distributions are different with the different similarity between aligned images,so the reference range should vary for different image compositions. However, we determine a fixed reference range by trial and error in this work, and will study an adaptive algorithm to select the reference range for further increasing the alignment accuracy in the future. In addition, we use the offband images composed of the 2D spectral data of FISS/GST to evaluate the alignment accuracy of algorithm.Those composed images are ideally strictly aligned. However, as the inevitable slight inaccuracy of the spectrometer system,the inclination or curvature of spectral lines always exists in the spectrum(Cai et al.2017).Although the curvature of spectral lines have been corrected in the post-processed spectrum (Chae 2004),there is still a residual inclination in the spatial direction (the offset is about 10 pixels in the total wavelength range 10 ?).This residual inclination will systematically shift the composed off-band images in one spatial direction,and different off-band wavelengths correspond to different translation offsets. However, only the data with the wavelength within±1 ? of spectrum are used, and the bandwidth between the composed monochromatic images of two adjacent wavelengths is 0.1 ?,which corresponds to an inclination only about 0.1 pixel.Therefore, to avoid the calculation error introduced by correcting residual inclination of spectral lines, the influence of the shifting is included in the values of RMSE,as shown in Table 2. Therefore, the actual alignment accuracy should be higher than the result proposed in this paper.

The authors are appreciated for all the help from the colleagues in the NVST team.We are also indebted to the GST teams for providing the FISS data. This work is supported by the National Natural Science Foundation of China (NSFC)under grant numbers 11903081, 12073077, 11973088,11873027, U2031140, 11833010, U1831210, Yunnan Natural Science Foundation of China (201901U070092), and the CAS Light of West China Program under numbers Y9XB015 and Y9XB019.

ORCID iDs

Yun-Fang Cai https://orcid.org/0000-0002-4956-4320

Xu Yang https://orcid.org/0000-0002-3238-0779

Kai-Fan Ji https://orcid.org/0000-0001-8950-3875