Guolu Go , Yng Li , Jiqi Li , Xueyun Zhou , Ziqin Zhou

a Yaan Meteorological Observatory, Sichuan Meteorological Bureau, Yaan, China

b College of Atmospheric Science, Chengdu University of Information Technology, Chengdu, China

c Leshan Meteorological Observatory, Sichuan Meteorological Bureau, Leshan, China

d Nanchong Meteorological Observatory, Sichuan Meteorological Bureau, Nanchong, China

Keywords:Rainstorm Short-term prediction method Back-propagation neural network Hybrid forecast model

A B S T R A C T Rainstorms are one of the most important types of natural disaster in China. In order to enhance the ability to forecast rainstorms in the short term, this paper explores how to combine a back-propagation neural network(BPNN) with synoptic diagnosis for predicting rainstorms, and analyzes the hit rates of rainstorms for the above two methods using the county of Tianquan as a case study. Results showed that the traditional synoptic diagnosis method still has an important referential meaning for most rainstorm types through synoptic typing and statistics of physical quantities based on historical cases, and the threat score (TS) of rainstorms was more than 0.75.However, the accuracy for two rainstorm types influenced by low-level easterly inverted troughs was less than 40%. The BPNN method efficiently forecasted these two rainstorm types; the TS and equitable threat score (ETS)of rainstorms were 0.80 and 0.79, respectively. The TS and ETS of the hybrid model that combined the BPNN and synoptic diagnosis methods exceeded the forecast score of multi-numerical simulations over the Sichuan Basin without exception. This kind of hybrid model enhanced the forecasting accuracy of rainstorms. The findings of this study provide certain reference value for the future development of refined forecast models with local features.

1. Introduction

Rainstorms and floods contribute about 68% of all natural disasters in China. Indeed, China is one of the nations most prominently influenced by rainstorms and floods ( Tao et al., 1979 ). For instance, an estimated 79 deaths and economic losses of 11.6 billion RMB were incurred as a result of the so-called “Beijing 721 extraordinary rainstorm ”( Wu et al., 2017 ). Research has shown that the intensity and frequency of extreme strong precipitation have increased owing to the rise in global average temperature, which potentially enhances the risk of flooding too( Westra et al., 2013 , 2014 ).

Before the 1990s, the method for forecasting rainstorms was mainly based on weather charts, hydrodynamic theory, and the forecaster’s experience. After the 1990s, numerical models gradually came to prominence along with the rapid development of computers. In recent years,artificial intelligence technology has developed at an astonishing speed,and is rapidly emerging within the field of meteorology.

Wheeling (2020) evaluated the manifestation of generative adversarial networks within the fields of weather prediction and climate forecasting, and found that they produce better results in climate models.Shen et al. (2020) used a long short-term memory neural network to realize the forecasting of summertime seasonal-scale precipitation. Google uses a neural weather model for precipitation forecasting (MetNet) to predict the probability of precipitation in the continental United States( S?nderby et al., 2020 ). However, to date, these applications of neural networks to predict precipitation have mainly concentrated on mediumto-long-term weather and climate forecasting. Short term precipitation forecasting, especially for heavy rainstorms, is still a considerable challenge.

Another important problem in using machine learning to predict precipitation is the lack of interpretability in physics ( Reichstein et al.,2019 ; Ebert-Uphoffand Hilburn, 2020 ). For example, a potential law for neural networks was found through fitting the input and output datasets,but this law only had mathematical meaning. In contrast, the traditional synoptic typing method has a distinct physical significance, but produces a more subjective forecast result for extreme weather processes.Hence, in order to cover the shortcomings of the above two methods,this paper explores the short-term (future 24 hours from 2000 LST to 2000 LST the following day) forecasting of rainstorms through combining the synoptic method and the machine learning method. Meanwhile,this study also adds the statistics of physical quantities based on the synoptic typing. Compared with previous synoptic diagnosis methods based on numerical prediction ( Doswell et al., 1996 ; Willem and Lisa, 2005 ;Gao et al., 2013 ), this study places more emphasis on real-time observational data.

This work used the county of Tianquan as the study area. Tianquan is located in the Hengduan mountains, at the junction of the Tibetan Plateau and Sichuan Basin, and is also known as “Ya-An-Tian-Lou ”, because it’s the precipitation center of the Sichuan Basin ( Peng et al., 1994 ;Yu et al., 1994 ; Zeng et al., 1994 ; Li and Zhang, 2011 ). The complex topography in this area makes rainstorm forecasting a difficult problem( Yu et al., 1997 ; Lu et al., 2009 ).

The remainder of the paper is structured as follows: Section 2 describes the data and methods used in this study. Section 3 shows how we combined the two methods to forecast rainstorms. Section 4 summarizes the principal conclusions and presents some further discussion.

2. Data and methods

2.1. Data

Hourly surface and twice-daily sounding observation datasets

in

situ

from 2016 to 2019 were used to analyze the warm-season precipitation (May—September) in Tianquan. The distribution of surface stations is shown in Fig. 1 (a). The sounding observatories around Tianquan include Wenjiang, Xichang, Ganzi, and Batang. These datasets can be accessed from the Meteorological Unified Service Interface Community( http://10.194.89.55/cimissapiweb ).

Fig. 1. Topography and precipitation in Tianquan. (a) Topography and the stations’ locations and elevations (dots refer to the locations of automatic stations;the star is the location of the city; and their elevations correspond to the righthand color bar). (b) Fitting curve between annual precipitation and elevation.(c) Accumulated rainstorm frequency during the warm season of 2016—19.

The warm season during 2016—19 contained 612 days. These days,bounded by 2000 local time, were divided into 564 samples (excluding missing data) and classified into four types on the basis of intensity and scope of precipitation, as follows: (i) regional rainstorms (greater than 3 stations with precipitation of 50 mm/day; total of 48 samples); (ii)scattered rainstorms (greater than 1 and fewer than 3 stations precipitation of with 50 mm/day; total of 28 samples); (iii) regional heavy rain (greater than 3 stations with precipitation of 25 mm/day; total of 49 samples); and (iv) ‘other’ (general weather processes aside from the above three types; total of 439 samples).

2.2. Synoptic system typing

This study adopted a similar synoptic typing method to previous research to classify the precipitation processes over Tianquan ( Peng et al.,1994 ; Zhou et al., 2015 ; Xiao et al., 2017 ; Yi et al., 2019 ). The principal synoptic systems and identification criteria were as follows: (1) Tibetan Plateau low-pressure system (TPL): the stations of Ganzi and Batang had negative variable pressure within 24 hours over 500 hPa; (2) southern vapor and instability low-level horizontal transport (SVT): Xichang station had strong southerly winds exceeding 8 m sover 700 hPa;and (3) low-level easterly inverted slot over the Sichuan Basin (EIT):Wenjiang station had an easterly wind component over 850 hPa. After synoptic typing had been carried out according to the above standards,the physical quantity thresholds corresponding to the synoptic patterns were calculated ( Table 1 ). Then, we searched all historical weather cases satisfying these synoptic patterns and physical quantity thresholds, and calculated the historical probability of producing a regional rainstorm,scattered rainstorm, and regional heavy rain ( Table 2 ).

Table 1 Thresholds of physical quantities over the different rainstorm synoptic types. VP surf and RH represent the water vapor pressure of the surface and relative humidity, respectively.

Table 2 Historical probability of different precipitation intensities satisfying the conditions of synoptic typing and physical thresholds.

Table 3 Score of predicted values corresponding with the real tag.

2.3. Back-propagation neural network

This study adopted a back-propagation neural network (BPNN)model of four layers, including the input layer (33 neurons), two hidden layers (7 ×9 neurons), and a fourth classified output layer. Firstly,the normalization physics features were transmitted into the input layer, and the tanh activation function nonlinearized these features with weights and bias to connect the next layer. Then, through comparing thedifference between the tagged real values and the predicted results of the output layer, the loss value between the fact and prediction was calculated by the loss function of cross entropy ( Ho and Wookey, 2020 ).Finally, the stochastic gradient descent with momentum method was adopted to update the weights of the last iteration to obtain an optimal classification result ( Bottou, 2010 ).

In order to improve the generalization ability and reduce the overfitting of the BPNN, aside from setting up a smaller L2 regularization coefficient (

α

= 1 ×10) to penalize the term with a larger weight factor, this study calculated the fitting score of the training set and test set in the different iterations. The predicted results obtained different scores in terms of the basis of the predicted type and real tag ( Table 3 ).Eq. (1) was used to evaluate the forecasting ability of the hybrid model:

The calculation of Eq. (1) could be divided into two steps. Firstly, the average scores of the four precipitation types could be gained according to Table 3 . Then, the average scores were multiplied by different weights to gain the ensemble score. The regional rainstorm type was paid a higher weight.

2.4. Forecast evaluation

The universal threat score (TS) and equitable threat score (ETS) verification techniques were adopted to assess the reliability of the BPNN and synoptic diagnostic methods:

Here,

N

,

N

, and

N

in Eq. (2) are the quantities of ‘correct’,‘false alarm’, and ‘missing alarm’, respectively; and

N

in Eq. (4) represents the quantities of ‘no forecast’ and ‘no appearance’ in reality. The ETS technique takes into account the situation of

N

more than the TS method, and largely punishes the ‘false alarm’ and ‘missing alarm’ rates,but a mass of

N

samples would cause the ETS to be obviously on the low side ( Wang and Yan, 2007 ; Sun et al., 2015 ). Therefore, this paper comprehensively refers to the TS and ETS results to evaluate the forecast quality, and compares them with the TS and ETS results of the precipitation forecasts of multiple numerical models over the Sichuan Basin during May—September 2019 (data from the China Meteorological Administration, http://10.1.64.154/areaHighResolution ).

3. Results

The rainstorms and cumulative annual precipitation over the Tianquan are affected by the complex valleys and rivers of the Hengduan Mountains ( Fig. 1 (a)). Through fitting the correlation of annual precipitation and terrain height, Fig. 1 (b) shows that the annual precipitation increased by 126.1 mm every 100 m, but when the elevation exceeded 1000 m it sharply reduced with the increase in height. This result is similar to previous findings on the orographic rainfall in Yaan region ( Peng et al., 1985 ). The areas where the accumulated rainstorm frequency exceeds 20 times are also mainly concentrated in the areas below 1000 m above sea level ( Fig. 1 (c)).

Fig. 2. (a) Correlation coefficients between the precipitation intensity and physical quantities in the situations of different types. (b) Loss curve and forecast score of the testing and training sets changing with the number of iterations.

Fig. 3. Confusion matrix of prediction type and real type over the (a) training set and (b) test set. The diagonal line separates the TS (blue) and ETS (green).

Through statistical analysis of Tianquan’s topography and rainstorms, the above results demonstrate that rainstorms in Tianquan had a higher frequency than other areas in the Sichuan Basin and a close relationship with topography. Hence, in order to predict rainstorms in Tianquan, first, the synoptic patterns of rainstorms were divided into seven types in terms of the configuration of the atmospheric circulation,and then the corresponding physical quantity thresholds were calculated( Table 1 ; methods in Section 2.2 ).

Through retrospectively analyzing all historical weather cases that satisfied the corresponding synoptic and physical conditions, the historical probabilities of regional rainstorms, scattered rainstorms, and regional heavy rain in the synoptic method were calculated ( Table 2 ). The TS exceeded 0.75 for five of the seven patterns. This result demonstrates that the synoptic diagnosis method carries a certain reference value.However, this statistical synoptic diagnosis method will make a large false alarm rate inevitably existed for some rainstorm weather processes( Tao et al., 1979 ). In particular, for the synoptic patterns influenced by low-level easterly winds, the TS was less than 0.4. However, low-level easterly winds are a non-negligible synoptic system for trumpet-shaped topographical precipitation ( Sun and Yang, 2008 ; Zhang et al., 2013 ;Sun et al., 2019 ). The cases with low-level easterly wind reached 40 cases in Tianquan, accounting for 83% of the total.

In these synoptic patterns, the primary physical difference was the local buoyancy condition of the air parcel. If easterly winds existed,the air lifting was easily satisfied by low-level orographic convergence uplift. Conversely, high convective available potential energy (CAPE) is needed as the energy source of air parcel lifting. The rainstorm process under the influence of easterly wind didn’t require too high local CAPE and water vapor, which makes it hard to distinguish the rainstorm and weaker precipitation process only by judging local physical condition.

In order to predict the rainstorm of EIT and TPL + EIT patterns,the BPNN model was applied into the rainstorm forecast (methods in Section 2.3 ). Fig. 2 (a) exhibited a poor correlation between the physics quantities and precipitation. Only the CIN in the TPL pattern could pass the significance testing of 0.05 level. This result demonstrated the precipitation as the result of the atmospheric dynamic and thermal function coupling could not be predicted by the simple linear relation, but BPNN could simulate this kind of nonlinear relation of precipitation and physics quantities.

The method outlined in Section 2.4 was used to assess the overfitting effect of the BPNN model. Fig. 2 (b) shows the convergence rate of the model and score curve of the forecast. When the iteration reached 1835 times, the forecast score of the test set had no obvious increase, but the forecast score of the training set was still growing, which means that the overfitting phenomenon appeared. Hence, 1835 iterations were selected as the optimal iteration time, and the weight coefficient in this iteration was also used as the final model configuration.

The classification forecast result of the BPNN showed high accuracy for the regional rainstorm type and ‘other’ type, but lacked enough understanding for the scattered rainstorm type and regional heavy rain type ( Fig. 3 ). The result between the test set and training set maintained consistency. Two residual types tended to be divided into general precipitation processes. This may be due to the tiny differences in physical quantities and synoptic situations among the ‘other’ type, scattered rainstorm type and regional heavy rain type. Relatively, the boundary between the regional rainstorm type and ‘other’ type could be more legible.

Fig. 4. Hybrid model based on the BPNN and synoptic diagnosis.

Compared with the TS and ETS of multiple models over the Sichuan Basin in 2019, the BPNN and synoptic diagnosis methods both had higher scores than the models, including GRAPES_MESO-3km, GRAPES_MESO-9km, GRAPES_GFS-25km, ECMWF-12.5km, and NCEP_GFS-25km. In the field over 50 mm, the highest TS and ETS were 0.171 and 0.164, both from ECWMF-12.5km. In the field under 25 mm, the highest TS and ETS were 0.673 and 0.319, both from GRAPES_MESO-3km. The hybrid model in this paper sharply improved the accuracy over the multi-model forecast at the local rainstorm forecast.

4. Conclusion and discussion

In order to improve the ability to forecast short-term rainstorms over complicated terrain, this study explored the application of synoptic diagnosis and BPNN, and analyzed their limitations and advantages in short-term rainstorm forecasting. Fig. 4 summarizes how to use the hybrid model to predict rainstorms via a flowchart. Firstly, the physical quantities including water vapor, energy, instability condition, and so on should be calculated after essential quality control (green box). Then,the weather needs to be classified into different patterns according to the circulation fields (blue box). And then, the weather is judged on whether it satisfies the physical quantity thresholds corresponding to the synoptic patterns, and the historical probability of rainstorms is obtained (yellow box). If the synoptic pattern belongs to the EIT and TPL + ETI pattern, the normalized physical quantities needs to be input into the BPNN model to return a BPNN prediction result (orange box). Finally, the short-term rainstorm forecast is produced (purple box).

The principal conclusions from this study are as follows:

(1) The trumpet-shaped topography had an obvious amplification effect on precipitation. Under the elevation of 1000 m on the windward slope, the annual precipitation increases by 126.1 mm per 100 m in elevation. The low-level easterly winds are an important influence for rainstorm processes over this topography; it is a non-negligible synoptic system and is an important reason behind the difficulty in precipitation forecasting.

(2) Traditional synoptic diagnosis still remains a better reference for five of the rainstorm process types, in which the TS even exceeded 0.75. However, this method produced a large false alarm rate for two residual rainstorm types influenced by the low-level easterly winds;the accuracy was less than 40%.

(3) Precipitation is the result of the atmospheric coupling of multiple complex systems, and an obvious linear correlation does not exist between precipitation intensity and physical quantities. The BPNN was able to effectively simulate the nonlinear process of precipitation. The forecast result of the test set showed that the TS and ETS of the regional rainstorm for the two residual types were 0.80 and 0.79.For the weaker precipitation process (

<

25 mm), the TS and ETS even reached 0.84 and 0.28, respectively, and these results were superior to the forecast score of multiple models over the Sichuan Basin.

Despite machine learning being able to sharply boost the accuracy of weather forecasts, many limitations still exist. The inexplicability,uncertainty and inevitable hyper-parameter in machine learning are still a big challenge ( Coyle and Weller, 2020 ). In particular, how to build a hybrid model and new climate model that deeply integrate machine learning and physics processes will be a crucial development direction in the future ( Schneider et al., 2017 ; Reichstein et al., 2019 ). This paper makes a certain exploration from the point of combining the BPNN and synoptic diagnosis methods, but how to integrate machine learning into complex earth system models still needs to be further investigated.

Funding

This work was jointly supported by the National Key Research and Development Program on Monitoring, Early Warning and Prevention of Major Natural Disasters [grant number 2018YFC1506006 ]and the National Natural Science Foundation of China [grant numbers 41805054 and U20A2097 ].