Jiancheng Zou ,Tianshu Chen and Xin Yang

Abstract:Heart rate is an important physiological parameter for clinical diagnosis,it can infer the health of the human body.Thus,efficient and accurate heart rate measurement is important for disease diagnosis and health monitoring.There are two ways to measure heart rate.One is contact type and the other is non-contact.Contact measurement methods include pulse cutting,electrocardiogram,etc.Because of the inconvenience of this method,a non-contact heart rate method has been proposed.Traditional non-contact measurement method based on image is collecting RGB three-channel signals in continuous video and selecting the average value of the green channel pixels as the heart rate signal for processing.However,this method is not accurate when the pixel values are changing greatly.To overcome this problem,non-contact real-time heart rate measurement method is proposed in this paper based on pixel standard deviation.Because of the changes in skin color caused by heart rate,the standard deviation signal of the green channel pixels in the region of interest (ROI) is filtered and extracted by the forward and inverse Fourier transform respectively,measuring the heart rate.The experimental results show that the improved algorithm can measure heart rate with faster speed and higher accuracy comparing to traditional methods.And we hope that our algorithm can apply in intelligent elder caring.

Keywords:Photoplethysmography,face detection,heart rate measurement.

1 Introduction

As the global aging trend increases,pensions have become problem for governments.According to the World Health Organization,Cardiovascular mortality has reached a record high,surpassing the mortality rate of cancer and other diseases.Cardiovascular disease killed nearly 17.5 million people in 2011,accounting for 31% of the total death toll in the world [Alwan (2011);Mathers and Loncar (2006)].Heart rate is an important parameter for monitoring cardiovascular disease,especially for the elderly and patients with chronic cardiovascular disease.Therefore,real-time monitoring of heart rate can gain valuable time for disease discovery and prevention.At present,the heart rate detection method is mainly based on contact types,such as electrocardiograph [Kilpatrick and Johnston (1994);Chen,Gyawali,Liu et al.(2017)],heart rate measurement watch and other equipment.However,this approach is inconvenient for the elderly.So a noncontact heart rate measurement is a candidate.

The main principle of non-contact heart rate measurement is photoplethysmography(PPG).The earliest PPG method for heart rate measurement was proposed by Wieringa F et al.[Wieringa and Mastik (2005)].They used PPG technology to extract heart rate signals at different wavelengths.Takano C et al.[Takano and Ohta (2007)] realized the extraction of heart rate signals under black light and white CCD.However,the measurement resultsoftwo methods are subjective to the change of illumination environment and motion.In 2008,Hu S et al.[Hu,Zheng et al.(2008)] proposed feasibility study of imaging photoelectric volume pulse wave tracing methods.It can collect human fingertip video under non-contact conditions by using LED light and imaging instrument,collecting the photoelectric volume change information of the fingertips under different light sources.In 2010,Poh et al.[Poh,M cduff and Picard(2010)] proposed a principle of non-connected heart rate measurement based on blind source separation [Zeng and Feng (2014);Yang,Li,Wang et al.(2011);Cardoso and Souloumiac (2000)] and applied it to a simple webcam to record images of face video.

The traditional PPG algorithm separates the pixel values of the e ROI extracted from the video into RGB three-channel.After separation,the average value of the green channel pixels is selected as the RGB value of the ROI in the video.We generalize the traditional PPG algorithm in this paper.Firstly,face is detected by Adaboost algorithm,and the nonface background area is removed,then ROI is located.Secondly,the RGB three-channel separation is performed on the ROI pixel,and the standard deviation of the green channel is selected for processing.Thirdly,the pixel information from the time domain is converted to the frequency domain by fast Fourier transformation to perform filtering processing.Finally,the frequency domain is transformed into the time domain by inverse Fourier transform.And the pulse wave signal and the heart rate are obtained.

The experimental results show that the improved algorithm can measure heart rate with faster speed,and is more accurate and less error than the traditional average extraction algorithm.This method can be applied in intelligent elder caring.

2 Related methods

Photoelectric volume description method,introduced by Hertzman [Hertzman (1938)],is a non-contact physiological signal detection technology.It has developed rapidly in recent years and has been widely used in the field of blood pressure,blood oxygen,heart rate measurement,etc.,and has become an emerging hotspot in the field of biomedical engineering [Cavalleri,Morstabilini and Reni (2005)].Video-based non-contact heart rate measurement technology achieves heart rate measurement by combining photoplethysmography with pattern recognition processing.The pulse waveform is traced by photoplethysmography to obtain a photoplethysmographic pulse wave.The essence of heart rate is the arterial pulse [Lin (2011);Zhao,Meng and Chen (2006)].The pulse can reflect the cardiovascular condition very well,so the heart rate can be obtained by obtaining the pulse wave.

The optical theoretical basis of PPG:Lamber-Beer describes the attenuation characteristics of light and uniform non-scattering media with absorption properties,which is the theoretical basis for PPG work [Downing and Stranieri (1980)].According to Lambert-Beer’s law,after the monochromatic light of intensity enters the medium,part of the light is absorbed by the medium,and part of the light is transmitted through the medium.According to the exponential decay law,the transmitted light intensity changes with the distance.

The Mathematical expression of Lambert-Beer's lawis:

Oxygenated hemoglobin in human capillaries can absorb certain light.Due to the periodic pulsation of the heart,the content of oxygenated hemoglobin in the blood vessels changes,thereby the blood absorbs light and reflects light have changed.The facial area of the human body contains the most abundant capillaries.In the color space,a periodic color change of the human face skin color is formed.As the concentration of hemoglobin changes when the light is irradiated onto the skin,the device captures color changes and physiological information of the human tissue,then heart rate can be calculated.

3 Proposed algorithm

In this section,the experimental process and PPG-standard deviation algorithm are explained in detail,the real-time heart rate is obtained through image acquisition and proposed algorithm.

3.1 Face recognition and positioning

Face recognition is the processing of each frame of the image.In order to achieve the goal,it is necessary to detect whether there is a face in the captured video,and if so,take an algorithm to calculate the coordinates of its position.

In this paper,Haar-like features [Hanai,Hori,Nishimura et al.(2009)] are used to extract facial features in images for calculating eigenvalues.There are four types of Haar-like features:edge features,linear features,center features,and diagonal features,which is combined into feature templates.The feature template has two rectangles,white and black.The feature value of the template is defined as the sum of the white rectangle pixel values minus the black rectangle pixel values.The Haar eigenvalue reflects the grayscale variation of the image.For example,some features of the e face can be described simply by rectangular features [Shengqun,Zhiping,Wencheng et al.(2018)],for example,the eyes should be darker than the cheeks.The sides of the e bridge are darker than the nose,and the mouth is darker than the surrounding.However,the rectangular feature is only sensitive to some simple graphic structures,such as edges and line segments,so it can only describe the structure of a specific direction (horizontal,vertical,diagonal).

Haar-like feature calculation is a fast algorithm that can find all the pixel sums in the image by traversing only one image,which greatly improves the efficiency of image feature value calculation.The main idea of the integral graph is to store the sum of the pixels of the e rectangular area formed by the image from the starting point to each point as an element of an array.When calculating the pixel of a certain area,the elements of the e array can be directly indexed,it is not necessary to recalculate the pixel sum of this area,thus speeding up the calculation (it is called the dynamic programming algorithm).The integral map can use the same time (constant time) to calculate different features at various scales,thus greatly improving the detection speed.

Then the classifiers are trained by the Adaboost algorithm [Cao,Miao,Liu et al.(2013)].each Haar feature weak classifier is trained with the same training set to extract facial features in the image.Then the Adaboost algorithm is used to classify the trained classifiers on different training sets to form a strong classification that can quickly locate and remove facial image.

Figure1:Collected face data set

3.2 ROI selection and positioning

According to the principle of photoplethysmography,the color of the skin can change with beating of the heart.Due to the richness of facial capillaries,the color change is more obvious than the rest of the body.

In 2008,Verkruysse et al.[Verkruysse,Svaasand,and Nelson (2008)] and others selected different regions of the e face as the ROI to extract the photoplethysmography pulse wave signals respectively.These experimental results show that the photoelectric pulse wave signal extracted from the large area of the forehead contains the least noise and is relatively clean.Therefore,the forehead area is selected as the ROI to achieve heart rate detection in this paper,which can reduce noise interference.

3.3 Acquisition of heart rate information

During the experiment,the subject sits 1m away from the camera and remains stationary to avoid large movements to obtain 200 frames of face image in video.Taking normal ambient light as the source of the experiment,extracting the ROI from each frame of the image in the video,and the ROI is obtained for RGB three-channel separation.After separation,the average value of the G channel pixels is obtained as the pixel value of the ROI.

Average function:

Figure2:Green channel

Figure3:Red channel

Figure4:Blue channel

As can be seen from Figs.2-4,the curve noise of the green channel is smaller among the mean curve of the RGB three-channel pixel values of the e ROI.According to physiological data,the melanin of the skin absorbs a large number of short-wavelength waves,and the water also absorbs a large amount of ultraviolet and infrared light.Comparing to red light,most of the green and yellow light entering the skin tissue is absorbed by erythrocyte.Green light can be absorbed by oxyhemoglobin and deoxyhemoglobin.In summary,it is more explicit to select pixel value of the green channel to extract the video heart rate signal.

Due to the movement of the measurement part,natural light,fluorescent lamps,and other interference,the curve will have a number of bumps which make it less smooth and affect the calculation of the subsequent extracted heart rate interval.Therefore,the expected(E(X)),variance (D(X)),and standard deviation (σ) of the green channel in the ROI are calculated separately for comparison.

Expected function:

Variance function:

Standard Deviation function:

The results show that the standard deviation curve of the pixel value is more stable than the mean curve,and the external interference is smaller.Therefore,the standard deviation curve of the ROI is selected for heart rate signal extraction.

Figure5:Distribution of regional pixel values

3.4 Heart rate signal filtering

The obtained heart rate signal still has more noise interference,so the filtering and denoising are required.The filtering process is performed by many methods.In this paper,the Fast Fourier Transform algorithm is selected,which transforms the signal from the time domain to the frequency domain for filtering,and then studies the spectral structure and variation of the signal.When a time domain continuous function is given,its Fourier transform is:

However,the pixel signals for each frame of the image are discrete in the time domain.The image is transformed through discrete time Fourier.First,we take samples in the continuous functionx(t),and assume that the sampling interval is Δt,the sequence of functions iss(t),so the resulting sampled signal xs (t)is:

Fourier transform is:

Simplify the result is:

The time domain interval is normalized,then x (nΔt)becomes a discrete sequence x[n],that is:

For the obtained heart rate signal,we filter by Eq.(10).Thereby the heart rate spectrum after noise reduction is obtained.

Figure6:Signal map in the frequency domain

3.5 Real-time heart rate acquisition

The green channel pixel values in ROI are filtered by a fast Fourier transform algorithm to convert information in the time domain to the frequency domain.Thereby reducing noise interference,and then reducing the filtered information to the time domain by inverse Fourier transform.As can be seen from Fig.6,the result shows a more obvious waveform,and it can be observed what the period of the skin color transformation is,that is,the value of the heart rate.

Figure7:Heart rate information comparison chart before and after treatment

According to Fig.7,if two photos are stored per second for processing,the real-time heart rate is calculated as:

HR= (138-18) ÷ 2 = 60(bmp)

4 Experimental results and analysis

During the experiment,the subject sits 1 m away from the camera and remains stationary to avoid large movements and it can obtain 200 frames of face image in video.The source of the experiment is ambient light.The experimental results are shown in Tab.1.The results show that compared with the traditional algorithm,the proposed algorithm has a smaller error and faster speed.This method can be applied in intelligent elder caring.

Table1:Experimental results

As can be seen from Tab.1,it shows that the errors in improved algorithm are smaller than the errors in traditional algorithm.PPG-standard algorithm has faster speed and higher accuracy than traditional algorithms.

Figure8:Experimental screenshot of real-time heart rate measurement by different testers

5 Conclusion

Heart rate,as one of the important physiological parameters for detecting physical health,plays an important role in the detection and prevention of diseases.The traditional heart rate detection method is mainly contact type,which brings inconvenience to the daily wear of the elderly.The non-contact real-time heart rate measurement based on the principle of photoelectric volume description method not only reduces the interference of external light source to the experiment,but also can measure without affecting the daily life of the measurer.This will be important for the development of wearable medical technology in the future.

In this paper,compared with the traditional PPG algorithm,the improved algorithm is more adaptable to the environment and has higher accuracy and robustness.

Acknowledgement:The work of this paper is supported by the National Natural Science Foundation of China under grant numbers 61572038,and the Innovation Project Foundation NCUT.