基于機(jī)器視覺的魚類模式生物在線監(jiān)測技術(shù)方法研究

水質(zhì)的生物式監(jiān)測方法靈敏度強(qiáng),適應(yīng)性廣,能夠適應(yīng)于多種場合的水質(zhì)監(jiān)測工作并廣泛地應(yīng)用在水環(huán)境的質(zhì)量檢測和水質(zhì)安全預(yù)警中,它能夠彌補(bǔ)理化監(jiān)測方法實(shí)時性和綜合性較差的缺點(diǎn),能夠產(chǎn)生巨大的社會和經(jīng)濟(jì)效益[1]。其基本原理是利用水生生物個體、種群的健康狀態(tài)、生理特征、運(yùn)動特征等的變化來指示水體環(huán)境質(zhì)量的變化,表征環(huán)境污染狀況,從生物學(xué)角度為水體環(huán)境質(zhì)量監(jiān)測和評價提供依據(jù)[2-3]。

魚類是最早運(yùn)用的模式生物。隨著機(jī)器視覺技術(shù)的不斷成熟和廣泛應(yīng)用,視覺輔助的水質(zhì)自動監(jiān)測成為一種可能。這種非接觸式的監(jiān)測手段的優(yōu)勢是無需破壞魚類的正常生活環(huán)境,在自然狀態(tài)下觀測魚類的生理特征和運(yùn)動特性。魚所表現(xiàn)出來的“逃避”、“呼吸加快”等異常行為顯然要遠(yuǎn)遠(yuǎn)提前于生物的病理損傷或死亡,為水環(huán)境的在線生物監(jiān)測技術(shù)的有效性提供有力的理論支持。通過利用機(jī)器視覺方法,目前國內(nèi)研究大部分僅將魚類的一部分運(yùn)動特征(魚類的游動速度、游動加速度、游動高度、轉(zhuǎn)彎次數(shù))為指標(biāo)應(yīng)用在水質(zhì)監(jiān)測中[4-21],缺少對胸鰭、尾鰭運(yùn)動等信息的統(tǒng)計,也缺少生理特征方面(呼吸頻率、呼吸深度等)的重要信息。這使得在一定程度上的局限了觀測數(shù)據(jù)的完整性,也勢必會降低水質(zhì)監(jiān)測的準(zhǔn)確性。

本文以青鳉魚為模式生物,在機(jī)器視覺的基礎(chǔ)上,實(shí)時監(jiān)控魚的胸鰭、尾鰭擺動頻率,魚的呼吸頻率、呼吸深度,達(dá)到獲得更有魯棒性、健碩性的觀測數(shù)據(jù)。

1 青鳉魚生理特征檢測方法(The detection method of medaka physiological characteristics)

魚鰓的呼吸對污染物質(zhì)十分敏感,對于半致死濃度的污染物在30 min內(nèi)便可被檢測到,亞致死的污染物也可在24 h內(nèi)被檢測出。在有污染物存在的情況下,魚鰓的呼吸會極速加快,并且變得無規(guī)律。Gerhardt等[21]將監(jiān)測電極直接安裝在魚鰓上,監(jiān)視當(dāng)前呼吸頻率,分析數(shù)據(jù)并與正常的呼吸頻率相比較,以判斷水是否被污染。Cairns等[22]在水箱中插入網(wǎng)電極,通過注入電極的信號來記錄魚的呼吸運(yùn)動。這2種方法雖然能一定程度地反映魚的真實(shí)生理狀況,然而無法排除裝置本身對魚類反應(yīng)的影響且得到的信號往往十分微弱,不直觀,容易形成較大的識別誤差。

基于視覺的方法是以非接觸式的監(jiān)測手段,通過電荷耦合器件(charge coupled device,CCD)相機(jī)在自然狀態(tài)下觀測魚類的生理特征和行為特性。本文通過分析青鳉魚魚鰓部位像素信息統(tǒng)計得到結(jié)果“顏色分布規(guī)律表(CDT)”,如表1所示。再結(jié)合機(jī)器學(xué)習(xí)算法中的線性分類器SVM(支持向量機(jī)),可準(zhǔn)確定位得到青鳉魚魚鰓區(qū)域,實(shí)驗(yàn)結(jié)果如圖1和2所示。其中SVM分類器是由顏色空間轉(zhuǎn)換后分別得到的S(HSV)通道、a(Lab)通道組合成的特征向量訓(xùn)練得來。

表1 顏色分布規(guī)律表Table 1 The color distribution table

圖1 顏色分布規(guī)律表(CDT)和支持向量機(jī)(SVM)Fig.1 The color distribution table(CDT)&support vector machine(SVM)

圖2 青鳉魚魚鰓提取實(shí)驗(yàn)結(jié)果Fig.2 Experiment result of extracting gill of medaka fish

根據(jù)魚鰓輪廓,我們可以計算其輪廓面積。在時間T內(nèi)魚的呼吸次數(shù)n可根據(jù)魚鰓輪廓面積在時間T內(nèi)的極大(小)值的個數(shù)來統(tǒng)計。青鱂魚的呼吸頻率f可由式(1)得出。

式中,n表示時間T內(nèi)魚的呼吸次數(shù),A表示魚鰓的輪廓面積。

2 青鳉魚運(yùn)動特征檢測方法(The detection method of medaka motion features)

本文通過對運(yùn)動魚進(jìn)行骨架提取,進(jìn)而通過提出青鳉魚骨架模型,可實(shí)時計算得到青鳉魚胸鰭尾鰭擺動頻率。

2.1 骨架提取

中軸變換(MAT)是一種用來確定物體、估計物體骨架的細(xì)化技術(shù)。理論上說,每個骨架點(diǎn)保持了與邊界點(diǎn)距離最小的性質(zhì),所以如果用以每個骨架點(diǎn)為中心的圓的集合,就可以恢復(fù)出原始的區(qū)域。具體就是以每個骨架點(diǎn)為圓心,以前述最小距離為半徑做圓。它們的包絡(luò)就構(gòu)成了區(qū)域的邊界,填充圓就得到這些區(qū)域。由上述可知,骨架就是用1個點(diǎn)與1個點(diǎn)集的最小距離來定義的,可寫成式(3):

式中,B為邊界,z為邊界點(diǎn),p為區(qū)域上任意一點(diǎn)。

在去噪后進(jìn)行骨架提取,最終得到單一的較為完整的魚體骨架結(jié)構(gòu)。圖3為經(jīng)過骨骼細(xì)化后,以及鏈接了斷裂連通域后得到的青鏘魚魚骨架。該骨架與青鏘魚的整體狀態(tài)保持一致,對之后采集相應(yīng)的數(shù)據(jù)起著關(guān)鍵的作用。

圖3 青鳉魚骨架提取結(jié)果Fig.3 Experiment result of skeleton extraction of medaka fish

圖4 青鳉魚骨架模型Fig.4 The skeleton model of medaka fish

2.2 青鳉魚骨架模型

本文提出了青鳉魚骨架模型如圖4,并標(biāo)定了6個關(guān)鍵點(diǎn)分別為:胸鰭參考點(diǎn)、胸鰭根部參考點(diǎn)、胸鰭末梢端點(diǎn)、魚尾參考點(diǎn)1、魚尾參考點(diǎn)2和魚尾端點(diǎn),以便于計算得到青鏘魚的胸鰭、尾鰭擺動頻率和幅值。胸鰭參考點(diǎn)以青鏘魚魚鰓與骨架的1個交點(diǎn)為參考點(diǎn),該點(diǎn)具有較好的穩(wěn)定性,可以適合作為參考點(diǎn)。得到參考點(diǎn)的同時,還能去除頭部無用的并可能會產(chǎn)生干擾的信息。其余各點(diǎn)以胸鰭參考點(diǎn)為初始點(diǎn),遍歷整個魚骨架,從而先后得胸鰭根部端點(diǎn)、胸鰭末梢端點(diǎn)、魚尾端點(diǎn)。結(jié)果顯示如圖5所示。

2.3 青鳉魚運(yùn)動特征計算

對于胸鰭擺動角度的計算,通過胸鰭參考點(diǎn)、胸鰭根部端點(diǎn)、胸鰭末點(diǎn)3點(diǎn)利用公式(4):

可以得到擺動角度θ,式中x為橫坐標(biāo),y為縱坐標(biāo),進(jìn)而利用公式(5):

可以判斷出是左胸鰭還是右胸鰭的擺動角度。同理,對于青鏘魚魚尾擺動角度,也可以據(jù)此方法得出。進(jìn)而通過統(tǒng)計一段時間T內(nèi),擺動角度的的峰值數(shù)量,可統(tǒng)計出擺動次數(shù)n。青鳉魚胸鰭和尾鰭擺動頻率計算方程如式(6):

圖5 青鳉魚骨架參考點(diǎn)Fig.5 The skeleton reference point of medaka fish

其中θimax和θimin為某個周期i內(nèi),青鳉魚胸鰭、尾鰭的最大擺動角度和最小擺動角度。

3 實(shí)驗(yàn)結(jié)果與分析(Experimental results and analysis)

為驗(yàn)證本文方法的有效性與可行性,做了如下實(shí)驗(yàn)。實(shí)驗(yàn)過程采用數(shù)碼攝像頭獲取正常水質(zhì)下的魚體運(yùn)動視頻圖像,每幀圖像大小為480×640,幀率為30 fps。

通過實(shí)驗(yàn)結(jié)果可得,CDT&SVM方法檢測得到的青鳉魚呼吸次數(shù)與實(shí)際青鳉魚呼吸次數(shù)誤差較小,并且對于幀率為30 fps的CCD相機(jī)的實(shí)時采集,算法能滿足實(shí)時性。實(shí)驗(yàn)在連續(xù)1 000幀圖像中得到的青鳉魚呼吸深度譜圖結(jié)果如圖6所示:

圖6 青鳉魚呼吸深度譜圖Fig.6 Spectrum of respiratory depth of medaka fish

青鳉魚胸鰭尾鰭擺動頻率實(shí)驗(yàn)結(jié)果如表3所示,其中通過算法檢測得到的尾鰭擺動數(shù)據(jù)準(zhǔn)確性較高。相較而言,由于青鳉魚胸鰭很難被準(zhǔn)確識別,從而導(dǎo)致胸鰭擺動數(shù)據(jù)的準(zhǔn)確性仍不夠準(zhǔn)確。算法具有較好的實(shí)時性,單幀耗時約23 ms,能滿足在線采集、分析的需求。

實(shí)驗(yàn)在連續(xù)1 000幀圖像中得到的青鳉魚胸鰭、尾鰭擺幅如圖7～9所示。

表2 青鳉魚呼吸頻率實(shí)測結(jié)果Table 2 Experiment result of medaka respiratory rate

表3 青鳉魚胸鰭尾鰭擺動頻率實(shí)測結(jié)果Table 3 Experiment result of medaka pectoral fins and tail beat frequency

圖7 青鳉魚尾鰭擺幅譜圖Fig.7 Spectrum of tail swinging amplitude of medaka fish

圖8 青鳉魚胸鰭(左側(cè))擺幅譜圖Fig.8 Spectrum of left pectoral fin swinging amplitude of medaka fish

圖9 青鳉魚胸鰭(右側(cè))擺幅譜圖Fig.9 Spectrum of right pectoral fin swinging amplitude of medaka fish

綜上所述,本文所提出的對青鳉魚的實(shí)時在線監(jiān)測方法,充分的利用了機(jī)器視覺的優(yōu)點(diǎn),在準(zhǔn)確監(jiān)測青鳉魚的運(yùn)動、生理特征的基礎(chǔ)上,仍能保持實(shí)時性,滿足了在線監(jiān)測的需求,也能為生物水質(zhì)監(jiān)測和預(yù)警的發(fā)展提供一定支持與參考。

[1]徐建瑜,崔紹榮,苗香雯,等.計算機(jī)視覺技術(shù)在水產(chǎn)養(yǎng)殖中的應(yīng)用與展望[J].農(nóng)業(yè)工程學(xué)報,2005,21(8): 174-178 Xu J Y,Cui S R,Miao X W,et al.Application and prospect of computer vision technology in aquaculture[J]. Transactions of the Chinese Society of Agricultural Engineering,2005,21(8):174-178(in Chinese)

[2]Heinz G H,Haseltine S D.Avoidance behaviour of young black ducks treated with chromium[J].Toxicology Letters,1981(8):307-310

[3]Steiner U K.Linking antipredator behaviour,ingestion, gut evacuation and costs of predator induced responses in tadpoles[J].Animal Behaviour,2007,74(5):1473-1479

[4]Nimkerdphol K,Nakagawa M.Effect of sodium hypochlorite on zebrafish swimming behavior estimated by fractal dimension analysis[J].Journal of Bioscience and Bioengineering,2008,105(5):486-492

[5]Hughes N F,Kelly L H.New techniques for 3D video tracking of fish swimming movements in still or flowing water[J].Canadian Journal of Fisheries and Aquatic Sciences,1996,53:2476-2483

[6]耿彩英.用于水質(zhì)監(jiān)測的活體魚視頻圖像理解[D].杭州:杭州電子科技大學(xué),2009:12 Geng C Y.The video image understanding of live fish used for water quality monitoring[D].Hangzhou:Hangzhou Dianzi University,2009:12(in Chinese)

[7]Zhu L Q,Weng W.Catadioptric stereo-vision system for the real-time monitoring of 3D behavior in aquatic animals[J].Physiology&Behavior,2007,91:106-119

[8]Wu G H,Yang Y,Zeng L J.Novel method based on video tracking system for simultaneous measurement of kinematics and flow in the wake of a freely swimming fish [J].Review of Scientific Instruments,2006,77:114302

[9]Wu G H,Zeng L J.Video tracking method for three-dimensional measurement of a free-swimming fish[J].Science in China Series G-Physics and Astronomy,2007,50: 779-786

[10]王麗娜.基于動態(tài)圖像理解的水質(zhì)生物監(jiān)測系統(tǒng)研究與應(yīng)用[D].杭州:杭州電子科技大學(xué),2011:1 Wang L N.Research and application on water quality biomonitoring system based on dynamic image understand-ing[D].Hangzhou:Hangzhou Dianzi University,2011:1 (in Chinese)

[11]胡江龍.多目標(biāo)跟蹤算法在水質(zhì)監(jiān)測中的應(yīng)用[D].杭州:杭州電子科技大學(xué),2012:5 Hu J L.Water quality monitoring using multi-object tracking algorithm[D].Hangzhou:Hangzhou Dianzi University,2012:5(in Chinese)

[12]馬欣.生物式水質(zhì)監(jiān)測的報警方法研究[D].杭州:杭州電子科技大學(xué),2012:3 Ma X.Research on alarming method of biological watermonitoring[D].Hangzhou:Hangzhou Dianzi University, 2012:3(in Chinese)

[13]Wu J,Xiao G,Zhang Y L,et al.Fish group tracking based on delaunay triangulation network[C].2011 4th International Congress on Image and Signal Processing (CISP),2011:534-537

[14]吳軍.基于視覺感知的魚群目標(biāo)檢測與跟蹤技術(shù)研究[D].杭州:浙江工業(yè)大學(xué),2011:4 Wu J.Research on detecting and tracking of fish group based on machine vision[D].Hangzhou:Zhejiang University of Technology,2011:4(in Chinese)

[15]于欣,侯曉嬌,盧煥達(dá),等.基于光流法與特征統(tǒng)計的魚群異常行為檢測[J].農(nóng)業(yè)工程學(xué)報,2014,30(2):162-168 Yu X,Hou X J,Lu H D,et al.Anomaly detection of fish school behavior based on features statistical and optical flow methods[J].Transactions of the Chinese Society of Agricultural Engineering,2014,30(2):162-168(in Chinese)

[16]Pinkiewicz T H,Purser G J,Williams R N.A computer vision system to analyse the swimming behavior of farmed fish in commercial aquaculture facilities:A case study using cage-held Atlantic salmon[J].Aquacultural Engineering,2011,45:20-27

[17]Lee J H,Wu M Y,Guo Z C.A tank fish recognition and tracking system using computer vision techniques[C]. Computer Science and Information Technology,2010,4: 528-532

[18]Serra-Toro C,Montoliu R,Traver V J,et al.Assessing water quality by video monitoring fish swimming behavior[C].20th International Conference on Pattern Recognition(ICPR),2010,8:23-26

[19]Christodoulou L,Kasparis T,Marques O.Advanced statistical and adaptive threshold techniques for moving object detection and segmentation[C].Digital Signal Processing(DSP),17thInternational Conference,2011:1-6

[20]Tan J S,Chang T K,Ooi P L,et al.Fast and robust zebrafish segmentation and detection algorithm under different spectrum conditions[C].Sensors Applications Symposium(SAS),2014:189-194

[21]Gerhardt A,Janssens D B L,Mo Z,et al.Short-term responses ofOryzias latipes(Pisces:Adrianichthyidae)and Macrobrachium nipponense(Crustacea:Palaemonidae)to municipal and pharmaceutical waste water in Beijing,China:Survival,behaviour,biochemical biomarkers[J]. Chemosphere,2002,47(1):35-47

[22]Cairns Jr J,Thompson K W,Hendricks A C.Effects of fluctuating sublethal applications of heavy metal solutions upon the gill ventilatory response of blue gills(Lepomis macrochirus)[M].Duluth,Minn.,Cincinnati,Ohio:U.S. Environmental Protection Agency,Research and Development,Environmental Research Laboratory,Center for Environmental Research Information,1981

quality monitoring provides a novel solution in this field.The water quality is detected by the biological response to reflect the direct or indirect pollution.However,the observation indexes and quantitative criteria are major problems to estimate in the complex water environment.In this paper,the medaka fish is chosen as the model organism,and the corresponding physiological characteristics and movement characteristics are observation indexes,such as breathing frequency,pectoral oscillation frequency,tail beat frequency,etc.By adopting machine vision based method,the real-time monitoring and analysis are achieved.Experimental results show that the proposed method can provide the support and reference for the development of biological water quality monitoring and early warning.The measured breathing frequency of medaka fish was 3.06 Hz,the pectoral oscillation frequency was 4.83 Hz and the tail beat frequency was 5.08 Hz.The results are consistent with the actual indexes.

biological water quality monitoring;real time;observation index

簡介:張融(1971—)，女，工程師，主要研究方向?yàn)榄h(huán)境工程、工業(yè)設(shè)計。

饒凱峰(1976—)，男，助理研究員，研究方向?yàn)樗鷳B(tài)毒理與監(jiān)測預(yù)警。