亚洲免费av电影一区二区三区,日韩爱爱视频,51精品视频一区二区三区,91视频爱爱,日韩欧美在线播放视频,中文字幕少妇AV,亚洲电影中文字幕,久久久久亚洲av成人网址,久久综合视频网站,国产在线不卡免费播放

        ?

        Facial expression recognition with contextualized histograms

        2015-04-22 07:49:26YUELei岳雷SHENTingzhi沈庭芝DUBuzhi杜部致ZHANGChao張超ZHAOShanyuan趙三元
        關(guān)鍵詞:刊刻張超書籍

        YUE Lei(岳雷), SHEN Ting-zhi(沈庭芝), DU Bu-zhi(杜部致)ZHANG Chao(張超) ZHAO Shan-yuan(趙三元)

        (1.School of Information and Electronics, Beijing Institute of Technology, Beijing 100081,China;2.School of Computer Science and Technology, Beijing Institute of Technology, Beijing 100081, China)

        ?

        Facial expression recognition with contextualized histograms

        YUE Lei(岳雷), SHEN Ting-zhi(沈庭芝)1, DU Bu-zhi(杜部致)1,ZHANG Chao(張超)1, ZHAO Shan-yuan(趙三元)2

        (1.School of Information and Electronics, Beijing Institute of Technology, Beijing 100081,China;2.School of Computer Science and Technology, Beijing Institute of Technology, Beijing 100081, China)

        A new algorithm taking the spatial context of local features into account by utilizing contextualized histograms was proposed to recognize facial expression. The contextualized histograms were extracted from two widely used descriptors—the local binary pattern (LBP) and weber local descriptor (WLD). The LBP and WLD feature histograms were extracted separately from each facial image, and contextualized histogram was generated as feature vectors to feed the classifier. In addition, the human face was divided into sub-blocks and each sub-block was assigned different weights by their different contributions to the intensity of facial expressions to improve the recognition rate. With the support vector machine (SVM) as classifier, the experimental results on the 2D texture images from the 3D-BUFE dataset indicated that contextualized histograms improved facial expression recognition performance when local features were employed.

        facial expression recognition; local binary pattern; weber local descriptor; spatial context; contextualized histogram

        Facial expressions play a key role in non-verbal face-to-face communication. Facial expressions have paramount impact on human interaction: about 55 percent of the effectiveness of a conversation relies on facial expressions, 38 percent is conveyed by voice intonation, and 7 percent by the spoken words[1]. Therefore, automatic facial expression is essential to unleash the potential of many applications such as intelligent human-computer interaction (HCI) and social analysis of human behavior.

        During the past decades, many visual features have been proposed to characterize facial expressions[2-4]. In particular, local image descriptors such as local binary pattern (LBP)[5]and weber local descriptor (WLD)[6]have demonstrated great effectiveness in facial expression recognition[7]. Ojala et al. initially proposed LBP to characterize texture units by exploring the spatial context around a given pixel. And inspired by Weber’s Law, Chen et al. proposed WLD which characterizes local features with two components, namely differential excitation and Orientation, to handle many tasks such as texture classification, action recognition, and face recognition. However, these two descriptors only explore the first order context at the pixel intensity level, rather than the pattern level. As a result, the contextual information among those local patterns or descriptors does not contribute to the discrimination of extracted descriptors and recognition performance could be compromised.

        In this paper, we explored spatial context among local descriptors during the formation of histograms with the recently introduced contextualized histogram[8]technique. That is, instead of simply calculating the distribution of each local pattern for a given image, more patterns at finer granularity were discovered to form a higher dimensional and more discriminative histogram. In order to further improve recognition performance, a weighted partition method was investigated.

        1 Local features

        1.1 LBP

        LBP was initially proposed by Ojala et al.[5]to characterize texture units by exploring the spatial context around a given pixel. That is, each pixel is converted into a LBP by referring to its neighborhood and the binary patterns of a given image are formed into a histogram which describes the distribution of those LBPs. In detail, the operator labels the pixels of an image by thresholding the 3×3 neighborhood of each pixel with the center value and considering the result as a binary number. By considering the 8-bit result as a binary number, a 256-bin histogram of the LBP labels computed over a region is used as a texture descriptor. Due to its effectiveness as well as simplicity, it has been widely used in many pattern classification problems such as face recognition and facial expression recognition.

        Formally, given a pixel at (xc,yc), the resulting LBP can be expressed in decimal form as

        (1)

        whereicandipare gray-level values of the central pixel and one of thePneighbor pixels with a radiusR, respectively. Functions(x) is defined as

        (2)

        Inaddition,thedimensionalityoftheoperatorwasfurtherreducedbyintroducingthenotionofauniformLBPwhichcontainsatmosttwobitwisetransitionsfrom0to1orviceversawhenthebinarystringisconsideredcircular.Asaresult,eachimagecanberepresentedwitha59-dimensionalhistogramwhereeachbinrepresentsthedistributionofoneLBPamongtheimage.

        Fig.1 Illustration of the LBP operator

        1.2WLD

        WLD[6]isbasedonthefactthathumanperceptionofapatterndependsonnotonlythechangeofastimulus(suchassound,lighting)butalsotheoriginalintensityofthestimulus.Specifically,WLDconsistsoftwocomponents:differentialexcitationandorientationforeachreferencepixelwithinitsneighbors(i.e.thecentralpixelofa3×3neighborhood).Thedifferentialexcitationcomponentisafunctionoftheratiobetweentwoterms:therelativeintensitydifferencebetweenareferencepixelanditsneighbors,andtheintensityofthereferencepixel.Theorientationcomponentisthegradientorientationofthereferencepixel.

        Specifically,thedifferentialexcitationξ(xc)ofareferencepixelxciscomputedinthreesteps.First,thedifferencesbetweenthereferencepixelanditsneighborsarecalculatedaftertheinputimageisfilteredwithf00asshowninFig.2.

        官箴文化在晚明有顯著的發(fā)展,呈現(xiàn)興盛局面,最直接、最明顯的表現(xiàn)就是官箴書數(shù)量的“驟增”。需說明的是,這里所指的官箴書包括三種情況:一是書籍的創(chuàng)作與刊刻都在晚明時期;二是書籍的創(chuàng)作在晚明,但可能刊刻于清代;三是無法判定其創(chuàng)作期,但最早的版本確定在晚明,這種情況一般不著撰者。以上三種情況,我們都可將其視為晚明官箴書。

        (3)

        (4)

        Fig.2 Masks of four Filters: f00, f01, f10 and f11

        TheorientationcomponentofWLD,i.e.thegradientorientation,iscomputedas

        (5)

        Asaresult,eachimagecanberepresentedwithaWLDhistogramfromthe2Dhistogramsofdifferentialexcitationsanddominantorientations.Thedimensionalityofthe2DhistogramisT×C,whereTisthenumberofdominantorientationsandCisthenumberofcells.EachcellisfurtherevenlydividedintoM×Ssegments,whereMisthenumberofsub-histogramsforeachorientationandSisthenumberofbinsofeachsub-histogram.Thatis,thedimensionoftheWLDhistogramisT×S×M.

        2 Histogram contextualization

        AssumethatanimageisrepresentedwithaK-binhistogram: h={hi}(i=1,2,…,K) (hisafeaturehistogramandKisthenumberofbinsofit).Histogramcontextualizationistoconstructaso-calledcontextualizedhistogramhs={hi(sj)}(i=1,2,…,K; j=1,2,…,M)basedonasetofpredefinedlocalcontextualstructuresdenotedas{sj}(j=1,2,…,M),wherehi(sj)isthenumberofpixelswhichtaketheithlocalfeaturevalueandsurroundedwiththelocalcontextualstructuresj.Thatis,foreachgivenlocalfeaturepattern(e.g.LBP),thereareMsub-patternsintermsofitsspatialcontext.Asaresult,thecontextualizedhistogramisofM×Kdimension.

        Foralocalcontextualstructurewithonlytwopixels,therearetwotypesofhomogeneities,namelyhomogeneousandinhomogeneous.Foraternarylocalcontextualstructure,thereare5typesoflocalhomogeneities.Therefore,theternarylocalcontextualstructuresareabletoencodemorecomplicatedandinformativelocalcontextualinformation.Fengetal.[8]defined30ternarycontextualstructuresbycombining5typesofhomogeneitiesand6shapes(Fig.3).Notethattheordershowninthefiguremeansthenumberofpixelsbelongingtothesamehistogrambin,anddifferentcolorsrepresentdifferenthistogrambins.Asaresult,thedimensionofthecontextualizedhistogramisM×K=30×K.

        Fig.3 Histogram contextualization with 30 ternary local contextual structures combining 5 homogeneities and 6 different shapes

        3 Experiments

        Weconductedexperimentsinfourstepswiththe2Dtextureimagesfrom3D-BUFEdataset: ①ExtractedlocalfeaturepatternsofLBPandWLDseparatelyfromtheoriginalexpressionimages; ②Dividedtheimagesintosub-block; ③Addedweightstoeachsub-block.; ④Impliedcontextualizedhistogramtothefeaturevectorsfromthethirdstep.Wegotexpressionclassificationresultsfromallthesefourtrialsandcomparedtherecognitionrateinthissection.

        3.1Experimentalsettings

        Weusedthe2Dtextureapexexpressionimagesfromthewidelyused3D-BUFEdataset[10].Therewere100subjectswhoparticipatedinfacescan,includingundergraduates,graduatesandfacultyfromvariousdepartmentsofBinghamtonUniversity(e.g.suchasPsychology,Arts,andEngineering)andStateUniversityofNewYork(e.g.ComputerScience,ElectricalEngineering,andMechanicalEngineering).ThemajorityofparticipantswereundergraduatesfromthePsychologyDepartment.Theresultingdatabaseconsistsofabout60%femaleand40%malesubjectswithavarietyofethnic/racialancestries,includingWhite,Black,East-Asian,Middle-eastAsian,HispanicLatino,andothers.Eachsubjectperformedsevenexpressionsinfrontofthe3Dfacescanner.Withtheexceptionoftheneutralexpression,eachofthesixprototypicexpressions(i.e.anger,disgust,fear,happiness,sadness,andsurprise)includesfourlevelsofintensity.Therefore,thereare25 3Dexpressionmodelsforeachsubject.Asaresult,thedatasetintotalcontains2 500 3Dfacialexpressionmodelsandeachmodelhasitscorresponding2Dtextureimage.SevensampleexpressionsofasubjectareshowninFig.4.Andwerandomlypicked80subjectsasthetrainingsetandtheother20subjectsasthetestingset.

        Fig.4 Samples of seven expressions (from left to right): anger, disgust, fear, happiness, neutral, sadness, and surprise

        Fig.5 Illustration of weighted sub-block based representation

        Inordertotaketheglobalspatialcontextintoaccount,wealsoinvestigatedtheimpactofsub-blockbasedrepresentationwithandwithoutweights.AsshowninFig.5,eachfaceimageispartitionedinto42 (6×7)sub-blocksandsquarecolorsofblack,darkgray,lightgrayandwhiteindicateweightsof0.0, 1.0, 2.0and4.0respectively[7].Asaresult,thefeaturevectorofeachimagewillbe42×K,whereKisthedimensionofthehistogramderivedfromalocaldescriptor(eg.59forLBPor192forWLDwhereT=8, S=4,andM=6).Wegotthesethreenumbers(T, SandM)byrepeatedlytryingdifferentcombinationsandtriedtoreachacompromisebetweencalculationefficiencyandrecognitioncorrectrates.

        Duetothesub-blockrepresentationandhistogramcontextualization,thefeaturedimensionisquitehigh.Forexample,thefeaturedimensionofacontextualizedWLDhistogramis241 920 (=192×42×30).Therefore,weperformdimensionreductionwithprinciplecomponentanalysis(PCA).Weemployedthewidelyusedsupportvectormachines(SVM)[11]astheclassifier.Thoughmanynewkernelsarebeingproposedbyresearchers,themostwidelyusedkernelfunctionsarethelinearfunction,polynomialfunction,radialbasisfunction(RBF),andSigmoidfunction.Inthispaper,weusedtheRBFkernelfunctiontocarryouttheexperiments.

        3.2Experimentalresultsanddiscussions

        Experimentswereconductedinfourstepstoevaluatetheperformanceofthecontextualizedhistogramsderivedfromlocaldescriptors.ThefourLBP-basedhistogramsareglobalhistogram(LBP),sub-blockbasedrepresentation(LBPwithsub-block),weightedsub-blockbasedrepresentation(LBPwithweightedsub-block),andcontextualizedhistogramsoverweightedsub-blockbasedrepresentation(LBPwithweightedsub-blockandcontext).AsshowninTab.1,thecontextualizedhistogramwasabletoachievethehighestrecognitionaccuracy,indicatingthatlocalcontextualstructurewashelpfulincharacterizingspatialcontext.Thesub-blockbasedrepresentationimprovedrecognitionperformancesignificantly,whichdemonstratedtheuseofglobalspatialcontextcouldimprovetherecognitionrateaswell.Assigndifferentweighttodifferentsub-blockswasalsohelpfulforfacialexpressionrecognition.Thebestperformanceforcontextualizedhistogramwasobtainedwhenthereducedfeaturedimensionwas200,afterseveralvalueswereattempted.

        Tab.1 Comparison of recognition accuracy among LBP based histograms

        The similar result was also observed for WLD-based histograms as shown in Tab.2. The classification based on original WLD histograms didn’t show competitive recognition capability. But when we first divided the face into sub-block and then assigned different weights to corresponding sub-blocks, the recognition rate improved significantly. We obtained the best results with the feature extract with WLD when we applied contextualized histogram on the vector generate from weighted sub-blocked human face.

        Tab.2 Comparison of recognition accuracy among WLD based histograms

        WLD outperformed LBP while a histogram was obtained globally (without partitioning an image into sub-blocks). This could be explained with the superiority of WLD. However, in other situations LBP slightly outperformed WLD. This could be explained with the high dimensionality of WLD based histograms. Dimension curse was still an imperfectly solved problem. So, efficient extraction of useful information from overlong vectors extracted with WLD may be able to help us to further improve the recognition rate.

        4 Conclusion

        In this paper we presented a solution to contextualize the histograms derived from local features such as LBP and WLD and utilize them for facial expression recognition. Since local contextual structures were taken into account when a histogram was derived from local features, the contextualized histogram was more discriminative for facial expression recognition. Experimental results on the 2D feature images from the 3D-BUFE dataset demonstrated the effectiveness of the proposed solution by adding the recognition rate when weighted sub-block and contextualized histograms were employed to the original local feature patterns. In the future, we will further investigate adaptive weighting strategies at both sub-lock level and the local contextual structure level, and try to introduce dimension reduction algorithm to improve the performance.

        [1] Mehrabian A. Communication without words[J]. Psychology Today, 1968, 2(4):53-56.

        [2] Fasel B, Luettin J. Automatic facial expressionanalysis: a survey[J]. IEEE Transactions on Pattern Recognition, 2003,36(1):259-275.

        [3] Krinidis S, Buciu I, Pitas I. Facial expressionanalysis and synthesis: a survey[C]∥The 10thInternational Conference on Human-Computer Interaction, Crete, Greece, 2003.

        [4] Pantic M, Rothkrantz L J. Automatic analysis offacial expressions the state of the art[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(12):1424-1445.

        [5] Ojala T, Pietikinen M, Harwood D. A comparative study of texture measures with classification based on featured distribution[J]. Pattern Recognition, 1996, 29(1):51-59.

        [6] Chen J, Shan S, He C,et al. Wld: A robust local image descriptor[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(9):1705-1720.

        [7] Shan C, Gong S, McOwan P. Facial expression recognition based on local binary patterns: A comprehensive study[J]. Image and Vision Computing, 2009, 27(6):803-816.

        [8] Feng J, Ni B, Xu D, et al. Histogram contextualization[J]. IEEE Transactions on Image Processing, 2012, 21(2):778-788.

        [9] Ojala T,Pietikainen M, Maenpaa T. Multi-resolution gray-scale and rotation invariant texture classification with local binary patterns[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(7):971-987.

        [10] Yin L, Wei X, Sun Y, et al. A 3d facialexpression database for facial behavior research[C]∥International Conference on Automatic Face and Gesture Recognition, Southampton, UK, 2006.

        [11] Chang C C, Lin C J. LibSVM: a library for support vector machines[J]. ACM Transactions on Intelligent Systems and Technology, 2011, 2(27):1-27.

        (Edited by Cai Jianying)

        10.15918/j.jbit1004-0579.201524.0317

        TP 37 Document code: A Article ID: 1004- 0579(2015)03- 0392- 06

        Received 2014- 02- 20

        Supported by the National Natural Science Foundation of China(60772066)

        E-mail: napoylei@163.com

        猜你喜歡
        刊刻張超書籍
        魯迅與“書籍代購”
        張超個人簡介
        散文百家(2021年11期)2021-11-12 03:06:38
        My New Invention
        How to Protect Us from Infectious Diseases
        張超個人簡介
        散文百家(2021年4期)2021-04-30 03:15:20
        宋人對三蘇文章的選編與刊刻
        國學(xué)(2020年1期)2020-06-29 15:14:38
        AOS在書籍編寫的應(yīng)用
        書籍
        論晚明元劇選刊刻現(xiàn)象及其文化審美意義
        戲曲研究(2018年3期)2018-03-19 08:47:18
        書籍是如何改變我們的
        国产精品泄火熟女| 亚洲一区二区一区二区免费视频| 水蜜桃网站视频在线观看| 亚洲va久久久噜噜噜久久天堂 | 丰满熟女高潮毛茸茸欧洲视频| 日本精品人妻无码77777| 久久亚洲日本免费高清一区 | 丝袜美腿视频一区二区| 极品嫩模高潮叫床| 国产高清无码在线| av在线男人的免费天堂| 一本色道久久亚洲综合| 国产性生大片免费观看性| 99久久人妻无码精品系列蜜桃| 亚洲国产免费一区二区| 亚洲国产精品久久久av| 精品国产人妻一区二区三区| 91成人午夜性a一级毛片| 国产精品毛片毛片av一区二区| 久久久久九九精品影院| 欧美与黑人午夜性猛交久久久| 亚洲国产精品综合久久20| 久久99精品综合国产女同| 国产精品精品自在线拍| 丰满少妇人妻无码专区| 少妇极品熟妇人妻高清| 亚洲av免费手机在线观看| 色播久久人人爽人人爽人人片av| 日韩精品永久免费播放平台| 久久久大少妇免费高潮特黄| 无码人妻丰满熟妇区五十路| 亚洲永久无码动态图| 九色精品国产亚洲av麻豆一 | 综合久久加勒比天然素人| 久久无码字幕中文久久无码| 99久久国语露脸精品国产| 成年毛片18成年毛片| 亚洲最好看的中文字幕| 特级婬片国产高清视频| 无码免费午夜福利片在线| 人妻少妇中文字幕,久久精品 |