亚洲免费av电影一区二区三区,日韩爱爱视频,51精品视频一区二区三区,91视频爱爱,日韩欧美在线播放视频,中文字幕少妇AV,亚洲电影中文字幕,久久久久亚洲av成人网址,久久综合视频网站,国产在线不卡免费播放

        ?

        Facial expression recognition with contextualized histograms

        2015-04-22 07:49:26YUELei岳雷SHENTingzhi沈庭芝DUBuzhi杜部致ZHANGChao張超ZHAOShanyuan趙三元
        關(guān)鍵詞:刊刻張超書籍

        YUE Lei(岳雷), SHEN Ting-zhi(沈庭芝), DU Bu-zhi(杜部致)ZHANG Chao(張超) ZHAO Shan-yuan(趙三元)

        (1.School of Information and Electronics, Beijing Institute of Technology, Beijing 100081,China;2.School of Computer Science and Technology, Beijing Institute of Technology, Beijing 100081, China)

        ?

        Facial expression recognition with contextualized histograms

        YUE Lei(岳雷), SHEN Ting-zhi(沈庭芝)1, DU Bu-zhi(杜部致)1,ZHANG Chao(張超)1, ZHAO Shan-yuan(趙三元)2

        (1.School of Information and Electronics, Beijing Institute of Technology, Beijing 100081,China;2.School of Computer Science and Technology, Beijing Institute of Technology, Beijing 100081, China)

        A new algorithm taking the spatial context of local features into account by utilizing contextualized histograms was proposed to recognize facial expression. The contextualized histograms were extracted from two widely used descriptors—the local binary pattern (LBP) and weber local descriptor (WLD). The LBP and WLD feature histograms were extracted separately from each facial image, and contextualized histogram was generated as feature vectors to feed the classifier. In addition, the human face was divided into sub-blocks and each sub-block was assigned different weights by their different contributions to the intensity of facial expressions to improve the recognition rate. With the support vector machine (SVM) as classifier, the experimental results on the 2D texture images from the 3D-BUFE dataset indicated that contextualized histograms improved facial expression recognition performance when local features were employed.

        facial expression recognition; local binary pattern; weber local descriptor; spatial context; contextualized histogram

        Facial expressions play a key role in non-verbal face-to-face communication. Facial expressions have paramount impact on human interaction: about 55 percent of the effectiveness of a conversation relies on facial expressions, 38 percent is conveyed by voice intonation, and 7 percent by the spoken words[1]. Therefore, automatic facial expression is essential to unleash the potential of many applications such as intelligent human-computer interaction (HCI) and social analysis of human behavior.

        During the past decades, many visual features have been proposed to characterize facial expressions[2-4]. In particular, local image descriptors such as local binary pattern (LBP)[5]and weber local descriptor (WLD)[6]have demonstrated great effectiveness in facial expression recognition[7]. Ojala et al. initially proposed LBP to characterize texture units by exploring the spatial context around a given pixel. And inspired by Weber’s Law, Chen et al. proposed WLD which characterizes local features with two components, namely differential excitation and Orientation, to handle many tasks such as texture classification, action recognition, and face recognition. However, these two descriptors only explore the first order context at the pixel intensity level, rather than the pattern level. As a result, the contextual information among those local patterns or descriptors does not contribute to the discrimination of extracted descriptors and recognition performance could be compromised.

        In this paper, we explored spatial context among local descriptors during the formation of histograms with the recently introduced contextualized histogram[8]technique. That is, instead of simply calculating the distribution of each local pattern for a given image, more patterns at finer granularity were discovered to form a higher dimensional and more discriminative histogram. In order to further improve recognition performance, a weighted partition method was investigated.

        1 Local features

        1.1 LBP

        LBP was initially proposed by Ojala et al.[5]to characterize texture units by exploring the spatial context around a given pixel. That is, each pixel is converted into a LBP by referring to its neighborhood and the binary patterns of a given image are formed into a histogram which describes the distribution of those LBPs. In detail, the operator labels the pixels of an image by thresholding the 3×3 neighborhood of each pixel with the center value and considering the result as a binary number. By considering the 8-bit result as a binary number, a 256-bin histogram of the LBP labels computed over a region is used as a texture descriptor. Due to its effectiveness as well as simplicity, it has been widely used in many pattern classification problems such as face recognition and facial expression recognition.

        Formally, given a pixel at (xc,yc), the resulting LBP can be expressed in decimal form as

        (1)

        whereicandipare gray-level values of the central pixel and one of thePneighbor pixels with a radiusR, respectively. Functions(x) is defined as

        (2)

        Inaddition,thedimensionalityoftheoperatorwasfurtherreducedbyintroducingthenotionofauniformLBPwhichcontainsatmosttwobitwisetransitionsfrom0to1orviceversawhenthebinarystringisconsideredcircular.Asaresult,eachimagecanberepresentedwitha59-dimensionalhistogramwhereeachbinrepresentsthedistributionofoneLBPamongtheimage.

        Fig.1 Illustration of the LBP operator

        1.2WLD

        WLD[6]isbasedonthefactthathumanperceptionofapatterndependsonnotonlythechangeofastimulus(suchassound,lighting)butalsotheoriginalintensityofthestimulus.Specifically,WLDconsistsoftwocomponents:differentialexcitationandorientationforeachreferencepixelwithinitsneighbors(i.e.thecentralpixelofa3×3neighborhood).Thedifferentialexcitationcomponentisafunctionoftheratiobetweentwoterms:therelativeintensitydifferencebetweenareferencepixelanditsneighbors,andtheintensityofthereferencepixel.Theorientationcomponentisthegradientorientationofthereferencepixel.

        Specifically,thedifferentialexcitationξ(xc)ofareferencepixelxciscomputedinthreesteps.First,thedifferencesbetweenthereferencepixelanditsneighborsarecalculatedaftertheinputimageisfilteredwithf00asshowninFig.2.

        官箴文化在晚明有顯著的發(fā)展,呈現(xiàn)興盛局面,最直接、最明顯的表現(xiàn)就是官箴書數(shù)量的“驟增”。需說明的是,這里所指的官箴書包括三種情況:一是書籍的創(chuàng)作與刊刻都在晚明時期;二是書籍的創(chuàng)作在晚明,但可能刊刻于清代;三是無法判定其創(chuàng)作期,但最早的版本確定在晚明,這種情況一般不著撰者。以上三種情況,我們都可將其視為晚明官箴書。

        (3)

        (4)

        Fig.2 Masks of four Filters: f00, f01, f10 and f11

        TheorientationcomponentofWLD,i.e.thegradientorientation,iscomputedas

        (5)

        Asaresult,eachimagecanberepresentedwithaWLDhistogramfromthe2Dhistogramsofdifferentialexcitationsanddominantorientations.Thedimensionalityofthe2DhistogramisT×C,whereTisthenumberofdominantorientationsandCisthenumberofcells.EachcellisfurtherevenlydividedintoM×Ssegments,whereMisthenumberofsub-histogramsforeachorientationandSisthenumberofbinsofeachsub-histogram.Thatis,thedimensionoftheWLDhistogramisT×S×M.

        2 Histogram contextualization

        AssumethatanimageisrepresentedwithaK-binhistogram: h={hi}(i=1,2,…,K) (hisafeaturehistogramandKisthenumberofbinsofit).Histogramcontextualizationistoconstructaso-calledcontextualizedhistogramhs={hi(sj)}(i=1,2,…,K; j=1,2,…,M)basedonasetofpredefinedlocalcontextualstructuresdenotedas{sj}(j=1,2,…,M),wherehi(sj)isthenumberofpixelswhichtaketheithlocalfeaturevalueandsurroundedwiththelocalcontextualstructuresj.Thatis,foreachgivenlocalfeaturepattern(e.g.LBP),thereareMsub-patternsintermsofitsspatialcontext.Asaresult,thecontextualizedhistogramisofM×Kdimension.

        Foralocalcontextualstructurewithonlytwopixels,therearetwotypesofhomogeneities,namelyhomogeneousandinhomogeneous.Foraternarylocalcontextualstructure,thereare5typesoflocalhomogeneities.Therefore,theternarylocalcontextualstructuresareabletoencodemorecomplicatedandinformativelocalcontextualinformation.Fengetal.[8]defined30ternarycontextualstructuresbycombining5typesofhomogeneitiesand6shapes(Fig.3).Notethattheordershowninthefiguremeansthenumberofpixelsbelongingtothesamehistogrambin,anddifferentcolorsrepresentdifferenthistogrambins.Asaresult,thedimensionofthecontextualizedhistogramisM×K=30×K.

        Fig.3 Histogram contextualization with 30 ternary local contextual structures combining 5 homogeneities and 6 different shapes

        3 Experiments

        Weconductedexperimentsinfourstepswiththe2Dtextureimagesfrom3D-BUFEdataset: ①ExtractedlocalfeaturepatternsofLBPandWLDseparatelyfromtheoriginalexpressionimages; ②Dividedtheimagesintosub-block; ③Addedweightstoeachsub-block.; ④Impliedcontextualizedhistogramtothefeaturevectorsfromthethirdstep.Wegotexpressionclassificationresultsfromallthesefourtrialsandcomparedtherecognitionrateinthissection.

        3.1Experimentalsettings

        Weusedthe2Dtextureapexexpressionimagesfromthewidelyused3D-BUFEdataset[10].Therewere100subjectswhoparticipatedinfacescan,includingundergraduates,graduatesandfacultyfromvariousdepartmentsofBinghamtonUniversity(e.g.suchasPsychology,Arts,andEngineering)andStateUniversityofNewYork(e.g.ComputerScience,ElectricalEngineering,andMechanicalEngineering).ThemajorityofparticipantswereundergraduatesfromthePsychologyDepartment.Theresultingdatabaseconsistsofabout60%femaleand40%malesubjectswithavarietyofethnic/racialancestries,includingWhite,Black,East-Asian,Middle-eastAsian,HispanicLatino,andothers.Eachsubjectperformedsevenexpressionsinfrontofthe3Dfacescanner.Withtheexceptionoftheneutralexpression,eachofthesixprototypicexpressions(i.e.anger,disgust,fear,happiness,sadness,andsurprise)includesfourlevelsofintensity.Therefore,thereare25 3Dexpressionmodelsforeachsubject.Asaresult,thedatasetintotalcontains2 500 3Dfacialexpressionmodelsandeachmodelhasitscorresponding2Dtextureimage.SevensampleexpressionsofasubjectareshowninFig.4.Andwerandomlypicked80subjectsasthetrainingsetandtheother20subjectsasthetestingset.

        Fig.4 Samples of seven expressions (from left to right): anger, disgust, fear, happiness, neutral, sadness, and surprise

        Fig.5 Illustration of weighted sub-block based representation

        Inordertotaketheglobalspatialcontextintoaccount,wealsoinvestigatedtheimpactofsub-blockbasedrepresentationwithandwithoutweights.AsshowninFig.5,eachfaceimageispartitionedinto42 (6×7)sub-blocksandsquarecolorsofblack,darkgray,lightgrayandwhiteindicateweightsof0.0, 1.0, 2.0and4.0respectively[7].Asaresult,thefeaturevectorofeachimagewillbe42×K,whereKisthedimensionofthehistogramderivedfromalocaldescriptor(eg.59forLBPor192forWLDwhereT=8, S=4,andM=6).Wegotthesethreenumbers(T, SandM)byrepeatedlytryingdifferentcombinationsandtriedtoreachacompromisebetweencalculationefficiencyandrecognitioncorrectrates.

        Duetothesub-blockrepresentationandhistogramcontextualization,thefeaturedimensionisquitehigh.Forexample,thefeaturedimensionofacontextualizedWLDhistogramis241 920 (=192×42×30).Therefore,weperformdimensionreductionwithprinciplecomponentanalysis(PCA).Weemployedthewidelyusedsupportvectormachines(SVM)[11]astheclassifier.Thoughmanynewkernelsarebeingproposedbyresearchers,themostwidelyusedkernelfunctionsarethelinearfunction,polynomialfunction,radialbasisfunction(RBF),andSigmoidfunction.Inthispaper,weusedtheRBFkernelfunctiontocarryouttheexperiments.

        3.2Experimentalresultsanddiscussions

        Experimentswereconductedinfourstepstoevaluatetheperformanceofthecontextualizedhistogramsderivedfromlocaldescriptors.ThefourLBP-basedhistogramsareglobalhistogram(LBP),sub-blockbasedrepresentation(LBPwithsub-block),weightedsub-blockbasedrepresentation(LBPwithweightedsub-block),andcontextualizedhistogramsoverweightedsub-blockbasedrepresentation(LBPwithweightedsub-blockandcontext).AsshowninTab.1,thecontextualizedhistogramwasabletoachievethehighestrecognitionaccuracy,indicatingthatlocalcontextualstructurewashelpfulincharacterizingspatialcontext.Thesub-blockbasedrepresentationimprovedrecognitionperformancesignificantly,whichdemonstratedtheuseofglobalspatialcontextcouldimprovetherecognitionrateaswell.Assigndifferentweighttodifferentsub-blockswasalsohelpfulforfacialexpressionrecognition.Thebestperformanceforcontextualizedhistogramwasobtainedwhenthereducedfeaturedimensionwas200,afterseveralvalueswereattempted.

        Tab.1 Comparison of recognition accuracy among LBP based histograms

        The similar result was also observed for WLD-based histograms as shown in Tab.2. The classification based on original WLD histograms didn’t show competitive recognition capability. But when we first divided the face into sub-block and then assigned different weights to corresponding sub-blocks, the recognition rate improved significantly. We obtained the best results with the feature extract with WLD when we applied contextualized histogram on the vector generate from weighted sub-blocked human face.

        Tab.2 Comparison of recognition accuracy among WLD based histograms

        WLD outperformed LBP while a histogram was obtained globally (without partitioning an image into sub-blocks). This could be explained with the superiority of WLD. However, in other situations LBP slightly outperformed WLD. This could be explained with the high dimensionality of WLD based histograms. Dimension curse was still an imperfectly solved problem. So, efficient extraction of useful information from overlong vectors extracted with WLD may be able to help us to further improve the recognition rate.

        4 Conclusion

        In this paper we presented a solution to contextualize the histograms derived from local features such as LBP and WLD and utilize them for facial expression recognition. Since local contextual structures were taken into account when a histogram was derived from local features, the contextualized histogram was more discriminative for facial expression recognition. Experimental results on the 2D feature images from the 3D-BUFE dataset demonstrated the effectiveness of the proposed solution by adding the recognition rate when weighted sub-block and contextualized histograms were employed to the original local feature patterns. In the future, we will further investigate adaptive weighting strategies at both sub-lock level and the local contextual structure level, and try to introduce dimension reduction algorithm to improve the performance.

        [1] Mehrabian A. Communication without words[J]. Psychology Today, 1968, 2(4):53-56.

        [2] Fasel B, Luettin J. Automatic facial expressionanalysis: a survey[J]. IEEE Transactions on Pattern Recognition, 2003,36(1):259-275.

        [3] Krinidis S, Buciu I, Pitas I. Facial expressionanalysis and synthesis: a survey[C]∥The 10thInternational Conference on Human-Computer Interaction, Crete, Greece, 2003.

        [4] Pantic M, Rothkrantz L J. Automatic analysis offacial expressions the state of the art[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(12):1424-1445.

        [5] Ojala T, Pietikinen M, Harwood D. A comparative study of texture measures with classification based on featured distribution[J]. Pattern Recognition, 1996, 29(1):51-59.

        [6] Chen J, Shan S, He C,et al. Wld: A robust local image descriptor[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(9):1705-1720.

        [7] Shan C, Gong S, McOwan P. Facial expression recognition based on local binary patterns: A comprehensive study[J]. Image and Vision Computing, 2009, 27(6):803-816.

        [8] Feng J, Ni B, Xu D, et al. Histogram contextualization[J]. IEEE Transactions on Image Processing, 2012, 21(2):778-788.

        [9] Ojala T,Pietikainen M, Maenpaa T. Multi-resolution gray-scale and rotation invariant texture classification with local binary patterns[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(7):971-987.

        [10] Yin L, Wei X, Sun Y, et al. A 3d facialexpression database for facial behavior research[C]∥International Conference on Automatic Face and Gesture Recognition, Southampton, UK, 2006.

        [11] Chang C C, Lin C J. LibSVM: a library for support vector machines[J]. ACM Transactions on Intelligent Systems and Technology, 2011, 2(27):1-27.

        (Edited by Cai Jianying)

        10.15918/j.jbit1004-0579.201524.0317

        TP 37 Document code: A Article ID: 1004- 0579(2015)03- 0392- 06

        Received 2014- 02- 20

        Supported by the National Natural Science Foundation of China(60772066)

        E-mail: napoylei@163.com

        猜你喜歡
        刊刻張超書籍
        魯迅與“書籍代購”
        張超個人簡介
        散文百家(2021年11期)2021-11-12 03:06:38
        My New Invention
        How to Protect Us from Infectious Diseases
        張超個人簡介
        散文百家(2021年4期)2021-04-30 03:15:20
        宋人對三蘇文章的選編與刊刻
        國學(xué)(2020年1期)2020-06-29 15:14:38
        AOS在書籍編寫的應(yīng)用
        書籍
        論晚明元劇選刊刻現(xiàn)象及其文化審美意義
        戲曲研究(2018年3期)2018-03-19 08:47:18
        書籍是如何改變我們的
        一区二区视频在线观看地址| 免费毛片性天堂| 波多野无码AV中文专区 | 国产精品亚洲日韩欧美色窝窝色欲| 天天插天天干天天操| 青青草视频在线播放观看| 18禁裸体动漫美女无遮挡网站| 日本丰满熟妇hd| 精品国产免费久久久久久| 亚洲国产丝袜美女在线| 国产av自拍视频在线观看| 国产成人涩涩涩视频在线观看| 精品国产群3p在线观看| 国产成人精品中文字幕| 国产极品裸体av在线激情网| 免费看黄色电影| 久久亚洲AV无码精品色午夜| 亚洲中文字幕一区高清在线| 无码a级毛片免费视频内谢5j| 人与禽交av在线播放| 免费一级国产大片| 国产韩国一区二区三区| 爱情岛论坛亚洲永久入口口| 久久久久久av无码免费看大片 | 亚洲成av人影院| 色老头一区二区三区| 色视频日本一区二区三区 | 九九久久精品一区二区三区av| 亚洲三区在线观看内射后入| 亚洲欧洲精品成人久久曰影片 | 黄 色 成 年 人 网 站免费| 中文字幕乱码亚洲一区二区三区 | 国产一区二区三区三区四区精品| 男女爽爽无遮挡午夜视频| 国产资源在线视频| 91精品亚洲熟妇少妇| 男女性杂交内射妇女bbwxz| 东京热久久综合久久88| 国内专区一区二区三区| 极品少妇xxxx精品少妇偷拍| 亚洲欧美日韩国产综合一区二区 |