亚洲免费av电影一区二区三区,日韩爱爱视频,51精品视频一区二区三区,91视频爱爱,日韩欧美在线播放视频,中文字幕少妇AV,亚洲电影中文字幕,久久久久亚洲av成人网址,久久综合视频网站,国产在线不卡免费播放

        ?

        Facial expression recognition with contextualized histograms

        2015-04-22 07:49:26YUELei岳雷SHENTingzhi沈庭芝DUBuzhi杜部致ZHANGChao張超ZHAOShanyuan趙三元
        關(guān)鍵詞:刊刻張超書籍

        YUE Lei(岳雷), SHEN Ting-zhi(沈庭芝), DU Bu-zhi(杜部致)ZHANG Chao(張超) ZHAO Shan-yuan(趙三元)

        (1.School of Information and Electronics, Beijing Institute of Technology, Beijing 100081,China;2.School of Computer Science and Technology, Beijing Institute of Technology, Beijing 100081, China)

        ?

        Facial expression recognition with contextualized histograms

        YUE Lei(岳雷), SHEN Ting-zhi(沈庭芝)1, DU Bu-zhi(杜部致)1,ZHANG Chao(張超)1, ZHAO Shan-yuan(趙三元)2

        (1.School of Information and Electronics, Beijing Institute of Technology, Beijing 100081,China;2.School of Computer Science and Technology, Beijing Institute of Technology, Beijing 100081, China)

        A new algorithm taking the spatial context of local features into account by utilizing contextualized histograms was proposed to recognize facial expression. The contextualized histograms were extracted from two widely used descriptors—the local binary pattern (LBP) and weber local descriptor (WLD). The LBP and WLD feature histograms were extracted separately from each facial image, and contextualized histogram was generated as feature vectors to feed the classifier. In addition, the human face was divided into sub-blocks and each sub-block was assigned different weights by their different contributions to the intensity of facial expressions to improve the recognition rate. With the support vector machine (SVM) as classifier, the experimental results on the 2D texture images from the 3D-BUFE dataset indicated that contextualized histograms improved facial expression recognition performance when local features were employed.

        facial expression recognition; local binary pattern; weber local descriptor; spatial context; contextualized histogram

        Facial expressions play a key role in non-verbal face-to-face communication. Facial expressions have paramount impact on human interaction: about 55 percent of the effectiveness of a conversation relies on facial expressions, 38 percent is conveyed by voice intonation, and 7 percent by the spoken words[1]. Therefore, automatic facial expression is essential to unleash the potential of many applications such as intelligent human-computer interaction (HCI) and social analysis of human behavior.

        During the past decades, many visual features have been proposed to characterize facial expressions[2-4]. In particular, local image descriptors such as local binary pattern (LBP)[5]and weber local descriptor (WLD)[6]have demonstrated great effectiveness in facial expression recognition[7]. Ojala et al. initially proposed LBP to characterize texture units by exploring the spatial context around a given pixel. And inspired by Weber’s Law, Chen et al. proposed WLD which characterizes local features with two components, namely differential excitation and Orientation, to handle many tasks such as texture classification, action recognition, and face recognition. However, these two descriptors only explore the first order context at the pixel intensity level, rather than the pattern level. As a result, the contextual information among those local patterns or descriptors does not contribute to the discrimination of extracted descriptors and recognition performance could be compromised.

        In this paper, we explored spatial context among local descriptors during the formation of histograms with the recently introduced contextualized histogram[8]technique. That is, instead of simply calculating the distribution of each local pattern for a given image, more patterns at finer granularity were discovered to form a higher dimensional and more discriminative histogram. In order to further improve recognition performance, a weighted partition method was investigated.

        1 Local features

        1.1 LBP

        LBP was initially proposed by Ojala et al.[5]to characterize texture units by exploring the spatial context around a given pixel. That is, each pixel is converted into a LBP by referring to its neighborhood and the binary patterns of a given image are formed into a histogram which describes the distribution of those LBPs. In detail, the operator labels the pixels of an image by thresholding the 3×3 neighborhood of each pixel with the center value and considering the result as a binary number. By considering the 8-bit result as a binary number, a 256-bin histogram of the LBP labels computed over a region is used as a texture descriptor. Due to its effectiveness as well as simplicity, it has been widely used in many pattern classification problems such as face recognition and facial expression recognition.

        Formally, given a pixel at (xc,yc), the resulting LBP can be expressed in decimal form as

        (1)

        whereicandipare gray-level values of the central pixel and one of thePneighbor pixels with a radiusR, respectively. Functions(x) is defined as

        (2)

        Inaddition,thedimensionalityoftheoperatorwasfurtherreducedbyintroducingthenotionofauniformLBPwhichcontainsatmosttwobitwisetransitionsfrom0to1orviceversawhenthebinarystringisconsideredcircular.Asaresult,eachimagecanberepresentedwitha59-dimensionalhistogramwhereeachbinrepresentsthedistributionofoneLBPamongtheimage.

        Fig.1 Illustration of the LBP operator

        1.2WLD

        WLD[6]isbasedonthefactthathumanperceptionofapatterndependsonnotonlythechangeofastimulus(suchassound,lighting)butalsotheoriginalintensityofthestimulus.Specifically,WLDconsistsoftwocomponents:differentialexcitationandorientationforeachreferencepixelwithinitsneighbors(i.e.thecentralpixelofa3×3neighborhood).Thedifferentialexcitationcomponentisafunctionoftheratiobetweentwoterms:therelativeintensitydifferencebetweenareferencepixelanditsneighbors,andtheintensityofthereferencepixel.Theorientationcomponentisthegradientorientationofthereferencepixel.

        Specifically,thedifferentialexcitationξ(xc)ofareferencepixelxciscomputedinthreesteps.First,thedifferencesbetweenthereferencepixelanditsneighborsarecalculatedaftertheinputimageisfilteredwithf00asshowninFig.2.

        官箴文化在晚明有顯著的發(fā)展,呈現(xiàn)興盛局面,最直接、最明顯的表現(xiàn)就是官箴書數(shù)量的“驟增”。需說明的是,這里所指的官箴書包括三種情況:一是書籍的創(chuàng)作與刊刻都在晚明時期;二是書籍的創(chuàng)作在晚明,但可能刊刻于清代;三是無法判定其創(chuàng)作期,但最早的版本確定在晚明,這種情況一般不著撰者。以上三種情況,我們都可將其視為晚明官箴書。

        (3)

        (4)

        Fig.2 Masks of four Filters: f00, f01, f10 and f11

        TheorientationcomponentofWLD,i.e.thegradientorientation,iscomputedas

        (5)

        Asaresult,eachimagecanberepresentedwithaWLDhistogramfromthe2Dhistogramsofdifferentialexcitationsanddominantorientations.Thedimensionalityofthe2DhistogramisT×C,whereTisthenumberofdominantorientationsandCisthenumberofcells.EachcellisfurtherevenlydividedintoM×Ssegments,whereMisthenumberofsub-histogramsforeachorientationandSisthenumberofbinsofeachsub-histogram.Thatis,thedimensionoftheWLDhistogramisT×S×M.

        2 Histogram contextualization

        AssumethatanimageisrepresentedwithaK-binhistogram: h={hi}(i=1,2,…,K) (hisafeaturehistogramandKisthenumberofbinsofit).Histogramcontextualizationistoconstructaso-calledcontextualizedhistogramhs={hi(sj)}(i=1,2,…,K; j=1,2,…,M)basedonasetofpredefinedlocalcontextualstructuresdenotedas{sj}(j=1,2,…,M),wherehi(sj)isthenumberofpixelswhichtaketheithlocalfeaturevalueandsurroundedwiththelocalcontextualstructuresj.Thatis,foreachgivenlocalfeaturepattern(e.g.LBP),thereareMsub-patternsintermsofitsspatialcontext.Asaresult,thecontextualizedhistogramisofM×Kdimension.

        Foralocalcontextualstructurewithonlytwopixels,therearetwotypesofhomogeneities,namelyhomogeneousandinhomogeneous.Foraternarylocalcontextualstructure,thereare5typesoflocalhomogeneities.Therefore,theternarylocalcontextualstructuresareabletoencodemorecomplicatedandinformativelocalcontextualinformation.Fengetal.[8]defined30ternarycontextualstructuresbycombining5typesofhomogeneitiesand6shapes(Fig.3).Notethattheordershowninthefiguremeansthenumberofpixelsbelongingtothesamehistogrambin,anddifferentcolorsrepresentdifferenthistogrambins.Asaresult,thedimensionofthecontextualizedhistogramisM×K=30×K.

        Fig.3 Histogram contextualization with 30 ternary local contextual structures combining 5 homogeneities and 6 different shapes

        3 Experiments

        Weconductedexperimentsinfourstepswiththe2Dtextureimagesfrom3D-BUFEdataset: ①ExtractedlocalfeaturepatternsofLBPandWLDseparatelyfromtheoriginalexpressionimages; ②Dividedtheimagesintosub-block; ③Addedweightstoeachsub-block.; ④Impliedcontextualizedhistogramtothefeaturevectorsfromthethirdstep.Wegotexpressionclassificationresultsfromallthesefourtrialsandcomparedtherecognitionrateinthissection.

        3.1Experimentalsettings

        Weusedthe2Dtextureapexexpressionimagesfromthewidelyused3D-BUFEdataset[10].Therewere100subjectswhoparticipatedinfacescan,includingundergraduates,graduatesandfacultyfromvariousdepartmentsofBinghamtonUniversity(e.g.suchasPsychology,Arts,andEngineering)andStateUniversityofNewYork(e.g.ComputerScience,ElectricalEngineering,andMechanicalEngineering).ThemajorityofparticipantswereundergraduatesfromthePsychologyDepartment.Theresultingdatabaseconsistsofabout60%femaleand40%malesubjectswithavarietyofethnic/racialancestries,includingWhite,Black,East-Asian,Middle-eastAsian,HispanicLatino,andothers.Eachsubjectperformedsevenexpressionsinfrontofthe3Dfacescanner.Withtheexceptionoftheneutralexpression,eachofthesixprototypicexpressions(i.e.anger,disgust,fear,happiness,sadness,andsurprise)includesfourlevelsofintensity.Therefore,thereare25 3Dexpressionmodelsforeachsubject.Asaresult,thedatasetintotalcontains2 500 3Dfacialexpressionmodelsandeachmodelhasitscorresponding2Dtextureimage.SevensampleexpressionsofasubjectareshowninFig.4.Andwerandomlypicked80subjectsasthetrainingsetandtheother20subjectsasthetestingset.

        Fig.4 Samples of seven expressions (from left to right): anger, disgust, fear, happiness, neutral, sadness, and surprise

        Fig.5 Illustration of weighted sub-block based representation

        Inordertotaketheglobalspatialcontextintoaccount,wealsoinvestigatedtheimpactofsub-blockbasedrepresentationwithandwithoutweights.AsshowninFig.5,eachfaceimageispartitionedinto42 (6×7)sub-blocksandsquarecolorsofblack,darkgray,lightgrayandwhiteindicateweightsof0.0, 1.0, 2.0and4.0respectively[7].Asaresult,thefeaturevectorofeachimagewillbe42×K,whereKisthedimensionofthehistogramderivedfromalocaldescriptor(eg.59forLBPor192forWLDwhereT=8, S=4,andM=6).Wegotthesethreenumbers(T, SandM)byrepeatedlytryingdifferentcombinationsandtriedtoreachacompromisebetweencalculationefficiencyandrecognitioncorrectrates.

        Duetothesub-blockrepresentationandhistogramcontextualization,thefeaturedimensionisquitehigh.Forexample,thefeaturedimensionofacontextualizedWLDhistogramis241 920 (=192×42×30).Therefore,weperformdimensionreductionwithprinciplecomponentanalysis(PCA).Weemployedthewidelyusedsupportvectormachines(SVM)[11]astheclassifier.Thoughmanynewkernelsarebeingproposedbyresearchers,themostwidelyusedkernelfunctionsarethelinearfunction,polynomialfunction,radialbasisfunction(RBF),andSigmoidfunction.Inthispaper,weusedtheRBFkernelfunctiontocarryouttheexperiments.

        3.2Experimentalresultsanddiscussions

        Experimentswereconductedinfourstepstoevaluatetheperformanceofthecontextualizedhistogramsderivedfromlocaldescriptors.ThefourLBP-basedhistogramsareglobalhistogram(LBP),sub-blockbasedrepresentation(LBPwithsub-block),weightedsub-blockbasedrepresentation(LBPwithweightedsub-block),andcontextualizedhistogramsoverweightedsub-blockbasedrepresentation(LBPwithweightedsub-blockandcontext).AsshowninTab.1,thecontextualizedhistogramwasabletoachievethehighestrecognitionaccuracy,indicatingthatlocalcontextualstructurewashelpfulincharacterizingspatialcontext.Thesub-blockbasedrepresentationimprovedrecognitionperformancesignificantly,whichdemonstratedtheuseofglobalspatialcontextcouldimprovetherecognitionrateaswell.Assigndifferentweighttodifferentsub-blockswasalsohelpfulforfacialexpressionrecognition.Thebestperformanceforcontextualizedhistogramwasobtainedwhenthereducedfeaturedimensionwas200,afterseveralvalueswereattempted.

        Tab.1 Comparison of recognition accuracy among LBP based histograms

        The similar result was also observed for WLD-based histograms as shown in Tab.2. The classification based on original WLD histograms didn’t show competitive recognition capability. But when we first divided the face into sub-block and then assigned different weights to corresponding sub-blocks, the recognition rate improved significantly. We obtained the best results with the feature extract with WLD when we applied contextualized histogram on the vector generate from weighted sub-blocked human face.

        Tab.2 Comparison of recognition accuracy among WLD based histograms

        WLD outperformed LBP while a histogram was obtained globally (without partitioning an image into sub-blocks). This could be explained with the superiority of WLD. However, in other situations LBP slightly outperformed WLD. This could be explained with the high dimensionality of WLD based histograms. Dimension curse was still an imperfectly solved problem. So, efficient extraction of useful information from overlong vectors extracted with WLD may be able to help us to further improve the recognition rate.

        4 Conclusion

        In this paper we presented a solution to contextualize the histograms derived from local features such as LBP and WLD and utilize them for facial expression recognition. Since local contextual structures were taken into account when a histogram was derived from local features, the contextualized histogram was more discriminative for facial expression recognition. Experimental results on the 2D feature images from the 3D-BUFE dataset demonstrated the effectiveness of the proposed solution by adding the recognition rate when weighted sub-block and contextualized histograms were employed to the original local feature patterns. In the future, we will further investigate adaptive weighting strategies at both sub-lock level and the local contextual structure level, and try to introduce dimension reduction algorithm to improve the performance.

        [1] Mehrabian A. Communication without words[J]. Psychology Today, 1968, 2(4):53-56.

        [2] Fasel B, Luettin J. Automatic facial expressionanalysis: a survey[J]. IEEE Transactions on Pattern Recognition, 2003,36(1):259-275.

        [3] Krinidis S, Buciu I, Pitas I. Facial expressionanalysis and synthesis: a survey[C]∥The 10thInternational Conference on Human-Computer Interaction, Crete, Greece, 2003.

        [4] Pantic M, Rothkrantz L J. Automatic analysis offacial expressions the state of the art[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(12):1424-1445.

        [5] Ojala T, Pietikinen M, Harwood D. A comparative study of texture measures with classification based on featured distribution[J]. Pattern Recognition, 1996, 29(1):51-59.

        [6] Chen J, Shan S, He C,et al. Wld: A robust local image descriptor[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(9):1705-1720.

        [7] Shan C, Gong S, McOwan P. Facial expression recognition based on local binary patterns: A comprehensive study[J]. Image and Vision Computing, 2009, 27(6):803-816.

        [8] Feng J, Ni B, Xu D, et al. Histogram contextualization[J]. IEEE Transactions on Image Processing, 2012, 21(2):778-788.

        [9] Ojala T,Pietikainen M, Maenpaa T. Multi-resolution gray-scale and rotation invariant texture classification with local binary patterns[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(7):971-987.

        [10] Yin L, Wei X, Sun Y, et al. A 3d facialexpression database for facial behavior research[C]∥International Conference on Automatic Face and Gesture Recognition, Southampton, UK, 2006.

        [11] Chang C C, Lin C J. LibSVM: a library for support vector machines[J]. ACM Transactions on Intelligent Systems and Technology, 2011, 2(27):1-27.

        (Edited by Cai Jianying)

        10.15918/j.jbit1004-0579.201524.0317

        TP 37 Document code: A Article ID: 1004- 0579(2015)03- 0392- 06

        Received 2014- 02- 20

        Supported by the National Natural Science Foundation of China(60772066)

        E-mail: napoylei@163.com

        猜你喜歡
        刊刻張超書籍
        魯迅與“書籍代購”
        張超個人簡介
        散文百家(2021年11期)2021-11-12 03:06:38
        My New Invention
        How to Protect Us from Infectious Diseases
        張超個人簡介
        散文百家(2021年4期)2021-04-30 03:15:20
        宋人對三蘇文章的選編與刊刻
        國學(xué)(2020年1期)2020-06-29 15:14:38
        AOS在書籍編寫的應(yīng)用
        書籍
        論晚明元劇選刊刻現(xiàn)象及其文化審美意義
        戲曲研究(2018年3期)2018-03-19 08:47:18
        書籍是如何改變我們的
        18禁无遮拦无码国产在线播放| 特黄三级一区二区三区| 一区二区三区在线观看视频 | 久久夜色国产精品噜噜亚洲av| 亚洲人精品午夜射精日韩| 亚洲男人的天堂在线aⅴ视频| 伊人22综合| 久久av一区二区三区下| 精品国产一区二区三区香| 欧洲成人一区二区三区| 国产无遮挡无码视频免费软件| 粗了大了 整进去好爽视频| 久久精品国产热久久精品国产亚洲| 日本免费a一区二区三区| 久久伊人亚洲精品视频| 丰满的人妻hd高清日本| 国产乱妇乱子视频在播放| 精品亚洲少妇一区二区三区| av成人资源在线观看| 91丝袜美腿亚洲一区二区| 国产成本人片无码免费2020| 国产欧美日韩专区| 久久伊人精品只有这里有 | 亚洲av综合色区无码一区| 纯爱无遮挡h肉动漫在线播放 | 亚洲∧v久久久无码精品| 久久亚洲av成人无码软件| 亚洲熟女熟妇另类中文| 国产七十六+老熟妇| 黑人巨大videos极度另类| 精品中文字幕日本久久久| 婷婷色国产精品视频二区| 成人精品一区二区三区中文字幕| 国产亚洲欧美精品一区| 国产女人av一级一区二区三区| 国产v片在线播放免费无码| 又爽又黄无遮挡高潮视频网站| 极品少妇被后入内射视| 狠狠综合久久av一区二区蜜桃 | 极品一区二区在线视频观看| 久久综合狠狠综合久久|