亚洲免费av电影一区二区三区,日韩爱爱视频,51精品视频一区二区三区,91视频爱爱,日韩欧美在线播放视频,中文字幕少妇AV,亚洲电影中文字幕,久久久久亚洲av成人网址,久久综合视频网站,国产在线不卡免费播放

        ?

        TrainingRobust Support Vector Machine Based on a New Loss Function

        2015-12-20 09:14:18LIUYeqing劉葉青

        LIU Yeqing(劉葉青)

        School of Mathematics and Statistics,Henan University of Science &Technology,Luoyang 471003,China

        Introduction

        Support vector machine (SVM)[1]was introduced by Vladimir Vapnik and colleagues.It is a relatively new learning method used for data classification.The basic idea is to find a hyperplane which separates the d-dimensional data perfectly into its two classes.Since example data were often not linearly separable,the notion of a“kernel induced feature space”was introduced which mapped the data into a higher dimensional space where the data were separable.Typically,casting into such a space would cause overfitting and calculation problems.The key technique used in SVM is that the higher-dimensional space doesn't need to be dealt with directly(as it turns out,only the formula for the dotproduct in that space is needed), which eliminates overfitting and calculation problems.Furthermore,the VCdimension(a measure of a system's likelihood to perform well on unseen data)of SVM can be explicitly calculated,unlike other learning methods such as neural networks,for which there is no measure.Overall,SVM is intuitive,theoretically well-founded,and has shown to be practically successful.

        SVM has been receiving increasing interest in many areas as a popular data classification tool.Unfortunately,researchers have shown that SVM is sensitive to the presence of outliers even though the slack technique is adopted.The central reason is that outliers are always playing dominant roles in determining the decision hyperplane since they tend to have the largest margin losses according to the character of the hinge loss function.

        In machine learning,the hinge loss is a loss function used for training classifiers.The hinge loss is used for the“maximum-margin”classification,most notably for SVM.For an intended output yi=±1and a classifier score f(xi)=wxi+b,the hinge loss of the prediction f(xi)is defined as l(yi·f(xi))=max(0,1-yi·f(xi)).It can be seen that when yiand f(xi)have the same sign,this means that f(xi)predicts the right class and|f(xi)|≥1,l(yi·f(xi))=0,but when they have opposite sign,l(yi·f(xi))increases linearly with f(xi)(one-sided error).SVM uses linear hinge loss function[2].According to the character of the linear hinge loss function,SVM is sensitive to outliers.In addition,the linear hinge loss function was not smooth.Thus, when used in SVM,it degrades the generalization performance of SVM.

        In this paper a new tangent loss function was proposed.As the tangent loss function was not smoothing in some interval,a polynomial smoothing function was used to approximate it in this interval.Based on the tangent loss function,the tangent SVM (TSVM)was proposed.The experimental results show that TSVM reduces the effects of outliers.So the proposed new loss function is effective.

        1 Related Works

        Consider a binary classification problem.Given a training dataset,where xi∈?nand yi∈{-1,+1}.The primal optimization of SVM is usually written as:

        where C >0is a penalty parameter;H1(u)is the linear hinge loss function,which has the form of H1(u)=max(0,1-u).It is obvious that the linear hinge loss function has no limit on the loss values of outliers,whose loss values may be very large when yi(w·xi+b)?1.Thus,of all training samples,the outliers will retain the maximal influences on the solution since they will normally have the largest hinge loss.This results in the decision hyperplane of SVM being inappropriately drawn toward outliers so that its generalization performance is degraded.

        Many loss functions[3-11]have been proposed to improve the generalization performance,increase the speed of optimization problem solving, or account for certain nonstandard situations.

        Lee and Mangasarian[3]proposed quadratic hinge loss function,

        Though quadratic hinge loss function is smooth,it enhances greatly the loss values of outliers.

        Shen et al.[7]proposed a new method that replaced the hinge loss in SVM by a loss function of the form,

        Although their method[7]outperforms SVM in nonseparable cases,the method is computationally much more complex than SVM.

        Lin et al.[8]extended SVM to nonstandard situations by adding an extra term to the hinge loss

        Here the extra termc(y)reflects two types of nonstandard situations:(1)misclassification costs are different for different classes,and(2)the sampling proportions of classes are different from their population proportions due to sampling bias.

        The above loss functions penalize only points with u <1.As a result,they all have the same problems as SVM,namely,they are sensitive to training samples and perform poorly.

        The Ramp loss function

        was used in SVM recently[5-6].The Ramp loss function has been investigated widely in the theoretical literature in order to improve the robustness of SVM,which limits its maximal loss value distinctly.Obviously,the Ramp loss function can put definite restrictions on the influences of outliers so that it is much less sensitive to their presence.Though the Ramp loss function limits the loss values of outliers,the losses caused by outliers are the same as those caused by any other misclassified samples.Thus,all misclassified samples are considered the same.Obviously,outliers and support vectors could not be treated equally.

        2 Tangent Loss Function

        To robustify the SVM,a new loss function is proposed.We call it tangent loss function,which has the form,

        The value of the function T(u)is in the interval[0,2).It is apparent that the loss function T(u)limits loss value of outliers.However,the tangent loss function is not smooth when u=1,and a polynomial function is used as the smooth approximation for T (u ) in the intervalwhere k >0.Thus,T(u)changes into a new smoothing loss function S(u)(see Fig.1),

        Thus,under the smoothing loss S(u),optimization problem(1)changes into the following problem,

        We call this SVM as TSVM.

        3 Experiments

        Fig.1 The function image of S(u)

        To verify the efficiency of the tangent loss function,experiments are performed on UCI data sets.We compare TSVM with SVM,which use the loss function of H2(u).Because the H2(u)quadratic hinge loss function is smooth function,the corresponding optimization problem can be solved directly.If we use the linear loss function H1(u),then the corresponding optimization problem can be solved only after being approximated using a smooth function.BFGS algorithm was used to solve the optimization problem(2).The algorithms are written in Matlab7.1,k =25,and other parameters are chosen for optimal performance.

        We demonstrate the effectiveness of the TSVM by comparing it numerically with SVM on UCI data sets.

        Those data sets include moderate sized data sets,monks-1,monks-2,breast-w,heart-statlog;balance and large data sets magic gamma telescope.In data sets monks-1 and monks-2,the training samples are draw-out from the testing samples.Their training samples are fixed.In other data sets,the training samples are chosen randomly,the remaining samples are testing samples.In the latter data sets,to demonstrate the capability of TSVM,training set sizes change.We performs 5-fold cross validation on each data set.Table 1shows the results of TSVM and SVM according to average testing accuracy,and p,q,and k are the numbers of training samples,testing samples,and outliers,respectively.

        Table 1 Comparing the testing accuracy of TSVM and SVM

        In order to compare the robustnesses of TSVM and SVM,we repeat above experiments on these training sets by adding 1% outliers.The outliers are produced by changing their labels.TSVM and SVM learn classification on such new training sets.The experimental results are also reported in Table 1.It is obvious that testing accuracy of TSVM is higher than that of SVM on all data sets.As a result of adding 1% outliers to the datasets,the testing accuracy of TSVM decreases no more than 1.5% while that of SVM decreases much more significantly by about 1.5%-7.5%.

        The experimental results show that TSVM is effective and is more robust to outliers than SVM.

        4 Conclusions and Future Research

        Because of using hinge loss function,conventional SVM is sensitive to outliers.A new loss function-tangent loss function was proposed.Since the tangent loss function was not smooth, a smoothing function was adopted to approximate it.The experimental results show the tangent loss function is more robust to outliers than linear hinge loss function.Therefore,the proposed new loss function is effective.

        There are several other research directions that need to be further pursued.Such as comparison with other large margin classifiers and other approaches to large sample bias problems would be interesting and generalized to nonlinear kernel functions needs to be investigated in the future.

        [1]Vapnik V.Satistical Learning Theory[M].New York:Wiley-Interscience,1998.

        [2]Chapelle O.Training a Support Vector Machine in the Primal[J].Neural Computation,2007,19(5):1155-1178.

        [3]Lee Y J,Mangasarian O L.SSVM:a Smooth Support Vector Machine for Classification[J].Computational Optimization and Applications,2001,22(1):5-21.

        [4]Wang S C,Jiang W,Tsui K L.Adjusted Support Vector Machines Based on a New Loss Function[J].Annals of Operations Research,2008,174(1):83-101.

        [5]Xu L,Crammer K,Schuurmans D.Robust Support Vector Machine Training via Convex Outlier Ablation [C].Proceedings of the 21st National Conference on Artificial Intelligence,Boston,2006:1321-1323.

        [6]Wang L,Jia H D,Li J.Training Robust Support Vector Machine with Smooth Ramp Loss in the Primal Space[J].Neurocomputing,2008,71(479):3020-3025.

        [7]Shen X,Tseng G C,Zhang X,et al.Onψ-Learning[J].Journal of American Statistical Association,2003,98(463):724-734.

        [8]Lin Y,Lee Y K,Wahba G.Support Vector Machines for Classification in Nonstandard Situations [J].Machine Learning,2002,46(2):191-202.

        [9]Wu Y C,Liu Y F.Non-crossing Large-Margin Probability Estimation and Its Application to Robust SVM via Preconditioning[J].Statistical Methodology,2011,8(1):56-67.

        [10]Ertekin S,Bottou L,Giles C.Nonconvex Online Support Vector Machines[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2010,33(2):368-381.

        [11]Zhong P.Training Robust Support Vector Regression with Smooth Non-convex Loss Function[J].Optimization Methods&Software,2012,27(6):1039-1058.

        日本在线播放不卡免费一区二区| 国产午夜精品一区二区三区不卡| 亚洲色图+国产精品| 岛国视频在线无码| 新久久国产色av免费看| 欧美村妇激情内射| 久久婷婷国产剧情内射白浆| 91在线区啪国自产网页| 尤物蜜桃视频一区二区三区| 极品少妇hdxx麻豆hdxx| 丰满人妻熟妇乱又伦精品视| 人妻少妇看A偷人无码电影| 国产情侣一区在线| 国产精品jizz视频| 欧洲女人性开放免费网站| 亚洲一区二区三区成人 | 国产白浆精品一区二区三区 | 日韩av无码一区二区三区不卡| 国产成人vr精品a视频| 偷拍区亚洲区一区二区| 日本精品一级二区三级| 精品国产三级a∨在线| 亚洲精品你懂的在线观看| 国产亚洲三级在线视频| 成年人观看视频在线播放| 亚洲午夜无码av毛片久久| 国产V日韩V亚洲欧美久久| 亚洲av熟女天堂久久天堂| 77777_亚洲午夜久久多人| 久久天天躁狠狠躁夜夜96流白浆| 青青草99久久精品国产综合| 国产一区白浆在线观看| 老司机亚洲精品影院| 91人妻无码成人精品一区91| 在线观看国产自拍视频| 久久婷婷五月综合色欧美| 久久久精品人妻一区亚美研究所 | 欧美在线不卡视频| 日本办公室三级在线看| 久久精品国产亚洲av果冻传媒| 夜夜揉揉日日人人|