View Full Document

A New Boosting Algorithm Using Input-Dependent Regularizer



View the full content.
View Full Document
View Full Document

11 views

Unformatted text preview:

The Twentieth International Conference on Machine Learning ICML 03 Washington DC August 21 24 2003 A New Boosting Algorithm Using Input Dependent Regularizer Rong Jin rong cs cmu edu Yan Liu yanliu cs cmu edu Luo Si lsi cs cmu edu Jaime Carbonell jgc cs cmu edu Alexander G Hauptmann alex cs cmu edu School of Computer Science Carnegie Mellon University Pittsburgh PA 15213 8213 USA Abstract AdaBoost has proved to be an effective method to improve the performance of base classifiers both theoretically and empirically However previous studies have shown that AdaBoost might suffer from the overfitting problem especially for noisy data In addition most current work on boosting assumes that the combination weights are fixed constants and therefore does not take particular input patterns into consideration In this paper we present a new boosting algorithm WeightBoost which tries to solve these two problems by introducing an inputdependent regularization factor to the combination weight Similarly to AdaBoost we derive a learning procedure for WeightBoost which is guaranteed to minimize training errors Empirical studies on eight different UCI data sets and one text categorization data set show that WeightBoost almost always achieves a considerably better classification accuracy than AdaBoost Furthermore experiments on data with artificially controlled noise indicate that the WeightBoost is more robust to noise than AdaBoost 1 Introduction As a generally effective algorithm to create a strong classifier out of a weak classifier boosting has gained popularity recently Boosting works by repeatedly running the weak classifier on various training examples sampled from the original training pool and combining the base classifiers produced by the week learner into a single composite classifier AdaBoost has been theoretically proved and empirically shown to be an effective method to improve the classification accuracy Through the particular weight updating procedure AdaBoost is able



Access the best Study Guides, Lecture Notes and Practice Exams

Loading Unlocking...
Login

Join to view A New Boosting Algorithm Using Input-Dependent Regularizer and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view A New Boosting Algorithm Using Input-Dependent Regularizer and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?