Ensembles of Binary SVM Decision Trees

Web proceedings papers

Authors

Gjorgji Madjarov , Dejan Gjorgjevikj and Tomche Delev

Abstract

Ensemble methods are able to improve the predictive performance of many base classifiers. In this paper, we consider two ensemble learning techniques, bagging and random forests, and apply them to Binary SVM Decision Tree (SVM- BDT). Binary SVM Decision Tree is a tree based architecture that utilizes support vector machines for solving multiclass problems. It takes advantage of both the efficient computation of the decision tree architecture and the high classification accuracy of SVMs. In this paper we empirically investigate the performance of ensembles of SVM-BDTs. Our most important conclusions are: (1) ensembles of SVM-BDTs yield noticeable better predictive performance than the base classifier (SVM-BDT), and (2) the random forests ensemble technique is more suitable than bagging for SVM- BDT.

Keywords

Ensembles, Bagging, Random Forests, Support Vector Machines, Binary decision tree.