Table of Accuracy of Learning Algorithms Over Datasets

Warning, this table is very out of date. Most of my algorithms have been improved (some quite significantly) since this table was generated. I am working to generate a new one, but it takes time to run this many tests.

I ran several of my learning algorithms with several datasets to try to determine which algorithm was the best. I have improved some of the algorithms since I made this table, so don't complain if you get slightly different results from these numbers. You can repeat any of these experiments with the following command


    waffles_learn crossvalidate [dataset] [algorithm]

, where [dataset] is shown in the top-most row, and [algorithm] is shown in the left-most column. (Also, you'll need to append ".arff" to the dataset name to specify the filename.) For example, if you run the command

    waffles_learn crossvalidate abalone.arff baseline

, then you should expect to obtain an accuracy of about 0.16495. The green (next to the left-most) column shows a "score" for each algorithm. This score was computed as follows:

For each dataset, the predictive accuracy scores were adjusted linearly such that baseline received a score of 0, and the most accurate algorithm received a score of 1. (Thus scores could not exceed 1, and would only be negative if the algorithm did worse than baseline, which shouldn't happen, but unfortunately does sometimes.) These scores were then averaged (excluding any scores less than -1) to produce the overall score for the algorithm.

If you want to understand how to parse the [algorithm], take a look at the usage page for the learn tool. You should be aware that there are several known problems with this chart, including:

So, my most accurate algorithm appears to be "bag 64 bucket decisiontree meanmarginstree end end". This algorithm is published in:

Gashler, M., Giraud-Carrier, C., and Martinez, T. R., Decision Tree Ensemble: Small Heterogeneous is Better than Large Homogeneous, to appear in Proceedings of ICMLA'08 (International Conference on Machine Learning Applications), 2008.

And without further ado, here's the table...
AlgorithmScoreabaloneadult-censusannealarrhythmiaaudiologyautosbadges2balance-scaleballoonsbreast-cancerbreast-wbupacarschesschess-KingRookVKingPawncoliccoloncredit-acredit-gdermatologydiabetesecoliglassheart-cheart-hheart-statloghepatitishypothyroidionosphereiriskr-vs-kpkroptlaborlensesletterlungCancerlymphlymphomaMagicTelescopemushroommusknurseryozonepost-operativePatientprimary-tumorsegmentsicksonarsoybeanspambasespectrometerspliceteachingAssistanttitanicvehiclevotevowelwaveform-5000wineyeastzoo
baseline00.164950.759190.761690.542040.219470.326830.714290.445120.440.70280.655220.579740.700230.162280.51320.630430.583870.555070.70.306010.651040.42560.328970.54460.639460.532590.793570.922850.641010.290670.51320.162280.649380.6250.037730.350.516220.479170.648370.517970.845860.331560.971210.711110.247740.133850.938760.51250.118310.605950.0877560.518810.293980.676960.236170.613830.167880.326280.398880.311990.40565
neuralnet0.36490.253910.551740.617590.383630.768140.1659710.875840.960.688810.898140.560660.892820.386510.97760.410330.538710.54580.45480.51530.551560.850.249530.500250.43810.509630.528850.456680.847290.940.972280.382140.568720.583330.556620.40.798650.270830.470650.999850.773840.92080.742590.591110.383510.139650.66670.767310.911570.605780.0429250.934920.328560.784550.244210.944370.694140.845760.334830.573580.91267
bucket neuralnet neuralnet -addlayer 4 neuralnet -addlayer 8 -momentum 0.8 neuralnet -addlayer 16 -momentum 0.8 end0.405780.264350.759190.761920.498230.737170.3022910.931840.656640.919840.52590.975810.558730.990110.619020.609680.563190.66040.542080.615890.841670.292520.493750.548980.534070.793560.922850.877510.9640.988170.55270.592980.750.517090.406250.779730.339580.628760.845860.97640.971210.631110.379360.249610.938760.791350.914790.583080.0813580.941070.337740.787550.253660.948030.923030.839120.385390.572640.91286
neuralnet -addlayer 40.412070.264260.759190.761250.540270.659290.3287310.917440.960.645450.948780.579690.956020.389420.988490.605430.645160.658260.70.535520.649740.835120.322430.526750.604080.501480.785810.922850.861540.957330.986110.392470.579310.6250.108920.46250.787840.38750.6484810.84620.963190.971210.537780.329740.22320.938760.768270.807590.605690.085870.930660.368140.785190.252250.93840.800810.833680.369660.563480.88486
neuralnet -addlayer 80.435410.265410.759190.761020.536730.742480.3238310.928010.960.653850.944770.580890.974650.479640.988740.58750.645160.668990.70.712570.648960.839290.334580.510270.637410.508890.785710.922850.862630.958670.987920.469560.519460.758330.446950.41250.824320.385420.6560510.845860.974310.971140.540.3770.325020.938760.754810.902190.583690.079850.938680.350930.786920.234990.944370.863840.835760.349440.582350.89278
graphcuttransducer 320.455170.00201090.833870.922050.548230.286730.187510.926530.865270.520.705590.96710.591910.700350.0999140.906820.620110.583870.837390.70080.753550.704950.771430.508410.832340.814290.828150.765220.935050.823930.920.91790.0999140.613920.550.760660.28750.725680.420830.826190.999380.946560.808010.970110.666670.325030.902770.945760.644230.591240.91480.0542470.80520.516630.786010.567380.344240.777720.959550.507950.41153
neuralnet -addlayer 160.467760.261240.759190.761690.538050.746020.3346110.95840.960.647550.946210.550650.98160.567170.988610.60870.645160.663770.70.842620.647660.852380.342990.508920.604080.520.766530.922850.86670.9560.989240.56970.572290.708330.705450.450.801350.352080.6623210.845950.974540.970580.573330.39410.395410.938760.804810.917130.546960.0892680.942630.321820.7880.25910.943430.90.83480.358430.585710.91486
decisiontree -random0.49070.190620.794320.413720.439820.594120.906120.77310.740.653150.937630.612170.735760.388490.823720.67120.632260.729280.65940.728420.666670.714290.584110.709580.668150.752380.932660.82390.9440.829540.388490.740640.533330.683190.33750.672970.429170.77060.995720.915940.761620.537780.303240.859390.934620.650960.660910.870720.303920.525890.512720.784650.881410.691720.594360.82360.435710.7958
meanmarginstree0.549280.201050.70030.824940.535840.676110.343450.816330.804470.940.655940.944770.566990.871180.47510.932420.632610.793550.624060.61040.534970.670050.779170.503740.574920.639460.580740.700630.922850.903120.930670.930410.471530.617730.758330.794460.40.70.627080.725460.999580.943530.940940.949920.548890.361070.806580.938760.770190.878780.712840.467440.91060.515390.784190.541130.937460.844650.772680.687640.486250.92871
graphcuttransducer 160.563720.0099590.834770.949220.548670.346020.367790.980270.863350.50.718880.966530.608180.707180.106380.934670.670650.645160.854490.70640.915850.717450.826790.57290.820440.812240.818520.767820.935150.834750.962670.937920.106380.613920.550.846590.293750.806760.420830.8335310.955350.904120.970110.666670.923380.958960.713460.805570.917540.100980.81210.512810.784640.657210.920.605860.784160.962920.525880.5938
knn 1 -equalweight0.609760.199570.796240.972160.56150.659290.65650.985710.782080.820.681120.94850.60640.799880.384880.891860.693480.748390.804930.68120.92240.69870.803570.662620.753780.773470.772590.787150.921470.855330.9480.900940.384880.767730.633330.937350.3750.793240.410420.8094410.930310.779140.950710.560.357490.949180.955510.824040.907460.895070.40830.737050.509960.725950.68960.915410.956360.70660.940450.504180.86537
knn 1 -equalweight -scalefeatures0.611940.199570.796240.972160.56150.659290.65650.985710.782080.820.681120.94850.60640.799880.384880.891860.693480.748390.804930.68120.92240.69870.805360.662620.743280.784350.772590.787150.921470.855330.9480.900940.384880.797170.633330.937350.3750.793240.410420.8094410.930310.779140.950710.560.357490.949180.955510.824040.895750.895070.40830.737050.509960.725950.685340.915410.956360.70660.940450.504180.86537
knn 10.616130.199570.796240.972160.563270.659290.655530.985710.782080.820.681120.94850.60640.799880.384880.891860.703260.748390.803480.68120.92240.69870.803570.662620.753780.787760.772590.782050.921740.855330.9480.900940.384880.70850.633330.937350.431250.793240.479170.8094410.930310.779140.951890.560.357490.949180.955510.824040.895750.895070.40830.737050.509960.725950.68960.915410.956360.70660.940450.504180.86537
knn 1 -scalefeatures0.618930.199570.796240.972160.563270.659290.655530.985710.784950.820.681120.94850.60930.799880.384880.891860.703260.748390.803480.68120.92240.69870.803570.662620.753780.787760.772590.782050.921740.847270.9480.900940.384880.70850.633330.937350.431250.793240.479170.8094410.930310.772580.951890.560.357490.949180.955510.824040.895750.895070.40830.737050.509960.68960.915410.952320.70660.940450.504180.86537
graphcuttransducer 20.645620.157670.804110.947880.589820.605310.649590.978910.717730.970.716780.95050.580290.761230.246730.912580.698910.745160.819420.69240.936070.708850.810120.684110.764310.792520.774070.79750.92630.863280.9240.921530.246730.747170.633330.932690.41250.775680.420830.8189310.955770.831820.961120.591110.356880.937660.954670.815380.913040.904670.326580.714420.50470.754380.67470.918620.966260.730120.947190.520620.92282
bag 8 decisiontree -random end0.657750.223030.831210.764370.555750.533630.652760.972790.822070.840.711190.957660.625480.810420.483110.904510.735330.60.829860.7120.880330.722660.804760.652340.777570.797280.79630.792190.950270.894030.940.915460.483110.765020.583330.848080.356250.736490.560420.837590.999830.938590.906930.971210.584440.383510.943720.957690.728850.786540.929060.36420.677550.496770.784550.690070.928720.826670.717120.93820.5310.85133
graphcuttransducer 80.658470.0314580.829230.964140.579650.447790.475250.994560.847980.810.732870.96710.599450.745140.122310.946560.703260.677420.851010.7160.940440.729170.835120.60.816490.801360.817040.782030.936530.841020.9560.94850.122310.627710.608330.902220.30.813510.420830.8393110.963380.872620.970030.664440.37690.934630.960920.784620.87760.918020.195130.803070.51540.785280.702360.928750.859190.779240.959550.539350.78227
graphcuttransducer 40.669380.0828830.819140.965920.603540.580530.610640.984350.821740.810.728670.961380.581450.725690.171990.942180.704890.719350.838550.71760.948630.741150.831550.660750.813180.804080.801480.793590.936690.853570.945330.945560.171990.733250.5750.930390.343750.806760.4250.8366210.965990.841680.968530.651110.392240.945450.958750.815380.901030.915150.257630.764390.505980.782730.704020.925990.956570.758920.952810.542860.87122
decisiontree0.672120.194490.787990.490270.526550.6478410.776320.920.663640.936190.620250.896990.522230.991180.693550.79130.6670.890710.756550.656070.716840.738780.748890.765020.960660.854740.945330.991180.522230.68670.633330.842510.51250.729730.570830.810670.99980.998970.966440.933750.535560.338090.952810.960760.720190.778060.902630.392470.894670.510.786280.695980.933790.829290.73940.895510.480860.41757
knn 7 -equalweight -scalefeatures0.674570.230310.830420.948330.586730.555750.52190.95170.878080.580.725170.965950.603510.852310.574420.932040.694570.674190.849860.7220.934430.72760.835710.610280.819110.802040.815560.785860.937220.829050.953330.935480.574420.575620.566670.92150.350.795950.404170.834310.999190.928860.9340.968850.657780.425920.927360.960980.729810.869410.896330.404890.798750.386810.684630.920930.677370.789680.946070.55930.72678
knn 7 -equalweight0.676080.230310.830420.948330.586730.555750.510340.95170.878080.580.725170.965950.603510.852310.574420.930350.694570.674190.834780.7220.934430.72760.835710.610280.819110.802040.815560.785860.937220.829630.953330.935480.574420.5580.566670.92150.350.795950.404170.833990.999190.927040.9340.968850.657780.425920.927360.960980.729810.869410.896330.404890.798750.386810.781190.684630.920930.677370.78580.946070.55930.72678
knn 3 -equalweight -scalefeatures0.677820.208670.819430.960580.600440.600880.53760.978230.829760.830.706990.961090.590190.821410.396140.92810.719570.741940.836810.70620.938250.728130.836310.634580.804620.785710.815560.794870.935260.841630.941330.935790.396140.673650.683330.929240.36250.79730.408330.8277810.939280.862180.967030.593330.38460.935580.959810.800960.88960.896630.392470.761070.392050.753280.687940.921390.859390.747840.948310.533560.8002
knn 3 -equalweight0.678190.208670.819430.960580.600440.600880.545470.978230.829760.830.706990.961090.590190.821410.396140.92760.704350.741940.836810.70620.938250.728130.836310.634580.804620.79660.815560.794870.935840.841630.941330.935790.396140.673650.683330.929240.36250.801350.408330.8277810.939280.862180.593330.38460.935580.959810.773080.88960.896630.392470.761070.40270.753280.701890.921390.859390.747840.948310.533560.8002
bag 1 decisiontree 1 meanmarginstree 1 knn 1 1 knn 5 1 neuralnet -addlayer 5 1 discretize naivebayes -ess 0.5 end0.678790.212160.811980.9020.563720.675220.585340.963270.840650.770.707690.947350.576250.882870.497450.979350.740220.709680.805510.860660.711720.827980.605610.753810.700680.749630.685130.922850.905970.921330.949310.495320.64150.658330.91360.43750.775680.483330.809620.999780.976390.957240.970980.533330.405260.943550.942310.788460.893120.83660.442570.446420.736140.676360.942530.889490.794920.901120.552830.85725
bucket decisiontree meanmarginstree end0.680860.193440.787970.797550.527430.70.6302910.794880.920.665030.941350.593580.891780.524740.991180.781520.751610.796810.67540.890710.680470.791070.642990.722760.714290.738520.776770.961930.905970.929330.991180.524710.6750.84570.43750.737840.616670.810370.999850.998420.968260.948190.560.346860.953330.963260.750960.874680.905450.488810.785460.701890.938390.850510.773520.902250.484370.90898
categorize knn 30.68780.202390.810890.959470.584070.584070.628130.984350.821760.860.672030.961370.598880.796060.629860.920650.689670.741940.830140.70180.934970.718750.830360.662620.774260.784350.810370.794910.931650.831910.940.914710.630670.659480.550.393750.774320.479170.828790.999850.974170.888780.615560.363990.946840.958320.785580.909810.409780.744260.536370.688650.920480.93960.748240.951690.525880.9189
knn 5 -equalweight -scalefeatures0.689030.219340.826450.952560.601330.590270.533540.968030.85920.830.721680.964520.593620.845950.517810.932040.698370.716130.842610.71680.938250.735420.843450.625230.81450.792520.824440.788430.93680.829640.950670.937050.517810.599750.608330.925910.350.795950.406250.831980.999610.936040.916510.968220.646670.408190.928230.961560.738460.887560.895460.406770.774860.394610.779190.683450.758790.772280.950560.55270.77051
knn 5 -equalweight0.693060.219340.826450.951890.601330.590270.533540.968030.85920.830.721680.964520.593620.845950.517810.932040.698370.716130.842610.71680.938250.735420.843450.625230.81450.792520.817780.788430.93680.829620.950670.937050.517810.599750.608330.925910.350.80.406250.831980.999610.936040.916510.644440.408190.929960.961560.738460.887560.895460.406770.774860.40540.779190.683450.920470.758790.772280.950560.55270.77051
knn 30.715270.202390.81160.967710.593360.669910.640930.983670.821760.870.716080.961370.598880.821410.396340.930160.707070.741940.83130.70640.939340.718750.830360.657940.79010.795240.810370.806510.932930.846750.9480.936980.403470.652340.70.939060.393750.817570.479170.8287910.945860.862220.967430.588890.376350.947710.960130.785580.9060.90550.431280.775240.503320.753280.702360.919550.942020.748240.951690.532350.85953
knn 3 -scalefeatures0.715430.202390.81160.967710.593360.669910.640930.983670.819510.870.716080.961370.601730.821410.396340.930160.707070.741940.83130.70640.939340.718750.830360.657940.79010.795240.810370.806510.932930.846750.9480.936980.396340.652340.70.939060.393750.817570.479170.8287910.945860.862220.967430.588890.376350.947710.960130.785580.905420.90550.431280.775240.504650.753280.702360.919550.942020.748240.951690.532350.85953
bag 8 decisiontree end0.751030.216180.809630.761690.559730.52920.6225210.79230.820.671330.953360.675940.888430.514010.987980.83370.696770.814780.71820.912020.73750.817260.683180.753120.782990.786670.788430.965850.912280.9440.987980.514010.810710.533330.890230.381250.772970.652080.859020.999930.998360.967150.969950.555560.400610.966490.969830.767310.785050.934280.455770.919250.516630.786190.718680.952650.841410.803480.937080.559430.41
knn 50.752350.214890.817810.966820.608410.667260.649660.982990.848320.890.720980.965380.60810.848610.518740.937050.702170.716130.834490.71760.940980.729950.844050.656070.805260.79660.815560.810390.9360.834770.9520.941610.518740.666380.641670.939220.368750.805410.479170.834980.999980.947680.917040.968690.624440.40410.945370.962090.7750.908920.908980.432010.801820.503330.780010.693620.921850.936360.772720.955060.553770.84365
knn 5 -scalefeatures0.758080.21240.817810.96570.608410.667260.649660.982990.848320.890.720980.965380.60810.848610.518740.936230.702170.716130.834490.71760.940980.729950.844050.656070.805260.79660.815560.810390.9360.834770.9520.941610.518740.666380.691670.939220.368750.805410.479170.8349810.948290.917040.624440.40410.945370.962090.7750.901020.908980.432010.801820.503330.780010.693850.921850.936360.772720.955060.553770.84365
knn 7 -scalefeatures0.764710.221450.822140.96570.603980.647790.644830.978910.87040.860.723080.966240.617390.862040.579180.940050.697830.683870.846670.72480.940440.726560.848210.656070.81780.802040.817040.811670.938020.831330.953330.944120.579180.662810.6250.937150.356250.808110.479170.838580.999950.947260.936980.969160.646670.419430.945110.960980.769230.896630.912720.428250.820630.507390.782010.693850.921850.932730.790120.952810.568870.83757
knn 70.768960.221450.822140.96570.603980.647790.644830.978910.87040.860.723080.966240.617390.862040.579180.940050.697830.683870.833330.72480.940440.729170.848210.656070.81780.802040.817040.811670.938020.831330.953330.944120.579180.688050.633330.937150.356250.808110.479170.838580.999950.947260.934350.969160.646670.419430.945110.962350.793270.896630.912720.428250.820630.507390.782010.693850.921850.932730.786160.952810.568870.83757
calibrator bag 32 decisiontree end0.774160.176780.810450.553980.630090.6272510.809910.840.660840.953930.685780.89630.442240.989490.833150.764520.820870.70840.910380.754430.829170.711210.771650.799320.809630.802630.968660.916830.9480.989490.442240.817610.658330.263740.456250.760810.647920.8709310.999210.970590.967510.582220.354510.972380.96930.770190.595020.94280.282480.925960.515320.787920.735220.953580.882220.825680.964040.55040.42161
bag 16 decisiontree end0.781810.227910.811680.761690.561060.55310.6528610.808650.860.672030.955080.67710.896640.525410.991610.82880.709680.82580.71240.912020.749220.833330.714020.759650.786390.802960.801230.967710.914530.9440.991610.525410.842240.583330.907840.368750.759460.683330.867170.99980.999090.971570.970580.591110.395850.972120.969880.781730.804680.939670.475360.925330.500720.782920.73050.953090.868890.820960.937080.576280.41835
bag 16 decisiontree -random end0.790740.230740.837650.761690.569470.576990.680980.98980.827520.820.709090.965380.663790.81470.545340.936480.760870.68710.837970.71680.922950.738540.829170.698130.793370.794560.81630.830920.954290.922510.940.935920.545340.88140.6750.892090.443750.754050.631250.8547410.946740.936480.971290.615560.400030.953330.960070.748080.841590.942490.425260.73530.511370.786460.714890.93380.900810.761640.956180.563070.89729
bag 8 bucket decisiontree meanmarginstree end end0.799330.224560.809390.832070.6991210.840950.760.70350.962810.635430.895950.531420.990550.764520.825510.896720.734380.820240.699070.746510.794560.790370.792240.966170.922510.9520.990550.532980.76490.683330.891410.406250.745950.654170.86030.99980.998360.965770.969160.577780.398290.969260.968820.771150.896650.937270.522010.929340.471580.784190.725060.950350.884240.809240.937080.552020.87718
bucket decisiontree meanmarginstree knn 1 knn 5 neuralnet -addlayer 5 discretize naivebayes -ess 0.5 end0.80820.209190.81810.967260.607520.652210.656710.886090.830.69720.964520.601770.92870.519450.98930.727720.817680.7240.926230.722660.832740.67290.808550.787070.801480.792290.96220.891720.9440.988670.520840.683330.937130.418750.786490.83550.999830.999640.964030.897790.491110.380450.947620.961560.818270.895770.910370.47120.915740.495460.785010.696450.943470.946260.754320.947190.559970.89333
bag 64 decisiontree -random end0.818620.243280.844110.761690.562390.578760.715240.991840.853440.870.717480.967670.695690.842010.619840.959450.789670.648390.855360.72060.957380.749220.841070.732710.821110.811560.817780.833650.955830.918510.9480.95870.619840.890760.633330.930960.393750.80.631250.8668510.948560.958290.971210.657780.445440.966670.962780.78750.871740.948710.442160.765330.483460.785370.728610.94760.932120.816320.970790.588270.90302
bag 32 decisiontree -random end0.820170.240080.840130.761690.566810.586730.702470.982310.840960.830.709090.969680.680620.839350.587670.950380.779890.619350.849860.72380.944260.741410.832740.733640.823750.816330.817780.832280.954350.924780.941330.952250.587670.855670.666670.917020.41250.766220.654170.861310.948290.951570.971290.617780.41590.963460.962460.773080.854180.946620.439180.761570.537750.785550.732150.938830.907880.796480.960670.58450.91278
bag 32 decisiontree end0.828460.234430.811020.761690.565490.596460.6507210.8092810.665730.961080.692760.899650.535180.990180.844570.774190.820580.70740.905460.754170.832740.738320.766950.80340.798520.803950.969720.90770.953330.990180.535180.814780.691670.91440.443750.74730.714580.871370.999660.999120.971680.97090.560.41950.971080.971470.795190.814660.94140.502480.927520.498110.785640.738060.956780.885450.830360.94270.573450.41208
bag 64 decisiontree end0.830610.23170.811580.761690.562830.632740.6331710.808970.90.658740.963090.685780.900930.535660.99130.849460.758060.824350.71880.914750.751040.819640.723360.763670.799320.801480.807840.968770.921380.945330.99130.535660.813790.708330.917660.38750.778380.731250.873210.999750.999120.972410.969950.591110.416520.975240.971210.80320.944790.512260.92840.496680.787460.74350.957230.886670.832840.949440.572240.41361
bag 32 bucket decisiontree meanmarginstree end end0.831960.234190.812520.848330.567260.716810.6273310.866240.920.679020.960520.680010.903120.557150.99180.848370.71080.915850.754170.839290.695330.767690.79320.817040.815520.96840.929910.946670.99180.550470.810470.70.914850.418750.870270.999930.999030.970590.96940.626670.428320.97160.970680.828850.916550.94240.562730.934360.467580.785190.743030.952630.907470.826120.95169
bag 64 bucket decisiontree meanmarginstree end end0.851930.233470.812070.838530.573890.742480.6184710.868480.880.69650.961660.683490.907870.557770.991740.81290.820870.71220.902730.746610.850.732710.761690.798640.805930.797590.969460.93050.9560.991740.559390.750.916630.4250.774320.73750.872620.999980.999120.972240.970350.622220.405950.973770.9710.838460.913030.944060.938430.483440.787640.750350.956780.914550.586660.92682
bag 16 bucket decisiontree meanmarginstree end end0.85710.811080.844770.703540.6282510.847040.870.69650.959370.662080.901970.542320.989550.833150.81290.826090.7090.896170.761980.816670.717760.761750.795920.80370.968450.931070.949330.989550.543590.856160.716670.905150.443750.737840.677080.865620.999730.998910.969370.967740.597780.420060.970480.97010.817310.908940.938970.548410.933040.478350.786640.952630.905450.819280.949440.580590.89075