svm.cpp: svm_predict() forcing one class

[cjlin1]

svm.cpp: svm_predict() forcing one class

[cjlin1]

This isn't a bug. Please see the following libsvm faq.

Q: Why using svm-predict -b 0 and -b 1 gives different accuracy values?

Let's just consider two-class classification here. After probability information is obtained in training, we do not have

prob > = 0.5 if and only if decision value >= 0.

So predictions may be different with -b 0 and 1.

Nick writes:

I am using libsvm 3.20. I have a dataset which causes svm_predict() and
svm_predict_probability() to give different results. In particular, svm_predict()
classifies everything to one class, which is definitely wrong for this dataset.

You can trigger it with the command line tools as follows

wget https://github.com/kousu/statasvm/raw/master/bugs/libsvm_classification/classification_bug.svmlight

svm-train -b 1 classification_bug.svmlight FIT >/dev/null &&

svm_predict(), incorrect

svm-predict -b 0 classification_bug.svmlight FIT P
cat P

svm_predict_probability(), correct (or at least, reasonable)

svm-predict -b 1 classification_bug.svmlight FIT P
cat P

Tabulating the results, I see

svm_predict(), incorrect

Model supports probability estimates, but disabled in prediction.
Accuracy = 53.5294% (91/170) (classification)
170 1

svm_predict_probability()

Accuracy = 84.7059% (144/170) (classification)
labels 0 1 2 3
61 0
100 1
9 2

Reference dataset and full test cases are at
https://github.com/kousu/statasvm/tree/master/bugs/libsvm_classification. This
showed up when run from my Stata wrapper in that repo, but it is also in sklearn
and in your command line tools.

I hit the svm_predict() bug a week ago, but I was even more surprised to see that
despite it, you can still good answers out of libsvm by tweaking parameters. Given
the huge number of machine learning projects that are dependent on your code,
there must be a lot of subtlely incorrect predictions that no one is catching. Do
you have any idea what would cause this?

—
Reply to this email directly or view it on GitHub.*

[kousu]

I missed that in the FAQ. It is short and hard to interpret. Perhaps incorporating some of the points made by sklearn would make this corner-case clearer.

[StevenLOL]

Agreed, sklearn has the best documentation.