ira20022/102017029_parameter_optimization_of_svm

SVM

What is a Support Vector Machine(SVM)? A Support Vector Machine is a supervised machine learning algorithm which can be used for both classification and regression problems. It follows a technique called the kernel trick to transform the data and based on these transformations, it finds an optimal boundary between the possible outputs.

In simple words, it does some extremely complex data transformations to figure out how to separate the data based on the labels or outputs defined.We will be looking only at the SVM classification algorithm in this article.

PARAMETER OPTIMIZATION

A fancy name for training: the selection of parameter values, which are optimal in some desired sense (eg. minimize an objective function you choose over a dataset you choose). The parameters are the weights and biases of the network.

DATASET

Data have been normalized by using the Z-normalization method and divided into two data sets: a training set containing 10437 samples, and a test set containing the 10437 samples.

CLASS DISTRIBUTION (training set) A: 4286
B: 5
C: 103
D: 352
E: 1095
F: 1961
G: 446
H: 519
I: 831
W: 44
X: 522
Y: 266

Attribute Information:

F1: intercolumnar distance
F2: upper margin
F3: lower margin
F4: exploitation
F5: row number
F6: modular ratio
F7: interlinear spacing
F8: weight
F9: peak number
F10: modular ratio/ interlinear spacing
Class: A, B, C, D, E, F, G, H, I, W, X, Y

Methodlogy

The dataset is split into training and testing set for 10 times and the following SVC classifier hyperparameter are selected for best accuracy:

Kernel - Different kernel functions to model non-linear relationships between the input variables and the output variableSelected from RBF, Polynomial, Linear and Sigmoid
C (Regularisation parameter) - It is used to set the tolerance of the model to allow the misclassification of data points in order to achieve lower generalization error. Higher the value of C, lesser is the tolerance and what is trained is a maximum-margin classifier. Smaller the value of C, larger is the tolerance of misclassificationRandom integer values from 1 to 6
Gamma (Kernel coefficient) - The gamma parameter defines how far the influence of a single training example reaches, with low values meaning ‘far’ and high values meaning ‘close’. The lower values of gamma result in models with lower accuracy and the same as the higher values of gamma. It is the intermediate values of gamma which gives a model with good decision boundaries.Random integer values from -1 to 6. If the value is less than 1, then gamma is randomly set as auto or scale. It is used only by rbf, poly and sigmoid kernel.
Degree - Random integer from 1 to 7. It is only used by poly kernel and represent the degree of polynomial kernel function.

OBSERVATIONS

Sample	Kernel	c	gamma	degree	Accuracy
0	rbf	6	auto	5	0.725791
1	rbf	2	4	4	0.708853
2	rbf	4	1	6	0.770214
3	rbf	5	1	1	0.771492
4	rbf	3	scale	4	0.709492
5	rbf	3	3	5	0.706296
6	rbf	1	4	7	0.639182
7	rbf	4	scale	1	0.736977
8	rbf	5	2	6	0.735059
9	rbf	4	2	3	0.744966

GRAPH

RESULT

KERNAL: RBF
C:5
GAMMA:1
DEGREE :1
Accuracy : 0.771492