Unexpected Meka Evaluation Result

[Mali-DS]

Unexpected Meka Evaluation Result

[fracpete]

From a quick glance, you seem to treat the data like you would for Weka. However, Meka works a bit different. See the following examples:

• preparing the class attributes - MekaClassAttributes filter
• train/predict - MLUtils.prepareData(Instances) method

Final remark, you only seem to have a single class attribute...

[Mali-DS]

Thanks for your answer, you mentioned good points, I changed my code and used Meka ways, now code is as under:
try {
ConverterUtils.DataSource dataSource = new ConverterUtils.DataSource(FILE_PATH); // original dataset
Instances preparedDataSet = dataSource.getDataSet();

        CRUpdateable classifier = new CRUpdateable();
        RandomForest randomForest = createRandomForest(1);  // random forest is not updatable classifier
        classifier.setClassifier(randomForest);

        Instances  trainingInstances = new Instances(dataSource.getStructure()); 
        Instances testInstances = new Instances(dataSource.getStructure());
        int countTestInstances = 0;
        int countTrainInstances = 0;
        boolean firstTrain = true;
        boolean benchTest = true;
        for(int row = 123; row < 5021; row++) {
                Instance trainingInstance = preparedDataSet.instance(row);
                trainingInstances.add(trainingInstance); // collect instances to use as training
                countTrainInstances++;
                if (firstTrain && countTrainInstances%100 == 0 ) { 
                    trainingInstances = PrepareClassAttributes(trainingInstances,"1,2,3,4,5,6,7");
                    firstTrain = false;
                    classifier.buildClassifier(trainingInstances);
                }
                if(!firstTrain){
                    benchTest = true;
                    classifier.updateClassifier(trainingInstance);
                    for(int j=row+1;j<row+101;j++){
                        if(benchTest && countTestInstances != 100) { 
                            Instance testInstance = preparedDataSet.instance(j);
                            testInstances.add(testInstance);
                            countTestInstances++;
                            if (countTestInstances % 100 == 0) {
                                testInstances = PrepareClassAttributes(testInstances,"1,2,3,4,5,6,7");
                                System.out.println("Evaluate CRUpdateable classifier on ");
                                String top = "PCut1"; 
                                String vop = "3";  
                                Result result = Evaluation.evaluateModel(classifier, trainingInstances , testInstances, top, vop);
                                System.out.println("Evaluation Info: " + result.toString());
                                countTestInstances = 0;
                                benchTest = false;
                                testInstances.delete();
                            }
                        }
                    }
                }
        }

    } catch (Exception e) {
        e.printStackTrace();
    }

but yet the Accuracy is zero, and the stats results are strange:

N(test) 100
L 7
Hamming score 0
Exact match 0
Hamming loss 1
ZeroOne loss 1
Levenshtein distance 1
Label indices [ 0 1 2 3 4 5 6 ]
Accuracy (per label) [ 0.000 0.000 0.000 0.000 0.000 0.000 0.000 ]

[jmread]

Actually the stats results make sense given that there are 0 correct predictions. Without being familiar with your data, it is difficult to know if this is 'strange' or not. Have you tried getting results using a simple test in the GUI first? Or to print out the prediction for each instance?