presnick/surveyequivalence

surveyequivalence's Issues

PrecisionScore.expected_score_anonymous_raters failed in test

this function only support label 'pos' and 'neg'

code is as follow:

    for (row, pred) in zip([row for _, row in W.iterrows()], classifier_predictions):
        # count frequency of each value
        counts = row.dropna().value_counts()
        freqs = counts/np.sum(counts)
        if len(counts) == 0:
            # no non-null labels for this item
            continue
        elif pred.value != "pos":
            # no impact on precision if classifier didn't predict positive
            continue
        tot += freqs['pos']
        ct += 1