pharo-ai/metrics

Implementation of Machine Learning metrics for Pharo

Description

This package is part of the Pharo AI project: It contains implementations, tests and documentation of different metrics for Machine Learning models. The evaluation metrics allows to assess how a trained Machine Learning model has performed.

For more information please refer to the wiki: https://github.com/pharo-ai/wiki/blob/master/wiki/DataExploration/Metrics.md

There is explained with more detail the available metrics and how to use them.

How to install it

EpMonitor disableDuring: [ 
   Metacello new
      baseline: 'AIMetrics';
      repository: 'github://pharo-ai/metrics';
      load ]

How to depend on it

spec 
   baseline: 'AIMetrics' 
   with: [ spec repository: 'github://pharo-ai/metrics' ].

Types of metrics:

• Clustering metrics
• Regression metrics
• Classification metrics

Example: Mean Squared Error (AIMeanSquaredError)

| yTrue yPredicted metric |
metric := AIMeanSquaredError new.
yTrue := #( 3 -0.5 2 7 ).
yPredicted := #( 2.5 0.0 2 8 ).
metric computeForActual: yTrue predicted: yPredicted "0.375"

Example: Accuracy Score (AIAccuracyScore)

| yTrue yPredicted metric |
metric := AIAccuracyScore new.
yTrue := #( 0 1 2 3 ).
yPredicted := #( 0 2 1 3 ).
metric computeForActual: yTrue predicted: yPredicted "0.5"

metrics's Issues

#'as yet unclassified'

Some class methods are not classified correctly

Should not we move testTake to Polymath?

Add Explained variance score to regression metrics

See: https://scikit-learn.org/stable/modules/model_evaluation.html#explained-variance-score

Replace prefix ML with AI

Having a dependency on the full polymath looks too large to me.

Why Array defines sumMatrix

sumMatrix
"Answer an with the sum of all the receiver's vectors"

^ self sum sum

should not be in PMVector?

Add precision, recall, and F-measurements

https://scikit-learn.org/stable/modules/model_evaluation.html#binary-classification

https://scikit-learn.org/stable/modules/generated/sklearn.metrics.precision_score.html#sklearn.metrics.precision_score

Can you explain how the metric is computed

Because in clustering we have

• population (dimension)
• distance
• radius

so how is the randindex computed?
Class comments are our friends.

initializeWithClusterA: firstCluster clusterB: secondCluster assertion logic

initializeWithClusterA: firstCluster clusterB: secondCluster
	"Private - Initialize the receiver"

	super initializeWithClusterA: firstCluster clusterB: secondCluster.
	self assert: [ firstCluster size = secondCluster size ].

why not

initializeWithClusterA: firstCluster clusterB: secondCluster
	"Private - Initialize the receiver"

        self assert: [ firstCluster size = secondCluster size ].
	super initializeWithClusterA: firstCluster clusterB: secondCluster.

Better readme please

Can you check the template of the Pharo-Containers?

Add Poisson, Gamma, Tweedie metrics to regression metrics

• Mean Poisson, Gamma, and Tweedie deviances

See: https://scikit-learn.org/stable/modules/model_evaluation.html#mean-poisson-gamma-and-tweedie-deviances

buildMatrix

buildMatrix: aCollection
	"Answer a <Collection> of points that are in the same vs different groups.
	Different groups are 1, same groups are 0"

	^ aCollection collect: [ :index | 
		(index - aCollection) abs 
			collect: [ :each | each min: 1 ] ]

I do not get this code what means index - aCollection ? this code totally confuses me.
So we should add a comment about this implementation