raffaello-camoriano/enitor

Enitor provides the MATLAB implementation of several large-scale kernel methods.

enitor's Issues

Add configuration parameter for validation part in experiment class

Incremental RF provides results which are incompatible with batch RF

fix asap

Remove filter hyperparameter cross validation logic from nystromUniformIncremental

https://github.com/Ommac/Enitor/blob/master/featureMap/nystrom/nystromUniformIncremental/nystromUniformIncremental.m#L294-L298

Add "formulation" tag to nrls algorithm

formulation = 1

M = K_{mm}

formulation = 2

M = eye(m)

Use key/value parameters

Parameters must be passed in a key/value fashion inside the kernel, filter and algorithm classes and subclasses. Otherwise, confusion can be made inside the methods. Code would be much cleaner.

Make the validationPart parameter explicit and easily accessible

Add perceptron loss

L(y,f) = (-y*f)+

Add "strict" option to "horizonSharpStop" stopping rule

if strict == 1 use < or >
if strict == 0 use <= or >=

In SGD, add option to compute CV performance only for the last step, instead of for every iteration

e. g. with an 'errorStorageMode' option, set to 'epoch' (default) or to 'iterations'

Fix incremental Nystrom stopping rule criterion

Now it stops the selection of lambda instead of m.

inputParser not compatible with cell array input arguments

Refactor code to avoid cell array input arguments.

Lambda selection for t >1

The training method only considers a single number as performance measure, even if the number of outputs is larger than 1. Another approach that should be considered for implementation is to compute the performance measure for each output and lambda guess and then select the best lambda guess according to a customizable function (e. g. mean, median).

Update the function signature of rfrls and ffrls

obj.errorStorageMode in kigdesc and ksgdesc shoud be a flag, not a string.

Adjust lambda range in experiments using Gaussian kernel

Maximum lambda should be <=10e0. Larger lambdas do not make much sense.

Add time/memory speedup options to computeSqDistMat@gaussianKernel

in incrementalNkrls, choosing numNysParGuesses == 1 provides unreliable results

Tested with

map = @nystromUniformIncremental;

and Covertype dataset

"Adult" dataset: numerical columns normalization

Numerical attributes (e.g. age) are currently not normalized/scaled/standardized. These attributes may dominate on binary dummy variables corresponding to categorical attributes (e.g. nationality).

fix mapParRangeSamples limit

it should not be limited to the number of training points.

add optional stopping rule to incrementalNKRLS

Add option for additional loss functions to dataset class

use try/catch for efficiency in SIsubGD_dual_hinge_loss && SsubGD_dual_hinge_loss

Add verbosity options

add iteration timing option to incremental nystrom and incremental random features

Restore the 'retraining' option

Add support for generating smaller datasets with a subset of classes in the constructor of Cifar10

Add Comparison between maximum size KRLS on a part of the dataset and DACKRLS, incremental NKRLS and RFKRLS on the full dataset

Unify randomFeaturesGaussianIncremental and nystromUniformIncremental compute() method

Make common parent class with abstract compute() method, to avoid repetitions.

Support for multiple kernel and filter parameters

Add "next" method and cell array with kernel/filter parameter lists, in order for the algorithm classes to be able to deal with more complex model selection strategies (such as non-isotropic Gaussian kernels sigmas or gradient descent on sigma/lambda)

Add inputParser

Add inputParser to class methods in order to rationalize input parameters.

See http://www.mathworks.it/help/matlab/ref/inputparser-class.html

retraining to be fixed in nrls

Add full kernel precomputation option to kigdesc

Keep previously sampled columns in batch experiments

Keep previously sampled columns in batch experiments.

Example

loop repetitions
shuffle dataset
loop nystrom low-rank, choosing the first m columns
loop Tikhonov
end
end
end

Add rebalancing methods

Datasets such as "sido" might require rebalancing.

add minimum rank option to incremental nystrom and incremental random features

set up error evaluation step in incRF and incNys

Add cross-validation mechanism to SGD

Parameters to be CV'd:

• eta (default: (0,1])
• theta (default: [0,1])

remove the 'hasRealValues' bottleneck

Datasets to be added

• Iris: http://archive.ics.uci.edu/ml/datasets/Iris (also builtin in MATLAB)
• Abalone http://archive.ics.uci.edu/ml/datasets/Abalone
• Wine http://archive.ics.uci.edu/ml/datasets/Wi
• MNIST2M
• MNIST-binary 500k/1M/2M
• housing
• SARCOS
• Landsat
• contraceptive
• iCubWorld28

add RFKRLS to Nystrom comparison experiment

Decouple loss function from dataset

losses:

• MSE
• RMSE
• classificationError
• hingeLoss

fix nystromUniform interface

Fix Hessenberger version of Tikhonov

Fix Hessenberger version of Tikhonov regularization filter.

Currently deactivated, generating a warning.

Add support for multiple guesses numbers for different hyperparameters in kernel and filter subclasses

Add full training, testing and validation performance matrix saving options to algorithms

The 1st guess for m in incremental and batch Nystrom should never be 1

split in numMGuesses+1 parts and disregard the 1st one.

Add option for rank-k update in randomFeaturesGaussianIncremental and nystromUniformIncremental

Goal: reducing overhead of function calls (see below)

merge S-IGD filters

add option 'ordering' for samples ordering:

• fixed: ordering is drawn once and kept (no repetitions)
• reshuffle_norep: ordering is drawn at each epoch (no repetitions)
• reshuffle_yesrep: ordering is drawn at each epoch (with repetitions)

Possible bug in incrementalrfrls

results with

numRFParGuesses = 10;

and

numRFParGuesses = 1;

do not match.

Dataset: InsuranceCompanyBenchmark

Add support for hyperparameter passing via map container in kernel.compute