Sparse PCA

[sjaustirni]

Sparse PCA

[bytesnake]

awesome 🎉 the current implementation of elasticnet only supports a single target, so you have to solve for every beta separately with Y=X alpha. Extension to multinomial regression can be found here https://statweb.stanford.edu/~jhf/ftp/noah.pdf

[bytesnake]

all of your items are hyperparameters, they are fixed prior to the training. The model parameter would be the eigen-decomposition. https://machinelearningmastery.com/difference-between-a-parameter-and-a-hyperparameter/
It would be awesome if you can re-use parts of the existing PCA infrastructure and fallback to the existing implementation if l1_penalty and l2_penalty are both zero. You can also use the elasticnet implementation in linfa-elasticnet.

[sjaustirni]

Hey @bytesnake thanks for your answer.

I was indeed confused about hyperparameters vs parameters, thank you for clearing it up :D

I want to, of course, use the existing PCA and ElasticNet implementation that we already have in linfa, especially since the linked Sparse PCA algorithm starts with an output from the ordinary PCA:

1. Let A start at V[,1:k], the loadings of the first k ordinary principal components.

I was wondering though whether the sparse PCA implementation should live in pca.rs under the sparse flag or whether SparsePCA should be a general wrapper around the existing PCA in its own class. I lean towards the latter, but I wanted to hear another opinion on this matter :)

Edit: I should figure that out from the penalty arguments, I did not get that first time around, but now I do 💪 :D