Getting an error when trying to fit factor loadings to original data

[educationaltestingservice]

Getting an error when trying to fit factor loadings to original data

[jbiggsets]

I like the option of giving the users the choice. Is that a lot of work?

No, I think it should be pretty easy to implement. I can work on it with this next release, if we're aiming for mid-April.

[jbiggsets]

Thanks for submitting this issue.

The way we're calculating the principal factors currently is using the randomized_svd() function from scikit-learn. This is a lot more efficient and resolves some of the intrinsic sign indeterminacy problems with SVD. However, it has the drawback that the number of factors is limited to K, where K = min(M, N). In your case, the number of rows is less than the number of columns, so it's constraining K to the number of rows.

An alternative is to use scipy's svd() function with full_matrix=True (the default). The drawback here that it's less efficient with larger data sets and, at least as far as I understand, does not have an easy way to resolve the sign indeterminacy the way randomized_svd() does when flip_sign=True. (If I'm wrong about that or you're aware of a work-around here, please let me know!)

Assuming I'm not missing anything, one way to resolve this might just be to implement both options and give users the ability to decide? @desilinguist, if you have thoughts, let me know.

[jbiggsets]

The more I look into this, the more I think that we shouldn't actually permit this use case. I still think we can allow users to choose between randomized_svd() and scipy.linalg.svd(), but scikit-learn's PCA() estimator also prevents users from trying to do PCA when the number of components is set higher than min(n_samples, n_features).

import numpy as np
from sklearn.decomposition import PCA

X = np.random.randn(5000).reshape(20, 250)

n = 21
for solver in ['full', 'randomized', 'arpack']:
    try:
        pca = PCA(n_components=n, svd_solver=solver)
        pca.fit(X)
    except ValueError as error:
        print(error); pass

Output:

n_components=21 must be between 0 and min(n_samples, n_features)=20 with svd_solver='full'
n_components=21 must be between 1 and min(n_samples, n_features)=20 with svd_solver='randomized'
n_components=21 must be between 1 and min(n_samples, n_features)=20 with svd_solver='arpack'

[jbiggsets]

Yeah, let's follow their lead and do the same then?

Yeah, I think that makes the most sense.

[desilinguist]

I like the option of giving the users the choice. Is that a lot of work?

[desilinguist]

Fantastic! Let's go for it.

[desilinguist]

Yeah, let's follow their lead and do the same then?