Clarification needed - How is the factor order determined?

[educationaltestingservice]

Clarification needed - How is the factor order determined?

[Fireblend]

Thanks for the thorough answer! It was of much help. I would also appreciate it if you added an option to order the factors post-rotation.

Thanks again!

[jbiggsets]

Hi @Fireblend -

Sorry for the delayed response. I had to look into what SAS is doing.

The short answer to your question is that, when SAS performs the varimax rotation on the factor loading matrix, it is re-ordering the columns according to the amount of variance accounted for by each factor after rotation. The factor_analyzer program is not doing that. It is maintaining the original order of the unrotated factor loading matrix after the rotation is performed.

If you compare the unrotated factor loading matrices in Python vs SAS, you'll see that the ordering of the factors is the same.

Python - Unrotated 3-Factor Loading Matrix, using MINRES (ULS)

df = pd.read_csv('../boston_housing.csv')

fa = FactorAnalyzer()
fa.analyze(df, rotation=None, method='minres')
fa.loadings

Variable	Factor1	Factor2	Factor3
crim	0.578795	-0.041037	0.299938
zn	-0.590944	-0.125879	0.347282
indus	0.830836	0.131870	-0.081245
chas	-0.010333	0.255487	-0.082585
nox	0.822241	0.284851	-0.108690
rm	-0.494985	0.518490	0.251530
age	0.740942	0.272030	-0.264950
dis	-0.764482	-0.458068	0.267981
rad	0.786369	0.091457	0.551421
tax	0.835811	0.048587	0.444062
ptratio	0.485985	-0.267325	0.076124
b	-0.455904	0.013064	-0.198389
lstat	0.777591	-0.282147	-0.160665
medv	-0.685469	0.647925	0.137074

SAS - Unrotated 3-Factor Loading Matrix, using MINRES (ULS)

PROC FACTOR METHOD=ULS
	DATA=WORK.BOSTON_HOUSING
	PRIORS=SMC
	NFACTORS=3
	ROTATE=NONE;
RUN;

var	Factor1	Factor2	Factor3
crim	0.57880	-0.04104	0.29994
zn	-0.59094	-0.12588	0.34728
indus	0.83084	0.13187	-0.08125
chas	-0.01033	0.25549	-0.08259
nox	0.82224	0.28485	-0.10869
rm	-0.49499	0.51850	0.25153
age	0.74094	0.27203	-0.26495
dis	-0.76448	-0.45806	0.26798
rad	0.78637	0.09146	0.55141
tax	0.83581	0.04859	0.44406
ptratio	0.48599	-0.26733	0.07613
b	-0.45590	0.01306	-0.19839
lstat	0.77759	-0.28215	-0.16066
medv	-0.68547	0.64792	0.13707

It's only after the rotation is performed that SAS re-orders the columns.

If this reordering is of interest, I don't think it would be particularly difficult to add. Let me know, and I'll file an issue. We can just make this an optional argument in the analyze() method. (Something like reorder_factors_after_rotation=True.)

One other thing to note is that you can select the ULS method in SAS, and you should get roughly the same output as method='minres' in the FactorAnalyzer package.

Hope this helps!