ctlab/itmo_fs

Feature selection library in python

ITMO_FS

Feature selection library in Python

Package information:

Install with

pip install ITMO_FS

Current available algorithms:

Supervised filters	Unsupervised filters	Wrappers	Hybrid	Embedded	Ensembles
Spearman correlation	Trace Ratio (Laplacian)	Add Del	Filter Wrapper	MOSNS	MeLiF
Pearson correlation	Multi-Cluster Feature Selection	Backward selection	IWSSr-SFLA	MOSS	Best goes first
Fit Criterion	Unsupervised Discriminative Feature Selection	Sequential Forward Selection		RFE	Best sum
F ratio		QPFS
Gini index		Hill climbing
Symmetric Uncertainty		Simulated Annealing
Fechner correlation		Recursive Elimination
Kendall correlation
Information Gain
ANOVA
Chi-squared
Relief
ReliefF
Laplacian score
Modified T-score
Mutual Information Maximization
Minimum Redundancy Maximum Relevance
Joint Mutual Information
Conditional Infomax Feature Extraction
Mutual Information Feature Selection
Conditional Mutual Info Maximization
Interaction Capping
Dynamic Change of Selected Feature
Composition of Feature Relevancy
Max-Relevance and Max-Independence
Interaction Weight
Double Input Symmetric Relevance
Fast Correlation
Statistical Inference Relief
Trace Ratio (Fisher)
Nonnegative Discriminative Feature Selection
Robust Feature Selection
Spectral Feature Selection
VDM
QPFS
MIMAGA

Documentation:

https://itmo-fs.readthedocs.io/en/latest/

itmo_fs's Issues

AttributeError: 'UnivariateFilter' object has no attribute 'run'

Hi,
I installed ITMO_FS successfully both Win10-python 3.7 and also Google Colab.

Run your code at both README.rst and also your paper (Future Internet 2020, 12, 54; doi:10.3390/fi12030054).

They gave the same error:
AttributeError: 'UnivariateFilter' object has no attribute 'run'

Any suggestion?

BUG :

Dear Develoer,

1. I got the following error when using the simple test on the documentation homepage, how can I fix it?

---

Traceback (most recent call last):

File ~\Documents\test\ITMO_FS-master\untitled0.py:13 in
sel = MOS()

TypeError: init() missing 2 required positional arguments: 'model' and 'weight_func'

---

2. The following question arises, do I need to remove the "smote=False" in the example?

---

TypeError: fit_transform() got an unexpected keyword argument 'smote'

ADD : may you add code examples fro each algorithm

may you add code examples fro each algorithm
like

Fast Correlation
Statistical Inference Relief
Trace Ratio (Fisher)
Nonnegative Discriminative Feature Selection
Robust Feature Selection
Spectral Feature Selection
VDM
QPFS
MIMAGA

VDM,QPFS and MIMAGA

I was trying the three methods (VDM,QPFS and MIMAGA) to select attributes from a csv file and I couldn't get it with any of them.
In VDM it tells me the following error:

Index_Error: arrays used as indices must be of integer (or boolean) type.

Can these methods be applied with floating data?

In the other methods I have not known how to apply them to a csv file, can you help me to do it?

Thank you very much.

BUG : Feature selection example from documentation not working

Describe the bug
Example "1.2. Problem statement regarding data sets with redundant features" from https://itmo-fs.readthedocs.io/en/latest/introduction.html#problem-statement-regarding-data-sets-with-redundant-features is not working for the both pypi and github versions of itmo_fs.

To Reproduce
Steps to reproduce the behaviour:

1. Install itmo_fs as pip install -U ITMO_FS
2. Run example from https://itmo-fs.readthedocs.io/en/latest/introduction.html#problem-statement-regarding-data-sets-with-redundant-features
3. Unknown parameter error (fig 1)
4. (if parameter removed) - attribute error (fig 2)
5. (if version from github master branch installed) - error TypeError (fig 3) for MOS

Expected behaviour
Two error metrics calculated and printed

Screenshots
fig 1:

fig 2:

fig 3

Desktop (please complete the following information):

• OS: Windows 10

Additional context

I wanted to try using itmo_fs as a building block for pipelines in the FEDOT framework. However, the example from the documentation is not working for me.

Different results

Dear Develoer,
I have been interested in feature selection for a while. GINI index results in ITMO_FS are quite different between 0.3.3 and 0.3 versions when using the same data set.
The values obtained according to the two versions are below

Feature-  Gini values at 0.33  -- 0.30
 'Q-1': 0.0171165   ---   0.017209
 'Q-2': 0.0302192   ---   0.030236
 'Q-3': 0.038750   ---   0.038718
 'MeanQ': 0.0224271   ---   0.040471  **
 'MTI': 0.0418138   ---   0.071110  ** 
 'Prep': 0.071110   ---   0.018591  **
Humid': 0.002983   ---   0.047469  ****
 'Temp': 0.051686   ---   0.024229  **
 'S': 0.035492569   ---   0.973375  ****
 'T': 0.009388788   ---   0.015001  ** 
'R': 0.032898059   ---   0.986235  ****

Is there a source explaining the difference between the two? Which Gini value should I take as the limit in the final version? How should I decide which feature to use or not in the next step?
I would be glad if you could help
Regards,

TypeError

First of all, congratulations on the work!
I installed your lib and I'm using Python 3.8 and Linux Mint 20.1.
I don't know if the lib supports both, but I can recognize imports.
I'm trying to use the documentation example, but the following error appears.

TypeError: fit_transform() got an unexpected keyword argument 'smote'

BUG : Spearman correlation error generated from missing otypes in np.vectorize

Description:
Python version: 3.8
numpy version: 1.22.3
ITMO_FS version: 0.3.3

When y.shape and x.shape are different, spearman_correlation fails because of missing "otypes" in np.vectorise.

This happens from np.vectorise - numpy/lib/function_base.py - _get_ufunc_and_otypes() - line 2365
otypes = ''.join([asarray(outputs[_k]).dtype.char for _k in range(nout)])

To Reproduce
Steps to reproduce the behaviour:

1. I have X - a numpy array shape (1774, 601) with float and nan values.
2. I have y - a numpy array shape (1774, ) with int values 0 and 1 only.
3. Just run: spearman_corr(X, y)

Expected behaviour
Returns the scores and doesn't fail.

Desktop (please complete the following information):

• OS: MacOS 13.2.1 (22D68)

Proposed fix:
I solved the issue by simply adding
otypes=[X.dtype]

So line 567 from filters/univariate/measures.py is now:
ranks_X[:, i] = np.vectorize(d.get, otypes=[X.dtype])(X[:, i])

Installation problem in ITMO_FS

Dear author,
While trying to install ITMO_FS inside python ver. 3.7, the terminal is throwing error:

Installing collected packages: Cython, quadprog, qpsolvers, numpy, joblib, scipy, scikit-learn, imbalanced-learn, imblea
rn, ITMO-FS
Running setup.py install for quadprog ... error
Complete output from command "C:\Users\SABUR\PycharmProjects\PhD Proj\venv\Scripts\python.exe" -u -c "import setupto
ols, tokenize;file='C:\Users\SABUR\AppData\Local\Temp\pip-install-ky9b_d4_\quadprog\setup.py';f=getattr(toke
nize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" insta
ll --record C:\Users\SABUR\AppData\Local\Temp\pip-record-khvrw0rg\install-record.txt --single-version-externally-managed
--compile --install-headers "C:\Users\SABUR\PycharmProjects\PhD Proj\venv\include\site\python3.7\quadprog":
running install
running build
running build_ext
skipping 'quadprog\quadprog.cpp' Cython extension (up-to-date)
building 'quadprog' extension
error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": http://landinghub.visu
alstudio.com/visual-cpp-build-tools

----------------------------------------

Command ""C:\Users\SABUR\PycharmProjects\PhD Proj\venv\Scripts\python.exe" -u -c "import setuptools, tokenize;file='
C:\Users\SABUR\AppData\Local\Temp\pip-install-ky9b_d4_\quadprog\setup.py';f=getattr(tokenize, 'open', open)(fi
le);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record C:\Users\SAB
UR\AppData\Local\Temp\pip-record-khvrw0rg\install-record.txt --single-version-externally-managed --compile --install-hea
ders "C:\Users\SABUR\PycharmProjects\PhD Proj\venv\include\site\python3.7\quadprog"" failed with error code 1 in C:\User
s\SABUR\AppData\Local\Temp\pip-install-ky9b_d4_\quadprog\

Though i have Microsoft Visual C++ 2015 Redistributable(x86) 14.0.23026 and Microsoft Visual Studio Installer both installed in my system. I am using windows 7, 32 bit OS.

Thank You
Emon

Allow installing with conda

Hey! Thanks a lot for sharing your work.

Would you please consider making this library available as a conda package?