zoj613/pyloras

Experimental implementations of several (over/under)-sampling techniques not yet available in the imbalanced-learn library.

pyloras's Issues

MAINT: Remove redundant copying of input data during resampling.

input data is unnecessarily copied twice in the _fit_resample method. First here:

pyloras/pyloras/_loras.py

Lines 144 to 145 in b896367

	X_res = X.copy()
	y_res = y.copy()

and then again here:

pyloras/pyloras/_loras.py

Lines 161 to 162 in b896367

	X_res = [X_res.copy()]
	y_res = [y_res.copy()]

It only needs to be done once. The second copy is not needed.

ENH: Allow passing in a pre-trained t-SNE embedding

Given that tSNE is the bottleneck. To speed things up we can allow a pre-trained TSNE embedding object (one that implements the transform method) to be passed into the constructor or the transform method. Either

loras = LORAS(..., pretrained_tsne=None)

or

loras.transform(X, y, pretrained_tsne=None)

DOC: Add visualization comparing the synthetic samples to those generated by SMOTE.

Something like this: https://imbalanced-learn.org/stable/over_sampling.html#a-practical-guide

ImportError: cannot import name delayed

When I use LORAS to resample the data, I receive an error stating that it cannot import name delayed even when I use pip to install delayed.

ENH: Implement the AMUS over-sampling algorithm.

Another new clustering-based oversampling algorithm that uses some Gaussian Mixture models can be found in [1]. would be interesting to see how it compared to ProWRAS.

References

[1] https://arxiv.org/pdf/2103.13823.pdf

DEV: Pin numpy version to >=1.17.0 and support `np.random.Generator` interface.

This is per NEP29 recommendations. It is also going to allow support for the new np.random.Generator interface which has faster random number generation and has more functionality that the old np.random.RandomState used by sklearn's check_random_state function.

ENH: Implement the `ProWRAS` oversampler.

This paper [1] presents ProWRAS, which is supposedly an improvement over LoRAS that also incorperates elements of the Proximity Weighted Synthetic oversampling (ProWSyn) algorithm.

References

[1] https://arxiv.org/pdf/2107.07349.pdf

MAINT/DOC: Add type information to arguments

Add some typing information to make arguments a little easier to read (maybe). I tend to prefer stub files (e.g. see here) over in-place typing since there is less distracting lines of code. However it means the file count doubles because every python source is accompanied by a .pyi file.

MAINT: Use public `bit_generator` instead of the private `_bit_generator`.

All occurrences of the ._bit_generator attribute in np.random.Generator should be changed to bit_generator since numpy made this attribute public since 1.17. See: scikit-learn/scikit-learn#20669 (comment)

ENH: Use a faster t-SNE implementation

Currently, scikit-learn's implementation is used. One of the following would likely perform better in terms of runtime:

• https://github.com/DmitryUlyanov/Multicore-TSNE
• https://github.com/pavlin-policar/openTSNE
• https://github.com/lvdmaaten/bhtsne

The current lines that would need to be changed are:

pyloras/pyloras/_loras.py

Lines 112 to 115 in b896367

	self.tsne_ = TSNE(n_components=2, n_jobs=self.n_jobs, random_state=rng)
	if self.embedding_params is not None:
	self.tsne_.set_params(**self.embedding_params)

pyloras/pyloras/_loras.py

Line 149 in b896367

X_embedded = self.tsne_.fit_transform(X_res)

ENH: Improve runtime of the re-sampling

Currently, the runtime is less than ideal. I think its possible to improve the speed by cythonizing the loop in

pyloras/pyloras/_loras.py

Lines 163 to 177 in b896367

	for class_sample, n_samples in self.sampling_strategy_.items():
	data_indices = np.flatnonzero(y == class_sample)
	# number of synthetic samples per neighborhood group
	n_gen = n_samples // data_indices.shape[0]
	neighborhood_groups = self.nn_.kneighbors(
	X_embedded[data_indices],
	return_distance=False
	)
	with parallel_backend('loky', n_jobs=self.n_jobs):
	samples = Parallel()(
	delayed(func)(X[i], class_sample, n_gen)
	for i in neighborhood_groups
	)
	X_res.extend(samples)
	y_res.extend([class_sample] * n_gen * neighborhood_groups.shape[0])

and maybe also rewriting

pyloras/pyloras/_loras.py

Line 184 in b896367

def _make_samples(

in Cython.

I will need to first profile and determine where the most time is spent.

ENH: Support sparse input data

Allow X to be a sparse array data type.

ENH: Add OTOS oversampling algorithm.

[1] Outlines an oversampling technique using Optimal Transport. Seems interesting enough to try out.

References

[1] https://www.aaai.org/ojs/index.php/AAAI/article/view/4503/4381

MAINT: Make copy of `manifold_learner` before using it, if passed to the constructor.

Currently no copy is made, and the passed learner is modified in-place.

pyloras/pyloras/_loras.py

Lines 129 to 131 in d39c5b5

	if self.manifold_learner:
	self._check_2d_manifold_learner()
	self.manifold_learner_ = self.manifold_learner

pyloras/pyloras/_loras.py

Lines 134 to 139 in d39c5b5

	if self.manifold_learner_params is not None:
	self.manifold_learner_.set_params(**self.manifold_learner_params)
	try:
	self.manifold_learner_.set_params(random_state=rng)
	except ValueError:
	pass

A copy should be made using sklearn's clone function so as to preserve the passed object.

ENH: Add MDO/AMDO oversampling algorithms

AMDO is outlined in [1] and is optimized for multi-class imbalanced datasets. The MDO algorithm seems a bit easier to implement so maybe that could be attempted first.

References

[1] https://www.researchgate.net/publication/320343439_AMDO_an_Over-Sampling_Technique_for_Multi-Class_Imbalanced_Problems