nyanp/optiver-realized-volatility-prediction

optiver-realized-volatility-prediction's Issues

bug(KNN FIT): Neighbors metric_params expecting 'VI', not 'V'

Hi,

I'm getting this from lightgbm-clustering-model.ipynb

Traceback (most recent call last):
  File "/examples/nyan-optiver/lightgbm-clustering-model.py", line 340, in <module>
    k_neighbors_p1 = Neighbors(pivot, 2, metric='mahalanobis', metric_params={'V':np.cov(pivot.values.T)})
  File "/examples/nyan-optiver/lightgbm-clustering-model.py", line 325, in __init__
    self.distances, self.neighbors = nn.kneighbors(pivot, return_distance=True)
  File "/opt/bitnami/python/lib/python3.10/site-packages/sklearn/neighbors/_base.py", line 752, in kneighbors
    chunked_results = list(
  File "/opt/bitnami/python/lib/python3.10/site-packages/sklearn/metrics/pairwise.py", line 1709, in pairwise_distances_chunked
    params = _precompute_metric_params(X, Y, metric=metric, **kwds)
  File "/opt/bitnami/python/lib/python3.10/site-packages/sklearn/metrics/pairwise.py", line 1551, in _precompute_metric_params
    raise ValueError(
ValueError: The 'VI' parameter is required for the mahalanobis metric when Y is passed.

Comes from your KNN Fit notebook:

k_neighbors_p1 = Neighbors(pivot, 2, metric='mahalanobis', metric_params={'V':np.cov(pivot.values.T)})
...
k_neighbors_size = Neighbors(pivot, 2, metric='mahalanobis', metric_params={'V':np.cov(pivot.values.T)})

Changing these from V -> VI per the error message resolves and runs. Is this supposed to be VI?

python --version
Python 3.10.4
...

pip list | grep 'sci\|sklearn'
scikit-learn             1.0.2
scipy                    1.8.0
sklearn                  0.0

Happy to submit a PR with the change, but wanted to ask first if this is correct.

Also, I've done some refactoring and Dockerizing for our purposes. If you like, I can submit a PR back with a Dockerfile and a docker compose yaml file to make this easier for people to bring up on their own machines.