y-ohr-n/optgbm

Optuna + LightGBM = OptGBM

optgbm's Issues

Record training and validation scores for each fold

test_fit_twice_without_study raises AssertionError

tmp_path = PosixPath('/tmp/pytest-of-root/pytest-6/test_fit_twice_without_study__0'), n_jobs = -1

    @pytest.mark.parametrize("n_jobs", [-1, 1])
    def test_fit_twice_without_study(tmp_path: pathlib.Path, n_jobs: int) -> None:
        X, y = load_breast_cancer(return_X_y=True)
    
        clf = OGBMClassifier(
            n_estimators=n_estimators,
            n_jobs=n_jobs,
            n_trials=n_trials,
            random_state=random_state,
            train_dir=tmp_path,
        )
    
        clf.fit(X, y)
    
        df = clf.study_.trials_dataframe()
        values = df["value"]
    
        clf = OGBMClassifier(
            bagging_fraction=1.0,
            bagging_freq=0,
            feature_fraction=1.0,
            lambda_l1=0.0,
            lambda_l2=0.0,
            min_data_in_leaf=20,
            n_estimators=n_estimators,
            n_jobs=n_jobs,
            n_trials=n_trials,
            random_state=random_state,
            train_dir=tmp_path,
        )
    
        clf.fit(X, y)
    
        df = clf.study_.trials_dataframe()
    
>       np.testing.assert_array_equal(values, df["value"])
E       AssertionError: 
E       Arrays are not equal
E       
E       Mismatched elements: 1 / 5 (20%)
E       Max absolute difference: 5.55111512e-17
E       Max relative difference: 1.84773566e-16
E        x: array([0.690443, 0.315822, 0.300428, 0.690443, 0.308651])
E        y: array([0.690443, 0.315822, 0.300428, 0.690443, 0.308651])

tests/test_sklearn.py:289: AssertionError

AttributeError: 'Booster' object has no attribute 'pandas_categorical'

Versions

LightGBM: 2.2.1

Implement Profiler

Add LGBMRanker

Allow users to set groups

Create documentation using Sphinx

FutureWarning: Passing attributes to check_is_fitted is deprecated and will be removed in 0.23. The attributes argument is ignored.

Versions

scikit-learn: 0.22

Control the necessity of weights with a parameter

AttributeError: module 'lightgbm.sklearn' has no attribute '_EvalFunctionWrapper'

Versions

• LightGBM: 2.2.3

Sort features based on feature importances

Allow users to set silent

Allow eval_metric to be a list of strings

Test with boosting_type other than "gbdt"

• dart
• gbdt
• goss
• rf

Extend search space

• classification
  • is_unbalance
  • scale_pos_weight
• regression
  • reg_sqrt
• both
  • boosting_type
  • cat_smooth
  • extra_trees
  • learning_rate
  • min_child_weight
  • min_data_per_group

Create "is_workingday" feature

Add optimize_kws as a fit parameter

Ignore predict parameters like pred_leaf

Support Python 3.5

Define the lower limit of the version for each package

AttributeError: module 'lightgbm.engine' has no attribute '_CVBooster'

Versions

LightGBM: 2.2.1

Implement CUI using Click

• Add predict_proba with mlflow models
• Generate a default config file
• Log parameters with mlflow tracking
• Manage an environment with mlflow projects
• Perform user-defined preprocessing
• Validate a recipe schema

Implement the model that searches catboost hyperparameters

Implement _VotingBooster.dump_model so that lgb.plot_tree can be used

Continuously Deploy the package to PyPI with CircleCI

AttributeError: folds should be a generator or iterator of (train_idx, test_idx)

Versions

lightgbm: 2.2.0

Implement a logging decorator

Set _evals_result so that lgb.plot_metric can be used

_evals_result: Dict[str, Dict[str, List[float]] = {
    "cv_agg": {eval_name: eval_hist[f"{eval_name}-mean"]}
}

_pickle.PicklingError: Could not pickle the task to send it to the workers

feature_importances_ is not working when n_jobs=-1.

Test reproducibility

Implement Selector with null importances

Test with MySQL

Test OGBMClassifier using check_estimator

Compute early_stopping_rounds automatically

• min(int(0.05 * n_estimators), 50)
• 10.0 / learning_rate

Implement Merger

Allow users to set metric

Decay learning rates

Is the use of arithmetic mean appropriate for probabilities?

Test with a version of Python other than 3.6

Allow objective to be a callable

Add plot_feature_importances to _BaseOGBMModel

• booster.feature_importance
• booster.feature_name
• booster.predict

Handle alias parameters

• feature_fraction
  • sub_feature
  • colsample_bytree
• max_depth
• num_leaves
  • num_leaf
  • max_leaves
  • max_leaf
• min_data_in_leaf
  • min_data_per_leaf
  • min_data
  • min_child_samples
• lambda_l1
  • reg_alpha
• lambda_l2
  • reg_lambda
  • lambda
• boosting
  • boosting_type
  • boost
• bagging_fraction
  • sub_row
  • subsample
  • bagging
• bagging_freq
  • subsample_freq
• seed
  • random_seed
  • random_state
• verbosity
  • verbose
• num_class
  • num_classes
• objective
  • objective_typ
  • app
  • application
• metric
  • metrics
  • metric_types
• etc.

Documentation Help?

Hello - I'm a big fan of this package. It quickly matches the performance I get from much more complicated tuning approaches. However, I was wondering if it would be possible to improve the documentation somewhat? Specifically, I'm interested in seeing how I can see the final parameters selected by the model, and how early stopping is handled? Any other user exposed parameters would also be useful to see documented.

Thanks!

test_fit_with_empty_param_distributions raises AssertionError

tmp_path = PosixPath('/tmp/pytest-of-root/pytest-0/test_fit_with_empty_param_dist0')

    def test_fit_with_empty_param_distributions(tmp_path: pathlib.Path) -> None:
        X, y = load_breast_cancer(return_X_y=True)
    
        clf = OGBMClassifier(
            colsample_bytree=0.1,
            n_estimators=n_estimators,
            n_trials=n_trials,
            param_distributions={},
            train_dir=tmp_path,
        )
    
        clf.fit(X, y)
    
        df = clf.study_.trials_dataframe()
        values = df["value"]
    
>       assert values.nunique() == 1
E       assert 2 == 1
E        +  where 2 = <bound method IndexOpsMixin.nunique of 0    0.302962\n1    0.302962\n2    0.302962\n3    0.302962\n4    0.302962\nName: value, dtype: float64>()
E        +    where <bound method IndexOpsMixin.nunique of 0    0.302962\n1    0.302962\n2    0.302962\n3    0.302962\n4    0.302962\nName: value, dtype: float64> = 0    0.302962\n1    0.302962\n2    0.302962\n3    0.302962\n4    0.302962\nName: value, dtype: float64.nunique

tests/test_sklearn.py:182: AssertionError

Enable random seed averaging

Add unsupported objectives to OBJECTIVE2METRIC

• lambdarank
• rank_xendcg
• xendcg
• xe_ndcg
• xe_ndcg_mart
• xendcg_mart
• cross_entropy
• xentropy
• cross_entropy_lambda
• xentlambda

LightGBMError: Cannot construct Dataset since there are no useful features

lightgbm.basic.LightGBMError: Cannot construct Dataset since there are no useful features.
It should be at least two unique rows.
If the num_row (num_data) is small, you can set min_data=1 and min_data_in_bin=1 to fix this.
Otherwise, please make sure you are using the right dataset

Versions

LightGBM: 2.2.1

Add make_eval_metric

Perform probability calibration

Ignore fit parameters like eval_set