cjlin1 / liblinear Goto Github PK
View Code? Open in Web Editor NEWLIBLINEAR -- A Library for Large Linear Classification
Home Page: https://www.csie.ntu.edu.tw/~cjlin/liblinear/
License: BSD 3-Clause "New" or "Revised" License
LIBLINEAR -- A Library for Large Linear Classification
Home Page: https://www.csie.ntu.edu.tw/~cjlin/liblinear/
License: BSD 3-Clause "New" or "Revised" License
LIBLINEAR is a simple package for solving large-scale regularized linear classification, regression and outlier detection. It currently supports - L2-regularized logistic regression/L2-loss support vector classification/L1-loss support vector classification - L1-regularized L2-loss support vector classification/L1-regularized logistic regression - L2-regularized L2-loss support vector regression/L1-loss support vector regression - one-class support vector machine. This document explains the usage of LIBLINEAR. To get started, please read the ``Quick Start'' section first. For developers, please check the ``Library Usage'' section to learn how to integrate LIBLINEAR in your software. Table of Contents ================= - When to use LIBLINEAR but not LIBSVM - Quick Start - Installation - `train' Usage - `predict' Usage - `svm-scale' Usage - Examples - Library Usage - Building Windows Binaries - MATLAB/OCTAVE interface - Python Interface - Additional Information When to use LIBLINEAR but not LIBSVM ==================================== There are some large data for which with/without nonlinear mappings gives similar performances. Without using kernels, one can efficiently train a much larger set via linear classification/regression. These data usually have a large number of features. Document classification is an example. Warning: While generally liblinear is very fast, its default solver may be slow under certain situations (e.g., data not scaled or C is large). See Appendix B of our SVM guide about how to handle such cases. http://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf Warning: If you are a beginner and your data sets are not large, you should consider LIBSVM first. LIBSVM page: http://www.csie.ntu.edu.tw/~cjlin/libsvm Quick Start =========== See the section ``Installation'' for installing LIBLINEAR. After installation, there are programs `train' and `predict' for training and testing, respectively. About the data format, please check the README file of LIBSVM. Note that feature index must start from 1 (but not 0). A sample classification data included in this package is `heart_scale'. Type `train heart_scale', and the program will read the training data and output the model file `heart_scale.model'. If you have a test set called heart_scale.t, then type `predict heart_scale.t heart_scale.model output' to see the prediction accuracy. The `output' file contains the predicted class labels. For more information about `train' and `predict', see the sections `train' Usage and `predict' Usage. To obtain good performances, sometimes one needs to scale the data. Please check the program `svm-scale' of LIBSVM. For large and sparse data, use `-l 0' to keep the sparsity. Installation ============ On Unix systems, type `make' to build the `train', `predict', and `svm-scale' programs. Run them without arguments to show the usages. On other systems, consult `Makefile' to build them (e.g., see 'Building Windows binaries' in this file) or use the pre-built binaries (Windows binaries are in the directory `windows'). This software uses some level-1 BLAS subroutines. The needed functions are included in this package. If a BLAS library is available on your machine, you may use it by modifying the Makefile: Unmark the following line #LIBS = -lblas and mark LIBS = blas/blas.a The tool `svm-scale', borrowed from LIBSVM, is for scaling input data file. `train' Usage ============= Usage: train [options] training_set_file [model_file] options: -s type : set type of solver (default 1) for multi-class classification 0 -- L2-regularized logistic regression (primal) 1 -- L2-regularized L2-loss support vector classification (dual) 2 -- L2-regularized L2-loss support vector classification (primal) 3 -- L2-regularized L1-loss support vector classification (dual) 4 -- support vector classification by Crammer and Singer 5 -- L1-regularized L2-loss support vector classification 6 -- L1-regularized logistic regression 7 -- L2-regularized logistic regression (dual) for regression 11 -- L2-regularized L2-loss support vector regression (primal) 12 -- L2-regularized L2-loss support vector regression (dual) 13 -- L2-regularized L1-loss support vector regression (dual) for outlier detection 21 -- one-class support vector machine (dual) -c cost : set the parameter C (default 1) -p epsilon : set the epsilon in loss function of epsilon-SVR (default 0.1) -n nu : set the parameter nu of one-class SVM (default 0.5) -e epsilon : set tolerance of termination criterion -s 0 and 2 |f'(w)|_2 <= eps*min(pos,neg)/l*|f'(w0)|_2, where f is the primal function and pos/neg are # of positive/negative data (default 0.01) -s 11 |f'(w)|_2 <= eps*|f'(w0)|_2 (default 0.0001) -s 1, 3, 4, 7, and 21 Dual maximal violation <= eps; similar to libsvm (default 0.1 except 0.01 for -s 21) -s 5 and 6 |f'(w)|_1 <= eps*min(pos,neg)/l*|f'(w0)|_1, where f is the primal function (default 0.01) -s 12 and 13 |f'(alpha)|_1 <= eps |f'(alpha0)|, where f is the dual function (default 0.1) -B bias : if bias >= 0, instance x becomes [x; bias]; if < 0, no bias term added (default -1) -R : not regularize the bias; must with -B 1 to have the bias; DON'T use this unless you know what it is (for -s 0, 2, 5, 6, 11) -wi weight: weights adjust the parameter C of different classes (see README for details) -v n: n-fold cross validation mode -C : find parameters (C for -s 0, 2 and C, p for -s 11) -q : quiet mode (no outputs) Option -v randomly splits the data into n parts and calculates cross validation accuracy on them. Option -C conducts cross validation under different parameters and finds the best one. This option is supported only by -s 0, -s 2 (for finding C) and -s 11 (for finding C, p). If the solver is not specified, -s 2 is used. Formulations: For L2-regularized logistic regression (-s 0), we solve min_w w^Tw/2 + C \sum log(1 + exp(-y_i w^Tx_i)) For L2-regularized L2-loss SVC dual (-s 1), we solve min_alpha 0.5(alpha^T (Q + I/2/C) alpha) - e^T alpha s.t. 0 <= alpha_i, For L2-regularized L2-loss SVC (-s 2), we solve min_w w^Tw/2 + C \sum max(0, 1- y_i w^Tx_i)^2 For L2-regularized L1-loss SVC dual (-s 3), we solve min_alpha 0.5(alpha^T Q alpha) - e^T alpha s.t. 0 <= alpha_i <= C, For L1-regularized L2-loss SVC (-s 5), we solve min_w \sum |w_j| + C \sum max(0, 1- y_i w^Tx_i)^2 For L1-regularized logistic regression (-s 6), we solve min_w \sum |w_j| + C \sum log(1 + exp(-y_i w^Tx_i)) For L2-regularized logistic regression (-s 7), we solve min_alpha 0.5(alpha^T Q alpha) + \sum alpha_i*log(alpha_i) + \sum (C-alpha_i)*log(C-alpha_i) - a constant s.t. 0 <= alpha_i <= C, where Q is a matrix with Q_ij = y_i y_j x_i^T x_j. For L2-regularized L2-loss SVR (-s 11), we solve min_w w^Tw/2 + C \sum max(0, |y_i-w^Tx_i|-epsilon)^2 For L2-regularized L2-loss SVR dual (-s 12), we solve min_beta 0.5(beta^T (Q + lambda I/2/C) beta) - y^T beta + \sum |beta_i| For L2-regularized L1-loss SVR dual (-s 13), we solve min_beta 0.5(beta^T Q beta) - y^T beta + \sum |beta_i| s.t. -C <= beta_i <= C, where Q is a matrix with Q_ij = x_i^T x_j. For one-class SVM dual (-s 21), we solve min_alpha 0.5(alpha^T Q alpha) s.t. 0 <= alpha_i <= 1 and \sum alpha_i = nu*l, where Q is a matrix with Q_ij = x_i^T x_j. If bias >= 0, w becomes [w; w_{n+1}] and x becomes [x; bias]. For example, L2-regularized logistic regression (-s 0) becomes min_w w^Tw/2 + (w_{n+1})^2/2 + C \sum log(1 + exp(-y_i [w; w_{n+1}]^T[x_i; bias])) Some may prefer not having (w_{n+1})^2/2 (i.e., bias variable not regularized). For primal solvers (-s 0, 2, 5, 6, 11), we provide an option -R to remove (w_{n+1})^2/2. However, -R is generally not needed as for most data with/without (w_{n+1})^2/2 give similar performances. The primal-dual relationship implies that -s 1 and -s 2 give the same model, -s 0 and -s 7 give the same, and -s 11 and -s 12 give the same. We implement 1-vs-the rest multi-class strategy for classification. In training i vs. non_i, their C parameters are (weight from -wi)*C and C, respectively. If there are only two classes, we train only one model. Thus weight1*C vs. weight2*C is used. See examples below. We also implement multi-class SVM by Crammer and Singer (-s 4): min_{w_m, \xi_i} 0.5 \sum_m ||w_m||^2 + C \sum_i \xi_i s.t. w^T_{y_i} x_i - w^T_m x_i >= \e^m_i - \xi_i \forall m,i where e^m_i = 0 if y_i = m, e^m_i = 1 if y_i != m, Here we solve the dual problem: min_{\alpha} 0.5 \sum_m ||w_m(\alpha)||^2 + \sum_i \sum_m e^m_i alpha^m_i s.t. \alpha^m_i <= C^m_i \forall m,i , \sum_m \alpha^m_i=0 \forall i where w_m(\alpha) = \sum_i \alpha^m_i x_i, and C^m_i = C if m = y_i, C^m_i = 0 if m != y_i. `predict' Usage =============== Usage: predict [options] test_file model_file output_file options: -b probability_estimates: whether to output probability estimates, 0 or 1 (default 0); currently for logistic regression only -q : quiet mode (no outputs) Note that -b is only needed in the prediction phase. This is different from the setting of LIBSVM. `svm-scale' Usage ================= See LIBSVM README. Examples ======== > train data_file Train linear SVM with L2-loss function. > train -s 0 data_file Train a logistic regression model. > train -s 21 -n 0.1 data_file Train a linear one-class SVM which selects roughly 10% data as outliers. > train -v 5 -e 0.001 data_file Do five-fold cross-validation using L2-loss SVM. Use a smaller stopping tolerance 0.001 than the default 0.1 if you want more accurate solutions. > train -C data_file ... Best C = 0.000488281 CV accuracy = 83.3333% > train -c 0.000488281 data_file Conduct cross validation many times by L2-loss SVM and find the parameter C which achieves the best cross validation accuracy. Then use the selected C to train the data for getting a model. > train -C -s 0 -v 3 -c 0.5 -e 0.0001 data_file For parameter selection by -C, users can specify other solvers (currently -s 0, -s 2 and -s 11 are supported) and different number of CV folds. Further, users can use the -c option to specify the smallest C value of the search range. This option is useful when users want to rerun the parameter selection procedure from a specified C under a different setting, such as a stricter stopping tolerance -e 0.0001 in the above example. Similarly, for -s 11, users can use the -p option to specify the maximal p value of the search range. > train -c 10 -w1 2 -w2 5 -w3 2 four_class_data_file Train four classifiers: positive negative Cp Cn class 1 class 2,3,4. 20 10 class 2 class 1,3,4. 50 10 class 3 class 1,2,4. 20 10 class 4 class 1,2,3. 10 10 > train -c 10 -w3 1 -w2 5 two_class_data_file If there are only two classes, we train ONE model. The C values for the two classes are 10 and 50. > predict -b 1 test_file data_file.model output_file Output probability estimates (for logistic regression only). Library Usage ============= These functions and structures are declared in the header file `linear.h'. You can see `train.c' and `predict.c' for examples showing how to use them. We define LIBLINEAR_VERSION and declare `extern int liblinear_version; ' in linear.h, so you can check the version number. - Function: model* train(const struct problem *prob, const struct parameter *param); This function constructs and returns a linear classification or regression model according to the given training data and parameters. struct problem describes the problem: struct problem { int l, n; double *y; struct feature_node **x; double bias; }; where `l' is the number of training data. If bias >= 0, we assume that one additional feature is added to the end of each data instance. `n' is the number of feature (including the bias feature if bias >= 0). `y' is an array containing the target values. (integers in classification, real numbers in regression) And `x' is an array of pointers, each of which points to a sparse representation (array of feature_node) of one training vector. For example, if we have the following training data: LABEL ATTR1 ATTR2 ATTR3 ATTR4 ATTR5 ----- ----- ----- ----- ----- ----- 1 0 0.1 0.2 0 0 2 0 0.1 0.3 -1.2 0 1 0.4 0 0 0 0 2 0 0.1 0 1.4 0.5 3 -0.1 -0.2 0.1 1.1 0.1 and bias = 1, then the components of problem are: l = 5 n = 6 y -> 1 2 1 2 3 x -> [ ] -> (2,0.1) (3,0.2) (6,1) (-1,?) [ ] -> (2,0.1) (3,0.3) (4,-1.2) (6,1) (-1,?) [ ] -> (1,0.4) (6,1) (-1,?) [ ] -> (2,0.1) (4,1.4) (5,0.5) (6,1) (-1,?) [ ] -> (1,-0.1) (2,-0.2) (3,0.1) (4,1.1) (5,0.1) (6,1) (-1,?) struct parameter describes the parameters of a linear classification or regression model: struct parameter { int solver_type; /* these are for training only */ double eps; /* stopping tolerance */ double C; double nu; /* one-class SVM only */ int nr_weight; int *weight_label; double* weight; double p; double *init_sol; }; solver_type can be one of L2R_LR, L2R_L2LOSS_SVC_DUAL, L2R_L2LOSS_SVC, L2R_L1LOSS_SVC_DUAL, MCSVM_CS, L1R_L2LOSS_SVC, L1R_LR, L2R_LR_DUAL, L2R_L2LOSS_SVR, L2R_L2LOSS_SVR_DUAL, L2R_L1LOSS_SVR_DUAL, ONECLASS_SVM. for classification L2R_LR L2-regularized logistic regression (primal) L2R_L2LOSS_SVC_DUAL L2-regularized L2-loss support vector classification (dual) L2R_L2LOSS_SVC L2-regularized L2-loss support vector classification (primal) L2R_L1LOSS_SVC_DUAL L2-regularized L1-loss support vector classification (dual) MCSVM_CS support vector classification by Crammer and Singer L1R_L2LOSS_SVC L1-regularized L2-loss support vector classification L1R_LR L1-regularized logistic regression L2R_LR_DUAL L2-regularized logistic regression (dual) for regression L2R_L2LOSS_SVR L2-regularized L2-loss support vector regression (primal) L2R_L2LOSS_SVR_DUAL L2-regularized L2-loss support vector regression (dual) L2R_L1LOSS_SVR_DUAL L2-regularized L1-loss support vector regression (dual) for outlier detection ONECLASS_SVM one-class support vector machine (dual) C is the cost of constraints violation. p is the sensitiveness of loss of support vector regression. nu in ONECLASS_SVM approximates the fraction of data as outliers. eps is the stopping criterion. nr_weight, weight_label, and weight are used to change the penalty for some classes (If the weight for a class is not changed, it is set to 1). This is useful for training classifier using unbalanced input data or with asymmetric misclassification cost. nr_weight is the number of elements in the array weight_label and weight. Each weight[i] corresponds to weight_label[i], meaning that the penalty of class weight_label[i] is scaled by a factor of weight[i]. If you do not want to change penalty for any of the classes, just set nr_weight to 0. init_sol includes the initial weight vectors (supported for only some solvers). See the explanation of the vector w in the model structure. *NOTE* To avoid wrong parameters, check_parameter() should be called before train(). struct model stores the model obtained from the training procedure: struct model { struct parameter param; int nr_class; /* number of classes */ int nr_feature; double *w; int *label; /* label of each class */ double bias; double rho; /* one-class SVM only */ }; param describes the parameters used to obtain the model. nr_class is the number of classes for classification. It is a non-negative integer with special cases of 0 (no training data at all) and 1 (all training data in one class). For regression and one-class SVM, nr_class = 2. nr_feature is the number of features. The array w gives feature weights. Its size is nr_feature*nr_class but is nr_feature if nr_class = 2 and the solver is not MCSVM_CS (see more explanation below). We use one against the rest for multi-class classification, so each feature index corresponds to nr_class weight values. Weights are organized in the following way +------------------+------------------+------------+ | nr_class weights | nr_class weights | ... | for 1st feature | for 2nd feature | +------------------+------------------+------------+ The array label stores class labels. When nr_class = 1 or 2, classification solvers (MCSVM_CS excluded) return a single vector of weights by considering label[0] as positive in training. If bias >= 0, x becomes [x; bias]. The number of features is increased by one, so w is a (nr_feature+1)*nr_class array. The value of bias is stored in the variable bias. rho is the bias term used in one-class SVM only. - Function: void cross_validation(const problem *prob, const parameter *param, int nr_fold, double *target); This function conducts cross validation. Data are separated to nr_fold folds. Under given parameters, sequentially each fold is validated using the model from training the remaining. Predicted labels in the validation process are stored in the array called target. The format of prob is same as that for train(). - Function: void find_parameters(const struct problem *prob, const struct parameter *param, int nr_fold, double start_C, double start_p, double *best_C, double *best_p, double *best_score); This function is similar to cross_validation. However, instead of conducting cross validation under specified parameters. For -s 0, 2, it conducts cross validation many times under parameters C = start_C, 2*start_C, 4*start_C, 8*start_C, ..., and finds the best one with the highest cross validation accuracy. For -s 11, it conducts cross validation many times with a two-fold loop. The outer loop considers a default sequence of p = 19/20*max_p, ..., 1/20*max_p, 0 and under each p value the inner loop considers a sequence of parameters C = start_C, 2*start_C, 4*start_C, ..., and finds the best one with the lowest mean squared error. If start_C <= 0, then this procedure calculates a small enough C for prob as the start_C. The procedure stops when the models of all folds become stable or C reaches max_C. If start_p <= 0, then this procedure calculates a maximal p for prob as the start_p. Otherwise, the procedure starts with the first i/20*max_p <= start_p so the outer sequence is i/20*max_p, (i-1)/20*max_p, ..., 0. The best C, the best p, and the corresponding accuracy (or MSE) are assigned to *best_C, *best_p and *best_score, respectively. For classification, *best_p is not used, and the returned value is -1. - Function: double predict(const model *model_, const feature_node *x); For a classification model, the predicted class for x is returned. For a regression model, the function value of x calculated using the model is returned. - Function: double predict_values(const struct model *model_, const struct feature_node *x, double* dec_values); This function gives nr_w decision values in the array dec_values. nr_w=1 if regression is applied or the number of classes is two. An exception is multi-class SVM by Crammer and Singer (-s 4), where nr_w = 2 if there are two classes. For all other situations, nr_w is the number of classes. We implement one-vs-the rest multi-class strategy (-s 0,1,2,3,5,6,7) and multi-class SVM by Crammer and Singer (-s 4) for multi-class SVM. The class with the highest decision value is returned. - Function: double predict_probability(const struct model *model_, const struct feature_node *x, double* prob_estimates); This function gives nr_class probability estimates in the array prob_estimates. nr_class can be obtained from the function get_nr_class. The class with the highest probability is returned. Currently, we support only the probability outputs of logistic regression. - Function: int get_nr_feature(const model *model_); The function gives the number of attributes of the model. - Function: int get_nr_class(const model *model_); The function gives the number of classes of the model. For a regression model, 2 is returned. - Function: void get_labels(const model *model_, int* label); This function outputs the name of labels into an array called label. For a regression model, label is unchanged. - Function: double get_decfun_coef(const struct model *model_, int feat_idx, int label_idx); This function gives the coefficient for the feature with feature index = feat_idx and the class with label index = label_idx. Note that feat_idx starts from 1, while label_idx starts from 0. If feat_idx is not in the valid range (1 to nr_feature), then a zero value will be returned. For classification models, if label_idx is not in the valid range (0 to nr_class-1), then a zero value will be returned; for regression models and one-class SVM models, label_idx is ignored. - Function: double get_decfun_bias(const struct model *model_, int label_idx); This function gives the bias term corresponding to the class with the label_idx. For classification models, if label_idx is not in a valid range (0 to nr_class-1), then a zero value will be returned; for regression models, label_idx is ignored. This function cannot be called for a one-class SVM model. - Function: double get_decfun_rho(const struct model *model_); This function gives rho, the bias term used in one-class SVM only. This function can only be called for a one-class SVM model. - Function: const char *check_parameter(const struct problem *prob, const struct parameter *param); This function checks whether the parameters are within the feasible range of the problem. This function should be called before calling train() and cross_validation(). It returns NULL if the parameters are feasible, otherwise an error message is returned. - Function: int check_probability_model(const struct model *model); This function returns 1 if the model supports probability output; otherwise, it returns 0. - Function: int check_regression_model(const struct model *model); This function returns 1 if the model is a regression model; otherwise it returns 0. - Function: int check_oneclass_model(const struct model *model); This function returns 1 if the model is a one-class SVM model; otherwise it returns 0. - Function: int save_model(const char *model_file_name, const struct model *model_); This function saves a model to a file; returns 0 on success, or -1 if an error occurs. - Function: struct model *load_model(const char *model_file_name); This function returns a pointer to the model read from the file, or a null pointer if the model could not be loaded. - Function: void free_model_content(struct model *model_ptr); This function frees the memory used by the entries in a model structure. - Function: void free_and_destroy_model(struct model **model_ptr_ptr); This function frees the memory used by a model and destroys the model structure. - Function: void destroy_param(struct parameter *param); This function frees the memory used by a parameter set. - Function: void set_print_string_function(void (*print_func)(const char *)); Users can specify their output format by a function. Use set_print_string_function(NULL); for default printing to stdout. Building Windows Binaries ========================= Windows binaries are available in the directory `windows'. To re-build them via Visual C++, use the following steps: 1. Open a dos command box and change to liblinear directory. If environment variables of VC++ have not been set, type "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvars64.bat" You may have to modify the above command according which version of VC++ or where it is installed. 2. Type nmake -f Makefile.win clean all 3. (optional) To build shared library liblinear.dll, type nmake -f Makefile.win lib 4. (Optional) To build 32-bit windows binaries, you must (1) Setup "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvars32.bat" instead of vcvars64.bat (2) Change CFLAGS in Makefile.win: /D _WIN64 to /D _WIN32 MATLAB/OCTAVE Interface ======================= Please check the file README in the directory `matlab'. Python Interface ================ Please check the file README in the directory `python'. Additional Information ====================== If you find LIBLINEAR helpful, please cite it as R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin. LIBLINEAR: A Library for Large Linear Classification, Journal of Machine Learning Research 9(2008), 1871-1874. Software available at http://www.csie.ntu.edu.tw/~cjlin/liblinear For any questions and comments, please send your email to [email protected]
I want to use python interface for 64bit python in windows, but the liblinear.dll in the /windows directory seems to be 32bit one. And i can not generate 64bit dll by "nmake -f Makefile.win clean all" , this just generate exe file in /windows directory. So how can i do that?
I am one of the mac user. However, when I typed make, this error occurred
clang: warning: no such sysroot directory: '/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk' [-Wmissing-sysroot]
libsvmread.c:1:10: fatal error: 'stdio.h' file not found
#include <stdio.h>
^~~~~~~~~
1 error generated.
warning: mkoctfile: building exited with failure status
Error: /opt/local/share/octave/6.2.0/m/miscellaneous/mex.m failed (line 54)
mex: building exited with failure status
=> Please check README for detailed instructions.
does anyone know the reason why? I installed the linear file onto the readable directory, but this error occurred. I don't know what is the problem. THanks
Hi, just want to know how can I read the support vectors out (in python interface)? Thanks.
Hi, I would like to work on merging the sample weights version. Can you provide some guidance in terms of your own requirements?
My final goal is having a sample weights version in R (based on https://cran.r-project.org/web/packages/LiblineaR/index.html)
I have done the majority of the work,
but there are a few sticking points:
a) adding sample weights 'breaks' the interface for eg matlab and python versions
b) should one change the code to always use sample weights, or is the added computational cost too great?
c) currently I have merged the c++ code using conditional compilation because of b), however, this does not work for python and R (no conditional compilation), so it raises the worry of python/R code calling the library with the wrong compilation options.
eg I could
Hi. Do you intend to act on the pull requests?
I am trying to use liblinear for python. I use Ubuntu 20.04 and I ran make in the python folder.
I’m trying to import it by running
from liblinearutil import *
However, I am still unable to successfully import it outside the liblinear directory. How can I fix this? Thank you
Hello,
I am trying to implement OvO multiclass logistic regression classifier in MATLAB, but the version on the webpage for the OvO code is older than the current version of LIBLINEAR (2.20 vs 2.30). Would it be possible to obtain an updated version for this code? Thanks in advance.
Hi,
I have to classifier my input data to multi class and I have trouble to use liblinear to classify it and give me the desired output can you point me please?
input data:
! COUNT !! LABEL !! PATTERN !! FEATURES
! 0.1 !! 42 !! NOUN DE NOUN !! 0:millión 1:de 2:euro
! 0.8 !! 43 !! NOUN DE NOUN !! 0:millión 1:de 2:euro
! 0.1 !! 44 !! NOUN DE NOUN !! 0:millión 1:de 2:euro
! 0.6 !! 42 !! NOUN DE NOUN !! 0:umbral 1:de 2:pobreza
.....
the desired output should look like l that, which is a list of weights for each feature/label
0:umbral/42 = 1.0054
1:de/42 = 0.0
2:pobreza/42 = 1.014
0:umbral/43 = 1.0044
1:de/43 = 0.0
2:pobreza/43 = 1.004
....
thanks for your time to guide me.
I am a little confusing while using this package for multi-classification. can anyone tell me how to do it ? Thanks.
what i had try:
train_labels=[[1,2], [2], [3]]
train_datas = [[1,1,0], [1,2,2], [1,1,1]]
prob = problem(train_labels, train_datas)
param = parameter('-s 0')
model = train(prob, param)
but it arise some errors:
Traceback (most recent call last):
File "C:\Users\Jiaming\Dropbox\Internship in ADSC\DeepWalk\experiments\classifier.py", line 69, in process
prob = problem(train_labels, train_datas)
File "C:\Users\Jiaming\Anaconda2\lib\site-packages\liblinear-210-py2.7.egg\liblinear\liblinear.py", line 107, in init
for i, yi in enumerate(y): self.y[i] = y[i]
TypeError: a float is required
It's possible to set both negative and positive class weights in a binary classification scenario. However, when a multi-class classification model is trained using a OVR solver, it's possible to set weight only for the positive (i.e. One) class, but the weight for the negative (i.e. Rest) class is always set to 1.
The difference can be seen in https://github.com/cjlin1/liblinear/blob/master/linear.cpp#L2552 where train_one uses both weights and https://github.com/cjlin1/liblinear/blob/master/linear.cpp#L2578 where param->C is used. That corresponds to always using 1 as a weight.
That doesn't allow class weight normalization, and unnormalized weights bias the C term.
I installed liblinear
through pip install liblinear
successfully. But got error ModuleNotFoundError: No module named 'liblinear'
.
I download the zip file from github and included the ../python
dir into sys.path
, the import error still remains.
How should I install it on windows for python. Thanks!
Hi,
It seems that there exists a potential integer overflow. Please find the following description:
I directly execute the "make" command in the unzipped directory, and got the following error:
g++ -Wall -Wconversion -O3 -fPIC -fopenmp -c -o newton.o newton.cpp
g++ -Wall -Wconversion -O3 -fPIC -fopenmp -c -o linear.o linear.cpp
linear.cpp: In member function 'virtual double l2r_erm_fun::fun(double*)':
linear.cpp:244:70: error: 'l2r_erm_fun::wTw' is not a variable in clause 'reduction'
#pragma omp parallel for private(i) reduction(+:wTw) schedule(static)
^
make: *** [linear.o] Error 1
The version I use is liblinear-multicore-2.43-2 from the official website.
How can I solve it, thank you.
I want use liblinear with python, but there find an error "LIBLINEAR library not found" with "from liblinear import *". However, the path of liblinear.dll is right, and it can be successful on linux, how to solve it?
Josephs-MacBook-Pro:~ joe$ brew irb
==> Interactive Homebrew Shell
Example commands available with: brew irb --examples
irb(main):001:0> Version.new("2.2") > Version.new("2.11")
=> false
irb(main):002:0>
This will cause a problem upgrading the Homebrew formula. Would it be possible to rename the release and tarball 2.20
instead of 2.2
or something along those lines?
I am computer science student from India.
I am used to play with SVM implementation of liblinear from sklearn library in python.
but recently I started converting my code from python to C++ and used LIBSVMs C_SVC it works perfectly for me giving me above 97% of accuracy.
But my data set is very large and training time is very slow on LIBSVM so I moved on LIBLINEAR to obtain multi core performance for training. and it is creating more furious problem for me that I am getting accuracy only around 15%.
DATASET:
2,50,000 Images of 7 different classes
dimension 128 X 128 px
calculate HOG features of all images, length of 1 feature vector is 1296
X* = 250000 x 1296
Y = 250000
whole data set is normalised in 0-1 range.
I am not using command line interface of LIBLINEAR because training file is getting very big in GBs.
I am including liblinear and performed all necessary steps in order to use all the classes and functions of it.
now I have to classify all images into 7 different classes
I am using param.s=2 param.e=0.0001 don't need to set weight of different classes
and perform cross fold validation 70 for 2,50,000 images to find value of C
it gives me value of C about 4.76837e-07 and CV accuracy = 16.3265%
what should I do??
If I made any mistake please direct me on the correct path. thank you.
train
in liblinear 2.11 crashes for me after applying the change in #36. The console output is below; observe the iteration counter. Valgrind detected an out-of-bounds read in linear.cpp:75-83
(axpy()
) and adding assert( y->index > 0 );
as the first statement in the while loop aborts the program on execution.
christoph:/tmp/tmp.8mis3wq6Em$ ./liblinear-2.11/train -B 1 -e 0.0001 training.txt svm-model
...*
optimization finished, #iter = 37
Objective value = -0.161526
nSV = 19
..*
optimization finished, #iter = 27
Objective value = -0.084889
nSV = 12
...*.
optimization finished, #iter = 40
Objective value = -0.110262
nSV = 14
...*
optimization finished, #iter = 39
Objective value = -0.185151
nSV = 17
...*
optimization finished, #iter = 37
Objective value = -0.089334
nSV = 13
Segmentation fault (core dumped)
christoph:~$ g++ --version
g++ (Ubuntu 5.4.0-6ubuntu1~16.04.5) 5.4.0 20160609
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Doing a routine scan of /windows/train.exe with Bitdefender revealed "Cloud.Malware.2158qCW@bmgxWG". My scans show that the malware is present since f0b2b38.
LibSVM currently has probability estimates for linear SVC. Could this be implemented in Liblinear aswell?
i need to get the value of coef(W) using liblinear
from sklearn.svm import LinearSVC
x_train=x_train.reshape(500,784)
#y_train=np.argmax(y_train,axis=1)
from sklearn.svm import LinearSVC,SVC
clf_weights =LinearSVC(random_state=0, tol=.01)
clf_weights.fit(x_train,y_train)
LinearSVC(C=1.0, class_weight=None, dual=True, fit_intercept=True,
intercept_scaling=1, loss='squared_hinge', max_iter=1000,
multi_class='ovr', penalty='l2', random_state=0, tol=0.01, verbose=0)
[[0. 0. 0. ... 0. 0. 0.]
[0. 0. 0. ... 0. 0. 0.]
[0. 0. 0. ... 0. 0. 0.]
...
[0. 0. 0. ... 0. 0. 0.]
[0. 0. 0. ... 0. 0. 0.]
[0. 0. 0. ... 0. 0. 0.]]
print(clf.intercept_)
output:
[-0.1940297 0.03668576 -0.34181232 -0.27452097 -0.10010669 -0.27315419
-0.29847159 -0.19633 -0.90375486 -0.29134017]
sv=clf_weights.support_vectors_
print(clf_weights.support_vectors_)
coef=clf_weights.coef_
i got coefficient as (45,784) and support vectors of (336,784)
also how i can get coef (W)
##########################################
is this correct?
for i in range(45):
w=w[i]+coef[i]*sv[i,:]
print(w)
im getting output with zeros
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
According to the LIBLINEAR website, liblinear and libsvm use the same input format. On my computer, train
signals an error when it reads an input file that is successfully parsed by libsvm. I traced the different behavior to line 395 in train.c
:
inst_max_index = 0; // strtol gives 0 if wrong format
In libsvm, the same variable is initialized to -1
and changing inst_max_index
allows train
to read the file successfully, too.
I am using a java version of liblinear which was autogenerated from the C++ code, so I am a little conflicted about where to ask this question, so I figured why not ask in both projects? The documentation on bias is a little unclear to me. I can see that when using the provided command-line executables, an additional bias term is automatically added to the feature set, but from what I can tell this does not appear to be the case if you use liblinear as a library for training and/or testing and prediction.
Intuitively one would assume that setting the bias param to some value would have an effect on the model, but it seems to me that if you set bias to 1, this would do nothing. In the executables the bias term is used as the synthetic feature's value, but if you are required to manually add the synthetic feature when you write your own code, is there any point in even setting bias? One could set bias to 1.0 and add a synthetic feature with any value. Additionally, it appears that this behavior, coupled with the way in which the dot product is calculated makes it difficult to even know when there is an issue as the model will typically work regardless of the input size (as I recall, it seems that features are simply being searched for as needed, and due to the sparse dot product missing features are ignored).
Maybe I am completely wrong in my analysis of the code, I am very rusty with C++. But it seems like the library does not take ownership of managing the bias despite that behavior being heavily implied. When constructing a problem it is suggested to set the bias param, manually increase the number of features, and then also manually add the feature to the inputs. So it seems the data format for a problem and of a model allows for an inconsistent state--where you are requesting bias be used, but not actually getting it or using some value other than the bias term as the synthetic feature
When I use L1R_LR, a discrimination ratio changes depending on a normalization method.
I tried two normalization method.
I read the program, but I did not find a reason.
I'm getting
make -C .. lib
if [ "Darwin" = "Darwin" ]; then \
SHARED_LIB_FLAG="-dynamiclib -Wl,-install_name,liblinear.so.3"; \
else \
SHARED_LIB_FLAG="-shared -Wl,-soname,liblinear.so.3"; \
fi; \
c++ ${SHARED_LIB_FLAG} linear.o tron.o blas/blas.a -o liblinear.so.3
ld: archive has no table of contents file 'blas/blas.a' for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [lib] Error 1
make: *** [lib] Error 2
when trying to make under python/ is there any solution for that? Running on mac mojave
I'm using Matlab 2015b. My compiler seems to working properly as I've compiled other .c files and the make
command returns without errors. I don't remember experiencing these problems on my last computer setup which used Matlab 2015a.
Thanks for your help!
Invalid MEX-file
'/home/mensen/matlab_toolboxes/liblinear-multicore-2.1-2/matlab/train.mexa64':
dlopen: cannot load any more object with static TLS
Error in mvpa_train>classif (line 39)
model = train(Y, sparse(double(X)), ['-s '
type ' -q -c ', num2str(best_lambda)]);
Dear Professor:
Recently I noticed that in your auto C search code, you set the min_C with:
1.0 / (matrix.length * max([Σ (line's value) ^ 2 for line in matrix]))
Could you explain why set starter C with this value? Is there any mathematical reason behind this value? I didn't find any explanation in your https://www.csie.ntu.edu.tw/~cjlin/papers/liblinear.pdf
Regards
I recently installed the LIBLINEAR and was testing the example scripts that were part of the README.weight
The following codes work until the problem() command
from liblinear.liblinearutil import *
import csv
y, x = svm_read_problem('./heart_scale', return_scipy=True)
W = [1] * len(y)
W[0] = 10
prob = problem(W, y, x) # Error occurs here
The error code is shown here:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/ksong4/miniconda3/lib/python3.7/site-packages/liblinear/liblinear.py", line 194, in __init__
tmp_xi, tmp_idx = gen_feature_nodearray(xi)
File "/home/ksong4/miniconda3/lib/python3.7/site-packages/liblinear/liblinear.py", line 106, in gen_feature_nodearray
raise TypeError('xi should be a dictionary, list, tuple, 1-d numpy array, or tuple of (index, data)')
TypeError: xi should be a dictionary, list, tuple, 1-d numpy array, or tuple of (index, data)
The imported x and y are both lists using the provided example heart_scale file.
The Makefile passes $(CFLAGS)
to $(CXX)
. It should pass $(CXXFLAGS)
instead.
Furthermore, it defines its own CFLAGS
, rather than honoring flags the user might supply when invoking make
. This causes the build to fail if the user wants to supply flags that are required for the build to succeed.
This was discovered because liblinear is included in nmap. See the nmap bug report here: nmap/nmap#1161 and the MacPorts project's bug report here: https://trac.macports.org/ticket/53995
In the Ubuntu terminal, I typed:
./train -s 2 -v 5 -e 0.001 -q train1.txt model1
Where "train1.txt" is my train sample file .
The result is:
Cross Validation Accuracy = 91.5398%
But I didn't find any file named "model1" in the current directory. What's the matter?
I was trying to implement a custom classifier in scikit-learn using the liblinear library, and cross-validate using the cross_validate()
method which can run multiple folds in parallel. It gives this error when running in parallel:
joblib.externals.loky.process_executor._RemoteTraceback:
"""
Traceback (most recent call last):
File "/home/akee511/src/emotion/.venv/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py", line 407, in _process_worker
call_item = call_queue.get(block=True, timeout=timeout)
File "/usr/lib/python3.8/multiprocessing/queues.py", line 116, in get
return _ForkingPickler.loads(res)
File "/home/akee511/src/emotion/src/ertk/sklearn/models/mtl.py", line 1, in <module>
import liblinear.liblinearutil as liblinearutil
ModuleNotFoundError: No module named 'liblinear.liblinearutil'; 'liblinear' is not a package
"""
I have determined that it's due to adding the directory to sys.path
in liblinearutil.py
which seems to be unnecessary.
Hello,
I was curious is there will be python 3 support given that python 2 is EOL soon?
Here's what I've done:
cd
to the downloaded folder.make
in the main folder, and then also in the /python
folder. This generates a liblinear.so.3
file in the /python
folder./python
folder.import liblinear
or import liblinearutil
, I get the following error: Exception: LIBLINEAR library not found.
If it helps: I need liblinear because I'm trying to use a different library that requires it.
So if I ran into an issue with liblinear where a feature present in class A is not biasing to that class. Isn't that weird?
If I incremented the weight from 1 to 10 for all features, then the prediction is better. Why?
More info here: https://stackoverflow.com/questions/60017644/is-there-a-reason-why-a-feature-only-present-in-a-given-class-is-not-being-predi
Tried to install in a VM running 16.04.1-Ubuntu, got a different error this time:
>>>sudo make
make -C .. lib
make[1]: Entering directory '/usr/local/liblinear-2.21'
g++ -Wall -Wconversion -O3 -fPIC -c -o linear.o linear.cpp
g++ -Wall -Wconversion -O3 -fPIC -c -o tron.o tron.cpp
make -C blas OPTFLAGS='-Wall -Wconversion -O3 -fPIC' CC='cc';
make[2]: Entering directory '/usr/local/liblinear-2.21/blas'
cc -Wall -Wconversion -O3 -fPIC -c dnrm2.c
cc -Wall -Wconversion -O3 -fPIC -c daxpy.c
cc -Wall -Wconversion -O3 -fPIC -c ddot.c
cc -Wall -Wconversion -O3 -fPIC -c dscal.c
ar rcv blas.a dnrm2.o daxpy.o ddot.o dscal.o
a - dnrm2.o
a - daxpy.o
a - ddot.o
a - dscal.o
ranlib blas.a
make[2]: Leaving directory '/usr/local/liblinear-2.21/blas'
if [ "Linux" = "Darwin" ]; then \
SHARED_LIB_FLAG="-dynamiclib -Wl,-install_name,liblinear.so.3"; \
else \
SHARED_LIB_FLAG="-shared -Wl,-soname,liblinear.so.3"; \
fi; \
g++ ${SHARED_LIB_FLAG} linear.o tron.o blas/blas.a -o liblinear.so.3
make[1]: Leaving directory '/usr/local/liblinear-2.21'
>>>```
Any hints on how to solve this?
Thanks.
I'm running liblinear on text classification using -s 0 and -s 6 in order to get probability estimates of a multiclass classification task.
I read through the guides, sites, documents, feature scaling etc but the accuracy of each classifier is always low ? e.g. 14%
How do I further improve the accuracy of the classifier?
Any help most useful
svm-train -s 3 -t 0 -c 1 -p 0.1 -e 0.001 -h 0 eunite2001 model.1 && svm-predict eunite2001.t model.1 prediction.1
liblinear-train -s 11 -c 1 -p 0.1 -e 0.001 eunite2001 model.11 && liblinear-predict eunite2001.t model.11 prediction.11
liblinear-train -s 12 -c 1 -p 0.1 -e 0.001 eunite2001 model.12 && liblinear-predict eunite2001.t model.12 prediction.12
liblinear-train -s 13 -c 1 -p 0.1 -e 0.001 eunite2001 model.13 && liblinear-predict eunite2001.t model.13 prediction.13
3. The results are here:
libsvm | liblinear -s 11 | liblinear -s 12 | liblinear -s 13 |
---|---|---|---|
754.219 | 711.818 | 714.293 | 655.209 |
735.951 | 695.675 | 703.196 | 651.262 |
745.716 | 606.048 | 601.496 | 628.192 |
756.885 | 721.134 | 721.481 | 652.914 |
758.048 | 704.657 | 705.966 | 644.363 |
758.296 | 703.099 | 703.878 | 644.147 |
756.88 | 680.706 | 688.226 | 629.164 |
753.174 | 681.003 | 682.531 | 631.114 |
733.147 | 666.063 | 668.37 | 617.042 |
743.909 | 606.234 | 599.665 | 605.601 |
... | ... | ... | ... |
When using scikit learn, Logistic Regression never returns on fitting with nearly degenerate data.
Scikit learn passed the blame on to liblinear.
import sklearn.linear_model
import numpy as np
model = sklearn.linear_model.LogisticRegression()
num_pts = 15
x = np.zeros((num_pts*2, 2))
x[3] = 3.7491010398553741e-208
y = np.append(np.zeros(num_pts), np.ones(num_pts))
model.fit(x, y)
Return or throw error.
Never returns.
Linux-2.6.32-573.18.1.el6.x86_64-x86_64-with-redhat-6.7-Carbon
('Python', '2.7.12 |Anaconda 2.0.1 (64-bit)| (default, Jul 2 2016, 17:42:40) \n[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]')
('NumPy', '1.11.0')
('SciPy', '0.17.0')
('Scikit-Learn', '0.17.1')
Is there any example of usage from C++ code?
Why is it that after the training, a lot of iteration information is output, but according to the example, only one should be output.
When I execute train(x, y)
, a lot of information appears on the console, as follows:
.......*
optimization finished, #iter = 74
Objective value = -61.281319
nSV = 103
.......**
optimization finished, #iter = 73
Objective value = -54.324909
nSV = 91
..*
optimization finished, #iter = 26
Objective value = -2.698234
nSV = 15
.*
optimization finished, #iter = 12
Objective value = -2.446329
nSV = 11
When I compile liblinear in Windows 10 with VS2019, I get the following error:
D:\liblinear-2.30>nmake -f Makefile.win lib
Microsoft (R) 程序维护实用工具 14.26.28805.0 版
版权所有 (C) Microsoft Corporation。 保留所有权利。
cl.exe /nologo /O2 /EHsc /I. /D _WIN64 /D _CRT_SECURE_NO_DEPRECATE -c tron.cpp
tron.cpp
tron.cpp(1): fatal error C1083: 无法打开包括文件: “math.h”: No such file or directory
NMAKE : fatal error U1077: “"C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.26.28801\bin\HostX86\x86\cl.exe"”: 返回代码“0x2”
Stop.
How to fix it?
Hello,
I am using liblinear to do an SVM classification and I am seeing output during training that I don't understand. Specifically,
...
iter 259 act 2.666e-02 pre 2.666e-02 delta 1.443e-03 f 8.453e+03 |g| 3.771e+01 CG 2
cg reaches trust region boundary
iter 260 act 3.524e-02 pre 3.522e-02 delta 1.451e-03 f 8.453e+03 |g| 7.068e+01 CG 3
cg reaches trust region boundary
iter 261 act 2.766e-02 pre 3.918e-02 delta 1.143e-03 f 8.453e+03 |g| 5.879e+01 CG 3
cg reaches trust region boundary
iter 262 act 3.855e-02 pre 3.855e-02 delta 1.299e-03 f 8.453e+03 |g| 1.061e+02 CG 3
cg reaches trust region boundary
iter 263 act 2.558e-02 pre 2.558e-02 delta 1.316e-03 f 8.453e+03 |g| 4.086e+01 CG 2
cg reaches trust region boundary
iter 264 act 3.885e-02 pre 3.885e-02 delta 1.442e-03 f 8.453e+03 |g| 1.712e+02 CG 3
...
I haven't been able to find documentation anywhere for what these values mean. Is it bad that the "trust boundary" is reached? Does it mean training isn't working? In the first few iterations the trust boundary is not always reached but later in training it seems to be. Is there a resource anywhere that can help me understand?
Thanks!
Jordan
Hi,
I am getting below error while trying to import liblinear in python. unable to figure out what is going wrong here.
AttributeError: /usr/lib/liblinear.so.1: undefined symbol: find_parameter_C
Hi, I'm a computer science student based in Milan.
I want to know if I can use this library (especially, with the Python interface/wrapper) for the ranking task. I want to learn a ranking function in Learning to Rank style.
It is possible?
Thanks for the answer!
When I run the code below:
y, x = [3, 3, 3], [{0: 1, 1: 1, 2: 1, 3: 1, 4: 1, 5: 1, 6: 1, 7: 1, 8: 1, 9: 1, 10: 1, 11: 1, 12: 1}, {0: 1, 1: 1, 2: 1, 4: 1, 5: 1, 6: 1, 7: 1, 8: 1, 10: 1, 11: 1, 13: 1, 14: 1, 15: 1}, {5: 1, 7: 1, 8: 1, 10: 1, 11: 1, 13: 1, 14: 1, 16: 1, 17: 1, 18: 1, 19: 1, 20: 1, 21: 1}]
prob = problem(y, x)
param = parameter('-c 4')
m = train(prob, param)
I got (pycharm)python.exe crash
Traceback (most recent call last):
File "D:/ProgramData/pycharm/workspace/Test/SlotFilling/main.py", line 95, in <module>
model.fit()
File "D:\ProgramData\pycharm\workspace\Test\SlotFilling\SVM.py", line 66, in fit
m = train(prob, param)
File "D:\ProgramData\Anaconda3\lib\site-packages\liblinearutil.py", line 155, in train
m = liblinear.train(prob, param)
OSError: exception: access violation reading 0x00000C084B86D008
Why ? Pls help me, THANKS
I wonder, if coefficients can be larger than 1 even on the normalized data ?
I am bench making liblinear with glmnet, and I see one of the coefficients is larger than one.
Thanks.
Hi, thanks for a great tool. With regard to this compilation warning:
linear.cpp: In function `void train_one(const problem*, const parameter*, double*, double, double)':
linear.cpp:1365: warning: 'loss_old' might be used uninitialized in this function
Can I just initialize "loss_old" to 0? Thanks.
Here is the issue, how can I saolve it?
Invalid MEX-file
'/home/kcho/Dropbox/ML/mii-stroke-deeplearn/013017_stroke_tissue_fate/liblinear-multicore/matlab/train.mexa64':
dlopen: cannot load any more object with static TLS.
It would be nice if the project also included cmake rules for the following reasons:
Adding support for cmake does not require dropping the existing the Makefiles.
There's been two PR that tried this already. There was #42, which unfortunately mixed the addition of cmake with other non-desired features and so was closed. And there is #26 which is now 5 years old and not been acted on. I also made my own cmake rules (it's pretty simple).
Would be nice if this feature could be added. I can take a look at this, no problem, if there is is interest on this feature. Please let me know.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.