Comments (3)
lorentzenchr
commented on August 25, 2024
2
It is often said that singular design matrices don't allow for a solution, but this is wrong, there are just infinitely many solutions. For OLS, there is a particular nice one called minimal norm solution, i.e. the solution/coefficients having minimal L2 norm among all solutions/coefficients.
It may by that this is of no high practical value, but in light of the discovered interpolation regime, it is at least interesting.
I have at least one PR for the line search in mind that could help at least with a few of those test failures.
from glum.
jtilly
commented on August 25, 2024
Thanks a lot! For future reference, these are the failing tests:
FAILED tests_glum.py::test_glm_regression_unpenalized[wide-GeneralizedLinearRegressor(family='binomial')-True-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized[wide-GeneralizedLinearRegressor(family='binomial')-True-irls-cd] - assert 0.690107820640591 == 1.2837501395684472 ± 6.4e-05
FAILED tests_glum.py::test_glm_regression_unpenalized[wide-GeneralizedLinearRegressor(family='binomial')-False-lbfgs] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized[wide-GeneralizedLinearRegressor(family='binomial')-False-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized[wide-GeneralizedLinearRegressor(family='binomial')-False-irls-cd] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized[wide-GeneralizedLinearRegressor(family='poisson')-True-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized[wide-GeneralizedLinearRegressor(family='poisson')-True-irls-cd] - assert 0.8533955861703721 == 1.2837501395684472 ± 6.4e-05
FAILED tests_glum.py::test_glm_regression_unpenalized[wide-GeneralizedLinearRegressor(family='poisson')-False-lbfgs] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized[wide-GeneralizedLinearRegressor(family='poisson')-False-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized[wide-GeneralizedLinearRegressor(family='poisson')-False-irls-cd] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized[wide-GeneralizedLinearRegressor(family='gamma')-True-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized[wide-GeneralizedLinearRegressor(family='gamma')-True-irls-cd] - assert 0.8051836315439316 == 1.2837501395684472 ± 6.4e-05
FAILED tests_glum.py::test_glm_regression_unpenalized[wide-GeneralizedLinearRegressor(family='gamma')-False-lbfgs] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized[wide-GeneralizedLinearRegressor(family='gamma')-False-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized[wide-GeneralizedLinearRegressor(family='gamma')-False-irls-cd] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized[wide-GeneralizedLinearRegressor(family='tweedie')-True-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized[wide-GeneralizedLinearRegressor(family='tweedie')-True-irls-cd] - assert 0.6292876313885498 == 1.2837501395684472 ± 6.4e-05
FAILED tests_glum.py::test_glm_regression_unpenalized[wide-GeneralizedLinearRegressor(family='tweedie')-False-lbfgs] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized[wide-GeneralizedLinearRegressor(family='tweedie')-False-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized[wide-GeneralizedLinearRegressor(family='tweedie')-False-irls-cd] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[long-GeneralizedLinearRegressor(family='binomial')-True-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[long-GeneralizedLinearRegressor(family='binomial')-True-irls-cd] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[long-GeneralizedLinearRegressor(family='binomial')-False-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[long-GeneralizedLinearRegressor(family='binomial')-False-irls-cd] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[long-GeneralizedLinearRegressor(family='poisson')-True-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[long-GeneralizedLinearRegressor(family='poisson')-True-irls-cd] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[long-GeneralizedLinearRegressor(family='poisson')-False-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[long-GeneralizedLinearRegressor(family='poisson')-False-irls-cd] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[long-GeneralizedLinearRegressor(family='gamma')-True-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[long-GeneralizedLinearRegressor(family='gamma')-True-irls-cd] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[long-GeneralizedLinearRegressor(family='gamma')-False-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[long-GeneralizedLinearRegressor(family='gamma')-False-irls-cd] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[long-GeneralizedLinearRegressor(family='tweedie')-True-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[long-GeneralizedLinearRegressor(family='tweedie')-True-irls-cd] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[long-GeneralizedLinearRegressor(family='tweedie')-False-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[long-GeneralizedLinearRegressor(family='tweedie')-False-irls-cd] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[wide-GeneralizedLinearRegressor(family='binomial')-True-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[wide-GeneralizedLinearRegressor(family='binomial')-True-irls-cd] - assert 1.3802141997400277 == 1.2837501395684472 ± 6.4e-06
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[wide-GeneralizedLinearRegressor(family='binomial')-False-lbfgs] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[wide-GeneralizedLinearRegressor(family='binomial')-False-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[wide-GeneralizedLinearRegressor(family='binomial')-False-irls-cd] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[wide-GeneralizedLinearRegressor(family='poisson')-True-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[wide-GeneralizedLinearRegressor(family='poisson')-True-irls-cd] - assert 1.706489240970734 == 1.2837501395684472 ± 6.4e-06
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[wide-GeneralizedLinearRegressor(family='poisson')-False-lbfgs] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[wide-GeneralizedLinearRegressor(family='poisson')-False-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[wide-GeneralizedLinearRegressor(family='poisson')-False-irls-cd] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[wide-GeneralizedLinearRegressor(family='gamma')-True-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[wide-GeneralizedLinearRegressor(family='gamma')-True-irls-cd] - assert 2.1915879526750373 == 1.2837501395684472 ± 6.4e-06
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[wide-GeneralizedLinearRegressor(family='gamma')-False-lbfgs] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[wide-GeneralizedLinearRegressor(family='gamma')-False-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[wide-GeneralizedLinearRegressor(family='gamma')-False-irls-cd] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[wide-GeneralizedLinearRegressor(family='tweedie')-True-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[wide-GeneralizedLinearRegressor(family='tweedie')-True-irls-cd] - assert 1.2585688960366177 == 1.2837501395684472 ± 6.4e-06
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[wide-GeneralizedLinearRegressor(family='tweedie')-False-lbfgs] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[wide-GeneralizedLinearRegressor(family='tweedie')-False-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized_hstacked_X[wide-GeneralizedLinearRegressor(family='tweedie')-False-irls-cd] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized_vstacked_X[wide-GeneralizedLinearRegressor(family='binomial')-True-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized_vstacked_X[wide-GeneralizedLinearRegressor(family='binomial')-True-irls-cd] - assert 0.6901078206405913 == 1.2837501395684472 ± 1.3e-04
FAILED tests_glum.py::test_glm_regression_unpenalized_vstacked_X[wide-GeneralizedLinearRegressor(family='binomial')-False-lbfgs] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized_vstacked_X[wide-GeneralizedLinearRegressor(family='binomial')-False-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized_vstacked_X[wide-GeneralizedLinearRegressor(family='binomial')-False-irls-cd] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized_vstacked_X[wide-GeneralizedLinearRegressor(family='poisson')-True-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized_vstacked_X[wide-GeneralizedLinearRegressor(family='poisson')-True-irls-cd] - assert 0.8533955861703684 == 1.2837501395684472 ± 1.3e-04
FAILED tests_glum.py::test_glm_regression_unpenalized_vstacked_X[wide-GeneralizedLinearRegressor(family='poisson')-False-lbfgs] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized_vstacked_X[wide-GeneralizedLinearRegressor(family='poisson')-False-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized_vstacked_X[wide-GeneralizedLinearRegressor(family='poisson')-False-irls-cd] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized_vstacked_X[wide-GeneralizedLinearRegressor(family='gamma')-True-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized_vstacked_X[wide-GeneralizedLinearRegressor(family='gamma')-True-irls-cd] - assert 0.8051836315439317 == 1.2837501395684472 ± 1.3e-04
FAILED tests_glum.py::test_glm_regression_unpenalized_vstacked_X[wide-GeneralizedLinearRegressor(family='gamma')-False-lbfgs] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized_vstacked_X[wide-GeneralizedLinearRegressor(family='gamma')-False-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized_vstacked_X[wide-GeneralizedLinearRegressor(family='gamma')-False-irls-cd] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized_vstacked_X[wide-GeneralizedLinearRegressor(family='tweedie')-True-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized_vstacked_X[wide-GeneralizedLinearRegressor(family='tweedie')-True-irls-cd] - assert 0.6292876313579576 == 1.2837501395684472 ± 1.3e-04
FAILED tests_glum.py::test_glm_regression_unpenalized_vstacked_X[wide-GeneralizedLinearRegressor(family='tweedie')-False-lbfgs] - AssertionError:
FAILED tests_glum.py::test_glm_regression_unpenalized_vstacked_X[wide-GeneralizedLinearRegressor(family='tweedie')-False-irls-ls] - numpy.linalg.LinAlgError: Matrix is singular.
FAILED tests_glum.py::test_glm_regression_unpenalized_vstacked_X[wide-GeneralizedLinearRegressor(family='tweedie')-False-irls-cd] - AssertionError:
================================================================== 76 failed, 212 passed, 107 warnings in 197.18s (0:03:17) ==================================================================
I can get the 12 failing L-BFGS related tests to pass by not standardizing the design matrix here.
64 failing tests to go.
from glum.
MarcAntoineSchmidtQC
commented on August 25, 2024
All the failing tests seem to be for unpenalized regression with a singular design matrix (either the wide problem: p=12, n=4, or the stacked problem where we duplicate all columns). Is that correct? Maybe this is a dumb question but what is the expected result in this case? I'm not surprised to see the tests failing in this case for glum, but in case we want to support this the tests are great!
from glum.
Related Issues (20)
- Daily run failure: Unit tests HOT 2
- Several warnings and erros in coef_table HOT 5
- BUG poisson GLM, offset and weighted versions, give wrong result HOT 4
- User facing API for specifying linear models terms HOT 2
- Trouble with importlib library when installing using pip3 HOT 2
- ElasticNet in Glum 2.6.0 is twice as slow as Scikit-learn 1.3.2 HOT 5
- Way to generate confidence interval for predictions? HOT 6
- Get p-value of a categorical model field? HOT 1
- Cross Validation for P2 HOT 3
- Clarify effect of `scale_predictors` on penalization in docstring HOT 7
- Significant difference in model coefficients for binomial distribution? HOT 8
- Daily run failure: Unit tests HOT 6
- Inconsistent coefficient values HOT 2
- Benchmark removal of libblas dependency HOT 4
- scikit-learn 1.5.0 breaks glum HOT 1
- Use pixi for setting up the dev environment HOT 2
- Daily run failure: Unit tests
- glum with ray HOT 3
- How to set `max_alpha` in `GeneralizedLinearRegressorCV`? HOT 2
- glum/tabmat is incompatible with scipy 1.14 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from glum.