mattloper / chumpy Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
Dear @mattloper , thank you for your work on the chumpy and the dogleg implementation. I have some questions about function beta_multiplier in optimization_interal.py in line#268. According to my understanding, \beta should satisfy "||d_sd + \beta*(d_gn-d_sd)||_2 == delta", but \beta is not calculated by solving the equation. Can you give me some hints about your calculation of \beta? Thank you.
Hi~
While using the method, one may have several terms to optimize. For example:
objs = {}
objs['j3d'] = obj_j3d(5e4)
objs['prior'] = pprior(0.8e2)
and I also know that the method has paras to set stopping conditions:
e_N are stopping conditions:
e_1 is gradient magnatude threshold
e_2 is step size magnatude threshold
e_3 is improvement threshold (as a ratio; 0.1 means it must improve by 10%% at each step)
Here I want to learn how can we set a specific term's stopping condition?
For example, we just want to ser a improvement threshold for the specific term objs['j3d'] = obj_j3d(5e4).
Thanks in advance!
Best,
Frank
I am trying to assign a 3x3 array as a subset of a 3x3x9 array, but it gives me
Exception: Can't assign a Ch objects as a subset of another.
The following code is troublesome:
k = ch.arange(81).reshape(3, 3, 9)
tmp = ch.eye(3)
k[:, :, 1] = tmp
The complete error message is the following:
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/dist-packages/chumpy/ch.py", line 468, in setitem
raise Exception("Can't assign a Ch objects as a subset of another."
Exception: Can't assign a Ch objects as a subset of another.
I had updated to pip 10.0.1 (to get rid of pip's update nag message), and then while trying to install a package that required chumpy its setup.py failed:
Collecting chumpy (from -r requirements.txt (line 5))
Downloading https://files.pythonhosted.org/packages/61/61/dc4e134b39674ecdd12a68139d2b82c4f9514133e023c070c973afced53a/chumpy-0.67.5.tar.gz (47kB)
100% |████████████████████████████████| 51kB 3.9MB/s
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-ERtEwy/chumpy/setup.py", line 8, in <module>
from pip.req import parse_requirements
ImportError: No module named req
[This stackoverflow answer(https://stackoverflow.com/a/49867265/69326) has a suggested fix.
I have some problem trying to try demo 'show_tree', The error message is like this? Have any idea why?
I use Python 2.7.12
Thanks a lot.
- - - - - - - - - - - <CODE> - - - - - - - - - - - -
import chumpy as ch
[x1, x2, x3] = ch.array(10), ch.array(20), ch.array(30)
for i in range(3): x2 = x1 + x2 + x3
x2.dr_wrt(x1) # pull cache
x2.dr_wrt(x3) # pull cache
x1.label='x1' # for clarity in show_tree()
x2.label='x2' # for clarity in show_tree()
x3.label='x3' # for clarity in show_tree()
x2.show_tree(cachelim=1e-4) # in MB
- - - - - - - - - - - </CODE> - - - - - - - - - - - -
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/chumpy/__init__.py", line 118, in demo
exec('global np\n' + demos[which], globals(), locals())
File "<string>", line 13, in <module>
File "/usr/local/lib/python2.7/dist-packages/chumpy/ch.py", line 1009, in show_tree
assert DEBUG, "Please use dr tree visualization functions in debug mode"
AssertionError: Please use dr tree visualization functions in debug mode
Is there anyway to get around the Python 3.10 error when installing?
The installation should install all the necessary packages that are listed in the requirements. However, the setup.py
has this line now:
version=importlib.import_module('chumpy').__version__
which makes an import of the file chumpy/__init__.py
, which in turn executes:
from ch import *
For the installation to work, all the packages required by Chumpy should then be installed prior to the installation of chumpy, which is, I believe, a problem.
lpc@lpc:~/Downloads/chumpy-master$ python setup.py
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'install_requires'
warnings.warn(msg)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: no commands supplied
Environment
Conda virtual environment
pip version: 20.1
Python version: 3.7.6
OS: Mac Catalina 10.15.4
Steps to Reproduce
Install anaconda 3.7.6
pip install chumpy
Output
Using cached chumpy-0.69.tar.gz (50 kB)
ERROR: Command errored out with exit status 1:
command: /Users/sandrews/.virtualenvs/py37cv4/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/d7/jhs0m4vn3k9bk6dy201nrbcw0000gn/T/pip-install-klqvusum/chumpy/setup.py'"'"'; __file__='"'"'/private/var/folders/d7/jhs0m4vn3k9bk6dy201nrbcw0000gn/T/pip-install-klqvusum/chumpy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/d7/jhs0m4vn3k9bk6dy201nrbcw0000gn/T/pip-pip-egg-info-9uh6xoy2
cwd: /private/var/folders/d7/jhs0m4vn3k9bk6dy201nrbcw0000gn/T/pip-install-klqvusum/chumpy/
Complete output (7 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/var/folders/d7/jhs0m4vn3k9bk6dy201nrbcw0000gn/T/pip-install-klqvusum/chumpy/setup.py", line 15, in <module>
install_requires = [str(ir.req) for ir in install_reqs]
File "/private/var/folders/d7/jhs0m4vn3k9bk6dy201nrbcw0000gn/T/pip-install-klqvusum/chumpy/setup.py", line 15, in <listcomp>
install_requires = [str(ir.req) for ir in install_reqs]
AttributeError: 'ParsedRequirement' object has no attribute 'req'
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.```
for example:
a = ch.array(5)
Rx = ch.array([[1, 0, 0],
[0, ch.cos(a), -ch.sin(a)],
[0, ch.sin(a), ch.cos(a)]])
but when I change a[0] ,the value in Rx will not change . It seems that my way of array assembly will break the forward-mode
So how can I assemble a new array and still hold the forward mode.
SyntaxError: Missing parentheses in call to 'print'. Did you mean print('Please indicate which demo you want, as follows:')?
Hello, first of all thank you for the useful library.
I see that computations are running on the CPU. In my work this causes a speed bottleneck and I would like to run these computations, or at least some parts on GPU. I was examining the code and my code uses minimize_dogleg function from ch.optimization. Lines with state.updateJ is where most of the is being spent.
I was trying to convert some operations to GPU using Numba library but I don't have much success so far. Does anyone tried this before, or if you can show some directions that would be appreciated.
ModuleNotFoundError: No module named 'pip'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
The current version on PyPI does not work with numpy>=1.24.0, since this is already fixed in master could you publish a new release? Thank you 🙂
Python 3.11.8 (main, Feb 26 2024, 21:39:34) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
import chumpy
Traceback (most recent call last):
File "", line 1, in
File "/home/preston/miniconda3/envs/manotorch/lib/python3.11/site-packages/chumpy/init.py", line 1, in
from .ch import *
File "/home/preston/miniconda3/envs/manotorch/lib/python3.11/site-packages/chumpy/ch.py", line 1319, in
from . import linalg
File "/home/preston/miniconda3/envs/manotorch/lib/python3.11/site-packages/chumpy/linalg.py", line 178, in
class SvdD(Ch):
File "/home/preston/miniconda3/envs/manotorch/lib/python3.11/site-packages/chumpy/linalg.py", line 181, in SvdD
@depends_on('x')
^^^^^^^^^^^^^^^
File "/home/preston/miniconda3/envs/manotorch/lib/python3.11/site-packages/chumpy/ch.py", line 1203, in _depends_on
want_out = 'out' in inspect.getargspec(func).args
^^^^^^^^^^^^^^^^^^
AttributeError: module 'inspect' has no attribute 'getargspec'. Did you mean: 'getargs'?
Method compute_r
in primitives serve as forward calculation. But I'm really curious about what letter r
stands for?
I'd appreciate it if you could answer this trivial question.
Thank you~
Could you please explain the reasoning behind explicitly calling the collect() method of the garbage collector in ch.py, line 802?
Have you encountered the issue of the memory blowing up during the optimization process?
I am asking this question because I am noticing a big difference in the time taken to optimize a function (especially when the memory is filled with additional objects managed by the collector), and I've tracked the bottleneck to the gc.collect() call.
Hello @mattloper ,
Thank you very much for this wonderful tool! I am trying to use chumpy in one of my project, and I met one problem when calling the ch.minimize function with method newton-cg. I dive into the source code of chumpy and found that the problem is caused by function ns_jacfunc in chumpy/optimization.py. To be more concrete, a function 'hstack' is called in this function, but actually it is not defined. I know both numpy and chumpy have hstack, but it seems you didn't import them in this py file. I have also tried to import them, but it didn't help (some other problems occur after it, so I guess you don't want to use np.hstack or ch.hstack here). Do you know how to fix this bug?
Best,
Yongchi
The computation order of Jacobian in Chumpy is nondeterministic. This in cases leads to different optimization results from run to run, since the numerical approximation errors are dependent on the computation order. Specifically, the function dr_wrt()
in class Ch
has the following code:
for k in set(self.dterms).intersection(propnames.union(set(self.__dict__.keys()))):
...
The set
is an unordered container in python, and what's even worse for science computation is that its access order is nondeterministic. A quick fix to this could be adding a call to sorted()
:
for k in sorted(set(self.dterms).intersection(propnames.union(set(self.__dict__.keys())))):
...
I tried following commands (snapshot attached) but I don't know why this error is coming (pdb breakpoint is encountered somehow).
I have shown 2 versions in my snapshot , one sequence for 1 D array (where .dr_wrt works) and one sequence with 2 D array where it is failing. Can you please check it out.
Thanks :)
Running ch.demo('optimization') and halted.
1.42e+02 | e1: 2.25e+01 | e2: 5.06e+01 | e3: 6.89e+01 |
/Library/Python/2.7/site-packages/scipy/sparse/linalg/dsolve/linsolve.py:145: MatrixRankWarning: Matrix is exactly singular
warn("Matrix is exactly singular", MatrixRankWarning)
Ctrl-c gives
/Users/chenyangliu/Desktop/BL-dev/chumpy/ch.pyc in __setattr__(self, name, value, itr)
342 if name == 'x':
343 self._dirty_vars.add(name)
--> 344 self.clear_cache(itr)
345 #else:
346 # import warnings
/Users/chenyangliu/Desktop/BL-dev/chumpy/ch.pyc in clear_cache(self, itr)
404
405 for parent, parent_dict in next._parents.items():
--> 406 object.__setattr__(parent, '_dirty_vars', parent._dirty_vars.union(parent_dict['varnames']))
407 parent._invalidate_cacheprop_names(parent_dict['varnames'])
408 todo.append(parent)
when i run pip install chumpy==0.69, there turns out
WARNING: Discarding https://files.pythonhosted.org/packages/87/81/d7a94ad0ff556b9dd1bf27b84c0b255d9ba22ad5952f20088dafacd9292e/chumpy-0.69.tar.gz#sha256=801bfdd53ffcdf386620b29129dc8d7f75f397010f31554c17a30d5a7b58c4df (from https://pypi.org/simple/chumpy/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output. ERROR: Could not find a version that satisfies the requirement chumpy==0.69 (from versions: 0.51, 0.52, 0.53, 0.54, 0.55, 0.56, 0.57, 0.58, 0.59, 0.60, 0.61, 0.62, 0.63, 0.64, 0.65, 0.66, 0.67.1, 0.67.2, 0.67.3, 0.67.4, 0.67.5, 0.67.6, 0.67.7, 0.68, 0.69, 0.70) ERROR: No matching distribution found for chumpy==0.69
Using pip 20.2.2 and setuptools 50.0.0 (released yesterday, August 30th)
Downgrading setuptools to 49.6.0 enabled me to workaround the issue.
Output:
Collecting chumpy
Downloading chumpy-0.70.tar.gz (50 kB)
ERROR: Command errored out with exit status 1:
command: /home/work/env/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-n2ix487y/chumpy/setup.py'"'"'; __file__='"'"'/tmp/pip-install-n2ix487y/chumpy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-gprw26td
cwd: /tmp/pip-install-n2ix487y/chumpy/
Complete output (15 lines):
/home/work/env/lib/python3.6/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-n2ix487y/chumpy/setup.py", line 56, in <module>
'Operating System :: POSIX :: Linux'
File "/home/work/env/lib/python3.6/site-packages/setuptools/_distutils/core.py", line 134, in setup
ok = dist.parse_command_line()
File "/home/work/env/lib/python3.6/site-packages/setuptools/_distutils/dist.py", line 484, in parse_command_line
args = self._parse_command_opts(parser, args)
File "/home/work/env/lib/python3.6/site-packages/setuptools/dist.py", line 903, in _parse_command_opts
nargs = _Distribution._parse_command_opts(self, parser, args)
File "/home/work/env/lib/python3.6/site-packages/setuptools/_distutils/dist.py", line 548, in _parse_command_opts
"command class %s must subclass Command" % cmd_class)
distutils.errors.DistutilsClassError: command class <class 'setuptools.command.egg_info.egg_info'> must subclass Command
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
The chumpy
package's setup.py
file currently attempts to import pip
during the installation process, as shown in the following lines:
try:
from pip._internal import main as pip_main
except ImportError:
from pip import main as pip_main
There are a couple of problems with this approach:
--use-pep517
flag with pip
(which will become the default behavior in the future), or when using other PEP 517-compliant build tools, the installation process fails with a ModuleNotFoundError: No module named 'pip' error. This is because pip is not guaranteed to be available during the installation process when PEP 517 is used.pip
availability: Even without using PEP 517, the assumption that pip
is installed and available during the package installation process is incorrect. Some package managers or environments may not include pip
by default, leading to the same ModuleNotFoundError
.chumpy
using pip
with the --use-pep517
flag:python -m venv myenv
source myenv/bin/activate
pip install chumpy --use-pep517
Traceback (most recent call last):
File "<string>", line 9, in <module>
ModuleNotFoundError: No module named 'pip'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
...
File "<string>", line 11, in <module>
ModuleNotFoundError: No module named 'pip'
Hello, The package seems works very well. With OpenDR and SMPLify etc. Still very difficult to use because cannot check the inside of objects. Problem come because the 'show_tree' doesnot work even after ch.DEBUG = True set successfully.
>>> import chumpy
/usr/local/lib/python3.9/site-packages/chumpy/__init__.py:11: FutureWarning: In the future `np.bool` will be defined as the corresponding NumPy scalar.
from numpy import bool, int, float, complex, object, unicode, str, nan, inf
/usr/local/lib/python3.9/site-packages/chumpy/__init__.py:11: FutureWarning: In the future `np.object` will be defined as the corresponding NumPy scalar.
from numpy import bool, int, float, complex, object, unicode, str, nan, inf
/usr/local/lib/python3.9/site-packages/chumpy/__init__.py:11: FutureWarning: In the future `np.str` will be defined as the corresponding NumPy scalar.
from numpy import bool, int, float, complex, object, unicode, str, nan, inf
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.9/site-packages/chumpy/__init__.py", line 11, in <module>
from numpy import bool, int, float, complex, object, unicode, str, nan, inf
ImportError: cannot import name 'bool' from 'numpy' (/usr/local/lib/python3.9/site-packages/numpy/__init__.py)
>>>
Version: 0.70
py39-numpy-1.25.0
Python-3.9
FreeBSD 14.0
Hi @mattloper , this awesome project is currently python 2.x based. Do you have the plan to support python 3.x?Thanks!
From numpy doc:
If a has more than two dimensions, then broadcasting rules apply [...]. This means that SVD is working in “stacked” mode: it iterates over all indices of the first a.ndim - 2 dimensions and for each combination SVD is applied to the last two indices.
https://docs.scipy.org/doc/numpy/reference/generated/numpy.linalg.svd.html
When calculating the derivatives for SVD chumpy assumes 2D arrays:
line 196 of linalg.py
:
result = np.einsum('ik,jk->kij', u, v)
line 214ff of linalg.py
mxsz = max(shp[0], shp[1])
D = np.zeros(mxsz)
D[:_D.size] = _D
The only workaround I see right now is looping over the dimensions outside of SVD (slow).
Is chumpy using Forward mode or Reverse mode automatic differentiation? A quick browse seems like forward mode...
This error occured when trying to load a pkl file.
I am using an Anacona environment with Python 3.11 and chumpy 0.70 installed- I'm guessing the current chumpy version isn't compatible with python 3.11 yet?
Traceback (most recent call last):
File "c:_MyDrive\Detectron\code\SMPL-densepose.py", line 8, in
data = pickle.load(f, encoding="latin1")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\anaconda3\envs\densepose\Lib\site-packages\chumpy_init_.py", line 1, in
from .ch import *
File "C:\Users\user\anaconda3\envs\densepose\Lib\site-packages\chumpy\ch.py", line 1319, in
from . import linalg
File "C:\Users\user\anaconda3\envs\densepose\Lib\site-packages\chumpy\linalg.py", line 178, in
class SvdD(Ch):
File "C:\Users\user\anaconda3\envs\densepose\Lib\site-packages\chumpy\linalg.py", line 181, in SvdD
@depends_on('x')
^^^^^^^^^^^^^^^
File "C:\Users\user\anaconda3\envs\densepose\Lib\site-packages\chumpy\ch.py", line 1203, in _depends_on
want_out = 'out' in inspect.getargspec(func).args
^^^^^^^^^^^^^^^^^^
AttributeError: module 'inspect' has no attribute 'getargspec'. Did you mean: 'getargs'?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.