Can't start (localshop celeryd -B -E fails)

[mvantellingen]

Can't start (localshop celeryd -B -E fails)

[007]

Same error here, on Ubuntu 10.04 / Python 2.6.

Haven't tested outside virtualenv, but fails within virtualenv.

[stuaxo]

Just found the same bug, this definitely fixes it.

[stuaxo]

OK, spoke to soon - django-celery==2.5.5 fixed a problem with this:
celery/celery#683

So 2.5.5 for celery AND django-celery

[jezdez]

As a work around you can use localshop celeryd -B -E -P solo which is a single process worker implementation.

[mvantellingen]

Thanks, I actually have the same issue. Not yet sure how to fix this yet (or what is causing this). Perhaps an incompatibility between logan and celery

[mvantellingen]

This should now be fixed in the develop branch (thanks @jezdez)

[victorhooi]

Hi,

I've just checked out the latest develop branch of localshop, and I seem to still be getting this issue:

localshop celeryd -B -E
Notice: Unable to load configuration file /home/vichoo/.localshop.py (No such file or directory), using default settings



 -------------- [email protected] v3.0.12 (Chiastic Slide)
---- **** ----- 
--- * ***  * -- [Configuration]
-- * - **** --- . broker:      django://localhost//
- ** ---------- . app:         default:0x18d1c3d0 (djcelery.loaders.DjangoLoader)
- ** ---------- . concurrency: 1 (processes)
- ** ---------- . events:      ON
- ** ---------- 
- *** --- * --- [Queues]
-- ******* ---- . celery:      exchange:celery(direct) binding:celery
--- ***** ----- 

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.19-py2.7-linux-x86_64.egg/billiard/forking.py", line 502, in main
    prepare(preparation_data)
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.19-py2.7-linux-x86_64.egg/billiard/forking.py", line 642, in prepare
    file, path_name, etc = imp.find_module(main_name, dirs)
ImportError: No module named localshop
[2013-01-02 05:24:14,366: WARNING/MainProcess] celery@dev_desktop_vm.aus.optiver.com ready.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.19-py2.7-linux-x86_64.egg/billiard/forking.py", line 502, in main
    prepare(preparation_data)
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.19-py2.7-linux-x86_64.egg/billiard/forking.py", line 642, in prepare
    file, path_name, etc = imp.find_module(main_name, dirs)
ImportError: No module named localshop
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.19-py2.7-linux-x86_64.egg/billiard/forking.py", line 502, in main
    prepare(preparation_data)
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.19-py2.7-linux-x86_64.egg/billiard/forking.py", line 642, in prepare
    file, path_name, etc = imp.find_module(main_name, dirs)
ImportError: No module named localshop
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.19-py2.7-linux-x86_64.egg/billiard/forking.py", line 502, in main
    prepare(preparation_data)
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.19-py2.7-linux-x86_64.egg/billiard/forking.py", line 642, in prepare
    file, path_name, etc = imp.find_module(main_name, dirs)
ImportError: No module named localshop
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.19-py2.7-linux-x86_64.egg/billiard/forking.py", line 502, in main
    prepare(preparation_data)
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.19-py2.7-linux-x86_64.egg/billiard/forking.py", line 642, in prepare
    file, path_name, etc = imp.find_module(main_name, dirs)
ImportError: No module named localshop
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.19-py2.7-linux-x86_64.egg/billiard/forking.py", line 502, in main
    prepare(preparation_data)
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.19-py2.7-linux-x86_64.egg/billiard/forking.py", line 642, in prepare
    file, path_name, etc = imp.find_module(main_name, dirs)
ImportError: No module named localshop
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.19-py2.7-linux-x86_64.egg/billiard/forking.py", line 502, in main
    prepare(preparation_data)
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.19-py2.7-linux-x86_64.egg/billiard/forking.py", line 642, in prepare
    file, path_name, etc = imp.find_module(main_name, dirs)
ImportError: No module named localshop
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.19-py2.7-linux-x86_64.egg/billiard/forking.py", line 502, in main
    prepare(preparation_data)
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.19-py2.7-linux-x86_64.egg/billiard/forking.py", line 642, in prepare
    file, path_name, etc = imp.find_module(main_name, dirs)
ImportError: No module named localshop
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.19-py2.7-linux-x86_64.egg/billiard/forking.py", line 502, in main
    prepare(preparation_data)
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.19-py2.7-linux-x86_64.egg/billiard/forking.py", line 642, in prepare
    file, path_name, etc = imp.find_module(main_name, dirs)
ImportError: No module named localshop
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.19-py2.7-linux-x86_64.egg/billiard/forking.py", line 502, in main
    prepare(preparation_data)
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.19-py2.7-linux-x86_64.egg/billiard/forking.py", line 642, in prepare
    file, path_name, etc = imp.find_module(main_name, dirs)
ImportError: No module named localshop
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.19-py2.7-linux-x86_64.egg/billiard/forking.py", line 502, in main
    prepare(preparation_data)
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.19-py2.7-linux-x86_64.egg/billiard/forking.py", line 642, in prepare
    file, path_name, etc = imp.find_module(main_name, dirs)
ImportError: No module named localshop
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.19-py2.7-linux-x86_64.egg/billiard/forking.py", line 502, in main
    prepare(preparation_data)
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.19-py2.7-linux-x86_64.egg/billiard/forking.py", line 642, in prepare
    file, path_name, etc = imp.find_module(main_name, dirs)
ImportError: No module named localshop

I can confirm that using localshop celeryd -B -E -P solo, as jezdez mentions seems to work - as in, it doesn't generate any errors on the console. I've yet to test actually downloading packages =).

Any thoughts on why this is still happening?

Cheers,
Victor

[mvantellingen]

Oke, well I can confirm that it still doesn't work. I'm going to look for a celery alternative...

[jezdez]

Hm, it works for me. Can you try python manage.py celeryd -B -E instead? Smells like Celery just stumbles over not finding the right binary to spawn the workers with.

[mvantellingen]

Well i've just created a fresh virtualenv, ran python setup.py develop
and then ./manage.py -B -E -c 1 but no luck.

django.core.exceptions.ImproperlyConfigured: django-configurations settings importer wasn't correctly installed. Please use one of the starter functions to install it as mentioned in the docs: http://django-configurations.readthedocs.org/

Will look at it some more tonight

On Wed, Jan 2, 2013 at 5:04 PM, Jannis Leidel [email protected]:

Hm, it works for me. Can you try python manage.py celeryd -B -E instead?
Smells like Celery just stumbles over not finding the right binary to spawn
the workers with.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/21#issuecomment-11812938.

[jezdez]

Make sure you use the manage.py that is included, as it requires to use the tools in configurations to work: https://github.com/mvantellingen/localshop/blob/develop/localshop/runner.py

[mvantellingen]

Yup, that's the one I've used. I just removed the usage of django-configurations as a test and then it works. So it looks like celery does something weird which causes problem with logan and django-configurations.

@jezdez Could you retry in a fresh virtualenv?

[jezdez]

@mvantellingen did you push it to develop?

[mvantellingen]

@jezdez yup. But I think I found the issue. The procfile contains '--no-execv'. With this option everything works. Without it you should also get the error

[jezdez]

@mvantellingen Yeah, it seems like this is an issue with celery, in case you want to set this in general you could also use CELERYD_FORCE_EXECV = False

[mvantellingen]

Yup already decided to just add it and be done with it :-) Thanks