mvantellingen/localshop

local pypi server (custom packages and auto-mirroring of pypi)

localshop's Issues

Avoid package overwrite

IMHO, i think if you submited a package to a broker like Localshop, I never can would be able to overwrite it, since it carries the sabe package version.

• upload package
• user installs it
• upload new package, same version

Now we face inconsistency. I fixed that on my simplepypi install by removing write permissions of the upload file. So, once uploaded, its uploaded.

About the other issue asking the ability to remove packages, we may go into this same inconsistency: if you remove, and then upload again? How to control that nobody already have it installed?

Best
Marco

Need a way to purge old development packages

I'm uploading every SVN revision of our product to our localshop instance. However, we are both quite big a package and also quite restricted in terms of hard drive space. There is also absolutely no need to keep development packages older than a week. Therefore, I'd need a way to purge development packages (with version numbers like '1.2.dev-r1234'), either automatically via Celery or with a management command.

When running localshop via supervisord uploads fail 500

Looking at the log file:

SuspiciousOperation: Attempted access to 'source/l/<path-to>.tar.gz' denied.

What folder is localshop trying to access.
and what perms do I need to set on it?

Unable to upload eggs, debug info

After successfully installing localshop using the mysql driver and creating a ~/.pypirc I am unable to upload eggs using:

python2.7 setup.py bdist_egg upload -r myrepo

I have also added my IP to the permissions list, but still no luck. I get "Upload failed (403): FORBIDDEN" every time. How would I turn on debugging to investigate this problem.

Thanks

SIP package not cached

Hi,
is it possible to cache packages such as SIP (same for PyQT) https://pypi.python.org/simple/SIP/.
The problem that localshop does not cache SIP is due to SIP on pypi just holds a reference to the download url's of the package instead of supporting direct downloads.

Is there any workaround for packages like this to get it cached by localshop?

Regards,

Jonas

Timestamp Differences

I uploaded a packag at 11am.
The machine is running on UTC, at 2pm.
Localshop says the package was released 4pm.
The file itself on the server has 2pm as timestamp.

Why localshop is using GMT+2 ?
How can I tell localshop to display using GMT-3?

Lovato

localshop celeryd throws OSError: [Errno 38] Function not implemented

Hi!

When executing localshop celeryd -B -E I got this error:

localshop celeryd -B -E &
[2] 58
(venv)localshop@8870c71dad26:~$ Notice: Unable to load configuration file /home/localshop/.localshop/localshop.conf.py (No such file or directory), using default settings



 -------------- celery@8870c71dad26 v3.0.12 (Chiastic Slide)
---- **** -----
--- * ***  * -- [Configuration]
-- * - **** --- . broker:      django://localhost//
- ** ---------- . app:         default:0x1ab2710 (djcelery.loaders.DjangoLoader)
- ** ---------- . concurrency: 4 (processes)
- ** ---------- . events:      ON
- ** ----------
- *** --- * --- [Queues]
-- ******* ---- . celery:      exchange:celery(direct) binding:celery
--- ***** -----

[2013-07-17 14:31:40,805: ERROR/MainProcess] Unrecoverable error: OSError(38, 'Function not implemented')
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/celery/worker/__init__.py", line 347, in start
    component.start()
  File "/usr/local/lib/python2.7/dist-packages/celery/concurrency/base.py", line 111, in start
    self.on_start()
  File "/usr/local/lib/python2.7/dist-packages/celery/concurrency/processes/__init__.py", line 77, in on_start
    **self.options)
  File "/usr/local/lib/python2.7/dist-packages/billiard/pool.py", line 809, in __init__
    self._setup_queues()
  File "/usr/local/lib/python2.7/dist-packages/billiard/pool.py", line 1070, in _setup_queues
    self._inqueue = SimpleQueue()
  File "/usr/local/lib/python2.7/dist-packages/billiard/queues.py", line 316, in __init__
    self._rlock = Lock()
  File "/usr/local/lib/python2.7/dist-packages/billiard/synchronize.py", line 157, in __init__
    SemLock.__init__(self, SEMAPHORE, 1, 1)
  File "/usr/local/lib/python2.7/dist-packages/billiard/synchronize.py", line 67, in __init__
    kind, value, maxvalue, self._make_name(), unlink_immediately)
OSError: [Errno 38] Function not implemented

any clues?

Can't start (localshop celeryd -B -E fails)

I've tried installing localshop in a virtualenv and directly, both result in the same traceback.
I'm able to import the localshop module from the command-line with "python -c 'import localshop'"

$ localshop celeryd -B -E
/usr/local/lib/python2.6/dist-packages/django/conf/__init__.py:110: DeprecationWarning: The SECRET_KEY setting must not be empty.
  warnings.warn("The SECRET_KEY setting must not be empty.", DeprecationWarning)

 -------------- celery@pages02 v3.0.0 (Chiastic Slide)
---- **** ----- 
--- * ***  * -- [Configuration]
-- * - **** --- . broker:      django://localhost//
- ** ---------- . app:         default:0x19d55d0 (djcelery.loaders.DjangoLoader)
- ** ---------- . concurrency: 1 (processes)
- ** ---------- . events:      ON
- ** ---------- 
- *** --- * --- [Queues]
-- ******* ---- . celery:      exchange:celery(direct) binding:celery
--- ***** ----- 

[2012-07-09 18:28:44,483: ERROR/MainProcess] Unrecoverable error: PicklingError("Can't pickle 'lock' object: <thread.lock object at 0x260f210>",)
Traceback (most recent call last):
  File "/usr/local/lib/python2.6/dist-packages/celery/worker/__init__.py", line 350, in start                                                                                      
    component.start()                                                                                                                                                              
  File "/usr/local/lib/python2.6/dist-packages/billiard/process.py", line 138, in start                                                                                            
    self._popen = Popen(self)                                                                                                                                                      
  File "/usr/local/lib/python2.6/dist-packages/billiard/forking.py", line 223, in __init__                                                                                         
    dump(process_obj, to_child, HIGHEST_PROTOCOL)                                                                                                                                  
  File "/usr/local/lib/python2.6/dist-packages/billiard/forking.py", line 143, in dump                                                                                             
    ForkingPickler(file, protocol).dump(obj)                                                                                                                                       
  File "/usr/lib/python2.6/pickle.py", line 224, in dump                                                                                                                           
    self.save(obj)                                                                                                                                                                 
  File "/usr/lib/python2.6/pickle.py", line 331, in save                                                                                                                           
    self.save_reduce(obj=obj, *rv)                                                                                                                                                 
  File "/usr/lib/python2.6/pickle.py", line 419, in save_reduce                                                                                                                    
    save(state)                                                                                                                                                                    
  File "/usr/lib/python2.6/pickle.py", line 286, in save                                                                                                                           
    f(self, obj) # Call unbound method with explicit self                                                                                                                          
  File "/usr/lib/python2.6/pickle.py", line 649, in save_dict                                                                                                                      
    self._batch_setitems(obj.iteritems())                                                                                                                                          
  File "/usr/lib/python2.6/pickle.py", line 681, in _batch_setitems                                                                                                                
    save(v)                                                                                                                                                                        
  File "/usr/lib/python2.6/pickle.py", line 331, in save                                                                                                                           
    self.save_reduce(obj=obj, *rv)                                                                                                                                                 
  File "/usr/lib/python2.6/pickle.py", line 419, in save_reduce
    save(state)
  File "/usr/lib/python2.6/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.6/pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "/usr/lib/python2.6/pickle.py", line 681, in _batch_setitems
    save(v)
  File "/usr/lib/python2.6/pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "/usr/lib/python2.6/pickle.py", line 419, in save_reduce
    save(state)
  File "/usr/lib/python2.6/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.6/pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "/usr/lib/python2.6/pickle.py", line 681, in _batch_setitems
    save(v)
  File "/usr/lib/python2.6/pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "/usr/lib/python2.6/pickle.py", line 419, in save_reduce
    save(state)
  File "/usr/lib/python2.6/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.6/pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "/usr/lib/python2.6/pickle.py", line 681, in _batch_setitems
    save(v)
  File "/usr/lib/python2.6/pickle.py", line 313, in save
    (t.__name__, obj))
PicklingError: Can't pickle 'lock' object: <thread.lock object at 0x260f210>
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python2.6/dist-packages/billiard/forking.py", line 470, in main
    prepare(preparation_data)
  File "/usr/local/lib/python2.6/dist-packages/billiard/forking.py", line 610, in prepare
    file, path_name, etc = imp.find_module(main_name, dirs)
ImportError: No module named localshop
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python2.6/dist-packages/billiard/forking.py", line 470, in main
    prepare(preparation_data)
  File "/usr/local/lib/python2.6/dist-packages/billiard/forking.py", line 610, in prepare
    file, path_name, etc = imp.find_module(main_name, dirs)
ImportError: No module named localshop

Error on running localshop

After a fresh install in a virtualenv with no site packages:

$ localshop init
Traceback (most recent call last):
  File "/home/yuri/virtualenvs/localshop/bin/localshop", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/home/yuri/virtualenvs/localshop/local/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 2607, in <module>
  File "/home/yuri/virtualenvs/localshop/local/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 569, in resolve
pkg_resources.VersionConflict: (kombu 2.2.1 (/home/yuri/virtualenvs/localshop/lib/python2.7/site-packages), Requirement.parse('kombu>=2.1.8,<2.2.0'))

Some package is requiring kombu < 2.2.0, but 2.2.1 gets installed on pip install localshop.

Scripted Install

Hello,

Thank you for providing such a useful app. I have a question about installation that I haven't been able to find an answer to. Hopefully it isn't out of line to ask here.

I would like to create a chef cookbook to automatically install/configure localshop. I'm not sure how to get around the prompts during 'localshop init' for username, email, and password though. Is there a way these values could be passed in from a configuration file or something so the installation can be fully scripted? Thanks.

Mirroring packages doesn't work?

Hi,

I've downloaded and installed Localshop.

I then add 127.0.0.1 to the list of allowed IPs.

I then start it up via Gunicorn:

(temp)victorhooi@mjolner:~/code/localshop$ localshop run_gunicorn -b 0.0.0.0:8000
2013-03-14 10:59:53 [30230] [INFO] Starting gunicorn 0.17.1
Starting gunicorn 0.17.1
2013-03-14 10:59:53 [30230] [INFO] Listening at: http://0.0.0.0:8000 (30230)
Listening at: http://0.0.0.0:8000 (30230)
2013-03-14 10:59:53 [30230] [INFO] Using worker: sync
Using worker: sync
2013-03-14 10:59:53 [30239] [INFO] Booting worker with pid: 30239
Booting worker with pid: 30239

I startup the Celery daemon as well:

(temp)victorhooi@mjolner:~/code/localshop$ localshop celeryd -B -E

 -------------- celery@mjolner v3.0.12 (Chiastic Slide)
---- **** -----
--- * ***  * -- [Configuration]
-- * - **** --- . broker:      django://localhost//
- ** ---------- . app:         default:0x2532590 (djcelery.loaders.DjangoLoader)
- ** ---------- . concurrency: 8 (processes)
- ** ---------- . events:      ON
- ** ----------
- *** --- * --- [Queues]
-- ******* ---- . celery:      exchange:celery(direct) binding:celery
--- ***** -----

[2013-03-14 10:43:41,536: WARNING/MainProcess] celery@mjolner ready.

I then try to install yolk via this new server:

(temp2)victorhooi@mjolner:~/code/localshop$ pip install -i http://localhost:8000/simple/ yolk
Downloading/unpacking yolk
User for localhost:8000: victorhooi
Password:
  Cannot fetch index base URL http://localhost:8000/simple/
  Could not find any downloads that satisfy the requirement yolk
No distributions at all found for yolk
Storing complete log in /tmp/tmpaOR9jo

It complains about not being able to find http://localhost:8000/simple - however, I can navigate to this fine (after I type in my username and password - which incidentally can be quite annoying at times and would be cool if there was the option to turn it off).

The contents of /tmp/tmpaOR9jo:

------------------------------------------------------------
/home/victorhooi/.venvs/temp2/bin/pip run on Thu Mar 14 20:51:00 2013
Downloading/unpacking yolk

  Getting page http://localhost:8000/simple/yolk/
  Could not fetch URL http://localhost:8000/simple/yolk/: timed out
  Will skip URL http://localhost:8000/simple/yolk/ when looking for download links for yolk
  Getting page http://localhost:8000/simple/
  Could not fetch URL http://localhost:8000/simple/: timed out
  Will skip URL http://localhost:8000/simple/ when looking for download links for yolk
  Cannot fetch index base URL http://localhost:8000/simple/

  URLs to search for versions for yolk:
  * http://localhost:8000/simple/yolk/
  Getting page http://localhost:8000/simple/yolk/
  Could not fetch URL http://localhost:8000/simple/yolk/: timed out
  Will skip URL http://localhost:8000/simple/yolk/ when looking for download links for yolk
  Could not find any downloads that satisfy the requirement yolk

No distributions at all found for yolk

Exception information:
Traceback (most recent call last):
  File "/home/victorhooi/.venvs/temp2/local/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg/pip/basecommand.py", line 139, in main
    status = self.run(options, args)
  File "/home/victorhooi/.venvs/temp2/local/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg/pip/commands/install.py", line 266, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/home/victorhooi/.venvs/temp2/local/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg/pip/req.py", line 1026, in prepare_files
    url = finder.find_requirement(req_to_install, upgrade=self.upgrade)
  File "/home/victorhooi/.venvs/temp2/local/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg/pip/index.py", line 171, in find_requirement
    raise DistributionNotFound('No distributions at all found for %s' % req)
DistributionNotFound: No distributions at all found for yolk

The Celery console doesn't show anything new.

Any thoughts on what's going on here?

Cheers,
Victor

Include upload/install documention in localshop

Since there are a few steps involved with setting up to upload a local package to localshop or install using pip/easy_install, it would be nice if there was documentation (or a link to documentation) available in the localshop web interface.

Localshop Frontpage Text Personalization

Suggestion:

add 2 new variables to conf.py (and mantain defaults with the strings currently available hardcoded inside the template):

• FRONTPAGE_TITLE
• FRONTPAGE_SUBTITLE

And use those on frontpage.html, here:

Localshop

<p>Welcome to your localshop!</p>

We have 2 localshop instalations, and its a good idea to identify which one is targeted for testing, and which one is production.

gunicorn won't start

setup in a virtualenv and installed from pip

http started with pid 22377
2012-03-08 21:13:25 [22377] [INFO] Starting gunicorn 0.14.1
2012-03-08 21:13:25 [22377] [INFO] Listening at: http://127.0.0.1:8900 (22377)
2012-03-08 21:13:25 [22377] [INFO] Using worker: sync
2012-03-08 21:13:25 [22380] [INFO] Booting worker with pid: 22380
2012-03-08 21:13:26 [22380] [ERROR] Exception in worker process:
Traceback (most recent call last):
File "/opt/localshop/lib/python2.6/site-packages/gunicorn/arbiter.py", line 453, in spawn_worker
worker.init_process()
File "/opt/localshop/lib/python2.6/site-packages/gunicorn/workers/base.py", line 99, in init_process
self.wsgi = self.app.wsgi()
File "/opt/localshop/lib/python2.6/site-packages/gunicorn/app/base.py", line 101, in wsgi
self.callable = self.load()
File "/opt/localshop/lib/python2.6/site-packages/localshop/services/http.py", line 32, in load
self.activate_translation()
AttributeError: 'LocalShopApplication' object has no attribute 'activate_translation'
2012-03-08 21:13:26 [22380] [INFO] Worker exiting (pid: 22380)
2012-03-08 21:13:26 [22377] [INFO] Shutting down: Master
2012-03-08 21:13:26 [22377] [INFO] Reason: Worker failed to boot.

Optional basic/digest permission check next to CIDR checks

I wonder if you'd be interested for a basic/digest auth based permission checks that could either be added on-top of the CIDR checks or alternatively (although the same could be achieve with a 0.0.0.0/0 CIDR).

Since pip supports basic auth for indexes automatically this could easily be used, too. Would you be interested in a patch?

Ability to delete packages?

Hi,

Is it possible to expose functionality through the Localshop webinterface that would let you remove packages?

I assume currently you can just delete the files, then remove all of the entries from the DB manually - however, it would be cleaner if this could be done via the Localshop web interface.

Cheers,
Victor

issue with ~ (tilde) symbol when mirroring from files hosted in home directories

The dulwich package url is http://samba.org/~jelmer/dulwich/dulwich-0.8.7.tar.gz but the tilde symbol gets converted to '%7e' and makes it fail.

Downloading/unpacking dulwich
  Getting page http://media.co.goldfish:8000/simple/dulwich
  URLs to search for versions for dulwich:
  * http://media.co.goldfish:8000/simple/dulwich/
  Getting page http://samba.org/%7ejelmer/dulwich
  Could not fetch URL http://samba.org/%7ejelmer/dulwich/dulwich-0.8.7.tar.gz (from http://media.co.goldfish:8000/simple/dulwich/): HTTP Error 301: The HTTP server returned a redirect error that would lead to an infinite loop.
The last 30x error message was:
Moved Permanently
  Will skip URL http://samba.org/%7ejelmer/dulwich/dulwich-0.8.7.tar.gz when looking for download links for dulwich

localshop fails to install on Python 3.x

Due to the fact that the latest released version uses Pillow<2.0.0 the installation fails.
If installed from source the installation fails due to django-userna.

Cannot fetch packages if name does not match

When I try to install packages that are known by different names, only one works.

For instance, line_profiler is known by the name line-profiler (and installs itself as that), but appears in the localshop list as line_profiler. Trying to install from a frozen pip requirements file fails, as it tries to install line-profiler instead of line_profiler.

You should do the same type of matching that pypi does to find alternate names.

Unable to install packages via Localshop?

Hi,

I've setup localshop (develop) branch on a machine, and I'm having issues installing packages through it.

I've started up the Gunicorn server:

localshop run_gunicorn -b 0.0.0.0:8000
Notice: Unable to load configuration file /home/vichoo/.localshop.py (No such file or directory), using default settings


/home/vichoo/.virtualenvs/localshop/lib/python2.6/site-packages/Django-1.4.2-py2.6.egg/django/conf/__init__.py:110: DeprecationWarning: The SECRET_KEY setting must not be empty.
  warnings.warn("The SECRET_KEY setting must not be empty.", DeprecationWarning)
2013-01-04 01:52:53 [16124] [INFO] Starting gunicorn 0.14.6
Starting gunicorn 0.14.6
2013-01-04 01:52:53 [16124] [INFO] Listening at: http://0.0.0.0:8000 (16124)
Listening at: http://0.0.0.0:8000 (16124)
2013-01-04 01:52:53 [16124] [INFO] Using worker: sync
Using worker: sync
2013-01-04 01:52:53 [16133] [INFO] Booting worker with pid: 16133
Booting worker with pid: 16133
/home/vichoo/.virtualenvs/localshop/lib/python2.6/site-packages/Django-1.4.2-py2.6.egg/django/template/defaulttags.py:1235: DeprecationWarning: The syntax for the url template tag is changing. Load the `url` tag from the `future` tag library to start using the new behavior.
  category=DeprecationWarning)
Not Found: /favicon.ico

And I've started up the celery job:

localshop celeryd -B -E -P solo
Notice: Unable to load configuration file /home/vichoo/.localshop.py (No such file or directory), using default settings


/home/vichoo/.virtualenvs/localshop/lib/python2.6/site-packages/Django-1.4.2-py2.6.egg/django/conf/__init__.py:110: DeprecationWarning: The SECRET_KEY setting must not be empty.
  warnings.warn("The SECRET_KEY setting must not be empty.", DeprecationWarning)

 -------------- celery@hostname v3.0.12 (Chiastic Slide)
---- **** -----
--- * ***  * -- [Configuration]
-- * - **** --- . broker:      django://localhost//
- ** ---------- . app:         default:0x1ee7d350 (djcelery.loaders.DjangoLoader)
- ** ---------- . concurrency: 1 (solo)
- ** ---------- . events:      ON
- ** ----------
- *** --- * --- [Queues]
-- ******* ---- . celery:      exchange:celery(direct) binding:celery
--- ***** -----

[2013-01-04 01:57:40,880: WARNING/MainProcess] celery@hostname ready.

I tried using a CIDR to provision my IP address, however, that still seemed to give me permission errors when I tried to retrieve the package listing:

> wget http://hostname:8000/simple/
--12:17:33--  http://hostname:8000/simple/
Resolving hostname... 10.68.129.37
Connecting to hostname|10.68.129.37|:8000... connected.
HTTP request sent, awaiting response... 401 UNAUTHORIZED
Authorization failed.

I then generated an access key, and seemed to have more luck with that:

> wget http://385cfd889b274a9ea3fb737fb4919d7e:a34ba2d73f32468f8b481290c3855d4e@hostname:8000/simple
--12:26:36--  http://385cfd889b274a9ea3fb737fb4919d7e:*password*@hostname:8000/simple
Resolving hostname... 10.68.129.37
Connecting to hostname|10.68.129.37|:8000... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: `simple.1'

[ <=>                                                                                                                  ] 69          --.-K/s   in 0s

12:26:36 (16.5 MB/s) - `simple.1' saved [69]

However, Pip is still having issues installing packages - it doesn't seem to be able to retrieve the /simple/ package listing, which is weird:

> pip install -i http://385cfd889b274a9ea3fb737fb4919d7e:a34ba2d73f32468f8b481290c3855d4e@hostname:8000/simple/ yolk
Downloading/unpacking yolk 
Cannot fetch index base URL http://385cfd889b274a9ea3fb737fb4919d7e:a34ba2d73f32468f8b481290c3855d4e@hostname:8000/simple/
Could not find any downloads that satisfy the requirement yolk
No distributions at all found for yolk
Storing complete log in /home/vichoo/.pip/pip.log

Also, on the Gunicorn console, I see:

2013-01-04 03:15:31 [17534] [CRITICAL] WORKER TIMEOUT (pid:17557)
WORKER TIMEOUT (pid:17557)  

Cheers,
Victor

PIP expects trailing slash in http://host:port/simple - Also unable to download packages

Hi,

I've setup localshop, and have managed to upload a package to it.

However, when I try to use pip on it, I get an error:

(localshop)[vichoo@dev_desktop_vm yolk]$ pip install -i http://localhost:8000/simple yolk
Downloading/unpacking yolk
  Cannot fetch index base URL http://localhost:8000/simple/
  Could not find any downloads that satisfy the requirement yolk
No distributions at all found for yolk
Storing complete log in /home/vichoo/.pip/pip.log

It seems in this case that PIP is trying http://localhost:8000/simple/, with a trailing slash after simple - even though I haven't given a trailing slash in the command-line.

Trying it with a trailing slash - pip install -i http://localhost:8000/simple/ yolk obviously doesn't work either.

Localshop works with /simple, but not /simple/ - /simple/ gives a Django 404 error:

I hacked the localshop/urls.py file (https://github.com/mvantellingen/localshop/blob/develop/localshop/urls.py) to add a trailing slash.

I.e.:

    url(r'^simple', include('localshop.apps.packages.urls_simple',

changed to:

    url(r'^simple/', include('localshop.apps.packages.urls_simple',

In this case, I can retrieve http://localhost:8000/simple/ now, and also pip is able to as well. So I seem to be one step closer.

However, I'm still not able to download packages:

(testlocalshop)[vichoo@dev_desktop_vm localshop]$ pip install -i http://localhost:8000/simple yolk
Downloading/unpacking yolk
  Real name of requirement yolk is yolk
  Could not find any downloads that satisfy the requirement yolk
No distributions at all found for yolk
Storing complete log in /home/vichoo/.pip/pip.log

Contents of my ~/.pip/pip.log:

------------------------------------------------------------
/home/vichoo/.virtualenvs/testlocalshop/bin/pip run on Wed Nov  7 16:13:08 2012
Downloading/unpacking yolk

  Getting page http://localhost:8000/simple/yolk
  Could not fetch URL http://localhost:8000/simple/yolk: HTTP Error 404: NOT FOUND
  Will skip URL http://localhost:8000/simple/yolk when looking for download links for yolk
  Getting page http://localhost:8000/simple/
  Real name of requirement yolk is yolk

  URLs to search for versions for yolk:
  * http://localhost:8000/simple/yolk/
  Getting page http://localhost:8000/simple/yolk/
  Could not fetch URL http://localhost:8000/simple/yolk/: HTTP Error 404: NOT FOUND
  Will skip URL http://localhost:8000/simple/yolk/ when looking for download links for yolk
  Could not find any downloads that satisfy the requirement yolk

No distributions at all found for yolk

Exception information:
Traceback (most recent call last):
  File "/home/vichoo/.virtualenvs/testlocalshop/lib/python2.7/site-packages/pip-1.2.1-py2.7.egg/pip/basecommand.py", line 107, in main
    status = self.run(options, args)
  File "/home/vichoo/.virtualenvs/testlocalshop/lib/python2.7/site-packages/pip-1.2.1-py2.7.egg/pip/commands/install.py", line 256, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/home/vichoo/.virtualenvs/testlocalshop/lib/python2.7/site-packages/pip-1.2.1-py2.7.egg/pip/req.py", line 1011, in prepare_files
    url = finder.find_requirement(req_to_install, upgrade=self.upgrade)
  File "/home/vichoo/.virtualenvs/testlocalshop/lib/python2.7/site-packages/pip-1.2.1-py2.7.egg/pip/index.py", line 157, in find_requirement
    raise DistributionNotFound('No distributions at all found for %s' % req)
DistributionNotFound: No distributions at all found for yolk

Any thoughts? Is anybody else able to get localshop working?

Cheers,
Victor

Hability to check if a package exists

Is there any way to check if localshop already has a package?
Like the way we can do with pypi.python.org...

Example:

https://pypi.python.org/pypi/surrealism/0.6.0 (ok, and tells there 0.7.0 is the latest)
https://pypi.python.org/pypi/surrealism/0.7.0 (ok, the latest)
https://pypi.python.org/pypi/surrealism/0.7.1 (HTTP 404)

This way I can use a script to determine if I can upload a newer package or not. If there is a match, I gave the user an error.

If there is not such feature, I can help develop it.

Lovato

Decouple the apps from the project

Localshop looks very interesting. I want to replace django-chishop and am not very happy with the direction it's fork (Django PyPi https://github.com/benliles/djangopypi) has taken.

My issue with localshop is that it doesn't seem to be designed to be hooked into a custom Django project. It seems that's it's a standalone Django project that needs to be used as is.

Am I correct? Are you planning on extracting the core functionality from the rest of the project in the future?

python-dev requirement

This should probably have been obvious to me but python-dev is required to install this. Might want to add that to the documentation. About to try it out!

Upgrade to the latest version of Django

Or at least to the latest patch version of 1.4.x.

Recent releases should show releasing in descending order

I would expect the "recent releases" list to show the most recent release first. This is not currently the case in 0.2.2.

Security

I love the security features provided by localshop.

But, is there a way of turning off separate security features, like:

• allowing users to download libraries even if their ip addresses are not added to the list of CIDRs
• only uploads needing authentication and not downloads?

Regards,
Radha.

Uploading packages doesn't work

Hi,

I'm trying to follow the instructions in the README to upload a package:

python setup.py upload -r local

I use python setup.py sdist, then try that, however, I get an error:

(localshop)[vichoo@dev_desktop_vm yolk]$ python setup.py upload -r local
running upload
error: No dist file created in earlier command

If I add the sdist command to use the source distribution, it seems to work:

(localshop)[vichoo@dev_desktop_vm yolk]$ python setup.py sdist upload -r local
running sdist
running egg_info
writing requirements to yolk.egg-info/requires.txt
writing yolk.egg-info/PKG-INFO
writing top-level names to yolk.egg-info/top_level.txt
writing dependency_links to yolk.egg-info/dependency_links.txt
writing entry points to yolk.egg-info/entry_points.txt
reading manifest file 'yolk.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.txt' under directory 'tests'
warning: no files found matching '*.conf' under directory 'docs'
warning: no files found matching '*.css' under directory 'docs'
warning: no files found matching '*.css_t' under directory 'docs'
warning: no files found matching 'indexsidebar.html' under directory 'docs'
warning: no files found matching 'tests/test_cli.py'
writing manifest file 'yolk.egg-info/SOURCES.txt'
creating yolk-0.4.3
creating yolk-0.4.3/docs
creating yolk-0.4.3/examples
creating yolk-0.4.3/examples/plugins
creating yolk-0.4.3/examples/plugins/yolk_pkg_manager
creating yolk-0.4.3/examples/plugins/yolk_portage
creating yolk-0.4.3/tests
creating yolk-0.4.3/yolk
creating yolk-0.4.3/yolk.egg-info
creating yolk-0.4.3/yolk/plugins
making hard links in yolk-0.4.3...
hard linking AUTHORS -> yolk-0.4.3
hard linking COPYING -> yolk-0.4.3
hard linking CREDITS -> yolk-0.4.3
hard linking ChangeLog -> yolk-0.4.3
hard linking FAQ -> yolk-0.4.3
hard linking INSTALL -> yolk-0.4.3
hard linking MANIFEST.in -> yolk-0.4.3
hard linking NEWS -> yolk-0.4.3
hard linking README -> yolk-0.4.3
hard linking README.rst -> yolk-0.4.3
hard linking THANKS -> yolk-0.4.3
hard linking TODO -> yolk-0.4.3
hard linking setup.py -> yolk-0.4.3
hard linking docs/Makefile -> yolk-0.4.3/docs
hard linking docs/conf.py -> yolk-0.4.3/docs
hard linking docs/tests.txt -> yolk-0.4.3/docs
hard linking docs/yolk-logo.png -> yolk-0.4.3/docs
hard linking docs/yolk.1 -> yolk-0.4.3/docs
hard linking examples/plugins/README -> yolk-0.4.3/examples/plugins
hard linking examples/plugins/yolk_pkg_manager/README -> yolk-0.4.3/examples/plugins/yolk_pkg_manager
hard linking examples/plugins/yolk_pkg_manager/setup.py -> yolk-0.4.3/examples/plugins/yolk_pkg_manager
hard linking examples/plugins/yolk_pkg_manager/yolk_acme.py -> yolk-0.4.3/examples/plugins/yolk_pkg_manager
hard linking examples/plugins/yolk_portage/README -> yolk-0.4.3/examples/plugins/yolk_portage
hard linking examples/plugins/yolk_portage/setup.py -> yolk-0.4.3/examples/plugins/yolk_portage
hard linking examples/plugins/yolk_portage/yolk_portage.py -> yolk-0.4.3/examples/plugins/yolk_portage
hard linking tests/rss_feed.py -> yolk-0.4.3/tests
hard linking tests/test_yolk_cli.py -> yolk-0.4.3/tests
hard linking yolk/__init__.py -> yolk-0.4.3/yolk
hard linking yolk/cli.py -> yolk-0.4.3/yolk
hard linking yolk/metadata.py -> yolk-0.4.3/yolk
hard linking yolk/pypi.py -> yolk-0.4.3/yolk
hard linking yolk/setuptools_support.py -> yolk-0.4.3/yolk
hard linking yolk/utils.py -> yolk-0.4.3/yolk
hard linking yolk/yolklib.py -> yolk-0.4.3/yolk
hard linking yolk.egg-info/PKG-INFO -> yolk-0.4.3/yolk.egg-info
hard linking yolk.egg-info/SOURCES.txt -> yolk-0.4.3/yolk.egg-info
hard linking yolk.egg-info/dependency_links.txt -> yolk-0.4.3/yolk.egg-info
hard linking yolk.egg-info/entry_points.txt -> yolk-0.4.3/yolk.egg-info
hard linking yolk.egg-info/requires.txt -> yolk-0.4.3/yolk.egg-info
hard linking yolk.egg-info/top_level.txt -> yolk-0.4.3/yolk.egg-info
hard linking yolk/plugins/__init__.py -> yolk-0.4.3/yolk/plugins
hard linking yolk/plugins/base.py -> yolk-0.4.3/yolk/plugins
Writing yolk-0.4.3/setup.cfg
Creating tar archive
removing 'yolk-0.4.3' (and everything under it)
running upload
Submitting dist/yolk-0.4.3.tar.gz to http://localhost:8000/simple/
error: None

Could we add a working command to the README, as well as an explanation of what works and what doesn't? (I'm sure many of us out there are not experts in Python packaging, myself included...lol).

Cheers,
Victor

Set up instructions

Hi there,

I am following your setup instructions, and it seems to not be correct.

And then start it via:
localshop run_gunicorn
localshop celeryd -B -E <-- The following does not work.

Push latest code to Github repo

Seems like the stuff released on PyPI isn't the current version on Github, would you mind pushing the changes? I'd like to help fix a few issues..

Swap home-made login/logout and profile handling

There are a few things not optimal, e.g. the login view is susceptible for the security issue we just fixed in Django.

Also there are a bunch of user views that don't serve a purpose right now, which I think is just an oversight or note-yet-done feature.

I'd like to suggest to switch to django-userena as it has a nice user management, also handles profiles and is well tested. If you'd be okay with that I'd be happy to do the grunt work.

Sections do not show up for restructured text on "Readme" tab

Auth System Connector

How complex is to bypass local auth system (CIDR need to be yet considered), but instead asking to local auth, ask to an external module?
It would be terrific if I can auth against my corporate LDAP server.

I have a python module to do that. So from my side would be to envelope it with some webservice wrapper, for instance. And at the configuration, I would be able to specify on Localshop this webservice URL, that need to implement a certain contract.

Best
Marco

userena breaks ldap authentication

Because userena and localshop use a different set of login parameters then the default this breaks thirdparty backends like django_auth_ldap.

The workaround is some additional code like this:

from django_auth_ldap.backend import LDAPBackend as Orig

class LDAPBackend(Orig):
    def authenticate(self, username=None, password=None, identification=None):
        if identification and not username:
            username = identification
        return super(LDAPBackend, self).authenticate(username, password)

But it's kind of silly that users have to jump through all these hoops for a django app that should just-work(tm).

AuthProfile is not automatically created, /accounts/username/edit/ raises 404

It seems that AuthProfile is not automatically created when a user is created.

Because of this /accounts/username/edit/ raises 404

Workaround:

def create_auth_profile(sender, **kwargs):
    user = kwargs['instance']
    if kwargs['created']:
        AuthProfile(user=user).save()

post_save.connect(create_auth_profile, sender=User, dispatch_uid="create_auth_profile")

Error Running localshop upgrade - django.db.utils.DatabaseError: near "*": syntax error

Hi,

I'm installing LocalShop on RHEL 5 with Python 2.6.

It seems to install fine, and I've run localshop init.

However, when I try to create the database and run the migrations:

./localshop upgrade
Syncing...
Creating tables ...
Installing custom SQL ...
Installing indexes ...
Installed 0 object(s) from 0 fixture(s)

Synced:
> django.contrib.auth
> django.contrib.contenttypes
> django.contrib.sessions
> django.contrib.sites
> django.contrib.messages
> django.contrib.staticfiles
> django.contrib.admin
> djkombu
> south

Not synced (use migrations):
- djcelery
- userena
- guardian
- localshop.apps.packages
- localshop.apps.permissions
(use ./manage.py migrate to migrate these)
Running migrations for djcelery:
- Migrating forwards to 0004_v30_changes.
> djcelery:0003_v26_changes
FATAL ERROR - The following SQL query failed: CREATE TABLE "_south_new_djcelery_crontabschedule" ("hour" varchar(64) NOT NULL, "day_of_month" varchar(64) NOT NULL DEFAULT *, "day_of_week" varchar(64) NOT NULL, "month_of_year" varchar(64) NOT NULL DEFAULT '*', "id" integer PRIMARY KEY, "minute" varchar(64) NOT NULL)
The error was: near "*": syntax error
! Error found during real run of migration! Aborting.

! Since you have a database that does not support running
! schema-altering statements in transactions, we have had 
 ! to leave it in an interim state between migrations.

! You *might* be able to recover with:
! The South developers regret this has happened, and would
! like to gently persuade you to consider a slightly
! easier-to-deal-with DBMS (one that supports DDL transactions)
! NOTE: The error which caused the migration to fail is further up.
Error in migration: djcelery:0003_v26_changes
Traceback (most recent call last):
  File "./localshop", line 9, in <module>
    load_entry_point('localshop==0.3', 'console_scripts', 'localshop')()
  File "/u01/pypi/localshop/lib/python2.6/site-packages/localshop-0.3-py2.6.egg/localshop/runner.py", line 53, in main
    settings_envvar='LOCALSHOP_CONF',
  File "/u01/pypi/localshop/lib/python2.6/site-packages/logan-0.5.0-py2.6.egg/logan/runner.py", line 135, in run_app
    management.execute_from_command_line([runner_name, command] + command_args)
  File "/u01/pypi/localshop/lib/python2.6/site-packages/Django-1.4.1-py2.6.egg/django/core/management/__init__.py", line 443, in execute_from_command_line
    utility.execute()
  File "/u01/pypi/localshop/lib/python2.6/site-packages/Django-1.4.1-py2.6.egg/django/core/management/__init__.py", line 382, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/u01/pypi/localshop/lib/python2.6/site-packages/Django-1.4.1-py2.6.egg/django/core/management/base.py", line 196, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/u01/pypi/localshop/lib/python2.6/site-packages/Django-1.4.1-py2.6.egg/django/core/management/base.py", line 232, in execute
    output = self.handle(*args, **options)
  File "/u01/pypi/localshop/lib/python2.6/site-packages/localshop-0.3-py2.6.egg/localshop/management/commands/upgrade.py", line 13, in handle
    delete_ghosts=True)
  File "/u01/pypi/localshop/lib/python2.6/site-packages/Django-1.4.1-py2.6.egg/django/core/management/__init__.py", line 150, in call_command
    return klass.execute(*args, **defaults)
  File "/u01/pypi/localshop/lib/python2.6/site-packages/Django-1.4.1-py2.6.egg/django/core/management/base.py", line 232, in execute
    output = self.handle(*args, **options)
  File "/u01/pypi/localshop/lib/python2.6/site-packages/south/management/commands/migrate.py", line 108, in handle
    ignore_ghosts = ignore_ghosts,
  File "/u01/pypi/localshop/lib/python2.6/site-packages/south/migration/__init__.py", line 213, in migrate_app
    success = migrator.migrate_many(target, workplan, database)
  File "/u01/pypi/localshop/lib/python2.6/site-packages/south/migration/migrators.py", line 235, in migrate_many
    result = migrator.__class__.migrate_many(migrator, target, migrations, database)
  File "/u01/pypi/localshop/lib/python2.6/site-packages/south/migration/migrators.py", line 310, in migrate_many
    result = self.migrate(migration, database)
  File "/u01/pypi/localshop/lib/python2.6/site-packages/south/migration/migrators.py", line 133, in migrate
    result = self.run(migration)
  File "/u01/pypi/localshop/lib/python2.6/site-packages/south/migration/migrators.py", line 107, in run
    return self.run_migration(migration)
  File "/u01/pypi/localshop/lib/python2.6/site-packages/south/migration/migrators.py", line 81, in run_migration
    migration_function()
  File "/u01/pypi/localshop/lib/python2.6/site-packages/south/migration/migrators.py", line 57, in <lambda>
    return (lambda: direction(orm))
  File "/u01/pypi/localshop/lib/python2.6/site-packages/django_celery-3.0.9-py2.6.egg/djcelery/migrations/0003_v26_changes.py", line 16, in forwards
    db.add_column('djcelery_crontabschedule', 'month_of_year', self.gf('django.db.models.fields.CharField')(default='*', max_length=64), keep_default=False)
  File "/u01/pypi/localshop/lib/python2.6/site-packages/south/db/sqlite3.py", line 31, in add_column
    field.column: self._column_sql_for_create(table_name, name, field, False),
  File "/u01/pypi/localshop/lib/python2.6/site-packages/south/db/generic.py", line 44, in _cache_clear
    return func(self, table, *args, **opts)
  File "/u01/pypi/localshop/lib/python2.6/site-packages/south/db/sqlite3.py", line 103, in _remake_table
    ", ".join(["%s %s" % (self.quote_name(cname), ctype) for cname, ctype in definitions.items()]),
  File "/u01/pypi/localshop/lib/python2.6/site-packages/south/db/generic.py", line 273, in execute
    cursor.execute(sql, params)
  File "/u01/pypi/localshop/lib/python2.6/site-packages/Django-1.4.1-py2.6.egg/django/db/backends/util.py", line 40, in execute
    return self.cursor.execute(sql, params)
  File "/u01/pypi/localshop/lib/python2.6/site-packages/Django-1.4.1-py2.6.egg/django/db/backends/sqlite3/base.py", line 337, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.DatabaseError: near "*": syntax error

Any thoughts?

Cheers,
Victor

Add wheel support for uploaded packages

https://crate.io/packages/wheel

Can't be installed on Windows

Hi there,

I was trying to install localshop on my Windows box using Python 2.7 and pip^but it failed with python-daemon which expects the resource package, only available on Unix.

http://docs.python.org/library/resource.html#module-resource

Is it a goald to support Windows or not at all at some point?

Thanks.

UI User error

Hi,
if i click on the link "$username -> Edit details" I created I get:

Not Found: /accounts/admin/edit/

On $username -> change password I get:

"Internal Server Error: /accounts/admin/password/
Traceback (most recent call last):
  File "/home/pypi/v_localshop/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 111, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
  File "/home/pypi/v_localshop/local/lib/python2.7/site-packages/userena/decorators.py", line 28, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/home/pypi/v_localshop/local/lib/python2.7/site-packages/guardian/decorators.py", line 103, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/home/pypi/v_localshop/local/lib/python2.7/site-packages/userena/views.py", line 508, in password_change
    extra_context['profile'] = user.get_profile()
  File "/home/pypi/v_localshop/local/lib/python2.7/site-packages/django/contrib/auth/models.py", line 400, in get_profile
    self._state.db).get(user__id__exact=self.id)
  File "/home/pypi/v_localshop/local/lib/python2.7/site-packages/django/db/models/query.py", line 366, in get
    % self.model._meta.object_name)
DoesNotExist: AuthProfile matching query does not exist.
Not Found: /accounts/admin/password/bootstrap/css/bootstrap.css
Not Found: /accounts/admin/password/bootstrap/css/bootstrap-responsive.css
Not Found: /accounts/admin/password/jquery-1.7.1.min.js
Not Found: /accounts/admin/password/bootstrap/js/bootstrap.js
Not Found: /accounts/admin/password/bootstrap/js/bootstrap.js

No access to initial installation on remote server

I installed localshop on an EC2 machine and started it using localshop run_gunicorn and it seems to start OK. I can curl http://localhost:8080 when I am SSHed to the machine. However, I can't access the admin interface from my machine, presumably because by default only localhost is authorized to access it.

Is there any way to enable access from external machines without needing to use the web interface?

...sort of a chicken and egg type problem.

Support for PyPI JSON API?

i.e. http://wiki.python.org/moin/PyPIJSON
e.g. http://pypi.python.org/pypi/BeautifulSoup/json

Page Not Found - http://localhost:8000/accounts/signin/?next=/

Hi,

I previously installed localshop from PyPI (version 0.3), and hit the issue with Celery (Issue #21).

Anyhow, I've just tried installing localshop from Git, and ran localshop init and localshop upgrade. I then tried starting it up via Gunicorn, and seem to hit some issue with getting a 404 on http://localhost:8000/accounts/signin/?next=/:

Page not found (404)
Request Method:     GET
Request URL:    http://localhost:8000/accounts/signin/?next=/

At the console:

(localshop)[vichoo@dev_desktop_vm localshop]$ localshop run_gunicorn
2012-10-10 01:03:39 [22315] [INFO] Starting gunicorn 0.14.6
Starting gunicorn 0.14.6
2012-10-10 01:03:39 [22315] [INFO] Listening at: http://127.0.0.1:8000 (22315)
Listening at: http://127.0.0.1:8000 (22315)
2012-10-10 01:03:39 [22315] [INFO] Using worker: sync
Using worker: sync
2012-10-10 01:03:39 [22324] [INFO] Booting worker with pid: 22324
Booting worker with pid: 22324
Not Found: /accounts/signin/
Not Found: /accounts/signin/

Any ideas/thoughts?

Cheers,
Victor

Easier fallback to PyPi with redirects

There is a much easier way to fallback to PyPi by using HTTP 302 redirects. See http://www.rackspace.com/blog/racker-hacker-create-a-local-pypi-repository-using-only-mod_rewrite/ for details.

Use pip to download packages instead of custom method

Downloading missing package using pip seems the most failsafe way instead of implementing our own methods

Installing packages from localshop fails with md5 error

I have a localshop instance running on http://192.168.1.20/, however when i try to install packages from It i get the following error:

$ pip install -ihttp://192.168.1.20/simple/ coverage
Downloading/unpacking coverage
  Downloading coverage-3.5.2.tar.gz (unknown size): 0bytes downloaded
  MD5 hash of the package http://192.168.1.20/packages/coverage/download/97/coverage-3.5.2.tar.gz#md5=28c43d41b13f8987ea14d7b1d4a4e3ec (from http://192.168.1.20/simple/coverage/) (d41d8cd98f00b204e9800998ecf8427e) doesn't match the expected hash 28c43d41b13f8987ea14d7b1d4a4e3ec!
Bad MD5 hash for package http://192.168.1.20/packages/coverage/download/97/coverage-3.5.2.tar.gz#md5=28c43d41b13f8987ea14d7b1d4a4e3ec (from http://192.168.1.20/simple/coverage/)
Storing complete log in /home/armonge/.pip/pip.log

According to the celery logs the package had previously been successfully downloaded

[2012-08-24 09:01:53,581: INFO/MainProcess] Got task from broker: localshop.apps.packages.tasks.download_file[e854d68d-812a-4172-8783-c6c65844357a]
[2012-08-24 09:01:53,627: DEBUG/MainProcess] Mediator: Running callback for task: localshop.apps.packages.tasks.download_file[e854d68d-812a-4172-8783-c6c65844357a]
[2012-08-24 09:01:53,628: DEBUG/MainProcess] TaskPool: Apply <function execute_and_trace at 0x9be8a3c> (args:('localshop.apps.packages.tasks.download_file', 'e854d68d-812a-4172-8783-c6c65844357
a', [], {'pk': 97}) kwargs:{'hostname': 'localhost.localdomain', 'request': {'retries': 0, 'task': 'localshop.apps.packages.tasks.download_file', 'utc': False, 'loglevel': 10, 'delivery_info': 
{'routing_key': u'celery', 'exchange': u'celery'}, 'args': [], 'expires': None, 'is_eager': False, 'eta': None, 'hostname': 'localhost.localdomain', 'kwargs': {'pk': 97}, 'logfile': None, 'id':
 'e854d68d-812a-4172-8783-c6c65844357a'}})
[2012-08-24 09:01:53,628: DEBUG/MainProcess] Task accepted: localshop.apps.packages.tasks.download_file[e854d68d-812a-4172-8783-c6c65844357a] pid:10546
[2012-08-24 09:01:53,735: INFO/PoolWorker-1] Downloading http://pypi.python.org/packages/source/c/coverage/coverage-3.5.2.tar.gz
[2012-08-24 09:01:53,737: INFO/PoolWorker-1] Starting new HTTP connection (1): pypi.python.org
[2012-08-24 09:01:54,046: DEBUG/PoolWorker-1] "GET /packages/source/c/coverage/coverage-3.5.2.tar.gz HTTP/1.1" 200 115497
[2012-08-24 09:01:57,436: INFO/PoolWorker-1] Complete
[2012-08-24 09:01:57,529: INFO/MainProcess] Task localshop.apps.packages.tasks.download_file[e854d68d-812a-4172-8783-c6c65844357a] succeeded in 3.90104198456s: None

The md5 of the file in cache seems to be:

md5sum .localshop/files/source/c/coverage/coverage-3.5.2.tar.gz 
d41d8cd98f00b204e9800998ecf8427e  .localshop/files/source/c/coverage/coverage-3.5.2.tar.gz

And more interesting is that the file seems to be empy

du -h .localshop/files/source/c/coverage/coverage-3.5.2.tar.gz
0   .localshop/files/source/c/coverage/coverage-3.5.2.tar.gz

For the record I am using the following software versions

Django          - 1.4.1        - active 
Python          - 2.7.3        - active development (/usr/lib/python2.7/lib-dynload)
South           - 0.7.6        - active 
amqplib         - 1.0.2        - active 
anyjson         - 0.3.3        - active 
celery          - 2.5.5        - active 
distribute      - 0.6.24       - active 
django-celery   - 2.4.2        - active 
django-kombu    - 0.9.4        - active 
django-model-utils - 1.1.0        - active 
django-picklefield - 0.2.1        - active 
docutils        - 0.8.1        - active 
gunicorn        - 0.14.6       - active 
kombu           - 2.1.8        - active 
localshop       - 0.3          - active 
lockfile        - 0.9.1        - active 
logan           - 0.3.1        - active 
netaddr         - 0.7.6        - active 
pip             - 1.1          - active 
python-daemon   - 1.6          - active 
python-dateutil - 1.5          - active 
requests        - 0.13.8       - active 
wsgiref         - 0.1.2        - active development (/usr/lib/python2.7)
yolk            - 0.4.3        - active 

Also I'd be willing to lend all the help i can, provided some direction

Celery Issues - AttributeError: 'NoneType' object has no attribute 'conf'

Hi,

I've downloaded the latest version of localshop from trunk (which uses django-celery==3.0.9) and when I try to run the localshop celeryd -B -E command I get:

(localshop)[vichoo@dev_desktop_vm localshop]$ localshop celeryd -B -E
Traceback (most recent call last):
  File "/home/vichoo/.virtualenvs/localshop/bin/localshop", line 9, in <module>
    load_entry_point('localshop==0.3', 'console_scripts', 'localshop')()
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/localshop-0.3-py2.7.egg/localshop/runner.py", line 53, in main
    settings_envvar='LOCALSHOP_CONF',
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/logan-0.5.0-py2.7.egg/logan/runner.py", line 135, in run_app
    management.execute_from_command_line([runner_name, command] + command_args)
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/Django-1.4.1-py2.7.egg/django/core/management/__init__.py", line 443, in execute_from_command_line
    utility.execute()
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/Django-1.4.1-py2.7.egg/django/core/management/__init__.py", line 382, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/Django-1.4.1-py2.7.egg/django/core/management/__init__.py", line 261, in fetch_command
    klass = load_command_class(app_name, subcommand)
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/Django-1.4.1-py2.7.egg/django/core/management/__init__.py", line 69, in load_command_class
    module = import_module('%s.management.commands.%s' % (app_name, name))
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/Django-1.4.1-py2.7.egg/django/utils/importlib.py", line 35, in import_module
    __import__(name)
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/django_celery-3.0.9-py2.7.egg/djcelery/management/commands/celeryd.py", line 16, in <module>
    class Command(CeleryCommand):
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/django_celery-3.0.9-py2.7.egg/djcelery/management/commands/celeryd.py", line 22, in Command
    + worker.preload_options)
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/celery-3.0.12-py2.7.egg/celery/bin/celeryd.py", line 161, in get_options
    conf = self.app.conf
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/celery-3.0.12-py2.7.egg/celery/local.py", line 169, in __getattr__
    return getattr(self._get_current_object(), name)
AttributeError: 'NoneType' object has no attribute 'conf'

This appears to be fixed in newer builds of django-celery:

celery/django-celery#185
celery/celery#995

Anyhow, I upgraded to django-celery==3.0.11 (which is the latest version from Pypi), and I get:

(localshop)[vichoo@dev_desktop_vm localshop]$ localshop celeryd -B -E

 -------------- celery@dev_desktop_vm.aus.optiver.com v3.0.12 (Chiastic Slide)
---- **** ----- 
--- * ***  * -- [Configuration]
-- * - **** --- . broker:      django://localhost//
- ** ---------- . app:         default:0xd951490 (djcelery.loaders.DjangoLoader)
- ** ---------- . concurrency: 1 (processes)
- ** ---------- . events:      ON
- ** ---------- 
- *** --- * --- [Queues]
-- ******* ---- . celery:      exchange:celery(direct) binding:celery
--- ***** ----- 

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.18-py2.7-linux-x86_64.egg/billiard/forking.py", line 502, in main
    prepare(preparation_data)
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.18-py2.7-linux-x86_64.egg/billiard/forking.py", line 642, in prepare
    file, path_name, etc = imp.find_module(main_name, dirs)
ImportError: No module named localshop
[2012-11-07 05:04:48,738: WARNING/MainProcess] /home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/djcelery/loaders.py:132: UserWarning: Using settings.DEBUG leads to a memory leak, never use this setting in production environments!
  warnings.warn("Using settings.DEBUG leads to a memory leak, never "
[2012-11-07 05:04:48,739: WARNING/MainProcess] celery@dev_desktop_vm.aus.optiver.com ready.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.18-py2.7-linux-x86_64.egg/billiard/forking.py", line 502, in main
    prepare(preparation_data)
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.18-py2.7-linux-x86_64.egg/billiard/forking.py", line 642, in prepare
    file, path_name, etc = imp.find_module(main_name, dirs)
ImportError: No module named localshop
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.18-py2.7-linux-x86_64.egg/billiard/forking.py", line 502, in main
    prepare(preparation_data)
  File "/home/vichoo/.virtualenvs/localshop/lib/python2.7/site-packages/billiard-2.7.3.18-py2.7-linux-x86_64.egg/billiard/forking.py", line 642, in prepare
    file, path_name, etc = imp.find_module(main_name, dirs)
ImportError: No module named localshop

That last ImportError seems to repeat indefinitely. It seems similar to this issue:

#21

but I don't know if it's related?

Is there any way of getting localshop to work with the latest django-celery (3.0.11)?

Cheers,
Victor

Turn off username/password checking?

Hi,

Is there a way to disable the authentication for downloading/uploading packages?

I assume for the CIDR, you could just add all ranges, which achieves the same thing.

However, it still seems to do username/password checking, and I can't find a way around that.

This is for a local deployment inside an intranet - so making it seamless and easy for users helps.

Also, this means that the example provided in the README could never work:

pip install -i http://localhost:8900/simple/ localshop

since this will always fail due to the lack of username/password.

Cheers,
Victor

Package mirroring does not work

We have a localshop running which we are using and relying mainly on the the pypi mirroring which sadly does not work.

The server is not running for a proxy and has direct access to the Internet.

What we are seeing in the localshop logs is that we are getting a GET request which is set as Queuing, but we then get a CRITICAL log message that the worker has timed out. I check the celeryd logs which is running in DEBUG and I don't see anything happening there. Once I restart the celery daemon, everything starts downloading.

Could you please let me know what we need to do to resolve this annoying issue.

Duplicate Package local package / pypi server

Hi,
it would be nice to have the opportunity to upload the same package version which exists on pypi to the localshop instance. Currently there is an error message:
~/v_localshop/local/lib/python2.7/site-packages/django/db/models/query.py", line 368, in get
% (self.model._meta.object_name, num, kwargs))
MultipleObjectsReturned: get() returned more than one Package -- it returned 2! Lookup parameters were {'name__iexact': u'pychecker'}

Can't localshop just handle local packages as a precedence prior pypi packages?

Regards,
Jonas

Running localshop start doesn't work with gunicorn >= 0.14

Console output:

env:localshop $ localshop start
http started with pid 83883
2012-05-25 10:22:06 [83883] [INFO] Starting gunicorn 0.14.3
2012-05-25 10:22:06 [83883] [INFO] Listening at: http://127.0.0.1:8000 (83883)
2012-05-25 10:22:06 [83883] [INFO] Using worker: sync
2012-05-25 10:22:06 [83884] [INFO] Booting worker with pid: 83884
2012-05-25 10:22:07 [83884] [INFO] Worker exiting (pid: 83884)
2012-05-25 10:22:07 [83883] [INFO] Shutting down: Master
2012-05-25 10:22:07 [83883] [INFO] Reason: Worker failed to boot.
env:localshop $