yunojuno-archive/django-package-monitor

A Django app for keeping track of dependency updates

django-package-monitor's Issues

KeyError: 'HTTP_REFERER'

/package_monitor/views.py in reload:

Traceback (most recent call last):
  File "/Users/erik/env/cargo/lib/python2.7/site-packages/django/core/handlers/base.py", line 132, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/Users/erik/env/cargo/lib/python2.7/site-packages/django/contrib/auth/decorators.py", line 22, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/Users/erik/env/cargo/lib/python2.7/site-packages/package_monitor/views.py", line 14, in reload
    url = request.META['HTTP_REFERER']
KeyError: 'HTTP_REFERER'

Handle semantic_version.parse errors

If one package in the list of requirements cannot be parsed (see #10), then the local requirements cannot be loaded.

Steps to reproduce:

1. Add Unideocode==0.04.7 to requirements.txt
2. Open the admin site / package_updates
3. Navigate to Package_Monitor > Package versions
4. Click on "Reload local requirements"

-or-

1. Add Unideocode==0.04.7 to requirements.txt
2. Run ./manage.py refresh_packages --local

Expected result:

All local packages are loaded into the database.

Actual result:

An exception is raised by the semantic_version parse function:

ValueError: Invalid leading zero in minor: '0.04.17'

Invalid leading zero

First of all, thank you very much for the previous fix 👍

We I tried with my own project packages there was another error that I would like to share:

ValueError at /package_monitor/reload/
Invalid leading zero in minor: '0.04.17'

The version is related to Unicode==0.04.17

Is it something related to semantic? If so, warn me if I am on the right repo.

Kind regards,

Andre Carpinteiro

Suggestion: configurable warnings about unwanted licences

(Low priority)

In the spirit of https://github.com/onur/cargo-license/, it would be interesting to be able to mark out certain licences as 'unwanted' in an application stack - eg GPL

The barest implementation could be just to make the admin changelist sortable by licence type.

Next level up might be to support a list of licence names (strings, I guess) which are used to highlight relevant packages in the admin and/or allow filtering by 'unwanted' licences in the changelist sidebar.

A management command could also be handy.

Strip comments from requirement lines

Pip supports inline comments in the requirements file:

social-auth-core==1.3.0   # via social-auth-app-django
sqlparse==0.2.3           # via django-request-token

As we parse the file into individual requirements, we should strip out any such comments. We can't split purely on # as this is used for appending package names to pinned commits, e.g.:

-e git+git://github.com/jsocol/django-waffle.git@a2c41676e4f9be54dab2522b10dec09189245e74#egg=django_waffle

Proposal is to split on ' #'.

Update failure if PyPI json failure

If an installed package doesnt exist on PyPI, the refresh process fails.

I have a patch for this, and possibly a few other PyPI errors I encountered.

DB field length overflow problem

We just found a DataError: value too long for type character varying(200) when trying to view the package-monitor results in the Admin. We don't have the context/stack available but it might be related to a pinned commit (git -e ...) as a dependency

017-06-02T10:25:28.949518+00:00 app[web.1]: ERROR Internal Server Error: /package_monitor/reload/
2017-06-02T10:25:28.949531+00:00 app[web.1]: Traceback (most recent call last):
2017-06-02T10:25:28.949533+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/django/core/handlers/exception.py", line 42, in inner
2017-06-02T10:25:28.949533+00:00 app[web.1]:     response = get_response(request)
2017-06-02T10:25:28.949534+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response
2017-06-02T10:25:28.949535+00:00 app[web.1]:     response = self.process_exception_by_middleware(e, request)
2017-06-02T10:25:28.949535+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response
2017-06-02T10:25:28.949536+00:00 app[web.1]:     response = wrapped_callback(request, *callback_args, **callback_kwargs)
2017-06-02T10:25:28.949537+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/django/contrib/auth/decorators.py", line 23, in _wrapped_view
2017-06-02T10:25:28.949537+00:00 app[web.1]:     return view_func(request, *args, **kwargs)
2017-06-02T10:25:28.949538+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/package_monitor/views.py", line 12, in reload
2017-06-02T10:25:28.949539+00:00 app[web.1]:     refresh_packages.local()
2017-06-02T10:25:28.949540+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/package_monitor/management/commands/refresh_packages.py", line 36, in local
2017-06-02T10:25:28.949540+00:00 app[web.1]:     create_package_version(r)
2017-06-02T10:25:28.949542+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/package_monitor/management/commands/refresh_packages.py", line 23, in create_package_version
2017-06-02T10:25:28.949542+00:00 app[web.1]:     PackageVersion(requirement=requirement).save()
2017-06-02T10:25:28.949543+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/django/db/models/base.py", line 796, in save
2017-06-02T10:25:28.949543+00:00 app[web.1]:     force_update=force_update, update_fields=update_fields)
2017-06-02T10:25:28.949544+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/django/db/models/base.py", line 824, in save_base
2017-06-02T10:25:28.949545+00:00 app[web.1]:     updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
2017-06-02T10:25:28.949546+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/django/db/models/base.py", line 908, in _save_table
2017-06-02T10:25:28.949546+00:00 app[web.1]:     result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
2017-06-02T10:25:28.949547+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/django/db/models/base.py", line 947, in _do_insert
2017-06-02T10:25:28.949548+00:00 app[web.1]:     using=using, raw=raw)
2017-06-02T10:25:28.949548+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/django/db/models/manager.py", line 85, in manager_method
2017-06-02T10:25:28.949552+00:00 app[web.1]:     return getattr(self.get_queryset(), name)(*args, **kwargs)
2017-06-02T10:25:28.949553+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/django/db/models/query.py", line 1045, in _insert
2017-06-02T10:25:28.949553+00:00 app[web.1]:     return query.get_compiler(using=using).execute_sql(return_id)
2017-06-02T10:25:28.949554+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 1054, in execute_sql
2017-06-02T10:25:28.949554+00:00 app[web.1]:     cursor.execute(sql, params)
2017-06-02T10:25:28.949555+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/django/db/backends/utils.py", line 79, in execute
2017-06-02T10:25:28.949556+00:00 app[web.1]:     return super(CursorDebugWrapper, self).execute(sql, params)
2017-06-02T10:25:28.949556+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
2017-06-02T10:25:28.949557+00:00 app[web.1]:     return self.cursor.execute(sql, params)
2017-06-02T10:25:28.949557+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/django/db/utils.py", line 94, in __exit__
2017-06-02T10:25:28.949558+00:00 app[web.1]:     six.reraise(dj_exc_type, dj_exc_value, traceback)
2017-06-02T10:25:28.949559+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
2017-06-02T10:25:28.949559+00:00 app[web.1]:     return self.cursor.execute(sql, params)
2017-06-02T10:25:28.949560+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/opbeat/instrumentation/packages/dbapi2.py", line 169, in execute
2017-06-02T10:25:28.949561+00:00 app[web.1]:     return self._trace_sql(self.__wrapped__.execute, sql, params)
2017-06-02T10:25:28.949561+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/opbeat/instrumentation/packages/dbapi2.py", line 182, in _trace_sql
2017-06-02T10:25:28.949562+00:00 app[web.1]:     return method(sql, params)
2017-06-02T10:25:28.949562+00:00 app[web.1]: DataError: value too long for type character varying(200)

Update requirements to use official version python-semanticversion (2.5.0)

Now that this issue has been fixed - rbarrois/python-semanticversion#33 and a new version released to PyPI, the fork should be replaced in requirements. New version is 2.5.0.

/cc @borfast

django-package-monitor

First of all, good evening.
I tried your app to get the feeling of the package itself and I have:

package_monitor_packageversion.licence may not be NULL

when trying to update all the packages.

Feel free to ask me for more details.

Kind regards,

Andre Carpinteiro

IndexError in models.py

I added django-package-monitor to my requirements.txt and it fails on python manage.py refresh_packages --clean --remote --local command now:

File "/Users/erik/env/cargo/lib/python2.7/site-packages/package_monitor/management/commands/refresh_packages.py", line 81, in handle
    local()
  File "/Users/erik/env/cargo/lib/python2.7/site-packages/package_monitor/management/commands/refresh_packages.py", line 34, in local
    create_package_version(r)
  File "/Users/erik/env/cargo/lib/python2.7/site-packages/package_monitor/management/commands/refresh_packages.py", line 21, in create_package_version
    PackageVersion(requirement=requirement).save()
  File "/Users/erik/env/cargo/lib/python2.7/site-packages/package_monitor/models.py", line 95, in __init__
    self.current_version = Version.coerce(requirement.specs[0][1])
IndexError: list index out of range

Without django-package-monitor in the requirements.txt works fine.

Tested on version 0.2.3

Apps aren't loaded yet

I try to run Django project with your app, but every time I run server with package_monitor in INSTALLED_APPS this exception raises:

  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/Users/erik/env/cargo/lib/python2.7/site-packages/django/core/management/__init__.py", line 351, in execute_from_command_line
    utility.execute()
  File "/Users/erik/env/cargo/lib/python2.7/site-packages/django/core/management/__init__.py", line 343, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/erik/env/cargo/lib/python2.7/site-packages/django/core/management/__init__.py", line 177, in fetch_command
    commands = get_commands()
  File "/Users/erik/env/cargo/lib/python2.7/site-packages/django/utils/lru_cache.py", line 101, in wrapper
    result = user_function(*args, **kwds)
  File "/Users/erik/env/cargo/lib/python2.7/site-packages/django/core/management/__init__.py", line 72, in get_commands
    for app_config in reversed(list(apps.get_app_configs())):
  File "/Users/erik/env/cargo/lib/python2.7/site-packages/django/apps/registry.py", line 137, in get_app_configs
    self.check_apps_ready()
  File "/Users/erik/env/cargo/lib/python2.7/site-packages/django/apps/registry.py", line 124, in check_apps_ready
    raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

Tested on Django==1.8.5.

Update project to use poetry / poetry-template

Ensure compatible with Django 1.9/1.10

We should make sure this works with the latest versions of Django so that apps that rely on this can be safely upgraded.

TypeError in models.py

Result of python manage.py refresh_packages --clean --local --remote:

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/Users/erik/env/cargo/lib/python2.7/site-packages/django/core/management/__init__.py", line 351, in execute_from_command_line
    utility.execute()
  File "/Users/erik/env/cargo/lib/python2.7/site-packages/django/core/management/__init__.py", line 343, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/erik/env/cargo/lib/python2.7/site-packages/django/core/management/base.py", line 394, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/erik/env/cargo/lib/python2.7/site-packages/django/core/management/base.py", line 445, in execute
    output = self.handle(*args, **options)
  File "/Users/erik/env/cargo/lib/python2.7/site-packages/package_monitor/management/commands/refresh_packages.py", line 83, in handle
    remote()
  File "/Users/erik/env/cargo/lib/python2.7/site-packages/package_monitor/management/commands/refresh_packages.py", line 40, in remote
    pv.update_from_pypi()
  File "/Users/erik/env/cargo/lib/python2.7/site-packages/package_monitor/models.py", line 117, in update_from_pypi
    self.licence = package_licence(info)[:100]
TypeError: 'NoneType' object has no attribute '__getitem__'