django-request/django-request

django-request is a statistics module for django. It stores requests in a database for admins to see, it can also be used to get statistics on who is online etc.

django-request

django-request is a statistics module for django. It stores requests in a database for admins to see, it can also be used to get statistics on who is online etc.

As well as a site statistics module, with the active_users template tag and manager method you can also use django-request to show who is online in a certain time:

Request.objects.active_users(minutes=15)

To find the request overview page, please click on Requests inside the admin, then “Overview” on the top right, next to “add request”.

Requirements

• Python: 3.6, 3.7, 3.8, 3.9, 3.10, 3.11, 3.12
• Django: 3.2, 4.0, 4.1, 4.2, 5.0
• python-dateutil

django-request 1.5.1 is the last version that supports Django 1.4, 1.5, 1.6.

django-request 1.5.4 is the last version that supports Django 1.7, 1.8, 1.9.

django-request 1.5.5 is the last version that supports Django 1.10.

django-request 1.5.6 is the last version that supports Django 1.11, 2.0, 2.1, 3.0, 3.1, and Python 2.7 and 3.4.

django-request 1.6.3 is the last version that supports Django 2.2, 4.0, and 4.1.

Installation

• Put 'request' in your INSTALLED_APPS setting.
• Run the command manage.py migrate.
• Add request.middleware.RequestMiddleware to MIDDLEWARE. If you use django.contrib.auth.middleware.AuthenticationMiddleware, place the RequestMiddleware after it. If you use django.contrib.flatpages.middleware.FlatpageFallbackMiddleware place request.middleware.RequestMiddleware before it else flatpages will be marked as error pages in the admin panel.
• Add REQUEST_BASE_URL to your settings with the base URL of your site (e.g. https://www.my.site/). This is used to calculate unique visitors and top referrers. REQUEST_BASE_URL defaults to 'http://%s' % Site.objects.get_current().domain.

Detailed documentation

For a detailed documentation of django-request, or how to install django-request please see: django-request or the docs/ directory.

django-request's Issues

To create a template tag to show the last active users in x amount of minutes

This can be done with a group by query, and filtering time to gte x amount of time ago. The tag should tag a argument, and return a list of users. Such as:

{% active_users in 5 minutes as active_user_list %}
{% active_users as active_user_list %}

Django 1.4 admin urls needs to end with a /

In request/admin.py, at the line 57:

url(r'^overview/traffic.json$', wrap(self.traffic), name='%s_%s_traffic' % info),

the pattern needs to be overview/traffic.json/ otherwise django throw a 404 because
it's searching for a primary key.

Not working with django 1.9.1

"request_request" table is not created on creating & applying migrations.

ModelAdmin for Request is created, which on clicking gives -

ProgrammingError at /admin/request/request/
(1146, "Table 'appname.request_request' doesn't exist")

cannot import name update_wrapper

It fails in 1.6 with the following error.

cannot import name update_wrapper

Django 1.7 compatibility

Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute
    django.setup()
  File "/usr/local/lib/python2.7/site-packages/django/__init__.py", line 21, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/local/lib/python2.7/site-packages/django/apps/registry.py", line 108, in populate
    app_config.import_models(all_models)
  File "/usr/local/lib/python2.7/site-packages/django/apps/config.py", line 197, in import_models
    self.models_module = import_module(models_module_name)
  File "/usr/local/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/local/lib/python2.7/site-packages/request/models.py", line 13, in <module>
    User = get_user_model()
  File "/usr/local/lib/python2.7/site-packages/django/contrib/auth/__init__.py", line 136, in get_user_model
    return django_apps.get_model(settings.AUTH_USER_MODEL)
  File "/usr/local/lib/python2.7/site-packages/django/apps/registry.py", line 199, in get_model
    self.check_models_ready()
  File "/usr/local/lib/python2.7/site-packages/django/apps/registry.py", line 131, in check_models_ready
    raise AppRegistryNotReady("Models aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Models aren't loaded yet.

Remove the dependency on external js

Remove the dependency on external js, such as google for jquery.

If USE_THOUSAND_SEPARATOR is True, URL must be unlocalized

If USE_THOUSAND_SEPARATOR in project settings.py is True, all URLs that contain a number > 1000 are localized adding a thousands separator.
This happens when the whole project need to be localized for different countries and the ID of a record is above 1000.

URL must be unlocalized using the template tag "unlocalize" (https://docs.djangoproject.com/en/1.6/topics/i18n/formatting/#unlocalize)

See for example the latestrequests.html template, line 15: the {{ request.path }} must be rewritten in {{ request.path|unlocalize }}.

Problem importing (using Django 1.6.1)

Could not import these both modules (using Django 1.6.1 and python 2.7.3 ) in admin.py:

from django.conf.urls.defaults import patterns, url
from django.utils.functional import update_wrapper

My solution was to replace both lines with:

from django.conf.urls import patterns, url
from functools import update_wrapper

add tag 3.1 to repository

Please update version in packages. Add tag to repo pls. Because in packages now version does not work with python 3.0 and django 1.8.

Change the resolution of the traffic graph

When you click to view the whole year, it would be much quicker to load do 12 separate queries for each month. Currently it will do 365 separate queries for each day to get a count. Thats 365 * amount of traffic modules. Which is a awful lot of queries.

migration package is not included in setup.py

This will prevent any migrations from being applied if you install as a python package

referer URLField length issue

Kyle,

Another length issue popped up.

the referrer field on the request model is a URLField which creates a column of type varying(200)

This raises an exception, "DataError: value too long for type character varying(200)"

null value in column "ip" violates not-null constraint

For some reason my IP address isn't picked up on the request and it's throwing an error. For now I have turned off IP Logging. Maybe think about a default on the required field in-case an IP is not found for some reason?

Create a command to remove old requests

Create a command to remove old requests

For example:

django-admin.py purgerequest 3 months

This command will delete all requests which are over three months old.

TypeError at /admin/request/request/overview/

Hi There

I installed django-requests today and got it working fine, that was until I tried to add REQUEST_IGNORE_PATHS to my setting it would bomb out with the warning below. I am using Django 1.6.10 and Python 2.7.8.

I traced this back to the file in question and the paths are now being past in a unicode, not str, so this was causing the problems.

If I place this in my settings.py file:

REQUEST_IGNORE_PATHS = (
    r'^admin/$',
    r'^admin/request/(\.*)+',
)

and then in my local router.py, just inside the for loop, line 20, I add a print line:

print(pattern, pattern__class__)

I get this output to my screen:

(u'^admin/$', <type 'unicode'>)
(u'^admin/request/(\\.*)+', <type 'unicode'>)

So based on this I fixed my local copy by changing line 21 in router.py from:

if pattern.__class__ == str:

to:

if pattern.__class__ == str or pattern.__class__ == unicode:

I don't know if this is the correct fix, but it works and correctly ignores the right paths.

Traceback Switch to copy-and-paste view

    ../lib/python2.7/site-packages/django/core/handlers/base.py in get_response

                        response = middleware_method(request, response)

        ...
    ▶ Local vars
    ../lib/python2.7/site-packages/request/middleware.py in process_response

                ignore = patterns(False, *settings.REQUEST_IGNORE_PATHS)

        ...
    ▶ Local vars
    ../lib/python2.7/site-packages/request/router.py in __init__

                        self.patterns.append(RegexPattern(*pattern))

Create a active users overview panel item.

opera and in production

this app has been very useful for finding 404s, but i'm experiencing two problems.

running the django development server (1.2.3, postgresql backend),
with django-request installed, when i load my site in Opera, i get:

DatabaseError: value too long for type character varying(25)

doesn't seem to produce the same thing with FF, Safari or Chrome.
strange.

also, not having realized the above, i moved my code into production with django-request active and most of the time, when people were clicking links from google, they were experiencing 500 errors in Chrome and Opera, but not an issue in FF. but only when clicking on the links ... if they refreshed, the page showed up fine.

i've since removed django-request and the problem is resolved.
sorry for the less-than-scientific explanations above
thanks for your help

Strange - I don't see the overview button..

Hi there,

Two sites; both with virtually identical settings files (difference is database mysql/sqlite). One site (sqlite) works no problem. The other site (mysql) does not. I see request under the admin. But there isn't an overview button.

Any help would be appreciated.

Thanks

Create a setting to exclude path's on a site (ex. /admin/*)

Create a setting to exclude path's on a site, such excluding the admin from being logged. Perhaps use regex to match the urls, like in urlconf.

Include the current day in the traffic graph

Include the current day in the traffic graph

jQuery is a 404

When overview admin uses jQuery from googlecode, it cannot find it, "http://jqueryjs.googlecode.com/files/jquery-1.4.2.min.js" is a 404.

Host field in overview

I really missing host field inside overview.
Since we have a web pages with several subdomains (each user profile have one), from current overview i can not see if someone goes to subdomain.example.com as first request (path will show "/").

Now i improvise, and watch further requests referral link.

Locally i have basic implementation of this feature, need to add few things and it is ready for pull request if you are interested.

But i have one question since i never had pull req like that.
I can't achieve that after update, the migrate will add new field. All i can do is drop whole table and create new one with migration, but this is terrible approach (for losing data). I've modified init migration file and create new migration file, but with no success. Any ideas how to properly create new field without dropping current table?

Language column length too short

I had users from Germany using Firefox were getting app exceptions. I checked the apache log:

DataError: value too long for type character varying(25)

This occured on:

File "/foo/django-request/request/models.py", line 62, in from_http_request
[error] self.save()

So the language column looks to be too short. Need to bump it up to something acceptable.

Method length too small?

Since the method-field is only a varchar(7), which fits perfectly for HTTP methods, it crashes on e.g. PROPFIND and PROPPATCH if some genious tries to accces your site with webdav.

This may not be a valid use case for django, but it raises an error 500 if someone does so. On one hand side, raising "false" errors in general should not happen, but this may also be a light vulnerability to the server.

fixed a bug where if i was already overriding extra_head in a previously included template it requests overview wouldnt work

fixed a bug where if i was already overriding extra_head in a previously included template it requests overview wouldnt work

AaronGhent@8b3d52b

i cant push due to an already pending pull request :)

How to enable search with Haystack?

Hi there,

My Searches shows empty. I'm wondering if this has something to do with my using Haystack with Solr. How are you gathering this data and do you have any thoughts on enabling search with Haystack?

Improve the management tool's to be able to delete requests based upon paths.

TypeError in utils.py

It seems like the specific code hasn't been pushed yet to Github (but is in the released 0.2.2), so excuse the patch-less ticket. See the traceback at http://dpaste.de/SmYs/ for details. It's just a missing comma on line 138.

Log GET/POST parameters

Don't include admin pages into stats

Requests entries and overview would be more readable if there'd be some way not to include admin paths into the analysis.

Improve the admin interface to include graphs, charts, etc for the requests.

Also show top referrer's, as well as top pages people visit, with detail statistics on each page on the site showing how the majority of users got there (from another page, search engine, etc).

Filtering by user__username not allowed

Since Django 1.2.4 filters that are not included in list_filter can't be used for securities sake. Clicking on a username fails since then with the exception:

SuspiciousOperation at /admin/request/request/
Filtering by user__username not allowed

But one can override the admin class:

def lookup_allowed(self, key):
if key == 'user__username':
return True
return super(RequestAdmin, self).lookup_allowed(key)

python 3.4 support, old format exceptions

https://i.imgur.com/yWVluCv.png
/request/traffic.py
/request/plugins.py
/request/management/commands/purgerequests.py

today should belong to this_week

the current date 2016-04-27 is not considered part of this_week

class RequestQuerySetTest(TestCase):
    def setUp(self):
        user = User.objects.create(username='foo')
        self.request = Request.objects.create(user=user, ip='1.2.3.4')

    def test_this_week_today(self):
        # setUp
        self.request.save()
        # Test
        qs = Request.objects.all().this_week()
        self.assertEqual(1, qs.count())

results in:

======================================================================
FAIL: test_this_week_today (tests.test_managers.RequestQuerySetTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/depaolim/Projects/django-request/tests/test_managers.py", line 186, in test_this_week_today
    self.assertEqual(1, qs.count())
AssertionError: 1 != 0

Turn the traffic graph into an overview plugin

URLField argument deprecated since Django 1.5

In the Request model object, the referer field is a URLField with an argument verify_exists set to False.
Since Django 1.5, this argument is deprecated.

It makes the install of django-request fail:

    $ ./manage.py syncdb
    TypeError: __init__() got an unexpected keyword argument 'verify_exists'

Django 1.5 - Support custom user models

Django 1.5 supports custom User models. Currently, syncdb breaks when syncing django-request models if a custom AUTH_USER_MODEL is defined in a projects settings.

See: https://docs.djangoproject.com/en/1.5/topics/auth/customizing/#referencing-the-user-model

Load the traffic graph separately

Load the traffic graph separately to the overview page via javascript to improve initial page load time, and also be able to change the scale.

Make django-request overview perform better

language field too short

The language field from the Request model is too short in some cases. It currently has a maximum length of 25.

It is filled with
self.language = request.META.get('HTTP_ACCEPT_LANGUAGE', '')
which, in my case, has for value : "fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3"

It of course raises a database error such as : 2010-06-27 15:13:39 CEST ERROR: value too long for type character varying(25)

So, either strip the HTTP_ACCEPT_LANGUAGE down or set a greater length for the language field.

Thank you !

ValueError at /admin/request/request/overview/

invalid literal for int() with base 10: ''
request\traffic.py in graph, line 48

%s is not valid format argument of strftime in next line:

return [{'data':[(int(day.strftime("%s"))*1000, module.count(qs)) for day, qs in days], 'label':ugettext(module.verbose_name_plural)} for module in self.modules]

Security issue with request overview

There are no permission checking on the request overview page.

Mixed content warning when overview page is served over HTTPS

django-request always uses an HTTP URL for Google Charts, which leads to the browser marking the page as insecure due to mixed content.

Fixing this ought to be a simple as changing
'http://chart.apis.google.com/chart?cht=p3&chd=t:%s&chs=%sx%s&chl=%s'
to
'//chart.apis.google.com/chart?cht=p3&chd=t:%s&chs=%sx%s&chl=%s'
in request_admin.py.

However, the google.com certificate doesn't include that sub-domain. To get around that, you can use the following instead:

'//chart.googleapis.com/chart?cht=p3&chd=t:%s&chs=%sx%s&chl=%s'

Add response_time field to Request

It would be interesting to log the elapsed time during the request. I'm using this code in my project to do so. I could later send the PR if it ends up been usefull

Python < 2.5 incompatibility with active_users template tag.

Python 2.5 handles dictionaries differently so

tokens = ['30', 'minutes']
{str(tokens.pop(0)): int(tokens.pop(0))}

Would pop them in the correct order, but other versions of python would pop them in opposite order.

Local host IP is shown (127.0.0.1) instead of real remote IP while using proxy

Hello,
I have found an issue trying to deploy the django request in a production-like environment. I have a simple Nginx + gunicorn configuration to process the remote requests.

Nginx is configured to properly set the header X-Real-IP with the remote_addr of the request, and it works properly (at least with django core and my apps).

With such environment, the IP shown does not take into account the proxy indirection, so we have 'localhost' shown in the statistics.

I have solved the issue patching the source code and using ipware package to retrieve the real IP (when possible).

request/models.py

+ from ipware.ip import get_ip, get_real_ip
class Request(models.Model):
[...]
- self.ip = request.META.get('REMOTE_ADDR', '')
+ self.ip = get_real_ip(request) if not None else ''

What do you think about the issue and the workaround/solution?

Static contex processor inclusion (missing in doc)

The context processor for serving static files
'django.core.context_processors.static'

must be included, otherwise the graph
http://mydomain/admin/request/request/overview/
won't be shown (as the static files of django-request won't be served properly):

There is no mention in the official doc about this, it would probably help to mention this for easing the installation...

TemplateSyntaxError at /admin/request/request/overview/

While trying to look at the overview page, I got this error.

Extra information:

Environment:

Request Method: GET
Request URL: http://localhost:8000/admin/request/request/overview/
Django Version: 1.1 rc 1 SVN-11332
Python Version: 2.5.2
Installed Applications:
['django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'request',
'django.contrib.admin',
'django.contrib.markup',
'django.contrib.comments',
'django.contrib.flatpages',
'lithium.blog',
'krisje8.projects',
'ipban',
'gravatar']
Installed Middleware:
('django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.flatpages.middleware.FlatpageFallbackMiddleware',
'request.middleware.RequestMiddleware',
'ipban.middleware.IPBanMiddleware')


Template error:
In template /root/python/request/templates/admin/request/overview.html, error at line 5
Caught an exception while rendering: Reverse for 'request_jquery_js' with arguments '()' and keyword arguments '{}' not found.
1 : {% extends "admin/base_site.html" %}


2 : {% load i18n %}


3 : 


4 : {% block extrahead %}


5 : <script language="javascript" type="text/javascript" src=" {% url request_jquery_js %} "></script>


6 : <script language="javascript" type="text/javascript" src="{% url request_jquery_flot_js %}"></script>


7 : <!--[if IE]><script src="{% url request_jquery_excanvas_flot_js %}" type="text/javascript"></script><![endif]-->


8 : 


9 : <script type="text/javascript">


10 : $(document).ready(function(){


11 :     $.plot($("#trafficgraph"), {{ traffic_graph|safe }}, { xaxis: { mode: "time" }} );


12 : });


13 : </script>


14 : {% endblock %}


15 : 


Traceback:
File "/root/python/django/core/handlers/base.py" in get_response
92.                 response = callback(request, *callback_args, **callback_kwargs)
File "/root/python/request/views.py" in overview
53.         'requests_url': '/admin/request/request/'
File "/root/python/django/shortcuts/__init__.py" in render_to_response
20.     return HttpResponse(loader.render_to_string(*args, **kwargs), **httpresponse_kwargs)
File "/root/python/django/template/loader.py" in render_to_string
108.     return t.render(context_instance)
File "/root/python/django/template/__init__.py" in render
178.         return self.nodelist.render(context)
File "/root/python/django/template/__init__.py" in render
779.                 bits.append(self.render_node(node, context))
File "/root/python/django/template/debug.py" in render_node
71.             result = node.render(context)
File "/root/python/django/template/loader_tags.py" in render
97.         return compiled_parent.render(context)
File "/root/python/django/template/__init__.py" in render
178.         return self.nodelist.render(context)
File "/root/python/django/template/__init__.py" in render
779.                 bits.append(self.render_node(node, context))
File "/root/python/django/template/debug.py" in render_node
71.             result = node.render(context)
File "/root/python/django/template/loader_tags.py" in render
97.         return compiled_parent.render(context)
File "/root/python/django/template/__init__.py" in render
178.         return self.nodelist.render(context)
File "/root/python/django/template/__init__.py" in render
779.                 bits.append(self.render_node(node, context))
File "/root/python/django/template/debug.py" in render_node
71.             result = node.render(context)
File "/root/python/django/template/loader_tags.py" in render
24.         result = self.nodelist.render(context)
File "/root/python/django/template/__init__.py" in render
779.                 bits.append(self.render_node(node, context))
File "/root/python/django/template/debug.py" in render_node
81.             raise wrapped

Exception Type: TemplateSyntaxError at /admin/request/request/overview/
Exception Value: Caught an exception while rendering: Reverse for 'request_jquery_js' with arguments '()' and keyword arguments '{}' not found.

Original Traceback (most recent call last):
File "/root/python/django/template/debug.py", line 71, in render_node
    result = node.render(context)
File "/root/python/django/template/defaulttags.py", line 382, in render
    raise e
NoReverseMatch: Reverse for 'request_jquery_js' with arguments '()' and keyword arguments '{}' not found.

explodes on syncdb

if you are using django-request on an initial install it raises an exception on syncdb looking for the Sites table. My fork added a commit to handle this more gracefully.

Make the admin panel more modular

Make the admin panel more modular and support re-arranging (with ajax) and customizing the admin panel. And save the state on a per-admin.