mongoengine/django-mongoengine

django mongoengine integration

Django-MongoEngine

THIS IS UNSTABLE PROJECT, IF YOU WANT TO USE IT - FIX WHAT YOU NEED

Right now we're targeting to get things working on Django 4.2;

WARNING: This project is not in good state, and is likely to break with django updates. It's better to use raw mongoengine.

Working / Django 4.2

• [ok] sessions
• [ok] models/fields, fields needs testing
• [ok] views
• [ok] auth
• [?] admin - partially working, some things broken

Current status

Many parts of projects rewritten/removed; Instead of copying django code i try to subclass/reuse/even monkey-patch; Everything listed above is working; admin - just base fuctions like changelist/edit, not tested with every form type; need's more work.

Some code just plaholder to make things work; django/forms/document_options.py - dirty hack absolutely required to get thigs work with django. It replaces mongo _meta on model/class and provide django-like interface. It get's replaced after class creation via some metaclass magick.

Fields notes

• Project uses mongoengine style argument required=False, not django style blank=False, to be compatible with mongo-types. All your fields are optional by default.

TODO

• Sync some files/docs that removed from mongoengine: https://github.com/seglberg/mongoengine/commit/a34f4c1beb93f430c37da20c8fd96ce02a0f20c1?diff=unified
• Add docs for integrating: https://github.com/hmarr/django-debug-toolbar-mongo
• Take a look at django-mongotools: https://github.com/wpjunior/django-mongotools

Connecting

In your settings.py file, add following lines:

MONGODB_DATABASES = {
    "default": {
        "name": database_name,
        "host": database_host,
        "password": database_password,
        "username": database_user,
        "tz_aware": True, # if you using timezones in django (USE_TZ = True)
    },
}

INSTALLED_APPS += ["django_mongoengine"]

Documents

Inhherit your documents from django_mongoengine.Document, and define fields using django_mongoengine.fields.:

from django_mongoengine import Document, EmbeddedDocument, fields

class Comment(EmbeddedDocument):
    created_at = fields.DateTimeField(
        default=datetime.datetime.now, editable=False,
    )
    author = fields.StringField(verbose_name="Name", max_length=255)
    email  = fields.EmailField(verbose_name="Email")
    body = fields.StringField(verbose_name="Comment")

class Post(Document):
    created_at = fields.DateTimeField(
        default=datetime.datetime.now, editable=False,
    )
    title = fields.StringField(max_length=255)
    slug = fields.StringField(max_length=255, primary_key=True)
    comments = fields.ListField(
        fields.EmbeddedDocumentField('Comment'), required=False,
    )

Sessions

Django allows the use of different backend stores for its sessions. MongoEngine provides a MongoDB-based session backend for Django, which allows you to use sessions in your Django application with just MongoDB. To enable the MongoEngine session backend, ensure that your settings module has 'django.contrib.sessions.middleware.SessionMiddleware' in the MIDDLEWARE_CLASSES field and 'django.contrib.sessions' in your INSTALLED_APPS. From there, all you need to do is add the following line into your settings module:

SESSION_ENGINE = 'django_mongoengine.sessions'
SESSION_SERIALIZER = 'django_mongoengine.sessions.BSONSerializer'

Django provides session cookie, which expires after `SESSION_COOKIE_AGE seconds, but doesn't delete cookie at sessions backend, so 'mongoengine.django.sessions' supports `mongodb TTL <http://docs.mongodb.org/manual/tutorial/expire-data/>_.

Note

SESSION_SERIALIZER is only necessary in Django>1.6 as the default serializer is based around JSON and doesn't know how to convert bson.objectid.ObjectId instances to strings.

How to run example app

poetry install
poetry run pip install -r example/tumblelog/requirements.txt
poetry run python example/tumblelog/manage.py runserver

How to run tests

poetry install
poetry run python -m pytest

django-mongoengine's Issues

update "type", "size" fields of Document ERROR

in one of my Document, there is two fields "type" and "size", update it as follow:
body = {'type': 'test4'}
customers = Customer.objects(name='test').update_one(**body)

error raised:
customers = Customer.objects(name='test').update_one(*_body)
File "/home/evan/code/hp/RESMGMT/rm_project/.tox/py34/lib/python3.4/site-packages/mongoengine/queryset/base.py", line 516, in update_one
upsert=upsert, multi=False, write_concern=write_concern, *_update)
File "/home/evan/code/hp/RESMGMT/rm_project/.tox/py34/lib/python3.4/site-packages/mongoengine/queryset/base.py", line 450, in update
update = transform.update(queryset._document, **update)
File "/home/evan/code/hp/RESMGMT/rm_project/.tox/py34/lib/python3.4/site-packages/mongoengine/queryset/transform.py", line 246, in update
field = cleaned_fields[-1]
IndexError: list index out of range

and I tracee it, found that both "type" and "size" is COMPARISON_OPERATORS keyword in file /mongoengine/queryset/transform.py

how can I update "type" and "size" fields of my Document?

Using applications relying on django models

Any ideas for approaches on using apps relying on django models? I was thinking something along the lines of monkey patching the models & replacing the relevant parts (eg. TextField => StringField), but I noticed you've been re-implementing them in this project (eg. the django auth).

What lessons have you learned so far and what do you think the best approach might be?

MONGODB_DATABASES setting

While still being undocumented, I found that your code does not necessarily adhere to both the mongoengine API (see django_mongoengine.utils.module where you will call register_connection), and to the conventions set forth by django for the settings.py.

The first requires the name parameter to be a positional argument rather than a keyword argument and the second requires that all keys are in upper case while the mongoengine API requires all keyword arguments in their lower case form.

My workaround for getting your dev branch (BTW why can I not just fetch and checkout that branch but instead have to pull for testing things out?), is as follows:

In the for loop near the end of the init method of MongoEngine, I inserted the following code

        if 'NAME' not in conn_settings:
            raise ImproperlyConfigured("Missing `NAME` in %s connection settings" % (alias,))
        name = conn_settings['NAME']
        del conn_settings['NAME']
        effective_settings = {k.lower() : v for k, v in conn_settings.items()}
        self.connection.register_connection(alias, name, **effective_settings)

This got it up and running so far, without having tested any further except for getting the admin module up and running.

Which django version will be supported

There is few different between django versions. So whice django version this module will support？

So if you didn't mind, I want to do some contribution for this module.

EmbeddedField support

Hello everyone.

I have been working a couple of days on EmbeddedField suport in forms and now I get it working.
I think that it could be a very interesting feature to include in the library, so if you are interested I can post the patch here for testing and (maybe) be included in mainline.

Have a nice day.

Unable to save empty list on a EmbeddedDocumentListField

I have models declaration as below:

class Tag (EmbeddedDocument):
    tag_object: fields.StringField(required=True)
    tag_line: fields.StringField(required=True)

class TagEntry (Document):
    email = fields.EmailField(required=True)
    tags = fields.EmbeddedDocumentListField(Tag, required=False)

Whenever I try to do:

 tag_entry.tags = []
 tag_entry.save()

I get a ValidationError like below:

ValidationError (TagEntry:56ffa40ee72569659e273281) (Field is required and cannot be empty: ['tags'])

What is wrong here?

authentification doesn't work

Hi I'm started django-mongoengine and found that the authenticate function doesn't work.
Here is my registration function that actually work:

def register(request):
    if request.method == 'POST':
    form = UserForm(request.POST)
    if form.is_valid():
        username = form.cleaned_data['username']
        email = form.cleaned_data['email']
        password = form.cleaned_data['password']
        try:
            user = User.objects.get(email=email)
            emailUsed = True
        except User.DoesNotExist:
            user = User(username=username, email=email)
            user.set_password(password)
            user.save()

            reussi = True
else:
    form = UserForm()
return render(request, 'myapp/index.html', locals())

Then I paste the login code from the doc and it doesn't work (always return invalid password):

def connection(request):
  if request.method == 'POST':
    form = LoginForm(request.POST)
    if form.is_valid():
        email = form.cleaned_data['email']
        password = form.cleaned_data['password']

        user = authenticate(email=email, password=password)
        if user is not None:
            if user.is_active:
                login(request, user)
                msg = 'you are logged !'
            else:
                msg = 'your acount have been desactivate'
        else:
            msg = 'invalid password'
else:
    form = LoginForm()
return render(request, 'myapp/login.html', locals())

Here is my settings.py just in case:

import os
from mongoengine import *
BASE_DIR = os.path.dirname(os.path.dirname(__file__))




SECRET_KEY = 'g(ht#57%e&@5h=&2ge_ngzz66ji#m%d6q=ziwztnezimwjs--j'


DEBUG = True

TEMPLATE_DEBUG = True

ALLOWED_HOSTS = []




INSTALLED_APPS = (
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'mongoengine.django.mongo_auth',
    'myapp',
)

AUTH_USER_MODEL = 'mongo_auth.MongoUser'
MONGOENGINE_USER_DOCUMENT = 'mongoengine.django.auth.User'

MIDDLEWARE_CLASSES = (
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
)

ROOT_URLCONF = 'blog.urls'

WSGI_APPLICATION = 'blog.wsgi.application'


DATABASES = {
    'default': {
        'ENGINE': '',
    }
}

connect('tumblelog')

SESSION_ENGINE = 'mongoengine.django.sessions'
SESSION_SERIALIZER = 'mongoengine.django.sessions.BSONSerializer'
AUTHENTIFICATION_BACKENDS = (
    'mongoengine.django.auth.MongoEngineBackend',
)


TEMPLATE_DIRS = (
    os.path.join(BASE_DIR, 'templates'),
)



LANGUAGE_CODE = 'en-us'

TIME_ZONE = 'UTC'

USE_I18N = True

USE_L10N = True

USE_TZ = True


STATIC_URL = '/static/'

Session Issue

if user login at first time without the session key, the session data will be overwrite by the SessionStore::save self._get_session(no_load=must_create).

so the user must re-login to give the correct session data

How to login a user using django-mongo-auth module

code look like this

try:
username = request.POST['username']
print username
pwd = request.POST['pwd']
print pwd

   Please help to provide  rest of the code which authenticate user using 
             django_mongoengine.mongo_auth.models import User 


    return redirect('/')

except DoesNotExist:
return redirect('/')

Erroneous imports in admin/options module

I found out that when trying to activate your admin 'module' in my settings.py / INSTALLED_APPS, starting the embedded server will fail.

It turned out that one of the import statements referenced the wrong module for importing the LOOKUP_SEP constant.

Changing

from django.db.models.sql.constants import LOOKUP_SEP, QUERY_TERMS

to

from django.db.models.constants import LOOKUP_SEP
from django.db.models.sql.constants import QUERY_TERMS

made it actually compile and load so far.

Settings for mongodb replicaset.

According to the docs, this is how one should define MONGODB_DATABASES.

MONGODB_DATABASES = {
    "default": {
        "name": database_name,
        "host": database_host,
        "password": database_password,
        "username": database_user,
        "tz_aware": True, # if you using timezones in django (USE_TZ = True)
    },
}

How should this be done if I have a replicaset instead of standalone mongo?

Add Django GridFS storage support

Cannot run the unit test for user login code

Hello, Every one, I write a "user login" code, and now I write a unit test for it, But the unit test cannot works well!

I write the unit test by using the django-test-addons

This is the main part for my settings.py

import os

# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/1.9/howto/deployment/checklist/

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = 'xkw=^1((9ae7cv0+)%b%jrq7e$+1xy_=&cpwk!lc*=8hqp9sdp'

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True

ALLOWED_HOSTS = []


# Application definition

INSTALLED_APPS = [
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'django.contrib.admin',

    # django mongo engine
    'django_mongoengine',
    'django_mongoengine.mongo_auth',
    'django_mongoengine.mongo_admin.sites',
    'django_mongoengine.mongo_admin',

    # Myself
    'passage',
    'account',
]

MIDDLEWARE_CLASSES = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

ROOT_URLCONF = 'BlogV2BackEnd.urls'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

WSGI_APPLICATION = 'BlogV2BackEnd.wsgi.application'

# Set the Django-mongoengien
MONGODB_DATABASES = {
    'default': {
        'name': 'blog',
        'host': 'localhost',
    }
}

AUTH_USER_MODEL = 'mongo_auth.MongoUser'

AUTHENTICATION_BACKENDS = (
    'django_mongoengine.mongo_auth.backends.MongoEngineBackend',
)

SESSION_ENGINE = 'django_mongoengine.sessions'

TEST_MONGO_DATABASE = {
    'db': 'blog_test',
    'host': ['localhost'],
    'port': 27017
}

# Internationalization
# https://docs.djangoproject.com/en/1.9/topics/i18n/

LANGUAGE_CODE = 'en-us'

TIME_ZONE = 'UTC'

USE_I18N = True

USE_L10N = True

USE_TZ = True

SITE_ID = 1

This is my unittest code:

# Create your tests here.
from django.utils.encoding import force_text
from django_mongoengine.mongo_auth.models import User

import test_addons
import time

class AuthTest(test_addons.MongoTestCase):

    def test_can_login_with_correct_user(self):
        User.objects.create(username='xff', password='abc123')
        data = {'username': 'xff', 'password': 'abc123'}
        # time.sleep(40)
        response = self.client.post('/account/login/', data=data)
        self.assertJSONEqual(force_text(response.content), {'status': 'success'})

This is my login code

#urls.py

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

from django.conf.urls import url
from account import views

urlpatterns = [
    url(r'^login/$', views.login),
]

# views.py
from django.http import JsonResponse, Http404
from django.contrib.auth import authenticate, login as django_login

# Create your views here.

def login(request):
    if request.method == 'POST':
        username = request.POST.get('username')
        password = request.POST.get('password')
        user = authenticate(username=username, password=password)
        if user is not None:
            if user.is_active:
                django_login(request, user)
                response = {
                    'status': 'success'
                }
            else:
                response = {
                    'status': 'deny'
                }
        else:
            response = {
                'status': 'failed'
            }
        return JsonResponse(response)
    else:
        raise Http404

This is my test result:

➜ /home/michael/gitroom/BlogV2BackEnd (dev) ✗ [blog]$ ./manage.py test account
F
======================================================================
FAIL: test_can_login_with_correct_user (account.tests.AuthTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/michael/gitroom/BlogV2BackEnd/account/tests.py", line 15, in test_can_login_with_correct_user
    self.assertJSONEqual(force_text(response.content), {'status': 'success'})
  File "/home/michael/.virtualenvs/blog/lib/python3.4/site-packages/django/test/testcases.py", line 759, in assertJSONEqual
    self.assertEqual(data, expected_data, msg=msg)
AssertionError: {'status': 'failed'} != {'status': 'success'}
- {'status': 'failed'}
+ {'status': 'success'}

----------------------------------------------------------------------
Ran 1 test in 0.070s

FAILED (failures=1)

I insert a sleep statement in my unit test code, and find the blog_test db in my mongodb, and the user which I created in the unit test, this is the mongo shell output:

➜ /home/michael/.virtualenvs/blog/lib/python3.4/site-packages/django_mongoengine/mongo_auth $ mongo
MongoDB shell version: 2.4.9
connecting to: test
> show dbs;
blog    0.203125GB
blog_test   0.203125GB
django_mongoengine  0.203125GB
> use blog_test
switched to db blog_test
> db.user.find();
{ "_id" : ObjectId("57344281f34a283ce4ff46de"), "_cls" : "User", "username" : "xff", "password" : "abc123", "is_staff" : false, "is_active" : true, "is_superuser" : false, "last_login" : ISODate("2016-05-12T08:44:49.706Z"), "date_joined" : ISODate("2016-05-12T08:44:49.706Z"), "user_permissions" : [ ] }

Descending sort '-_id' is not working [mongodb 2.6.11]

I came across a strange issue; sorting on id field with '-' is not working and does not return anything where as without '-' is working fine.

This does not work query_set.order_by('-_id')
where as this is working query_set.order_by('_id')

CheckGrappelli WTF? :D

I mean, come on, you can't be serious. Checking the availability using a templatetag?

Just add dependencies towards Grapelli, if you really need it, and raise an Exception in case that it cannot be found on the include path. But, hell, no tag for that, please.
Besides, the tag alone will not help as you actually need two separate code bases or paths in your views to either create grappelli based dashboards etc. or the standard, more generic admin views and presentations thereof.

So, please, refactor the existing tag into something more obsolete and then reconsider, perhaps by introducing a function
that will return whether or not grapelli was enabled in INSTALLED_APPS.

From there, you can then evolve by deciding to either render the standard admin templates or the grapelli enabled ones... in your view/controller that is, with no tag being involved.

Example:

def view(....):
    if grapelli_is_enabled:
        return render('grapelli_view.xhtml'...)
    return render('standard_view.xhtml'...)

Does it work with python 3?

what's the next milestone?

Thank for create this repo, i want to ask that what's the next milestone and what's the development plans? May i fork and pull now?

Superuser has no permission

When I login to django admin with super user I get the following message.

The user_permissions array is also empty in mongodb.

{
"_id": ObjectId("57dc5db686w5c77e351e2e1c"),
"_cls": "User.Member",
"email": "[email protected]",
"password": "pbkdf2_sha256$24C000$7vNWGxjVxbr$LvauNaXnjlwPrGdG+3b1Cz04vpoW7f0kxV/0nkW6Abg=",
"is_active": true,
"is_superuser": true,
"last_login": ISODate("2016-08-11T12:03:58.895Z"),
"date_joined": ISODate("2016-08-11T11:12:53.376Z"),
"user_permissions": [],
"is_staff": true,
"username": "djadmin"
}

Is there anyway to fix this?

Create new Admin

Why not write a new Admin to work with MongoEngine?

Django Admin was made to relate database, Django-MongoEngine a code is not pretty to keep.

PyPI release?

There seems to be 0.01 on PyPI, any chance of getting the more recent version out?

Thanks!

Merge with django-mongotools

https://github.com/wpjunior/django-mongotools.

it's good idea ?

Ensure user is sparse

MongoEngine/mongoengine/issues/128

Django mongoengine Connection Support

I am trying to use django-mongoengine with my project but I am facing some connectivity issues regarding auth and sessions. I am using django v1.9.2. I only want to use sessions and auth.

This I added in my settings:

MONGODB_DATABASES = {
    "default": {
        "name": 'picknbox',
        "host": '127.0.0.1',
        # "password": database_password,
        # "username": database_user,
         "tz_aware": True, # if you using timezones in django (USE_TZ = True)
    },
}

DATABASES = {
    'default': {'ENGINE': 'django.db.backends.dummy'}
}

INSTALLED_APPS += ["django_mongoengine"]
SESSION_ENGINE = 'django_mongoengine.sessions'
SESSION_SERIALIZER = 'django_mongoengine.sessions.BSONSerializer'

Where can I see the auth_user and django_sessions collections? I cant see them in my mongodb instance?
Also When I run python manage.py migrate, it throws an error regarding Database Improperly Configured.

Also if you can provide some more instructions that would be great.
Thanks

user._meta has no attribute 'pk' causes error while storing session

I think this issue was already present in the old project, and was closed when the split occured.
It seemed that with the latest github build of this project, and django 1.9.2 this still isn't fixed.
I get the Exception
AttributeError: 'MetaDict' object has no attribute 'pk'
On the login function in: django.contrib.auth.init.py"

The error is on request.session[SESSION_KEY] = user._meta.pk.value_to_string(user)

the _meta dict doesn't have the .pk attribute.
Is there any quick fix I can implement my self?

Add fixtures support

see: hmarr/mongoengine#493

Is this project stil live?

I'm seeing the last change on the dev branch as of 8 months ago.

Any prospects of the development to continue?

http://docs.mongoengine.org/en/latest/django.html -- 404

field_generator ignores required property of FileField and ImageField.

Currently all field generators check if field is or not required except for FileField and ImageField

The sollution is quite simple:

diff -rupN django-mongoengine.orig/django_mongoengine/forms/field_generator.py django-mongoengine/django_mongoengine/forms/field_generator.py
--- django-mongoengine.orig/django_mongoengine/forms/field_generator.py 2015-09-23 16:28:34.090070123 +0200
+++ django-mongoengine/django_mongoengine/forms/field_generator.py  2015-09-23 17:50:35.980082981 +0200
@@ -258,9 +258,15 @@ class MongoFormFieldGenerator(object):
             return f

     def generate_filefield(self, field, **kwargs):
+        defaults = {
+            'required': field.required
+        }
         return forms.FileField(**kwargs)

     def generate_imagefield(self, field, **kwargs):
+        defaults = {
+            'required': field.required
+        }
         return forms.ImageField(**kwargs)

     def generate_dictfield(self, field, **kwargs):

Check cascade works as expected

See: http://stackoverflow.com/questions/14621235/django-mongodbforms-keyerror-metacascade

Empty Query Set not really empty

I have a use case where I need to return empty query set. I used MyModel.objects.none() query. But, the resulting queryset doesnt give proper result for functions like .sum()

For eg.
I have a model called Plot with a field area.
If I have a queryset with n number of plots, and do .sum('area'), it gives me sum of area of objects in the qureyset as expected. But, when the queryset is empty, it returns sum of area of all objects in Plot, rather than the expected 0.

AttributeError at /admin/trends/trend/

# urls.py
from django.conf.urls import url, include
from django_mongoengine.mongo_admin import site
from django.contrib import admin
admin.autodiscover()

urlpatterns = [
    url(r'^admin/', include(site.urls)),
]

#admin.py
# -*- coding: utf-8 -*-

from django_mongoengine import mongo_admin as admin

from trends.models import Trend


class TrendAdmin(admin.DocumentAdmin):
    pass


admin.site.register(Trend, TrendAdmin)

# -*- coding: utf-8 -*-

from django_mongoengine import Document
from django_mongoengine import fields

import datetime

# Create your models here.


class Trend(Document):
    name = fields.StringField(required=True)
    url = fields.StringField(required=True)
    source = fields.StringField(required=True)
    location = fields.StringField(required=True)
    woeid = fields.LongField(required=True)
    as_of = fields.DateTimeField(required=True)
    created_on = fields.DateTimeField(required=True,
                                      default=datetime.datetime.utcnow)
    updated_at = fields.DateTimeField(required=True,
                                      default=datetime.datetime.utcnow)
    raw = fields.DictField()
    is_reappeared = fields.BooleanField(default=False)
    reappeared_as_of = fields.DateTimeField()

    meta = {
        'ordering': ['updated_at'],
        'indexes': ['name', '$name', 'source']
    }

Accessing the admin url for trend results in AttributeError. Is get_queryset needs to be implemented ?

Environment:


Request Method: GET
Request URL: http://localhost:8000/admin/trends/trend/

Django Version: 1.8.8
Python Version: 2.7.10
Installed Applications:
['django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'trends']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware']


Traceback:
File "/Users/krace/.virtualenvs/trendsporter2.7/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  132.                     response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/Users/krace/.virtualenvs/trendsporter2.7/lib/python2.7/site-packages/django_mongoengine/mongo_admin/options.py" in wrapper
  402.                 return self.admin_site.admin_view(view)(*args, **kwargs)
File "/Users/krace/.virtualenvs/trendsporter2.7/lib/python2.7/site-packages/django/utils/decorators.py" in _wrapped_view
  110.                     response = view_func(request, *args, **kwargs)
File "/Users/krace/.virtualenvs/trendsporter2.7/lib/python2.7/site-packages/django/views/decorators/cache.py" in _wrapped_view_func
  57.         response = view_func(request, *args, **kwargs)
File "/Users/krace/.virtualenvs/trendsporter2.7/lib/python2.7/site-packages/django_mongoengine/mongo_admin/sites.py" in inner
  202.             return view(request, *args, **kwargs)
File "/Users/krace/.virtualenvs/trendsporter2.7/lib/python2.7/site-packages/django/utils/decorators.py" in _wrapper
  34.             return bound_func(*args, **kwargs)
File "/Users/krace/.virtualenvs/trendsporter2.7/lib/python2.7/site-packages/django/utils/decorators.py" in _wrapped_view
  110.                     response = view_func(request, *args, **kwargs)
File "/Users/krace/.virtualenvs/trendsporter2.7/lib/python2.7/site-packages/django/utils/decorators.py" in bound_func
  30.                 return func.__get__(self, type(self))(*args2, **kwargs2)
File "/Users/krace/.virtualenvs/trendsporter2.7/lib/python2.7/site-packages/django_mongoengine/mongo_admin/options.py" in changelist_view
  1212.                 self)
File "/Users/krace/.virtualenvs/trendsporter2.7/lib/python2.7/site-packages/django_mongoengine/mongo_admin/views.py" in __init__
  22.                 list_per_page, list_max_show_all, list_editable, model_admin)
File "/Users/krace/.virtualenvs/trendsporter2.7/lib/python2.7/site-packages/django/contrib/admin/views/main.py" in __init__
  47.         self.root_queryset = model_admin.get_queryset(request)

Exception Type: AttributeError at /admin/trends/trend/
Exception Value: 'TrendAdmin' object has no attribute 'get_queryset'

Usage with django_tables2

I encountered a wee problem with using django_tables2. When passing an array of documents into the table from a queryset the column headings fall back to the default for verbose_name (None). I think the reason could be this part of the DjangoField constructor:

if self.verbose_name is None and self.name:
            self.verbose_name = self.name.replace('_', ' ')

self.verbose_name will never be set as self.name is undefined at that stage.

Does that make sense?

Cheers

DocumentForm deletion within DocumentFormSet

Not sure that the method used to check if a form has been deleted is right in this file. Django formset use the following test:

if form in formset.deleted_forms

In the case when data doesn't validate and is part of a deleted form, form.cleaned_data does not exist, and the way we test it can lead to problems (and in the worst case won't do everything we want...).

lambda invalid syntax

.../django-mongoengine/django_mongoengine/utils/wrappers.py", line 52
f = lambda (k, v): (
^
SyntaxError: invalid syntax

https://www.python.org/dev/peps/pep-3113/

Support for basic mongoengine integration with Django

Not sure if this helps but, I've made a simple django-mongoengine which can support handling Mongoengine connections via the Django connection manager, and adds setup/teardown unit test support when using mongo databases.

There is some nasty monkeypatching to make this work, but so far it seems to work pretty well out of the box. Although it doesn't add support for admin panels or anything, it works great for those that just want to use a mongodb connection in Django without any fuss.

Happy to do a better PR if you feel it's worthy enough for inclusion.

Using ModelChoiceField with mongo queryset triggers AttributeError

Hi there,

I am trying to build a form with ModelChoiceField

document = forms.ModelChoiceField(label="Document", queryset=Document.objects.all(), to_field_name="name")

and I get AttributeError: 'QuerySet' object has no attribute '_prefetch_related_lookups'.

Is this a known issue? I found on your code that you've done some functions mapping in queryset.py, is that suppose to address issue like this?

Please let me know what to look at to help with this issue, or is there anything I've not done right.

Thanks!

ImproperlyConfigured: settings.DATABASES is improperly configured.

I set settings.DATABASES to

{
'default': {
'ENGINE': 'django.db.backends.dummy'
}
}

I'm still getting

ImproperlyConfigured: settings.DATABASES is improperly configured. Please supply the ENGINE value. Check settings documentation for more details.

I'm on Django 1.6.2.

Fix Travis Build

Right now it seems like our travis builds are failing for reasons that are not the fault of any pull requesters.

We should try and get this fixed soon.

Create admin login credentials

Hello,

I have downloaded your example and it works well with mongoDB on my machine. But now i want to create admin login credential. is there a way to do it or is it just broken for now?

ObjectId('55d3f4b756c02c3f3a0febc1') is not JSON serializable

vagrant@precise64:~/wwwroot/xxxx$ django-admin --version
1.7.8
In [3]: mongoengine.VERSION
Out[3]: (0, 10, 0)
In [5]: pymongo.version
Out[5]: '3.0.3'

Setting file error:

MONGODB_DATABASES = {
        'default': {'name': 'XXX'}
}
AUTHENTICATION_BACKENDS = (
    'django_mongoengine.auth.backends.MongoEngineBackend',
)
SESSION_ENGINE = 'django_mongoengine.sessions'
# SESSION_SERIALIZER = 'django_mongoengine.sessions.BSONSerializer'

User login error

TypeError at /login/
ObjectId('55d3f4b756c02c3f3a0febc1') is not JSON serializable
Request Method: POST
Request URL:    http://192.168.33.10:8888/login/?next=/loginout/
Django Version: 1.7.8
Exception Type: TypeError
Exception Value:    
ObjectId('55d3f4b756c02c3f3a0febc1') is not JSON serializable
Exception Location: /usr/lib/python2.7/json/encoder.py in default, line 184
Python Executable:  /usr/bin/python
Python Version: 2.7.6

Fix this error, Setting file:

MONGODB_DATABASES = {
        'default': {'name': 'XXXXX'}
}
AUTHENTICATION_BACKENDS = (
    'django_mongoengine.auth.backends.MongoEngineBackend',
)
SESSION_ENGINE = 'django_mongoengine.sessions'
SESSION_SERIALIZER = 'django_mongoengine.sessions.BSONSerializer'

but new django_mongoengine not found django_mongoengine.sessions.BSONSerializer, Add code to django_mongoengine/sessions.py

from bson import json_util

class BSONSerializer(object):
    """
    Serializer that can handle BSON types (eg ObjectId).
    """
    def dumps(self, obj):
        return json_util.dumps(obj, separators=(',', ':')).encode('ascii')

    def loads(self, data):
        return json_util.loads(data.decode('ascii'))

Why do you want to delete BSONSerializer?
Please check your code to include the example/tumblelog.

General question on overall progress so far

First things first: a very good job, integration wise, so far.

What is your estimate on when we can try out django-mongosession? Are you short on committers/contributers?

For a current project of mine, I require a working document form and a working backend admin site and standard user management including standard authentication.

If you need help, I would be glad to contribute to your project.

Error updating objects with FileFields

BaseDocumentForm perform an update instead of save the hole object witch is great, but if the form contains a FileField (or ImageField) it won't be converted to a GridFS file, so mongoengine raises an AttributeError since object is InMemoryUploadedFile.

This bug can be confirmed with the example application editing an image post.

Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/django/core/handlers/base.py", line 132, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/lib64/python2.7/site-packages/django/views/generic/base.py", line 71, in view
    return self.dispatch(request, *args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/django/views/generic/base.py", line 89, in dispatch
    return handler(request, *args, **kwargs)
  File "/home/darkstar/django-mongoengine/django_mongoengine/views/edit.py", line 214, in post
    return super(BaseUpdateView, self).post(request, *args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/django/views/generic/edit.py", line 215, in post
    return self.form_valid(form)
  File "/home/darkstar/django-mongoengine/django_mongoengine/views/edit.py", line 62, in form_valid
    self.object = form.save()
  File "/home/darkstar/django-mongoengine/django_mongoengine/forms/documents.py", line 424, in save
    self.instance.update(**update)
  File "/usr/lib64/python2.7/site-packages/mongoengine/document.py", line 465, in update
    return self._qs.filter(**self._object_key).update_one(**kwargs)
  File "/usr/lib64/python2.7/site-packages/mongoengine/queryset/base.py", line 487, in update_one
    upsert=upsert, multi=False, write_concern=write_concern, **update)
  File "/usr/lib64/python2.7/site-packages/mongoengine/queryset/base.py", line 448, in update
    update = transform.update(queryset._document, **update)
  File "/usr/lib64/python2.7/site-packages/mongoengine/queryset/transform.py", line 249, in update
    value = field.prepare_query_value(op, value)
  File "/usr/lib64/python2.7/site-packages/mongoengine/base/fields.py", line 160, in prepare_query_value
    self.validate(value)
  File "/usr/lib64/python2.7/site-packages/mongoengine/fields.py", line 1461, in validate
    if value.grid_id is not None:
AttributeError: 'InMemoryUploadedFile' object has no attribute 'grid_id'

An easy sollution for this issue can be replace save function on BaseDocumentForm. Here you have a patch:

diff -rupN django-mongoengine.orig/django_mongoengine/forms/documents.py django-mongoengine/django_mongoengine/forms/documents.py
--- django-mongoengine.orig/django_mongoengine/forms/documents.py   2015-09-23 16:28:34.090070123 +0200
+++ django-mongoengine/django_mongoengine/forms/documents.py    2015-09-23 16:29:46.080070311 +0200
@@ -391,37 +391,15 @@ class BaseDocumentForm(BaseForm):
             raise ValueError("The %s could not be %s because the data didn't"
                              " validate." % (self.instance.__class__.__name__, fail_message))

-        if self.instance._created:
-            self.instance = construct_instance(self, self.instance, self.fields, self._meta.exclude)
+        self.instance = construct_instance(self, self.instance, self.fields, self._meta.exclude)

-            # Validate uniqueness if needed.
-            if self._validate_unique:
-                self.validate_unique()
-
-            if commit:
-                self.instance.save()
-        else:
-            update = {}
-            for name, data in self.cleaned_data.iteritems():
+        # Validate uniqueness if needed.
+        if self._validate_unique:
+            self.validate_unique()

-                try:
-                    if isinstance(data, datetime.datetime):
-                        data = data.replace(tzinfo=None)
-                    if getattr(self.instance, name) != data:
-                        update['set__' + name] = data
-                        setattr(self.instance, name, data)
-                except AttributeError:
-                    raise Exception('Model %s has not attr %s but form %s has' \
-                                    % (type(self.instance),
-                                      name,
-                                      type(self)))
-
-            # Validate uniqueness if needed.
-            if self._validate_unique:
-                self.validate_unique()
+        if commit:
+            self.instance.save()

-            if commit and update:
-                self.instance.update(**update)
         return self.instance
     save.alters_data = True

mongo_admin is not working.

How to register document model?
I am unable to register it using mongo_admin.site.register or using register decorator.

default param used but no defined in MongoFormFieldGenerator.generate()

I really don't know why it has been working until now, maybe embedded fields triggers this function... anyway, this variable doesn't exist...

  File "/usr/lib64/python2.7/site-packages/django_mongoengine/forms/documents.py", line 240, in __new__
    opts.exclude, opts.widgets, formfield_callback, formfield_generator)
  File "/usr/lib64/python2.7/site-packages/django_mongoengine/forms/documents.py", line 180, in fields_for_document
    form_field = field_generator.generate(f, **kwargs)
  File "/usr/lib64/python2.7/site-packages/django_mongoengine/forms/field_generator.py", line 37, in generate
    if default:
NameError: global name 'default' is not defined

I supose that default parameter reffers to charfield_default parameter, so this patch should solve this problem:

diff -rupN django-mongoengine.orig/django_mongoengine/forms/field_generator.py django-mongoengine/django_mongoengine/forms/field_generator.py
--- django-mongoengine.orig/django_mongoengine/forms/field_generator.py 2015-09-23 16:28:34.090070123 +0200
+++ django-mongoengine/django_mongoengine/forms/field_generator.py  2015-09-24 18:35:52.060088655 +0200
@@ -23,7 +23,7 @@ class MongoFormFieldGenerator(object):
         field-classname) and raises a NotImplementedError of no generator
         can be found.

-        :param default: Default to a CharField?
+        :param charfield_default: Default to a CharField?
         """
         field_name = field.__class__.__name__.lower()
         if hasattr(self, 'generate_%s' % field_name):
@@ -34,7 +34,7 @@ class MongoFormFieldGenerator(object):
             if hasattr(self, 'generate_%s' % cls_name):
                 return getattr(self, 'generate_%s' % cls_name)(field, **kwargs)

-        if default:
+        if charfield_default:
             # Default to a normal CharField
             # TODO: Somehow add a warning
             defaults = {'required': field.required}

Toolbar issue

I apologize if this is the wrong group for this bug report-- this was the only place where I could find discussion of a django-debug-toolbar-mongo bug.

I ran into a problem with the django debug toolbar because operation_tracker's _update (and insert and remove) method specifies the deprecated "safe" option, with a default value of False. Not only is the option deprecated, but if it is not specified at all, mongo uses a default value of None, which ultimately gets transformed into True. The safe property has been replaced by WriteConcern.

Our application had defined a WriteConcern of 1 at the collection level, but it was getting overriden on individual calls by the debug toolbar's safe=False option. I had to explicitly pass w=1 on each insert, update, and remove call.

This bug exists with the most recent version, 1.9.

Thanks,

Christina Roberts

Admin fails at ReferenceField with following error

init__() takes at least 2 arguments (4 given)

/home/ubuntu/workspace/django-mongoengine/django_mongoengine/fields/djangoflavor.py in formfield

            return form_class(**defaults)

<django_mongoengine.fields.ReferenceField object at 0x7fd88686a0d0>
kwargs  {}
choices_form_class  None
defaults    = {'help_text': None, 'label': None, 'required': False}
<class 'django_mongoengine.forms.fields.ReferenceField'>

do this have a proposed release date !!

Hello guys, I love mongoengine and been using it for a while, but recently you stopped support for django . Im ok to use 0.9 version, but some how i could not use pymongo 3.2 and latest drf versions because of mongoengine 0.9.

Im glad that you guys are working on this .. So when can we expect the support back

cache solution?

anyone know if django-mongoengine has a preferred cache solution?

There are a few systems available for django, but it would be interesting to hear some success stories.

https://docs.djangoproject.com/en/dev/topics/cache/

Multiple DB connection error.

Commit <8ad246f2178b0e9c08a0f5346e3f6278f5c2dbbc> : simple module loading

Is it possible the above commit can break multiple DB connection below?
It works with the previous versions, but from the above commit version, it throw errors about Connection with alias "advising" has not been defined.

MONGODB_DATABASES = {
'default': {
'name': 'django_mongoengine',
# 'host': '127.0.0.1',
# "password": database_password,
# "username": database_user,
'tz_aware': True, # if you using timezones in django (USE_TZ = True)
},
'advising': {
'name': 'ctAdvising',
'host': 'localhost',
'tz_aware': True, # if you using timezones in django (USE_TZ = True)
},
}

---

ConnectionError at /email_sent_logs/
Connection with alias "advising" has not been defined
Request Method: GET
Request URL: http://127.0.0.1:7000/email_sent_logs/
Django Version: 1.8.5
Exception Type: ConnectionError
Exception Value:
Connection with alias "advising" has not been defined

Exception Location: /usr/local/lib/python2.7/dist-packages/mongoengine/connection.py in get_connection, line 101

/srv/kaizen_api/views.py in EmailSentLogViewSet
queryset = EmailSentLog.objects.all() ...
/usr/local/lib/python2.7/dist-packages/mongoengine/queryset/manager.py in get
queryset = queryset_class(owner, owner._get_collection()) ...
/usr/local/lib/python2.7/dist-packages/mongoengine/document.py in _get_collection
db = cls._get_db() ...
/usr/local/lib/python2.7/dist-packages/mongoengine/document.py in _get_db
return get_db(cls._meta.get("db_alias", DEFAULT_CONNECTION_NAME)) ...
/usr/local/lib/python2.7/dist-packages/mongoengine/connection.py in get_db
conn = get_connection(alias) ...
▼ Local vars
Variable Value
alias 'advising'
reconnect False
/usr/local/lib/python2.7/dist-packages/mongoengine/connection.py in get_connection
raise ConnectionError(msg) ...
▼ Local vars
Variable Value
msg 'Connection with alias "advising" has not been defined'
alias 'advising'
reconnect False

Subclass Django's exceptions

see: hmarr/mongoengine#502