QuerySets evaporate on successive iteration

[eykd]

QuerySets evaporate on successive iteration

[iapain]

+1 we solved this problem by adding cache in queryset.

[hmarr]

There has been a discussion about this at http://groups.google.co.uk/group/mongoengine-dev/browse_thread/thread/b75fdfaed1440997 - a concern is that documents can get pretty large (especially when you got lots of embedded document), so caching by default could be a memory hog. It seems like the most popular option would be to cache by default, but make it easy to disable the caching behaviour, like Django does.

[iapain]

You'r right about memory hog and since its mongo untill and unless there are lots of DBRef query read should be pretty similar to memcached read. Anyways there is always a work around by calling list() on queryset but then we lost Queryset method chainging on it and its not a recommended way.

@eykd try:
docs = list(Foo.objects())

[eykd]

The memory concerns make perfect sense, but since you're trying to mimic Django's QuerySet behavior and interface, the current behavior breaks the Principle of Least Surprise. This crucial difference should at least be clearly documented. It's not a completely foreign concept at least--mongoengine's QuerySets are to Django's as generator comprehensions are to list comprehensions. As long as the behavior is documented, then it's no more surprising than a generator "evaporating" after the first evaluation.

Also, something should be done about that exception. If it's going to act like a generator, it should merely evaporate, not explode. :)