ivelum/djangoql

Advanced search language for Django

djangoql's Issues

using "in" with an Integer field

I have a PositiveSmallIntegerField with choices specified. The choices are specified as a subclass of django.db.models.IntegerChoices, which I believe is new in Django 3.0.

my_field=1 or my_field=2 works fine. However, my_field in (1,2) returns an empty QuerySet. Has anyone else encountered this?

Couldn't create 'djangoql.parsetab'. Permission denied

The package version of djangoql (in Fedora) somehow gets this warning:

WARNING: Couldn't create 'djangoql.parsetab'. [Errno 13] Permission denied: '/usr/lib/python3.7/site-packages/djangoql/parsetab.py'

Do you know how or why this file is being created/modified?

The version is 0.13.0.

Thank you.

PS: I've opened a bug in Fedora (for myslelf) https://bugzilla.redhat.com/show_bug.cgi?id=1760757

Allow non-quoted strings as right hand values

I'm wondering if it would be somehow possible to allow non-quoted strings as right hand values for single word searches. With this you could do queries like name ~ anssi or phone = 0505968123. Strings with spaces would of course need to be quoted, so name = "Anssi Kääriäinen" would be the search for full name.

I know this leads to some complications, like case name = None (could be said that name = "" should match both name is none and name is empty string cases) and maybe some complications I don't even know in the lexer and parser parts. On the other hand I believe users would prefer the ability to do simple searches without quoting.

Support of Django 3.0

Hello,

Would it be possible to support this package for Django 3 version?

Little background: Next Fedora (32) distribution is upgrading to Django 3.

Here is the error when tests are run with Django 3.0a0:

+ /usr/bin/python3 test_project/manage.py test core.tests
Creating test database for alias 'default'...
Destroying test database for alias 'default'...
Traceback (most recent call last):
  File "test_project/manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/lib/python3.8/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python3.8/site-packages/django/core/management/__init__.py", line 395, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/lib/python3.8/site-packages/django/core/management/commands/test.py", line 23, in run_from_argv
    super().run_from_argv(argv)
  File "/usr/lib/python3.8/site-packages/django/core/management/base.py", line 328, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/lib/python3.8/site-packages/django/core/management/base.py", line 369, in execute
    output = self.handle(*args, **options)
  File "/usr/lib/python3.8/site-packages/django/core/management/commands/test.py", line 53, in handle
    failures = test_runner.run_tests(test_labels)
  File "/usr/lib/python3.8/site-packages/django/test/runner.py", line 687, in run_tests
    self.run_checks()
  File "/usr/lib/python3.8/site-packages/django/test/runner.py", line 625, in run_checks
    call_command('check', verbosity=self.verbosity)
  File "/usr/lib/python3.8/site-packages/django/core/management/__init__.py", line 168, in call_command
    return command.execute(*args, **defaults)
  File "/usr/lib/python3.8/site-packages/django/core/management/base.py", line 369, in execute
    output = self.handle(*args, **options)
  File "/usr/lib/python3.8/site-packages/django/core/management/commands/check.py", line 59, in handle
    self.check(
  File "/usr/lib/python3.8/site-packages/django/core/management/base.py", line 392, in check
    all_issues = self._run_checks(
  File "/usr/lib/python3.8/site-packages/django/core/management/base.py", line 382, in _run_checks
    return checks.run_checks(**kwargs)
  File "/usr/lib/python3.8/site-packages/django/core/checks/registry.py", line 72, in run_checks
    new_errors = check(app_configs=app_configs)
  File "/usr/lib/python3.8/site-packages/django/core/checks/urls.py", line 13, in check_url_config
    return check_resolver(resolver)
  File "/usr/lib/python3.8/site-packages/django/core/checks/urls.py", line 23, in check_resolver
    return check_method()
  File "/usr/lib/python3.8/site-packages/django/urls/resolvers.py", line 406, in check
    for pattern in self.url_patterns:
  File "/usr/lib/python3.8/site-packages/django/utils/functional.py", line 48, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/usr/lib/python3.8/site-packages/django/urls/resolvers.py", line 587, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/usr/lib/python3.8/site-packages/django/utils/functional.py", line 48, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/usr/lib/python3.8/site-packages/django/urls/resolvers.py", line 580, in urlconf_module
    return import_module(self.urlconf_name)
  File "/usr/lib64/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/builddir/build/BUILD/djangoql-0.13.0/test_project/test_project/urls.py", line 20, in <module>
    from core.views import completion_demo
  File "/builddir/build/BUILD/djangoql-0.13.0/test_project/core/views.py", line 4, in <module>
    from django.shortcuts import render_to_response
ImportError: cannot import name 'render_to_response' from 'django.shortcuts' (/usr/lib/python3.8/site-packages/django/shortcuts.py)

Thank you.

PS: I can try to submit patches.

case insensitive search

It would be nice if the search was case insensitive by default. I keep on accidentally using the wrong case. I might be able to get company open source time to submit a PR for this but probably not anytime soon.

Support of Django 4.0

Hello,

Would it be possible to support this package for Django 4 version?

Here is the imports error that happening because in Django 4.0 from django.conf.urls import re_path was deprecated and new usage is from django.urls import re_path:

try:
    from django.conf.urls import re_path
except ImportError:  # Django <2.0
    from django.conf.urls import url as re_path

Exception:

File "python3.10/site-packages/djangoql/admin.py", line 28, in <module>
from django.conf.urls import url as re_path
ImportError: cannot import name 'url' from 'django.conf.urls'

Maybe some more exceptions happening, but this one is obvious.

So can we add support of Django 4.0?

Revert to Django builtin search?

Hi,

Is it currently a feature to switch between django (simple) and djangoql (advanced) search? Making an invalid search of ?q=foo just gives Unexpected end of input, maybe if a query is invalid, assume its a simple search?

Auto-resize after auto-completion

textarea auto-resize is not happen after an completion widget item is selected.
Example: https://jsfiddle.net/p7xwnpvk/

Steps to bundle the completion

Hello,

The completion JS is bundled within the PyPi.
Would it be possible to share the steps how to bundle it?
We'd like to package it to RPM as well, to make it compatible.

Thank you.

error: MySQL backend does not support timezone-aware datetimes when USE_TZ is False.

I think this change: 87958ad

Broke date searching if the project doesn't use TimeZones (USE_TZ = False)

Is there a work around?

Test - using 0.8.8, set USE_TZ = False and do a date search.

File "/Users/leehinde/Documents/Clients/rnrvirtual/venv-python2.7-django/lib/python2.7/site-packages/django/db/models/fields/init.py", line 1483, in get_db_prep_value
return connection.ops.value_to_db_datetime(value)
File "/Users/leehinde/Documents/Clients/rnrvirtual/venv-python2.7-django/lib/python2.7/site-packages/django/db/backends/mysql/operations.py", line 150, in value_to_db_datetime
raise ValueError("MySQL backend does not support timezone-aware datetimes when USE_TZ is False.")

Search does not work with GenericIPAddress fields

Unfortunately, for now, djangoql doesn't work with GenericIPAddress fields.

I am getting an exception:
"Field "field_name" has "unknown" type. It can be compared to , but not to '10.45.26.93'"

Does search in DjangoQLField.get_options() work?

I know that in the changelog it says that search in get_options() is 'breaking' in version 0.14.0, but does it now work in version 0.14.1?

I am asking because, after updating to 0.14.1, if I change one of my fields from this code

class SearchFieldOptUsername(StrField):
    """Create a list of unique users' usernames for search"""

    model = User
    name = 'username'
    suggest_options = True
    id_list = []

    def get_options(self):

        return super(SearchFieldOptUsername, self).get_options().\
        filter(id__in=self.id_list).\
        order_by(self.name).\
        values_list(self.name, flat=True)

to this code

class SearchFieldOptUsername(StrField):
    """Create a list of unique users' usernames for search"""

    model = User
    name = 'username'
    suggest_options = True
    id_list = []

    def get_options(self, search):

        return self.model.\
        filter(id__in=self.id_list).\
        filter(username__icontains=search).\
        order_by(self.name).\
        values_list(self.name, flat=True)

I stop getting suggestions.

Thanks!

Autocomplete disappears for search with spaces

For example i'm looking for author name = "Leo Tolstoy" and autocomplete dropdown disappears after Leo and space.
And it doesn't matter was string quoted or not.

How to handle proxy models?

Say I have proxy models BarA and BarB like:

class Foo(models.Model):
    bar = models.ForeignKey(Bar)

    ql_fields = ('bar',)

class Bar(models.Model):
    name = models.CharField()
    a_stuff = models.CharField()
    b_stuff = models.CharField()

class BarA(Bar):

    ql_fields = ('a_stuff',)

    class Meta:
        proxy = True

class BarB(Bar):

    ql_fields = ('b_stuff',)

    class Meta:
        proxy = True

When specifying the lookup fields for Foo.bar (all of which are on the base model), how can I handle distinct field lookups on each proxy?

Would it be possible to specify in Foo as ql_fields = ('bara', 'barb') and for DjangoQL to find the proxies? Or would it make more sense to create a set of all the ql_fields of all proxy models?

How to limit the number of values shown for autocomplete field?

First of all, I'd like to say good job on creating DjangoQL! It's a great app.

I have a quick question about your autocomplete function. I use it to show suggestions for a field that has many possible values (>1000), like so

class SearchFieldOptPartDescription(StrField):
    """Create a list of unique part descriptions for search"""

    model = order_management_Order
    name = 'part_description'
    suggest_options = True

    def get_options(self):
        return super(SearchFieldOptPartDescription, self).\
        get_options().all().distinct()

I wonder if you could point me in the right direction as to how I could reduce the number of shown options to, let's say, the top 10 after a person has started typing 3 letters. Right now, it shows all available values right from the start.

For example for jquery autocomplete function I would use slice(), like so

$('#id_part_description').autocomplete({
            source: function(request, response) {
                    var results = $.ui.autocomplete.filter(product_names, request.term);
                    response(results.slice(0, 10));

I suspect I would have to change DjangoQL's javascript to accomplish this. But I don't know where to find the relevant lines of code that I need to modify.

Thanks a lot in advance!

Sometimes got an error

Very rarely while searching I got an error FieldError: Cannot resolve keyword '_djangoql' into field. does it happen to anybody else? Is it known issue?

Stacktrace:

FieldError: Cannot resolve keyword '_djangoql' into field. Choices are: a_user, a_user_id, answer_count_by_question_ids, answer_with_anyone_comment_count_by_question_ids, answer_with_reviewer_comment_count_by_question_ids, answer_with_reviewer_like_count_by_question_ids, answered_question_ids, answers_count, answers_with_reviewer_comments_count, anyone_comment_count_by_question_ids, biweekly_due_day_which_week, carried_over_goals_count, company_id, completed_goals_count, due_date_ts, due_day, group_ids, has_comment,...
  File "django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "django/core/handlers/base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "django/core/handlers/base.py", line 113, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "django/contrib/admin/options.py", line 606, in wrapper
    return self.admin_site.admin_view(view)(*args, **kwargs)
  File "django/utils/decorators.py", line 142, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "django/views/decorators/cache.py", line 44, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "django/contrib/admin/sites.py", line 223, in inner
    return view(request, *args, **kwargs)
  File "django/utils/decorators.py", line 45, in _wrapper
    return bound_method(*args, **kwargs)
  File "django/utils/decorators.py", line 142, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "django/contrib/admin/options.py", line 1685, in changelist_view
    cl = self.get_changelist_instance(request)
  File "django/contrib/admin/options.py", line 744, in get_changelist_instance
    sortable_by,
  File "django/contrib/admin/views/main.py", line 81, in __init__
    self.queryset = self.get_queryset(request)
  File "django/contrib/admin/views/main.py", line 439, in get_queryset
    qs, search_use_distinct = self.model_admin.get_search_results(request, qs, self.query)
  File "ff/mods/admin/utils.py", line 102, in get_search_results
    return super().get_search_results(request, queryset, search_term)
  File "djangoql/admin.py", line 63, in get_search_results
    search_term=search_term,
  File "django/contrib/admin/options.py", line 1023, in get_search_results
    queryset = queryset.filter(reduce(operator.or_, or_queries))
  File "django/db/models/query.py", line 892, in filter
    return self._filter_or_exclude(False, *args, **kwargs)
  File "django/db/models/query.py", line 910, in _filter_or_exclude
    clone.query.add_q(Q(*args, **kwargs))
  File "django/db/models/sql/query.py", line 1290, in add_q
    clause, _ = self._add_q(q_object, self.used_aliases)
  File "django/db/models/sql/query.py", line 1312, in _add_q
    current_negated, allow_joins, split_subq, simple_col)
  File "django/db/models/sql/query.py", line 1318, in _add_q
    split_subq=split_subq, simple_col=simple_col,
  File "django/db/models/sql/query.py", line 1190, in build_filter
    lookups, parts, reffed_expression = self.solve_lookup_type(arg)
  File "django/db/models/sql/query.py", line 1049, in solve_lookup_type
    _, field, _, lookup_parts = self.names_to_path(lookup_splitted, self.get_meta())
  File "django/db/models/sql/query.py", line 1420, in names_to_path
    "Choices are: %s" % (name, ", ".join(available)))

Airquote disappears

djangoql 0.14.1

Hello,

After upgrading to version 0.14.1 and updating my code so that get_options() takes the new search parameter, I have experienced an odd behaviour. I am not entirely sure whether this is a bug with djangoql or something that I did wrong.

When I select my choice from a list of string options, the second airquote disappears, which causes an error when I hit the search button

This is the underlying code

class SearchFieldOptSupplier(StrField):

    model = Order
    name = 'supplier'
    suggest_options = True

    def get_options(self, search):

        if len(search) < 3:
            return ["Type 3 or more characters to see suggestions"]
        else:
            return self.model.objects.filter(supplier__icontains=search).\
                        distinct()[:10].values_list(self.name, flat=True)

The strange thing is that I have other similar classes that do NOT show the same problem, such as the following

class SearchFieldOptCostUnit(StrField):

    model = CostUnit
    name = 'cost_unit'
    suggest_options = True

    def get_options(self, search):
        return self.model.objects.filter(name__icontains=search).order_by('name').\
        values_list('name', flat=True)

Any ideas?

Add operator to limit result count

Currently the limits for query results are set by Django admin pagination, with the option to "select all" across the pages when performing an action on the results. Some actions have limits of their own, so the "select all" group is too big, but the default pagination size of 100 results means having to manually loop through select 100, apply action, select next 100, [...] until the action has been applied to the full set.

Could a djangoql operator be added which can set a limit on the number of returned rows? Something exactly like the SQL "LIMIT" would be ideal.

Thanks!

Different results when order is changed.

I get wired behaver:

While changing the order of AND conditions:

I get this user appear.
While user have both many_to_many "he" and "ar"

This is the code I changed:

class FollowerLangField(StrField):
    name = 'language'
    model = Follower

    # def get_lookup_name(self):
    #     print(f'language__id')
    #     return 'language__id'

    # def get_lookup_value(self, value):
    #     lang_id = Language.objects.filter(code_name=value).first().id
    #     print(f'lang_id {lang_id}')
    #     return lang_id

    def get_lookup(self, path, operator, value):
        _search = 'language__id' #'__'.join(path + [self.get_lookup_name()])
        _op, invert = self.get_operator(operator)
        _value = Language.objects.filter(code_name=value).first().id
        print('debug get_lookup:',_search, _op, invert, _value)
        if operator == '!=':
            q = models.Q(**{'%s%s' % (_search, _op): _value})
        else:
            q = models.Q(**{'%s%s' % (_search, _op): _value})
        return ~q if invert else q


...
model:
class Follower(models.Model):
    language = models.ManyToManyField(
            "instagram_data.Language", verbose_name=_("language_code_name"), blank=True)


class Language(models.Model):
    code_name = models.CharField(max_length=6, null=True, blank=True)

    def __str__(self):
        return self.code_name

It seems to ignore the second "language" condition.

Y?

Saved search queries

Hi,

First of all, kudos, this app fills a gap in django admin UX.

It would be nice to have a save query button, and a select menu in the search bar to re-use them.

Keep the good work!

Add "Syntax Help" link to the error messages that get populate

Another "nice to have" from others that have been using DjangoQL.

Error messages are rather cryptic to new users and users can very easily miss the "Syntax Help" link in the auto-complete. It would be great if it was also in the error messages that also get created at the top of the page.

Use more user-friendly field names in auto-complete

Hi @stebunovd

Firstly, great feature, many thanks.

Secondly, is it possible for the auto-complete field names to be more user-friendly or pleasing to the eye? For example, instead of the auto-complete showing the model field name of user_registration_date, for it show, say, the more readable Registration Date? Perhaps to use the verbose_name of the given model field?

Thanks

Support for "NOT" operator

Awesome project!

Is there a way to put a "not" in front of a long complex query?

If not, I'd recommend a simple "exclude" parameters in the djangoql() manager method e.g., MyModel.objects.djangoql('mycolumn>True', exclude=True).

Use of verbose_name for filters

Is it possible to use verbose_name of a model attribute instead of the attribute itself?

E.g.

 'Title ~ "war" and Author.Last-Name = "Tolstoy"'

Inability to use previous search implementations

If a field isn't supported, or if certain search features have been implemented before, there should be a way to "escape" back to previous search functionality.

For example, fields like uuids aren't searchable in DjangoQL. While there are ways to make them searchable, it would be convenient to have an escape character, like a backslash, to disable DjangoQL for that search.

Where to execute SQL statement instead of ORM ?

I want to be able to execute and SQL statement instead of django one, can it be done ?
*While a user search in djangoQL ...

How much does DjangoQL improve performance?

I'm presuming djangoql searches are much faster than regular django searches as you can select on the field you want rather than django's crazy "search everything" approach. But you do have any hard stats on how much performance is improved by? It would vary a lot based on amount of data and schema of course, but it would still be good to know.

Multiple model filtering

I am not sure if this is a feature request or just my lack of understanding about this package.

But I wanted to create a page that would allow me to filter across two or more models.
For example, for table A I want to check one field is true and then reference another (linked) table to make sure another field has a certain value.

RFE: Lookup two field values

(Feature Request)

Hello,

Would it be possible to extend the query language to support lookup by other fields value?

For example:

name ~ title

would translate to

Q(name__icontains=F('title'))

The suggestions can also offer the field name in the context where a value is expected.

Thank you for consideration.

Contains on M2M models produces same result multiple times

I don't know if this is expected behavior:

class Author(models.Model):
    name = models.CharField(max_length=50)
    books = models.ManyToManyField(Book)

class Book(model.Model):
    name = models.CharField(max_length=50)

@admin.register(Author)
class AuthorAdmin(DjangoQLSearchMixin, admin.ModelAdmin):
    pass

Even if I only have a single author with multiple books, when using the search query:

books.name ~ "The"

I get the same author listed multiple times in the change_list view of the admin site.

Layout issue

I am having a layout issue. Any ideas?

Can I change once search with another?

I need to replace user search for "chars" into an "id"
replacing:
qs1.filter(~Q(language__code_name='ar'))
with:
lang_id = Language.objects.filter(code_name='ar').first().id
qs1.filter(~Q(language__id=lang_id))

It fetching results much quicker for me (avoiding nested loops).
was not sure what to change.

Thanks for the awsome project.

Duplicate results in reverse ForeignKey lookup

My model structure:

class Car(models.Model):
	name = models.CharField(max_length=50)
	
class Tyre(models.Model):
	name = models.CharField(max_length=50)
	car = models.ForeignKey(Car, related_name="tyres")

objects:

car = Car(name="car 1")
tyre1 = Tyre(name="tyre 1", car=car)
tyre2 = Tyre(name="tyre 2", car=car)
tyre3 = Tyre(name="tyre 3", car=car)
tyre4 = Tyre(name="tyre 4", car=car)

So when I lookup for tyres.name ~ "tyre" in the CarAdmin I get 4 results of the same car.
I wanted to know if it is possible to filter out duplicate results on admin.

Show only the operators that I need

It is possible to only show the operators that I need, example field.other_field and for this only show < and >

Import error in Django 3.1

Compatibility issue with Django 3.1b1.

  File ".../lib/python3.8/site-packages/djangoql/schema.py", line 8, in <module>
    from django.db.models import FieldDoesNotExist, ManyToManyRel, ManyToOneRel
ImportError: cannot import name 'FieldDoesNotExist' from 'django.db.models' (.../lib/python3.8/site-packages/django/db/models/__init__.py)

Getting rid of RuntimeWarning: DateTimeField received a naive datetime + is there something like TimeField?

First of all thank you for this package, it saved me huge amount of time. To the point:

I had a test in which I was filtering with some datetime:

self.client.get(
    '/users/user_1/meals/',
    {'filter': 'datetime_created < "2017-01-02 00:00:00"'},
    content_type='application/json',
)

When running ./manage.py test (Django 1.11) I received such warnings:

/usr/local/lib/python3.5/dist-packages/django/db/models/fields/__init__.py:1451: RuntimeWarning: DateTimeField Meal.datetime_created received a naive datetime (2017-01-02 00:00:00) while time zone support is active.

So, I changed from naive datatime "2017-01-02 00:00:00" to timezone aware datetime: "2017-01-02 00:00:00Z". Z (from "Zulu") in the end means UTC, this is the timezone of my Django project (TIME_ZONE = 'UTC' in settings.py). This is a good read from Django docs about time zones.

But then, I was getting an error from DjangoQL:

(...)
File "/usr/local/lib/python3.5/dist-packages/djangoql/schema.py", line 215, in validate
    repr(value),
djangoql.exceptions.DjangoQLSchemaError: Field "datetime_created" can be compared to timestamps in "YYYY-MM-DD HH:MM" format, but not to '2017-01-01T00:00Z'

Essentially, validate in DjangoQL's DateTimeField doesn't allow adding this Z in the end.

I ended up copy-pasting it and tweaking slightly, so that it allows this Z:

class UTCDateTimeField(DateTimeField):
    name = 'datetime_created'
    value_types_description = 'timestamps in "YYYY-MM-DD[ HH:MM[:SS[Z]]" format'

    def validate(self, value):
        super(DateTimeField, self).validate(value)
        mask = '%Y-%m-%d'
        if len(value) > 10:
            mask += ' %H:%M'
        if len(value) > 16:
            mask += ':%S'
        if len(value) > 19:
            mask += 'Z'
        try:
            datetime.datetime.strptime(value, mask)
        except ValueError:
            raise DjangoQLSchemaError(
                'Field "%s" can be compared to timestamps in '
                '"YYYY-MM-DD[ HH:MM[:SS[Z]]" format, but not to %s' % (
                    self.name,
                    repr(value),
                )
            )

Using such custom field works, no warnings.

I thought I will post here for 3 reasons:

1. People with the same problem can get a hand.
2. Maybe a pull request to DjangoQL can be created to improve DateTimeField.
3. Maybe there is a different way of doing it?

Bonus question: I'm missing TimeField, to be able to filter by "HH:MM[:SS]" only (for example, to get meals from 12:00 o'clock... tea etc.), should I also use custom field?

Small improvement in docs regarding customizing fields

In my situation I had a model containing a created datetime field, which I wanted to customize to search on date only. I got confused because the equivalent of the example code pasted below didn't seem to work.

class UserDateJoinedYear(IntField):
    name = 'date_joined_year'

    def get_lookup_name(self):
        return 'date_joined__year'


class UserQLSchema(DjangoQLSchema):
    def get_fields(self, model):
        fields = super(UserQLSchema, self).get_fields(model)
        if model == User:
            fields = [UserDateJoinedYear()] + fields
        return fields

It turned out it didn't work because in order to overwrite a field name that already exists, I needed to change the line:
fields = [UserDateJoinedYear()] + fields
to:
fields = fields + [UserDateJoinedYear()]

Makes perfect sense it works this way. Maybe you should make the same change in your example to avoid confusion.

Custom query fields

First, I just tried this library, and this is just amazing. Really nice work!

I've been thinking that it would be nice to have some form of custom fields available. For example, you might want to filter on user's full name when you have first_name and last_name in the model, or you might want to filter on some aggregate (say, num_books >= 10).

It seems this would require changes to both the schema and build_filter parts. I'll probably write a quick hack for this so that it's possible to check what exactly this requires.

Support for groups using parentheses

Hi! We need to make some slightly complicated queries in our project that require parentheses. For example:

(a and b) or (b and c)

I do not know how complicated it would be to support this feature.

Thanks :)

DjangoQL as deafult search option in Django Admin even when search_fields is set

At the moment, as stated in the docs "If you define search_fields in a ModelAdmin class, DjangoQL integration would automatically recognize this and let users choose between a standard Django search (that you specified with search_fields) and Advanced Search with DjangoQL".

The issue that, under these circumstances, the standard Django search option is shown as default. If I want to use DjangoQL, I need to manually turn it on. I would prefer the opposite, whereby DjangoQL is chosen as the default search option.

I have no choice but to include search_fields in my ModelAdmin, although I don't really care for it, because I've implemented some autocomplete fields using Django's 2.0 autocomplete_fields, which require search_fields to be set, as described here.

What would be the easiest way to set DjangoQL as the main deafult search option in the change_list view even when search_fields is set?

AttributeError: 'QuerySet' object has no attribute 'explain'

I ran into the error:

File "/.../lib/python3.5/site-packages/djangoql/admin.py", line 87, in get_search_results
    qs.explain()
AttributeError: 'QuerySet' object has no attribute 'explain'

using Django 2.0.8

$ python -c 'import django; print(django.__version__)'
2.0.8

It looks like QuerySet.explain() was added in 2.1

I regrettably can't offer a different solution to the problem that the usage of explain solves.

Thank you for this great project!

Not implemented on Production but do sees it local

If I run the same code on local pc - http://127.0.0.1:8000 - I will see the search working really cool.
While running as a service on linux production env - I don't see the feature working at all !

Should I remove any files ?

Abnormal scroll behavior for a long autocomplete list.

I failed to use scroll in autocomplete dropdown box for a large sets of data. Here some details:

1. Sometimes mouse cursor starts to be tied with scrollbar. (Usually after drag and drop in scrollbar)
2. I'm also failed to use arrow down key. When you press it on latest visible element it scrolls to top and selection also jumps to any value from top.
3. Mouse scroll works in the same way like arrow down key.

Ability to search for a DateTimeField without entering seconds

Let's say I have a DateTimeField on the User model called last_sign_in. I want to find all Users who signed in on May 4th 2020.

If I enter last_sign_in = "2020-05-04" then the results just show Users where the last_sign_in was at midnight on the 4th. So as a workaround I need to enter something like last_sign_in >= "2020-05-04" and last_sign_in < "2020-05-05".

It would be nice if there was a shorthand so the workaround wasn't necessary. Thanks!

print SQL executed

I am looking for a way to handle the SQL executed in djangoql filters.

I can change apply_search but is there an official way ? same as I do explain on QS ?

This is working:
def apply_search(queryset, search, schema=None):
"""
Applies search written in DjangoQL mini-language to given queryset
"""
ast = DjangoQLParser().parse(search)
schema = schema or DjangoQLSchema
schema_instance = schema(queryset.model)
schema_instance.validate(ast)
qs = queryset.filter(build_filter(ast, schema_instance))
print('djangoql_sql: ', qs.query)
print('djangoql_explain: ', qs.explain())
return qs

ability to order by a field

It would be cool if you could order your results by a field

Collectstatic on Django 4 fails due to missing map file

Trying to use whitenoise's collectfiles fails with

The JS file 'djangoql/js/completion.js' references a file which could not be found:
djangoql/js/completion.js.map

since the djangoql package apparently doesn't ship the .map file. Django 4 changed file discovery to also find sourcemap files, see e.g. https://code.djangoproject.com/ticket/33353.

Could

• the .map file be shipped with the package or
• the sourcemap reference comment //# sourceMappingURL=completion.js.map be removed?

DjangoQL Parser does not support unicode string such as chinese?

It seems that when i type the expression below in the django admin search

name = "年年有余"

but the parser encode the unicode string ( eg: Chinese Sentence ) badly

    def p_string(self, p):
        """
        string : STRING_VALUE
        """
        if PY2:
            value = p[1].decode('string_escape')
        else:
            value = bytes(p[1], 'utf8').decode('unicode_escape')
            print(p[1])  #  "年年有余" this is the actual value
            print(value) # å¹´å¹´æ
        p[0] = Const(value=value)

My Dev Env

• Python 3
• django 1.10.3

Add Startswith Operator

It would be great to add support for the __startswith QuerySet feature. ~ is great, but it does an icontains which completely bypasses a index on the DB. Starts with would allow us to maintain the index on a large tables for indexed fields.

Unsupported lookup 'iexact' for CharField or join on the field not permitted, perhaps you meant iexact or exact?

Hi there,

I've started receiving the following error from djangoql (v0.13.1)

File "/home/lib/python3.6/site-packages/django/core/handlers/exception.py" in inner
  34.             response = get_response(request)

File "/home/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
  115.                 response = self.process_exception_by_middleware(e, request)

File "/home/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
  113.                 response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/home/lib/python3.6/site-packages/django/utils/decorators.py" in _wrapped_view
  142.                     response = view_func(request, *args, **kwargs)

File "/home/lib/python3.6/site-packages/django/views/decorators/cache.py" in _wrapped_view_func
  44.         response = view_func(request, *args, **kwargs)

File "/home/lib/python3.6/site-packages/django/contrib/admin/sites.py" in inner
  223.             return view(request, *args, **kwargs)

File "/home/lib/python3.6/site-packages/djangoql/admin.py" in introspect
  130.         response = self.djangoql_schema(self.model).as_dict()

File "/home/lib/python3.6/site-packages/djangoql/schema.py" in as_dict
  429.                 [(name, field.as_dict()) for name, field in fields.items()]

File "/home/lib/python3.6/site-packages/djangoql/schema.py" in <listcomp>
  429.                 [(name, field.as_dict()) for name, field in fields.items()]

File "/home/lib/python3.6/site-packages/djangoql/schema.py" in as_dict
  44.             'options': list(self.get_options()) if self.suggest_options else [],

File "./collection_management/admin.py" in get_options
  361.             exclude(username__iexact__in=["AnonymousUser","guest","admin"]).\

File "/home/lib/python3.6/site-packages/django/db/models/query.py" in exclude
  899.         return self._filter_or_exclude(True, *args, **kwargs)

File "/home/lib/python3.6/site-packages/django/db/models/query.py" in _filter_or_exclude
  908.             clone.query.add_q(~Q(*args, **kwargs))

File "/home/lib/python3.6/site-packages/django/db/models/sql/query.py" in add_q
  1290.         clause, _ = self._add_q(q_object, self.used_aliases)

File "/home/lib/python3.6/site-packages/django/db/models/sql/query.py" in _add_q
  1318.                     split_subq=split_subq, simple_col=simple_col,

File "/home/lib/python3.6/site-packages/django/db/models/sql/query.py" in build_filter
  1251.         condition = self.build_lookup(lookups, col, value)

File "/home/lib/python3.6/site-packages/django/db/models/sql/query.py" in build_lookup
  1101.             lhs = self.try_transform(lhs, name)

File "/home/lib/python3.6/site-packages/django/db/models/sql/query.py" in try_transform
  1151.                 "permitted%s" % (name, output_field.__name__, suggestion)

Exception Type: FieldError at /collection_management/oligo/introspect/
Exception Value: Unsupported lookup 'iexact' for CharField or join on the field not permitted, perhaps you meant iexact or exact?

It seems to be becoming from this line of code, strangely enough only when the corresponding model is empty, that is, it doesn't have any records in it.

Any ideas?

Thanks in advance!