beniwohli/django-cms-search

This package provides multilingual search indexes for easy Haystack integration with django CMS

django-cms-search

Warning

This package is deprecated. Please use its successor, aldryn-search

This package provides multilingual search indexes for easy Haystack integration with django CMS.

Usage

After installing django-cms-search through your package manager of choice, add cms_search to your INSTALLED_APPS. That's it.

For setting up Haystack, please refer to their documentation.

For more docs, see the docs folder or the online documentation.

Warning

Since version 0.5, the HaystackSearchApphook is not registered automatically anymore. If you want do use the default app hook provided by django-cms-search, add this (e.g. in models.py):

from cms_search.cms_app import HaystackSearchApphook
apphook_pool.register(HaystackSearchApphook)

django-cms-search's Issues

info about multilang indexes

Hello. Sorry to write this as an issue, but github provides no way of emailing you

You made a great job making multilanguage indexes for haystack. I'm trying to figure it out because I need to adapt it to other translated apps, and I'm not being able to follow the whole thing. For example, in indexes.py:L38 it says if 'HaystackTrans' in getattr(attrs['HaystackTrans'], 'fields', []): but searching for 'HaystackTrans' doesn't throw any results.

I wonder how django-cms-specific is the MultiLanguageIndexBase metaclass. I see the prepare field thing, which seems to work with cms_search.search_indexes.page_index_factory._PageIndex.

I haven't been able to make a breakpoint in the metaclass stop, in order to debug what's going on.

Thanks

indexing public and draft pages

It's taken me all day to get to the bottom of this! I might be losing my mind here, but given the latest version of django-cms-search and django-cms 2.4.1 (and following the tutorials of both to get set up), haystack appears to index every page twice. That is, do a search for a common term on a typical site and in the results list of pages you'll see (for example) 'Home', 'Home', 'About Us', 'About Us', 'Contact Us', 'Contact Us', etc.

From what I can tell, haystack is indexing both the public and draft versions of the page objects. I'm new to django-cms but I get the impression the moderation system has changed recently. Anyway, there's a method in the cms PageManager class called 'public' which appears to return only the public (as opposed to draft) versions of the page. So in search_indexes.py

If you change:

qs = base_qs.published(site=site_obj.id).filter(

to

qs = base_qs.public().published(site=site_obj.id).filter(

Only the public versions of the pages are indexed.

Peace out!

AttributeError in cms_search_tags.py

I just experienced something weird which I don't really understand what might be the cause.

On a new live site, while searching for a given term (here it was "Produkte", which should definitely give some results on the respective page), it produced this error:

File "/var/www/domains/******/project/cms_search/search_helpers/templatetags/cms_search_tags.py", line 48, in get_value
untranslated = getattr(obj, field_name)

AttributeError: 'NoneType' object has no attribute 'url'

Everything else seemed to work very fine. It might very well be that there is something i did wrong in my models that are searchable as well and that this issue has nothing really to do with cms-search itself. I am using haystack with whoosh.

However, this is my hotfix, I think it might be helpful and does not do any damage to the rest of the system:

(cms_search_tags.py line 48 onward)

    untranslated = ""
    try:
        untranslated = getattr(obj, field_name)
    except AttributeError:
        pass
    if self._is_truthy(untranslated):

etc.

django-cms-search can't index contents in several languages

Based on the docs, django-cms-search is supposed to handle multilingual content properly, but I only obtain search results in the first my settings.LANGUAGES. No results are ever found in the other languages.

Suppose the order of my languages is 'fr', 'en', 'es' like in the example below. If I make a search in English (/search/?models=cms_search.page_en&q=foo), I get no results. If I switch French and English in the list, re-index, and refresh the page, I get some results. I think this is a bug.

I'm using the whoosh backend. Here is some information about my setup.

settings.py

LANGUAGES = (
    ('fr', _('French')),
    ('en', _('English')),
    ('es', _('Spanish')),
)

HAYSTACK_SEARCH_ENGINE = 'whoosh'
HAYSTACK_SITECONF = 'www.custom_cms.search_sites'
HAYSTACK_WHOOSH_PATH = os.path.join(ROOT_DIR, 'cache', 'whoosh', 'www_index')

www/custom_cms/search_sites.py

import haystack
from cms.models import monkeypatch_reverse
from cms.plugin_pool import plugin_pool

plugin_pool.get_plugin('TextPlugin').search_fulltext = True

monkeypatch_reverse()    
haystack.autodiscover()

./manage.py rebuild_index

WARNING: This will irreparably remove EVERYTHING from your search index.
Your choices after this are to restore from backups or rebuild via the `rebuild_index` command.
Are you sure you wish to continue? [y/N] y

Removing all documents from your index because you said so.
All documents removed.
Indexing 35 pages (French).
Indexing 35 pages (English).
Indexing 35 pages (Spanish).

./manage.py shell

>>> from haystack.query import SearchQuerySet
>>> from cms_search import models
>>> SearchQuerySet().models(models.Page_Fr).all()
[<SearchResult: cms_search.page_fr (pk=u'1')>, <SearchResult: cms_search.page_fr (pk=u'2')>, <SearchResult: cms_search.page_fr (pk=u'3')>, <SearchResult: cms_search.page_fr (pk=u'4')>, <SearchResult: cms_search.page_fr (pk=u'5')>, <SearchResult: cms_search.page_fr (pk=u'6')>, <SearchResult: cms_search.page_fr (pk=u'7')>, <SearchResult: cms_search.page_fr (pk=u'8')>, <SearchResult: cms_search.page_fr (pk=u'9')>, <SearchResult: cms_search.page_fr (pk=u'10')>, <SearchResult: cms_search.page_fr (pk=u'11')>, <SearchResult: cms_search.page_fr (pk=u'12')>, <SearchResult: cms_search.page_fr (pk=u'13')>, <SearchResult: cms_search.page_fr (pk=u'14')>, <SearchResult: cms_search.page_fr (pk=u'15')>, <SearchResult: cms_search.page_fr (pk=u'16')>, <SearchResult: cms_search.page_fr (pk=u'17')>, <SearchResult: cms_search.page_fr (pk=u'18')>, <SearchResult: cms_search.page_fr (pk=u'19')>, '...(remaining elements truncated)...']
>>> SearchQuerySet().models(models.Page_En).all()
[]
>>> SearchQuerySet().models(models.Page_Es).all()
[]

Does not index all sites

Currently only pages on Site.objects.get_current() are indexed. This is due to how django-cms falls back to using the default site on it's querysets. (https://github.com/divio/django-cms/blob/develop/cms/models/query.py#L16)

Can't search in different languages, besides english.

I'm indexing the pages in two different languages (Portuguese and English), but i'm only allowed to retrieve the pages through english search, and not when searching for them in portuguese.

Also, when i search in english, i get the both pages. Instead of only one.

Problem with non-unicode strings in settings.LANGUAGES

If, as in this case (settings.py):

gettext_noop = lambda s:s

LANGUAGES = (
   ('es', gettext_noop('Castellano')),
   ('en', gettext_noop('English')),
   ('pt', gettext_noop('Português')),
)

There is a non-ascii character provided in a string that is not unicode by default (python 2.x) syncdb fails for cms_search:

Traceback (most recent call last):
File "./manage.py", line 14, in
execute_manager(settings)
File "/home/rafadev/www/suntech/bootstrap/lib/python2.6/site-packages/django/core/management/init.py", line 438, in execute_manager
utility.execute()
File "/home/rafadev/www/suntech/bootstrap/lib/python2.6/site-packages/django/core/management/init.py", line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/rafadev/www/suntech/bootstrap/lib/python2.6/site-packages/django/core/management/base.py", line 191, in run_from_argv
self.execute(_args, *_options.dict)
File "/home/rafadev/www/suntech/bootstrap/lib/python2.6/site-packages/django/core/management/base.py", line 220, in execute
output = self.handle(_args, *_options)
File "/home/rafadev/www/suntech/bootstrap/lib/python2.6/site-packages/django/core/management/base.py", line 351, in handle
return self.handle_noargs(**options)
File "/home/rafadev/www/suntech/bootstrap/lib/python2.6/site-packages/south/management/commands/syncdb.py", line 90, in handle_noargs
syncdb.Command().execute(**options)
File "/home/rafadev/www/suntech/bootstrap/lib/python2.6/site-packages/django/core/management/base.py", line 220, in execute
output = self.handle(_args, _options)
File "/home/rafadev/www/suntech/bootstrap/lib/python2.6/site-packages/django/core/management/base.py", line 351, in handle
return self.handle_noargs(_options)
File "/home/rafadev/www/suntech/bootstrap/lib/python2.6/site-packages/django/core/management/commands/syncdb.py", line 107, in handle_noargs
emit_post_sync_signal(created_models, verbosity, interactive, db)
File "/home/rafadev/www/suntech/bootstrap/lib/python2.6/site-packages/django/core/management/sql.py", line 182, in emit_post_sync_signal
interactive=interactive, db=db)
File "/home/rafadev/www/suntech/bootstrap/lib/python2.6/site-packages/django/dispatch/dispatcher.py", line 172, in send
response = receiver(signal=self, sender=sender, *_named)
File "/home/rafadev/www/suntech/bootstrap/lib/python2.6/site-packages/django/contrib/auth/management/init.py", line 26, in create_permissions
for codename, name in _get_all_permissions(klass._meta):
File "/home/rafadev/www/suntech/bootstrap/lib/python2.6/site-packages/django/contrib/auth/management/init.py", line 15, in _get_all_permissions
perms.append((_get_permission_codename(action, opts), u'Can %s %s' % (action, opts.verbose_name_raw)))
File "/home/rafadev/www/suntech/bootstrap/lib/python2.6/site-packages/django/db/models/options.py", line 196, in verbose_name_raw
raw = force_unicode(self.verbose_name)
File "/home/rafadev/www/suntech/bootstrap/lib/python2.6/site-packages/django/utils/encoding.py", line 88, in force_unicode
raise DjangoUnicodeDecodeError(s, *e.args)
django.utils.encoding.DjangoUnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 7: ordinal not in range(128). You passed in <django.utils.functional.proxy object at 0x3584c50> (<class 'django.utils.functional.proxy'>)

A temporary fix, in my case was changing LANGUAGES to:

LANGUAGES = (
   ('es', gettext_noop('Castellano')),
   ('en', gettext_noop('English')),
   ('pt', gettext_noop(u'Português')),
)

But we asume that, as no other of my installed applications had the same issue, its probably a bug in cms_search.

Include site in SearchIndex

As the djanog-cms supports multiple sites, it would be nice to include the site in the SearchIndex so one could filter the SearchQuerySet by site to only display results for the current site.

Customize PageIndex

Which way would you recommend to customize the page index created by cms_search? I'd like for example to set the text field to EdgeNgramField or make the title field indexable.

"The apphook is not registered any more..."

I'm new to haystack + django cms; so not sure what an AppHook is.

What does this mean; do I need to do something to make the search work ?

strip_tags removes tags and does not put in a space

In search_indexes.py on lines 51 and 53, strip tags is used. strip_tags removes all tags with nothing, when indexing text plugins with
in it (can happen pretty often), the last word of the line will be concatenated with the first word on the next line. This generates wrong indexing files (experienced this with haystack an whoosh).

this is my solution

import re

def custom_strip_tags(value):
"""Returns the given HTML with all tags stripped."""
return re.sub(r'<[^>]*?>', ' ', force_unicode(value))

then replace the two instances of strip_tags with the custom one.

Best!

not all pages get indexed

we filter out all pages that have redirect__isnull=False. Unfortunately redirect sometimes is also an empty string.

pull request in the works :-)

rebuild_index showing: Indexing 2 <django.utils.functional.__proxy__ object at 0x10afcf910>.

Hey I am having an issue in rebuilding the index

I do not see what other outputs people are having but get this:

Indexing 2 < django.utils.functional.proxy object at 0x10afcf910 >.

Is this an issue with django 1.5.5?

search_fields doesn't handle relations

https://github.com/piquadrat/django-cms-search/blob/develop/cms_search/search_indexes.py#L75

This line handles indexing of only simple attributes. Plugins with relations spanning multiple models (like django-cms' snippet) don't get indexed.

I think snippet's idea of using django's __ notation for marking foreignkey search_fields relations is right one and it would be great if it was also supported.

fields.py should use CMS_SITE_LANGUAGES

CMS_SITE_LANGUAGES allows different languages to be configured per site. This should be used instead in fields.py so only those languages configured for a particular site are displayed to users.

TypeError: coercing to Unicode: need string or buffer, NoneType found

Hello,

I thank you for the plugin in django-cms.

I have installed the version 0.6.3. I was the problem for the indexation with the command django: rebuild_index

The problem come here : text += obj.get_meta_keywords() if hasattr(obj, 'get_meta_keywords') else u'' in search_indexes.py.

I replace this line by text += obj.get_meta_keywords() or u''
It's good after.

Thanks.