djangonauts/django-hstore

PostgreSQL HStore support for Django.

django-hstore

---

You need dynamic columns in your tables. What do you do?

• Create lots of tables to handle it. Nice, now you'll need more models and lots of additional sqls. Insertion and selection will be slow as hell.
• Use a noSQL database just for this issue. Good luck.
• Create a serialized column. Nice, insertion will be fine, and reading data from a record too. But, what if you have a condition in your select that includes serialized data? Yeah, regular expressions.

---

Documentation - Mailing List

---

Projects using this package

• django-rest-framework-hstore: django-rest-framework tools for django-hstore
• Libreborme: django web platform that opens data from the Spanish Companies Register by providing also a search engine
• Nodeshot: extensible django web application for management of community-led georeferenced data - some features of django-hstore, like the schema-mode have been developed for this project

django-hstore's Issues

register_hstore not getting called properly during tests

We have an application with a test that looks roughly like this:

from django.test import TestCase
from things.models import Thing

class ThingCreationTestCase(TestCase):
    """Ensure hstore fields work."""
    def test_create_thing(self):
        thing1 = Thing(name='new-thing', data={'testkey': 'testvalue'})
        thing1.save()
        thing2 = Thing.objects.all()[0]
        self.assertEqual(thing2.data['testkey'], thing1.data['testkey'])

After updating to django_hstore >= 1.2.1, this test fails with the following error:

======================================================================
ERROR: Ensure hstore fields work
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/projects/myproject/python/django/things/tests.py", line 95, in test_create_thing
    self.assertEqual(thing2.data['testkey'], thing1.data['testkey'])
TypeError: string indices must be integers, not str

The application itself runs fine, however, and hstore fields on models work properly--only the tests fail.

I was able to trace the error to this set of commits: 8ac21be...2e9164f

In other words, this test works with versions of django_hstore prior to Jan. 12, 2014, while it fails with versions after Jan. 12, 2014.

I was able to resolve the issue by placing the following at the top of the test:

from django.test import TestCase
from things.models import Thing

class ThingCreationTestCase(TestCase):
    """Ensure hstore fields work."""
    def test_create_thing(self):
        from django.db import connection
        from psycopg2.extras import register_hstore
        register_hstore(connection.connection)
        thing1 = Thing(name='new-thing', data={'testkey': 'testvalue'})
        thing1.save()
        thing2 = Thing.objects.all()[0]
        self.assertEqual(thing2.data['testkey'], thing1.data['testkey'])

It looks like django_hstore isn't calling register_hstore properly at the start of each test case; I suspect that this is due to the way the connection_created signal is being handled in django_hstore 1.2.1 and after, but I wasn't able to figure out exactly why register_hstore isn't being called.

We are using the django.contrib.gis.db.backends.postgis database engine. We store and deal with spatial data, so we can't switch to django.db.backends.postgresql_psycopg2 as suggested in #79. When running tests under GeoDjango, the database is created from template_postgis (link), and we have the hstore extension installed on that database:

vagrant@vagrant-ubuntu-trusty-64:~$ sudo -u postgres psql template_postgis
psql (9.4.0, server 9.3.5)
Type "help" for help.

template_postgis=# CREATE EXTENSION hstore;
ERROR:  extension "hstore" already exists
template_postgis=#

Add reload schema feature

It would be very useful to be able to change the SCHEMA of an hstore field (in schema mode) in runtime for advanced manipulation.

Hstore with DRF serializer

Thanks guys for keeping up with this project. it's cool! are you planning to make a django-jsonb for the new versions of postgres?
@nemesisdesign I am using your code https://github.com/ninuxorg/nodeshot/blob/master/nodeshot/core/base/fields.py#L142 in my model here:
https://gist.github.com/pazooki/96a7d0c79e0d17caceac#file-gistfile1-py-L51
but I haven't been able to get it to work. before using your code and just using hstore.DictionaryField I was getting this error: http://dpaste.com/3WF8PW7 then I realized that I have to write a Field class for hstore so the serializer can handle it properly, then I found your code.

Now I'm facing this problem:
http://dpaste.com/03ARC3Q

Would you kindly let me know if you know a way to go around this?
Thanks!

"""
pip freeze
Django==1.6.5
Markdown==2.4.1
PyJWT==0.2.1
South==1.0
django-braces==1.4.0
django-cors-headers==0.12
django-extensions==1.3.8
django-filter==0.7
django-guardian==1.2.2
django-hstore==1.2.5
django-oauth-toolkit==0.7.1
djangorestframework==2.3.14
djangorestframework-jwt==0.1.5
drf-extensions==0.2.4
oauthlib==0.6.3
phonenumbers==6.2.0
psycopg2==2.5.3
python-openid==2.2.5
python-social-auth==0.1.26
requests==2.3.0
requests-oauthlib==0.4.1
six==1.7.3
uWSGI==2.0.6
ujson==1.33
wsgiref==0.1.2
"""

Filtered queryset not used in hslice manager method

https://github.com/jordanm/django-hstore/blob/88196eac5220b7031c63e0a37cf82c4ff2a556a8/django_hstore/hstore.py#L100-101

Compatibility with django.postgres.HStoreField

How about stealing some improvements from this PR?
django/django#3219

:-)

It looks like the implementation is intentionally simple so to avoid complex code to achieve certain features.

So it probably make sense to take some improvements back into django-hstore.

Then if in PostgreSQL, the binary json type will really be merged with the hstore 2.0 features as it was announced few months ago, it could make sense to implement support for this new feature (binary json with hstore lookups and indexes) directly in this package which is already widely used.

update_query breaks transactions in django 1.7

django_hstore.query.update_query uses transaction.is_managed which was completely deprecated in Django 1.7, rendering whole HStoreManager incompatbile with new transaction management.

Basically in 1.7 transaction.is_managed does not do or return anything and following code:

if not transaction.is_managed(using=self.db):
        transaction.enter_transaction_management(using=self.db)

always tries to enter transaction management even if it's already inside atomic block.

The result is:

django.db.transaction.TransactionManagementError: This is forbidden when an 'atomic' block is active.

EDIT:
transaction.commit_unless_managed is used there too and is also deprecated.

Is there a way to reload schema just for one instance?

Hi django-hstore developers!

I'm wondering if there is a way to apply schema not to the whole model class, but rather for one model instance.

My situation is following:

I have a model that has a set of fields that are common for all instances.
This model has a 'category'.
Depending on category (e.g. Hotel, Parking, Theatre) I'd like to add a fixed set of additional attributes (different for every category). For example, Hotel should have a star_rating, while Parking should have a lots_number but it shouldn't have a star_rating.
I can't create separate models for each category, I'd like it to be as much dynamic as possible.

So, the solution I see is to attach schema name to a Category model and then apply this schema to 'attributes' field on instance creation.
However, if I do it in the way described in docs:

field = SchemaDataBag._meta.get_field('data')
# load a different schema
field.reload_schema([
    {
        'name': 'url',
        'class': 'URLField'
    }
])
# turn off schema mode
field.reload_schema(None)

Schema is getting updated for the whole model class (all instances).

Incompatibility with Django 1.6

django-hstore seems to be not compatible with Django 1.6

C:\Python27\lib\site-packages\django_orm\postgresql\hstore\forms.py:2: Deprecati
onWarning: django.utils.simplejson is deprecated; use json instead.
from django.utils import simplejson as json

Unhandled exception in thread started by <function wrapper at 0x02E3B3B0>
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\django\utils\autoreload.py", line 93, in w
rapper
fn(_args, *_kwargs)
File "C:\Python27\lib\site-packages\django\core\management\commands\runserver.
py", line 98, in inner_run
self.validate(display_num_errors=True)
File "C:\Python27\lib\site-packages\django\core\management\base.py", line 310,
in validate
num_errors = get_validation_errors(s, app)
File "C:\Python27\lib\site-packages\django\core\management\validation.py", lin
e 34, in get_validation_errors
for (app_name, error) in get_app_errors().items():
File "C:\Python27\lib\site-packages\django\db\models\loading.py", line 196, in
get_app_errors
self.populate()
File "C:\Python27\lib\site-packages\django\db\models\loading.py", line 78, in
populate
self.load_app(app_name)
File "C:\Python27\lib\site-packages\django\db\models\loading.py", line 99, in
load_app
models = import_module('%s.models' % app_name)
File "C:\Python27\lib\site-packages\django\utils\importlib.py", line 40, in im
port_module
import(name)
File "c:\mygeosite\geodjango\basqui\models.py", line 13, in
from shapefile.models import *
File "c:\mygeosite\geodjango\shapefile\models.py", line 3, in
from django_orm.postgresql import hstore
File "C:\Python27\lib\site-packages\django_orm\postgresql\hstore__init.py",
line 2, in
from django_orm.postgresql.hstore.managers import HStoreManager
File "C:\Python27\lib\site-packages\django_orm\postgresql\hstore\managers.py",
line 2, in
from django_orm.postgresql.hstore.query import HStoreQuerySet
File "C:\Python27\lib\site-packages\django_orm\postgresql\hstore\query.py", li
ne 54, in
from django_orm.cache.query import CachedQuerySet
File "C:\Python27\lib\site-packages\django_orm\cache\query.py", line 4, in
from django.db.models.query import ITER_CHUNK_SIZE
ImportError: cannot import name ITER_CHUNK_SIZE

Schema mode is incompatible with django-autocomplete-light

I made a repo showing the issue at https://github.com/bufke/autocomplete_hstore

When using autocomplete-light with schema mode I get AttributeError: 'VirtualField' object has no attribute 'fk_field'

If you remove the schema from here it will run fine.

I'm not sure if this should be a report for autocomplete instead. Let me know what you think. I'll try thinking of a fix too.

web_1 | Traceback (most recent call last):
web_1 |   File "manage.py", line 10, in <module>
web_1 |     execute_from_command_line(sys.argv)
web_1 |   File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
web_1 |     utility.execute()
web_1 |   File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute
web_1 |     django.setup()
web_1 |   File "/usr/local/lib/python2.7/site-packages/django/__init__.py", line 21, in setup
web_1 |     apps.populate(settings.INSTALLED_APPS)
web_1 |   File "/usr/local/lib/python2.7/site-packages/django/apps/registry.py", line 115, in populate
web_1 |     app_config.ready()
web_1 |   File "/usr/local/lib/python2.7/site-packages/django/contrib/admin/apps.py", line 22, in ready
web_1 |     self.module.autodiscover()
web_1 |   File "/usr/local/lib/python2.7/site-packages/django/contrib/admin/__init__.py", line 23, in autodiscover
web_1 |     autodiscover_modules('admin', register_to=site)
web_1 |   File "/usr/local/lib/python2.7/site-packages/django/utils/module_loading.py", line 74, in autodiscover_modules
web_1 |     import_module('%s.%s' % (app_config.name, module_to_search))
web_1 |   File "/usr/local/lib/python2.7/importlib/__init__.py", line 37, in import_module
web_1 |     __import__(name)
web_1 |   File "/code/demo/admin.py", line 6, in <module>
web_1 |     class SomethingAdmin(admin.ModelAdmin):
web_1 |   File "/code/demo/admin.py", line 7, in SomethingAdmin
web_1 |     form = autocomplete_light.modelform_factory(SomethingWithSchema)
web_1 |   File "/usr/local/lib/python2.7/site-packages/autocomplete_light/forms.py", line 443, in modelform_factory
web_1 |     {'Meta': Meta})
web_1 |   File "/usr/local/lib/python2.7/site-packages/autocomplete_light/forms.py", line 269, in __new__
web_1 |     cls.post_new(new_class, meta)
web_1 |   File "/usr/local/lib/python2.7/site-packages/autocomplete_light/forms.py", line 339, in post_new
web_1 |     cls.add_generic_fk_fields(new_class, meta)
web_1 |   File "/usr/local/lib/python2.7/site-packages/autocomplete_light/forms.py", line 358, in add_generic_fk_fields
web_1 |     field.fk_field)
web_1 | AttributeError: 'VirtualField' object has no attribute 'fk_field'

Feature request: support keyword args in create for schema hstore

It would be really nice to be able to do:

Order.objects.create(send_emails=True, send_texts=False)

Where send_emails and send_texts are defined in my schema like so:

props = hstore.DictionaryField(schema=[
    {
        'name': 'send_texts',
        'class': 'BooleanField',
        'kwargs': {
            'default': True
        }
    },
    {
        'name': 'send_emails',
        'class': 'BooleanField',
        'kwargs': {
            'default': True
        }
    },
])

Admin errorlist not shown in case of validation error

Relevant snippet from the model:

class Language(models.Model):
    driver = models.ForeignKey('Driver')
    data = hstore.DictionaryField()

    objects = hstore.HStoreManager()

class Driver(models.Model):
    name = models.CharField(max_length=255)
    surname = models.CharField(max_length=255)

relevant part from the admin:

class LangInline(admin.StackedInline):
    max_num = 1
    extra = 1
    model = models.Language

admin.site.register(models.Driver, inlines=(LangInline,))

If I try to save a new driver without adding any row to the "data" field it fails saying "Please correct the error below." but no error is printed.
If I add a row to the field the Driver and Language models are created but then If I try to remove the Language model from the Driver's admin page I get the message "Please correct the error below." again.

Why forcing DatetimeField to utcnow() ?

Hi!

We've started using your library @komoot and we've rapidly hit the problem that it is impossible to give a None value to a DateTimeField in schema mode.

This part of the code:

django-hstore/django_hstore/virtual.py

Line 102 in c5d9b46

kwargs['default'] = datetime.datetime.utcnow()

Seems to indicate that any default value set to None will be converted to datetime.utcnow(). Is there a reason for this specific behavior? If I'm not mistaken django ORM authorizes None values for DateTimeField if null=True.

Our workaround for now is to set 'kwargs': {'default': ''}, however it forces our schema to be in a quiet inconsistent state...

Thanks for your explanation :-)

Is null filtering is not working on hstore field

These cases are not working

Something.objects.filter(data=None)
Something.objects.filter(data__exact=None)
Something.objects.filter(data__isnull=False)
Something.objects.filter(data__isnull=True)

result

TypeError: invalid lookup type

Liskov Violation in `HStoreDict`

First off, I am using this library with schema_mode turned off.

Now, it is very likely that my code will have the built-in assumption that,

For all x
If isinstance(d, dict),
and I type d["hi"] = x
then d["hi"] == x

This is guaranteed in general by dictionaries, but not by HStoreDict (since everything gets converted to strings). This can cause very subtle errors.

In my case, the Voluptuous library was failing to properly coerce strings to floats, because it was consuming an HStoreDict. The workaround is just to convert my HStoreDict back to a standard dict before passing it on to Voluptuous.

I am not sure if there is a sensible way to fix this problem. I imagine that the this behavior exists because the hstore column stores only strings; if they are going to have to be saved in the DB as strings, we might as well get that work done as early as possible.

Perhaps we could move to a strategy where we convert values to strings on save, rather than on dictionary-update? Or maybe we could raise an exception if you try to insert a non-string into the dictionary?

TypeError when using hstore.DictionaryField in a unique_together clause

I'm using Django 1.7 (latest pull from stable/1.7 branch), but can recreate the issue with Django 1.6.

Full traceback below:

Traceback:
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/core/handlers/base.py" in get_response
  113.                     response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/Users/dave/Workspace/willhill_bingo/venv/lib/python2.7/site-packages/newrelic-2.18.1.15/newrelic/hooks/framework_django.py" in wrapper
  492.                 return wrapped(*args, **kwargs)
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/contrib/admin/options.py" in wrapper
  550.                 return self.admin_site.admin_view(view)(*args, **kwargs)
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/utils/decorators.py" in _wrapped_view
  105.                     response = view_func(request, *args, **kwargs)
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/views/decorators/cache.py" in _wrapped_view_func
  52.         response = view_func(request, *args, **kwargs)
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/contrib/admin/sites.py" in inner
  204.             return view(request, *args, **kwargs)
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/contrib/admin/options.py" in add_view
  1417.         return self.changeform_view(request, None, form_url, extra_context)
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/utils/decorators.py" in _wrapper
  29.             return bound_func(*args, **kwargs)
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/utils/decorators.py" in _wrapped_view
  105.                     response = view_func(request, *args, **kwargs)
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/utils/decorators.py" in bound_func
  25.                 return func.__get__(self, type(self))(*args2, **kwargs2)
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/db/transaction.py" in inner
  394.                 return func(*args, **kwargs)
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/contrib/admin/options.py" in changeform_view
  1359.             if form.is_valid():
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/forms/forms.py" in is_valid
  162.         return self.is_bound and not bool(self.errors)
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/forms/forms.py" in errors
  154.             self.full_clean()
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/forms/forms.py" in full_clean
  355.         self._post_clean()
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/forms/models.py" in _post_clean
  430.             self.validate_unique()
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/forms/models.py" in validate_unique
  439.             self.instance.validate_unique(exclude=exclude)
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/db/models/base.py" in validate_unique
  802.         errors = self._perform_unique_checks(unique_checks)
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/db/models/base.py" in _perform_unique_checks
  886.             qs = model_class._default_manager.filter(**lookup_kwargs)
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/db/models/manager.py" in manager_method
  92.                 return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/db/models/query.py" in filter
  689.         return self._filter_or_exclude(False, *args, **kwargs)
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/db/models/query.py" in _filter_or_exclude
  707.             clone.query.add_q(Q(*args, **kwargs))
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/db/models/sql/query.py" in add_q
  1287.         clause, require_inner = self._add_q(where_part, self.used_aliases)
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/db/models/sql/query.py" in _add_q
  1314.                     current_negated=current_negated, connector=connector)
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/db/models/sql/query.py" in build_filter
  1186.             condition = self.build_lookup(lookups, col, value)
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/db/models/sql/query.py" in build_lookup
  1094.                     return final_lookup(lhs, rhs)
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/db/models/lookups.py" in __init__
  80.         self.rhs = self.get_prep_lookup()
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/db/models/lookups.py" in get_prep_lookup
  83.         return self.lhs.output_type.get_prep_lookup(self.lookup_name, self.rhs)
File "/Users/dave/Workspace/willhill_bingo/venv/src/django/django/db/models/fields/__init__.py" in get_prep_lookup
  632.             return value.prepare()

Hstore adapter registration relies on connection_created signal

Currently, django_hstore relies the connection_created signal in order to register the hstore adapters, unfortunately, if TransactionMiddleware is enabled, the signal gets called before the django_hstore app has been loaded, with means the adapter does not get loaded.

Since I'm not sure whether django, or django_hstore is at fault, I think at minimum, this should be documented until a fix in either project can be found.

More information (including test project) at: https://code.djangoproject.com/ticket/22297

Behaviour when null=True really sucks

When the null attribute is set to True in a DictionaryField, the behaviour sucks.

Because the default value of the dictionary becomes None, trying to setting or getting keys from the dictionary returns a TypeError exception:

set example:

n.data['test'] = 'test'

get example:

n.data.get('test', 'default value')

These two lines both generate an exception.

To mitigate this, one would always have to do something like:

n.data = n.data or {}
n.data['test'] = 'test'

or:

try:
    n.data['test'] = 'test'
except TypeError:
    n.data = {}
    n.data['test'] = 'test'

which generates a lot of redundant, useless, ugly code which also slows down development.

One solution could be to not use null=True at all, but at the present moment the only way i could find to add an hstore field to an existing database table is setting the default value to NULL, which makes possible to add the field in the table but renders development clunky.

I propose that the fallback value of an HStoreField must always be an empty HStoreDict, even when null values are allowed.

HSTORE adapter registration in psycopg2 conflicts with SQLAlchemy

If both django_hstore and SQLAlchemy are used in the same Python process with psycopg2, then django_hstore's adapter registration will cause psycopg2 to return HStoreDict instances to SQLAlchemy, which expects str. This causes errors like AttributeError: 'HStoreDict' object has no attribute 'decode' in sqlalchemy/dialects/postgresql/hstore.py:

    def process(value):
        if value is not None:
            return _parse_hstore(value.decode(encoding))

Some solution for the two to coexist would be good, for people who are using both Django and SQLAlchemy together.

One way might be to register the adapter only per connection/cursor, rather than globally (probably per cursor would be required for people using https://github.com/kennethreitz/django-postgrespool since the connections get shared).

HSTORE adapter registration intermittently not applied

Every now and then, when one of my apps starts up after a period of inactivity, the HSTORE adapter fails to register, resulting in the following error traces:

Traceback (most recent call last):

 File "/home/ubuntu/webapps/guardian_advertising_jobs/venv/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 112, in get_response
   response = wrapped_callback(request, *callback_args, **callback_kwargs)

 File "/usr/local/lib/python2.7/dist-packages/newrelic-2.14.0.11/newrelic/hooks/framework_django.py", line 492, in wrapper
   return wrapped(*args, **kwargs)

 File "/home/ubuntu/webapps/guardian_advertising_jobs/venv/local/lib/python2.7/site-packages/django/views/generic/base.py", line 69, in view
   return self.dispatch(request, *args, **kwargs)

 File "/home/ubuntu/webapps/guardian_advertising_jobs/guardian_advertising/apps/articles/views.py", line 110, in dispatch
   return super(AppMixin, self).dispatch(request, *args, **kwargs)

 File "/usr/local/lib/python2.7/dist-packages/newrelic-2.14.0.11/newrelic/hooks/framework_django.py", line 829, in wrapper
   return wrapped(*args, **kwargs)

 File "/home/ubuntu/webapps/guardian_advertising_jobs/venv/local/lib/python2.7/site-packages/django/views/generic/base.py", line 87, in dispatch
   return handler(request, *args, **kwargs)

 File "/home/ubuntu/webapps/guardian_advertising_jobs/guardian_advertising/apps/articles/views.py", line 198, in get
   return super(ArticleMixin, self).get(request, *args, **kwargs)

 File "/home/ubuntu/webapps/guardian_advertising_jobs/venv/local/lib/python2.7/site-packages/django/views/generic/detail.py", line 111, in get
   context = self.get_context_data(object=self.object)

 File "/home/ubuntu/webapps/guardian_advertising_jobs/guardian_advertising/apps/articles/views.py", line 256, in get_context_data
   context["title"] = unicode(self.object)

 File "/home/ubuntu/webapps/guardian_advertising_jobs/guardian_advertising/apps/articles/models.py", line 237, in __unicode__
   return unicode(self.content)

 File "/home/ubuntu/webapps/guardian_advertising_jobs/guardian_advertising/apps/articles/content/models.py", line 256, in __unicode__
   return self.title

 File "/home/ubuntu/webapps/guardian_advertising_jobs/guardian_advertising/apps/articles/content/fields.py", line 59, in __get__
   return self.value_from_instance(obj)

 File "/home/ubuntu/webapps/guardian_advertising_jobs/guardian_advertising/apps/articles/content/fields.py", line 42, in value_from_instance
   value = self.raw_value_from_instance(obj)

 File "/home/ubuntu/webapps/guardian_advertising_jobs/guardian_advertising/apps/articles/content/fields.py", line 35, in raw_value_from_instance
   return obj._article.fields.get(self.name, "")

AttributeError: 'str' object has no attribute 'get'

This also occurs in some background tasks, with the following trace:

Traceback (most recent call last):

 File "/home/ubuntu/webapps/guardian_advertising_jobs/guardian_advertising/apps/utils/tasks.py", line 43, in trigger_task
   signal.send(None, timestamp=timestamp)

 File "/home/ubuntu/webapps/guardian_advertising_jobs/venv/local/lib/python2.7/site-packages/django/dispatch/dispatcher.py", line 185, in send
   response = receiver(signal=self, sender=sender, **named)

 File "/home/ubuntu/webapps/guardian_advertising_jobs/venv/local/lib/python2.7/site-packages/django/db/transaction.py", line 371, in inner
   return func(*args, **kwargs)

 File "/home/ubuntu/webapps/guardian_advertising_jobs/guardian_advertising/apps/utils/tasks.py", line 66, in do_site_refresh_task
   return func(timestamp)

 File "/home/ubuntu/webapps/guardian_advertising_jobs/guardian_advertising/apps/feeds/tasks.py", line 42, in import_twitter_accounts
   content = tweet_data["text"],

 File "/home/ubuntu/webapps/guardian_advertising_jobs/guardian_advertising/apps/feeds/models.py", line 37, in create_article
   defaults = defaults,

 File "/home/ubuntu/webapps/guardian_advertising_jobs/venv/local/lib/python2.7/site-packages/django/db/models/manager.py", line 154, in get_or_create
   return self.get_queryset().get_or_create(**kwargs)

 File "/home/ubuntu/webapps/guardian_advertising_jobs/venv/local/lib/python2.7/site-packages/django/db/models/query.py", line 391, in get_or_create
   six.reraise(*exc_info)

 File "/home/ubuntu/webapps/guardian_advertising_jobs/venv/local/lib/python2.7/site-packages/django/db/models/query.py", line 383, in get_or_create
   obj.save(force_insert=True, using=self.db)

 File "/home/ubuntu/webapps/guardian_advertising_jobs/venv/local/lib/python2.7/site-packages/django/db/models/base.py", line 545, in save
   force_update=force_update, update_fields=update_fields)

 File "/home/ubuntu/webapps/guardian_advertising_jobs/venv/local/lib/python2.7/site-packages/django/db/models/base.py", line 573, in save_base
   updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)

 File "/home/ubuntu/webapps/guardian_advertising_jobs/venv/local/lib/python2.7/site-packages/django/db/models/base.py", line 654, in _save_table
   result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)

 File "/home/ubuntu/webapps/guardian_advertising_jobs/venv/local/lib/python2.7/site-packages/django/db/models/base.py", line 687, in _do_insert
   using=using, raw=raw)

 File "/home/ubuntu/webapps/guardian_advertising_jobs/venv/local/lib/python2.7/site-packages/django/db/models/manager.py", line 232, in _insert
   return insert_query(self.model, objs, fields, **kwargs)

 File "/home/ubuntu/webapps/guardian_advertising_jobs/venv/local/lib/python2.7/site-packages/django/db/models/query.py", line 1514, in insert_query
   return query.get_compiler(using=using).execute_sql(return_id)

 File "/home/ubuntu/webapps/guardian_advertising_jobs/venv/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 903, in execute_sql
   cursor.execute(sql, params)

 File "/home/ubuntu/webapps/guardian_advertising_jobs/venv/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
   return self.cursor.execute(sql, params)

 File "/home/ubuntu/webapps/guardian_advertising_jobs/venv/local/lib/python2.7/site-packages/django/db/utils.py", line 99, in __exit__
   six.reraise(dj_exc_type, dj_exc_value, traceback)

 File "/home/ubuntu/webapps/guardian_advertising_jobs/venv/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
   return self.cursor.execute(sql, params)

 File "/usr/local/lib/python2.7/dist-packages/newrelic-2.14.0.11/newrelic/hooks/database_dbapi2.py", line 21, in execute
   return self.__wrapped__.execute(sql, parameters)

ProgrammingError: can't adapt type 'HStoreDict'

Libraries used by the site:

Django==1.6.5
Markdown==2.3.1
Pillow==2.3.0
South==0.8.4
astroid==1.0.1
beautifulsoup4==4.3.2
django-historylinks==1.0.3
django-hstore==1.2.3
django-pipeline==1.3.21
django-reversion==1.8.0
django-usertools==1.0.3
feedparser==5.1.3
futures==2.1.6
logilab-common==0.61.0
mock==1.0.1
psycopg2==2.5.2
python-dateutil==2.2
pytz==2013.9
six==1.5.2
uWSGI==2.0.1
wsgiref==0.1.2

Of most interest is the uWSGI library, which might (at a hunch), be doing something unexpected to the startup process (it's pre-forking). That's just a hunch, however.

Question: DictionaryField in schema mode + ForeignKey?

Is it possible to use ForeignKey type with DictionaryField in schema mode?

I am trying to do that as follows:

{
    "name": "testimonial_1_photo",
    "class": "ForeignKey",
    "kwargs": {
        "to": "filer.Image",
        "verbose_name": "Photo",
        "blank": true,
        "null": true
    }
}

I have the form for this model in Django-admin. When opening the page, it returns the following error:

ValueError: Cannot create form field for u'testimonial_1_photo' yet, because its related model u'filer.Image' has not been loaded yet

It seems that filer.Image path is not resolved to the actual model.

Actually I am trying to use django-filer reference fields with DictionaryField with a workaround that instructs django-hstore to accept them in schema. They do not work and I do not know why. But django-filer reference field is a subclass of ForeignKey, so trying to make ForegnKey working first.

Request to move the project

I have a proposal.

Since this project is used by some people, and has attracted attention recently after the growth of interest in schemaless databases, it would be a good idea to let it grow organically.

Unfortunately it seems that the original author does not have much time to review changes. Same applies for the repository that is listed on pypi: https://github.com/aino/django-hstore - this repo is even more outdated.

I propose to create an organization, fork the repository there, assign privileges to 2-3 mantainers, create a mailing list, release a new version on pypi and write a news about these changes.

I would like to know your opinion @jordanm

Thanks
Federico Capoano

How to use with order_by or ordering?

I'd expect to be able to do something like:
Model.objects.all().order_by('data->door_order_no')

Could this be implemented somehow?

Invalid registring hstore extension in a transaction block using atomic context manager.

moving the report by @waypointsoftware on a separate issue:

Do you have any solution for using the extension with a transaction?

def handle_noargs(self, **options):
    with transaction.atomic():
        streams = StreamData.objects.filter(processed=False, duplicate=False)[:10]
        for s in streams:
            print type(s.value_store)

This code produces "str" for each row when a transaction is used, it produces HStoreDict when not in a transaction

Contains lookup doesn't seem to support LIKE 'value%' SQL [was ORM Equivalent of SQL]

Hello, I'm having trouble forming a django ORM query for a query. In plain english, the query sounds like: "select a row where data[my_key] contains the string I'm looking for."

Concretely, consider the following model:

from django.db import models
from django_hstore import hstore


class TestHstore(models.Model):
    name = models.CharField(max_length=250)
    data = hstore.DictionaryField(db_index=True)
    objects = hstore.HStoreManager()

    def __unicode__(self):
        return self.name

The following model instance:

instance = TestHstore.objects.create(name="test", data={"en_us": "english", "french": "fries"})

I'd like to make use the Django ORM that makes a query equivalent to the following SQL:

SELECT * 
FROM hstore_test_testhstore 
WHERE data->'french' LIKE 'fr%';

I can't seem to find a way to do that within the ORM. TestHstore.objects.filter(data__contains={'en_us': "english"}) works for exact matches of the value in the key/value pair, but fails to return a database row when the value doesn't match completely (TestHstore.objects.filter(data__contains={'en_us': "eng"}))

Any Suggestions?

Thanks for this cool package--really great work!

Willingness to accept a PEP8 pull request?

The documentation mentions "Follow PEP8, Style Guide for Python Code" but there are multiple violations in the code base.

Would you be willing to accept a pull request that fixes all of the PEP8 violations?

GeoDjango incompatability

Is my understanding correct that there is no way to use the custom hstore fields in the same model with geospatial fields, which rely on django postgis database backend?

I looked in the docs, and an older version of the doc said this was true, but the current version doesn't seem to mention anything about it, so I wasn't sure if this was an issue.

Values evaluated and stored to string in Admin

Given a model:

class Something(models.Model):
    name = models.CharField(max_length=32)
    data = hstore.DictionaryField() 

If I store a list:

instance = Something.objects.create(name='something', data={'first_key': 'value', 'second_key': [1,2,3]})

when I save using the admin interface, the value the list gets converted to a string:

>>> instance = Something.objects.get(name='something')
>>> instance['second_key']
    u'[1, 2, 3]'

So the values are not preserved in their native type. There is already a fix or somebody working on this?

Schema mode and custom fields

I'm trying to use schema mode with a custom field, but no luck so far.

I'm stripping down the code to make a simple example to post here. Is it supposed to work if schema contains "class" key pointing to a model field class?

HStoreDict get method does not follow the standard python dict get implementation

Python dict method never raises an exception while HStoreDict does.

Support ordering with Django 1.7's custom lookups

This is re-opening issue #31, as we should be able to support ordering now in Django 1.7 which is almost at a stable release, with a custom lookup:
https://docs.djangoproject.com/en/dev/howto/custom-lookups/

Feature proposal: Hstore Lookups with Django 1.7

Django 1.7 comes with awesome new feature: custom lookups https://docs.djangoproject.com/en/dev/ref/models/custom-lookups/

I want implement custom lookups for hstore fields.

This shouldn't break any compatibility because it will be activated only if django-hstore is used with django 1.7

Key name unique feature

Hi,

I did not find it in the docs, but found by testing that through the range of all hstore fields in the Model key names must be unique or the idea of virtual fields get broken.
I wished to try to use Hstore as a multilingual data storage. That means usage of the same language code keys like 'en', 'es' for EVERY field, requiring translation, like title, detail...
As result the model generated is correct, but useless as django-hstore does not understand the field to use.
Sure, the solution could be making one Hstore field with all translations in it, but it would a total code mess for me..

Alas..
Anyway the django-hstore is a nice app, thanks :)

Iho

cannot import name register_hstore

Seems like I'm doing something obvious and wrong, but missing it. I created a new project, set the adapter to

    'ENGINE': 'django_hstore.postgresql_psycopg2',

and add django_hstore to installed_apps

and then try to run dbshell or syncdb or anything, and I get this:

django.core.exceptions.ImproperlyConfigured: 'django_hstore.postgresql_psycopg2' isn't an available database backend.
Try using django.db.backends.XXX, where XXX is one of:
'dummy', 'mysql', 'oracle', 'postgresql', 'postgresql_psycopg2', 'sqlite3'
Error was: cannot import name register_hstore

I tried django 1.3.1 and django 1.2.7

Am I missing something obvious?

Multidatabase setup: other non-postgresql databases should not register HSTORE

in a situation in which postgresql+hstore is the main DB and another non-postgresql relational DB is a secondary the HSTORE function should probably be disabled by default without needing to add HAS_HSTORE = False.

What do you think and if you agree what would be the best way to do it in your opinion?

HStoreDict.__unicode__() returns empty string when dict is empty

Don't know if this is a bug, but the when an HStoreDict does not have any elements in it, the unicode method returns '' rather than '{}'. The main manifestation of this is under django-rest-framework - the browsable api will not render an empty dict in the HTML Form for user editing even though the model field has been initialized with an empty dict (and not a null).

HStoreDict does not convert long integers to strings.

Python "long" integers are not converted to strings by HStoreDict.ensure_acceptable_value(). Usual integers are silently promoted to infinite-sized long integers by Python, and surprisingly long is not a subclass of int.

>>> from django_hstore.dict import HStoreDict as HSD
>>> HSD({'a': 123})
{'a': u'123'}
>>> HSD({'a': 12345678901234567890})
{'a': 12345678901234567890L}
>>> type(HSD({'a': 12345678901234567890})['a'])
<type 'long'>
>>> isinstance(1, int)
True
>>> isinstance(12345678901234567890, int)
False
>>> isinstance(12345678901234567890, long)
True

This can eventually result in a mixed string and integer PostgreSQL array being passed to the hstore() constructor, and PostgreSQL complaining about strings not being integers.

A proposed fix is to add "long" to the list of types in https://github.com/djangonauts/django-hstore/blob/1.3.0/django_hstore/dict.py#L123

Version 1.2.3 will not install from PyPi

pip install --upgrade django-hstore==1.2.3
Downloading/unpacking django-hstore==1.2.3
  Could not find a version that satisfies the requirement django-hstore==1.2.3 (from versions: 1.1.0, 1.1.1, 1.2.1, 1.2.2)
Cleaning up...
No distributions matching the version for django-hstore==1.2.3

And yet it claims to be available on PyPi:

https://pypi.python.org/pypi/django-hstore/1.2.3

Potentially it needs to have a download registered with it?

schema mode does not populate __dict__ correctly

Given this model declaration (based on the documentation):

class SomethingWithJsonSchema(models.Model):
    name = models.CharField(max_length=32)
    data = hstore.DictionaryField(schema=[
        {
            'name': 'number',
            'class': 'IntegerField',
            'kwargs': {
                'default': 0
            }
        },
        {
            'name': 'float',
            'class': 'FloatField',
            'kwargs': {
                'default': 1.0
            }
        },
        {
            'name': 'char',
            'class': 'CharField',
            'kwargs': {
                'default': 'test', 'blank': True, 'max_length': 10
            }
        },
        {
            'name': 'text',
            'class': 'TextField',
            'kwargs': {
                'blank': True
            }
        },
        {
            'name': 'choice',
            'class': 'CharField',
            'kwargs': {
                'blank': True,
                'max_length': 10,
                'choices': (('choice1', 'choice1'), ('choice2', 'choice2'))
            }
        },
        {
            'name': 'json',
            'class': JSONField,
            'kwargs': {
                'blank': True,
                'default': {},
            }
        }

    ])

    objects = hstore.HStoreManager()

I added a virtualfield named json whose class value is jsonfield class. This wasn't documented but looking at the code, you're specifically handling this form of passing in a class object.

Attempting to go through the documentation example:

In [1]: from test.models import SomethingWithJsonSchema

In [2]: obj = SomethingWithJsonSchema()

In [3]: obj.number
Out[3]: 0

In [4]: obj.float
Out[4]: 1.0

In [5]: obj.json
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-5-c1aff2e6837d> in <module>()
----> 1 obj.json

/src/test/env/lib/python2.7/site-packages/jsonfield/subclassing.pyc in __get__(self, obj, type)
     32         if obj is None:
     33             raise AttributeError('Can only be accessed via an instance.')
---> 34         return obj.__dict__[self.field.name]
     35
     36     def __set__(self, obj, value):

KeyError: u'json'

the jsonfield code that's in the stacktrace (link) expects there to be a json key in the object's __dict__ and will specifically write things to __dict__.

Here's an example of the difference in jsonfield vs a normal field for setting:

In [8]: obj.float = 9.99

In [9]: obj.__dict__
Out[9]:
{'_state': <django.db.models.base.ModelState at 0x38a2990>,
 'data': {u'float': 9.99},
 'id': None,
 'name': u''}

In [10]: obj.json = {}

In [11]: obj.__dict__
Out[11]:
{'_state': <django.db.models.base.ModelState at 0x38a2990>,
 'data': {u'float': 9.99},
 'id': None,
 u'json': {},
 'name': u''}

Since jsonfield is a commonly used project, I'd expect one of two things:

• (ideally) coordinate with django-jsonfield to support this kind of field in django-hstore
  or
• update the documentation to note jsonfield as one of the fields that wouldn't "just work", along with FileField, ImageField, and BinaryField.

Add ability to change destination for underscore js media

django_hstore/widgets.py

There is hardcoded external_js to cloudflare. Should be able to easily override to use internal CDN f required or other source.

class BaseAdminHStoreWidget(AdminTextareaWidget):
    """
    Base admin widget class for default-admin and grappelli-admin widgets
    """
    admin_style = 'default'
    @property
    def media(self):
    # load underscore from CDNJS (popular javascript content delivery network)
    external_js = [
    "//cdnjs.cloudflare.com/ajax/libs/underscore.js/1.5.2/underscore-min.js"
    ]
    internal_js = [
    "django_hstore/hstore-widget.js"
    ]

Psycopg2 required version

The register process seems not to work on a psycopg2.4 version:

"AttributeError: module object has no attribute 'new_array_type'"

Updating my psycopg2 version to last one (2.5.2) resolved the issue. Indeed, the function has been introduced in 2.4.3 version:

https://github.com/psycopg/psycopg2/blob/7b82be936dae074217b7c5b9cefe0ec2b227160c/NEWS#L153

I guess a change on prerequisites list should be ok?

Can't adapt type 'HStoreDictionary'

I'm getting a DatabaseError exception, "can't adapt type 'HStoreDictionary'" with a simple test app.

# people/models.py
from django.db import models
from django_hstore import hstore

class Person(models.Model):
    name = models.CharField(max_length=64)
    details = hstore.DictionaryField(db_index=True)
    objects = hstore.Manager()

The db is synced and I can insert records directly with SQL at a psql prompt, including hstore values.

The exception comes when I try to create a record in Python:

>>> Person.objects.create(name='John Doe', details={'email': '[email protected]', 'age': 34})
...
DatabaseError: can't adapt type 'HStoreDictionary'

Ubuntu 10.10, PostgreSQL 9.0.4, psycopg 2.4, Django 1.3.

Failing Tests

Trying to get a stable dev environment up and running, but can't get the tests to pass.

Workflow:

$ v.mk test_env  # Make new virtualenv
$ git clone https://github.com/djangonauts/django-hstore.git
$ cd django-hstore
$ pip install -r requirements.txt
$ pip install Django==1.7
$ python runtests.py
Creating test database for alias 'default'...
.E/Users/anthony/.virtualenvs/test/lib/python2.7/site-packages/django/db/models/base.py:1030: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  if f.blank and raw_value in f.empty_values:

/Users/anthony/.virtualenvs/test/lib/python2.7/site-packages/django/db/models/fields/__init__.py:458: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  if value in self.empty_values:

EE..E...........E...E.......EE.E.........FE....F........EE..........EF...FE....F...F./Users/anthony/Projects/test/django-hstore/django_hstore/query.py:57: RemovedInDjango18Warning: 'is_managed' is deprecated.
  if not transaction.is_managed(using=self.db):

F.......EE.........EF......F.FF
======================================================================
ERROR: test_admin_add (tests.django_hstore_tests.tests.SchemaTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 718, in test_admin_add
    self.assertEqual(d.number, 3)
  File "/Users/anthony/Projects/test/django-hstore/django_hstore/virtual.py", line 49, in __get__
    return getattr(instance, self.hstore_field_name).get(self.name, self.default)
AttributeError: 'str' object has no attribute 'get'

======================================================================
ERROR: test_admin_add_utf8 (tests.django_hstore_tests.tests.SchemaTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 730, in test_admin_add_utf8
    self.assertEqual(d.number, 3)
  File "/Users/anthony/Projects/test/django-hstore/django_hstore/virtual.py", line 49, in __get__
    return getattr(instance, self.hstore_field_name).get(self.name, self.default)
AttributeError: 'str' object has no attribute 'get'

======================================================================
ERROR: test_admin_change (tests.django_hstore_tests.tests.SchemaTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 741, in test_admin_change
    response = self.client.get(url)
  File "/Users/anthony/.virtualenvs/test/lib/python2.7/site-packages/django/test/client.py", line 467, in get
    **extra)
  File "/Users/anthony/.virtualenvs/test/lib/python2.7/site-packages/django/test/client.py", line 285, in get
    return self.generic('GET', path, secure=secure, **r)
  File "/Users/anthony/.virtualenvs/test/lib/python2.7/site-packages/django/test/client.py", line 355, in generic
    return self.request(**r)
  File "/Users/anthony/.virtualenvs/test/lib/python2.7/site-packages/django/test/client.py", line 437, in request
    six.reraise(*exc_info)
  File "/Users/anthony/.virtualenvs/test/lib/python2.7/site-packages/django/core/handlers/base.py", line 111, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/Users/anthony/.virtualenvs/test/lib/python2.7/site-packages/django/contrib/admin/options.py", line 567, in wrapper
    return self.admin_site.admin_view(view)(*args, **kwargs)
  File "/Users/anthony/.virtualenvs/test/lib/python2.7/site-packages/django/utils/decorators.py", line 105, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/Users/anthony/.virtualenvs/test/lib/python2.7/site-packages/django/views/decorators/cache.py", line 52, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "/Users/anthony/.virtualenvs/test/lib/python2.7/site-packages/django/contrib/admin/sites.py", line 204, in inner
    return view(request, *args, **kwargs)
  File "/Users/anthony/.virtualenvs/test/lib/python2.7/site-packages/django/contrib/admin/options.py", line 1440, in change_view
    return self.changeform_view(request, object_id, form_url, extra_context)
  File "/Users/anthony/.virtualenvs/test/lib/python2.7/site-packages/django/utils/decorators.py", line 29, in _wrapper
    return bound_func(*args, **kwargs)
  File "/Users/anthony/.virtualenvs/test/lib/python2.7/site-packages/django/utils/decorators.py", line 105, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/Users/anthony/.virtualenvs/test/lib/python2.7/site-packages/django/utils/decorators.py", line 25, in bound_func
    return func.__get__(self, type(self))(*args2, **kwargs2)
  File "/Users/anthony/.virtualenvs/test/lib/python2.7/site-packages/django/db/transaction.py", line 394, in inner
    return func(*args, **kwargs)
  File "/Users/anthony/.virtualenvs/test/lib/python2.7/site-packages/django/contrib/admin/options.py", line 1403, in changeform_view
    form = ModelForm(instance=obj)
  File "/Users/anthony/.virtualenvs/test/lib/python2.7/site-packages/django/forms/models.py", line 319, in __init__
    object_data = model_to_dict(instance, opts.fields, opts.exclude)
  File "/Users/anthony/.virtualenvs/test/lib/python2.7/site-packages/django/forms/models.py", line 151, in model_to_dict
    data[f.name] = f.value_from_object(instance)
  File "/Users/anthony/.virtualenvs/test/lib/python2.7/site-packages/django/db/models/fields/__init__.py", line 848, in value_from_object
    return getattr(obj, self.attname)
  File "/Users/anthony/Projects/test/django-hstore/django_hstore/virtual.py", line 49, in __get__
    return getattr(instance, self.hstore_field_name).get(self.name, self.default)
AttributeError: 'str' object has no attribute 'get'

======================================================================
ERROR: test_create (tests.django_hstore_tests.tests.SchemaTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 844, in test_create
    self.assertEqual(s1.number, 2)
  File "/Users/anthony/Projects/test/django-hstore/django_hstore/virtual.py", line 49, in __get__
    return getattr(instance, self.hstore_field_name).get(self.name, self.default)
AttributeError: 'str' object has no attribute 'get'

======================================================================
ERROR: test_schemadatabag_save (tests.django_hstore_tests.tests.SchemaTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 680, in test_schemadatabag_save
    self.assertEqual(d.number, 4)
  File "/Users/anthony/Projects/test/django-hstore/django_hstore/virtual.py", line 49, in __get__
    return getattr(instance, self.hstore_field_name).get(self.name, self.default)
AttributeError: 'str' object has no attribute 'get'

======================================================================
ERROR: test_utf8 (tests.django_hstore_tests.tests.SchemaTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 821, in test_utf8
    self.assertEqual(d.char, u'è')
  File "/Users/anthony/Projects/test/django-hstore/django_hstore/virtual.py", line 49, in __get__
    return getattr(instance, self.hstore_field_name).get(self.name, self.default)
AttributeError: 'str' object has no attribute 'get'

======================================================================
ERROR: test_boolean (tests.django_hstore_tests.tests.TestDictionaryField)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 110, in test_boolean
    self.assertEqual(json.loads(databag.data['boolean']), True)
TypeError: string indices must be integers, not str

======================================================================
ERROR: test_decimal (tests.django_hstore_tests.tests.TestDictionaryField)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 62, in test_decimal
    self.assertEqual(databag.data['dec'], force_text(Decimal('1.01')))
TypeError: string indices must be integers, not str

======================================================================
ERROR: test_dictionary (tests.django_hstore_tests.tests.TestDictionaryField)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 101, in test_dictionary
    self.assertEqual(json.loads(databag.data['dict']), {'subkey': 'subvalue'})
TypeError: string indices must be integers, not str

======================================================================
ERROR: test_hslice (tests.django_hstore_tests.tests.TestDictionaryField)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 405, in test_hslice
    self.assertEqual(DataBag.objects.hslice(id=alpha.id, attr='data', keys=['v']), {'v': '1'})
  File "/Users/anthony/Projects/test/django-hstore/django_hstore/managers.py", line 33, in hslice
    return self.filter(**params).hslice(attr, keys)
  File "/Users/anthony/Projects/test/django-hstore/django_hstore/query.py", line 46, in selector
    return method(self, query, *args, **params)
  File "/Users/anthony/Projects/test/django-hstore/django_hstore/query.py", line 294, in hslice
    return dict((key, field._value_to_python(value)) for key, value in result[0].items())
AttributeError: 'str' object has no attribute 'items'

======================================================================
ERROR: test_list (tests.django_hstore_tests.tests.TestDictionaryField)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 96, in test_list
    self.assertEqual(json.loads(databag.data['list']), ['a', 'b', 'c'])
TypeError: string indices must be integers, not str

======================================================================
ERROR: test_long (tests.django_hstore_tests.tests.TestDictionaryField)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 76, in test_long
    self.assertEqual(databag.data['long'], force_text(l))
TypeError: string indices must be integers, not str

======================================================================
ERROR: test_number (tests.django_hstore_tests.tests.TestDictionaryField)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 88, in test_number
    self.assertEqual(databag.data['num'], '1')
TypeError: string indices must be integers, not str

======================================================================
ERROR: test_serialization_deserialization (tests.django_hstore_tests.tests.TestDictionaryField)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 430, in test_serialization_deserialization
    self.assertEqual(json.loads(str(DataBag.objects.get(name='alpha').data)), json.loads(str(alpha.data)))
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 368, in decode
    raise ValueError(errmsg("Extra data", s, end, len(s)))
ValueError: Extra data: line 1 column 4 - line 1 column 20 (char 3 - 19)

======================================================================
ERROR: test_hremove (tests.django_hstore_tests.tests.TestReferencesField)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 1057, in test_hremove
    self.assertEqual(RefsBag.objects.get(name='alpha').refs['0'], alpha.refs['0'])
  File "/Users/anthony/Projects/test/django-hstore/django_hstore/dict.py", line 149, in __getitem__
    value = super(self.__class__, self).__getitem__(*args, **kwargs)
  File "/Users/anthony/Projects/test/django-hstore/django_hstore/dict.py", line 73, in __getitem__
    value = super(HStoreDict, self).__getitem__(*args, **kwargs)
KeyError: '0'

======================================================================
ERROR: test_hslice (tests.django_hstore_tests.tests.TestReferencesField)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 1071, in test_hslice
    self.assertEqual(RefsBag.objects.hslice(id=alpha.id, attr='refs', keys=['0']), {'0': refs[0]})
  File "/Users/anthony/Projects/test/django-hstore/django_hstore/managers.py", line 33, in hslice
    return self.filter(**params).hslice(attr, keys)
  File "/Users/anthony/Projects/test/django-hstore/django_hstore/query.py", line 46, in selector
    return method(self, query, *args, **params)
  File "/Users/anthony/Projects/test/django-hstore/django_hstore/query.py", line 294, in hslice
    return dict((key, field._value_to_python(value)) for key, value in result[0].items())
AttributeError: 'str' object has no attribute 'items'

======================================================================
ERROR: test_simple_retrieval (tests.django_hstore_tests.tests.TestReferencesField)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 995, in test_simple_retrieval
    self.assertEqual(Ref.objects.get(name='0'), alpha.refs['0'])
  File "/Users/anthony/Projects/test/django-hstore/django_hstore/dict.py", line 149, in __getitem__
    value = super(self.__class__, self).__getitem__(*args, **kwargs)
  File "/Users/anthony/Projects/test/django-hstore/django_hstore/dict.py", line 73, in __getitem__
    value = super(HStoreDict, self).__getitem__(*args, **kwargs)
KeyError: '0'

======================================================================
FAIL: test_hremove (tests.django_hstore_tests.tests.TestDictionaryField)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 395, in test_hremove
    self.assertEqual(DataBag.objects.get(name='alpha').data, alpha.data)
AssertionError: '"v"=>"1", "v2"=>"3"' != {'v2': '3', 'v': '1'}

======================================================================
FAIL: test_hupdate (tests.django_hstore_tests.tests.TestDictionaryField)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 411, in test_hupdate
    self.assertEqual(DataBag.objects.get(name='alpha').data, alpha.data)
AssertionError: '"v"=>"1", "v2"=>"3"' != {'v2': '3', 'v': '1'}

======================================================================
FAIL: test_properties_hstore (tests.django_hstore_tests.tests.TestDictionaryField)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 589, in test_properties_hstore
    self.assertEqual(type(instance.data), HStoreDict)
AssertionError: <type 'str'> != <class 'django_hstore.dict.HStoreDict'>

======================================================================
FAIL: test_replace_full_dictionary (tests.django_hstore_tests.tests.TestDictionaryField)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 229, in test_replace_full_dictionary
    self.assertEqual(replacement, DataBag.objects.get(name='foo').data)
AssertionError: {'added': 'new', 'change': 'new value'} != '"added"=>"new", "change"=>"new value"'

======================================================================
FAIL: test_unicode_processing (tests.django_hstore_tests.tests.TestDictionaryField)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 207, in test_unicode_processing
    self.assertEqual(greets, DataBag.objects.get(name='multilang').data)
AssertionError: {u'en': u'hello, world', u'zh': u'\u4f60\u597d\uff0c\u4e16\u754c', u'de': u'Gr\xfc\xdfe, Welt', u'jp': u'\u3053\u3093\u306b\u3061\u306f\u3001\u4e16\u754c', u'es': u'hola, ma\xf1ana', u'he': u'\u05e9\u05dc\u05d5\u05dd, \u05e2\u05d5\u05dc\u05dd'} != '"de"=>"Gr\xc3\xbc\xc3\x9fe, Welt", "en"=>"hello, world", "es"=>"hola, ma\xc3\xb1ana", "he"=>"\xd7\xa9\xd7\x9c\xd7\x95\xd7\x9d, \xd7\xa2\xd7\x95\xd7\x9c\xd7\x9d", "jp"=>"\xe3\x81\x93\xe3\x82\x93\xe3\x81\xab\xe3\x81\xa1\xe3\x81\xaf\xe3\x80\x81\xe4\xb8\x96\xe7\x95\x8c", "zh"=>"\xe4\xbd\xa0\xe5\xa5\xbd\xef\xbc\x8c\xe4\xb8\x96\xe7\x95\x8c"'

======================================================================
FAIL: test_location_create (tests.django_hstore_tests.tests.TestDictionaryFieldPlusGIS)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 1216, in test_location_create
    self.assertEqual(other_loc.data, {'prop1': '1', 'prop2': 'test_value'})
AssertionError: '"prop1"=>"1", "prop2"=>"test_value"' != {'prop1': '1', 'prop2': 'test_value'}

======================================================================
FAIL: test_location_hupdate (tests.django_hstore_tests.tests.TestDictionaryFieldPlusGIS)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 1222, in test_location_hupdate
    self.assertEqual(loc.data, {'prop1': '2', 'prop2': 'test_value'})
AssertionError: '"prop1"=>"2", "prop2"=>"test_value"' != {'prop1': '2', 'prop2': 'test_value'}

======================================================================
FAIL: test_simple_retrieval_get (tests.django_hstore_tests.tests.TestReferencesField)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 1000, in test_simple_retrieval_get
    self.assertEqual(Ref.objects.get(name='0'), alpha.refs.get('0'))
AssertionError: <Ref: Ref object> != None

======================================================================
FAIL: test_location_create (tests.django_hstore_tests.tests.TestReferencesFieldPlusGIS)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 1248, in test_location_create
    self.assertEqual(loc_1.data, {'prop1': '1', 'prop2': 'test_value'})
AssertionError: '"prop1"=>"1", "prop2"=>"test_value"' != {'prop1': '1', 'prop2': 'test_value'}

======================================================================
FAIL: test_location_hupdate (tests.django_hstore_tests.tests.TestReferencesFieldPlusGIS)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 1256, in test_location_hupdate
    self.assertEqual(loc.data, {'prop1': '2', 'prop2': 'test_value'})
AssertionError: '"prop1"=>"2", "prop2"=>"test_value"' != {'prop1': '2', 'prop2': 'test_value'}

======================================================================
FAIL: test_hstore_registring_in_transaction_block (tests.django_hstore_tests.tests.NotTransactionalTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/anthony/Projects/test/django-hstore/tests/django_hstore_tests/tests.py", line 942, in test_hstore_registring_in_transaction_block
    self.assertIsInstance(qs[0].data, HStoreDict)
AssertionError: '"v"=>"1", "v2"=>"3"' is not an instance of <class 'django_hstore.dict.HStoreDict'>

----------------------------------------------------------------------
Ran 118 tests in 2.052s

FAILED (failures=11, errors=17)
Destroying test database for alias 'default'...

This is using Python 2.7.6. Postgres 9.3.4.

Any ideas of what I'm missing?

Add a test suite for migrations

Seems like proper behavior when dealing with migrations have been troubling you. Maybe it'd be useful adding a test suite around proper migration output for south/1.7?

I could pitch in later this week if you think a migration test suite belongs to the project (I think it does).

DictionaryField + south schemamigration results in wrong default field value.

I'm using the latest 1.3.0 release and noticed something odd using the DictionaryField and South (0.8.4) schemamigrations.

I have the following field:

stats = hstore.DictionaryField(
    null=True, schema=[{
        'name': 'total', 'class': 'IntegerField', 'kwargs': {'default': 0}}])

Now when I add a schemamigration for this new field, I end up with a wrong default value on the migration:

def forwards(self, orm):
    # Adding field 'User.stats'
    db.add_column(
        u'users_user', 'stats', self.gf(u'django_hstore.fields.DictionaryField')(default={}, null=True), keep_default=False)

Running this migration results in:

DEBUG 2014-08-06 12:51:22,180 django.db.backends (0.000) ALTER TABLE "users_user" ADD COLUMN "stats" hstore NULL DEFAULT ;; args=[]
FATAL ERROR - The following SQL query failed: ALTER TABLE "users_user" ADD COLUMN "stats" hstore NULL DEFAULT ;
The error was: syntax error at or near ";"
LINE 1: ... TABLE "users_user" ADD COLUMN "stats" hstore NULL DEFAULT ;

The default value on the migration should be a None instead of an empty Dict {}.

To fix this I have to change the default value to None in the forwards migration and in the frozen models.

def forwards(self, orm):
    # Adding field 'User.stats'
    db.add_column(
        u'users_user', 'stats', self.gf(u'django_hstore.fields.DictionaryField')(default=None, null=True), keep_default=False)

Do you know what might be causing this?

How can I use "update()", not "save()" with hstore field?

I'm looking for something like:

Model.objects.filter(pk=f.id).update(debug__v=v)

Where debug is a hstore field, and __v is a key: debug["v"].
I'm looking for this because I want to update hstore field, but do not call post_save functions. (in case of save())

support theme Django Suit

I added hot fix for support theme django suit.

django_hstore/widgets.py

class SuitAdminHStoreWidget(BaseAdminHStoreWidget):
    """
    Widget that displays the HStore contents
    in the suit django-admin with a nice interactive UI
    """
    admin_style = 'suit'

templates/hstore_suit_widget.html

{% load i18n %}
<script type="text/html" id="hstore-row-template-{{ field_name }}" class="hstore-row-template-inline">
    <div class="form-row field-data" style="padding: 5px 15px;">
        <div>
            <input type="text" placeholder="{% trans 'key' %}" value="<%= key %>" style="min-width:150px;">
            &nbsp; <strong>:</strong> &nbsp;&nbsp;
            <input style="min-width:300px;"
                   value="<%= _.escape(value) %>"
                   type="text"
                   placeholder="{% trans 'value' %}">
            &nbsp;&nbsp;
            <a href="#" class="remove-row" title="{% trans 'remove row' %}">
                <img src="{{ STATIC_URL }}admin/img/icon_deletelink.gif" width="10" height="10">
            </a>
        </div>
    </div>
</script>

<script type="text/html" id="hstore-ui-template-{{ field_name }}"{% if '__prefix__' in field_name %} class="hstore-ui-template-inline"{% endif %}>

    <div class="hstore inline-group inline-related" id="hstore-{{ field_name }}">
        <h2><%= label %></h2>
        <h3><%= label %></h3>

        <% if(help && help != '') { %>
            <div class="form-row">
                <p class="help" style="margin:0;padding:0"><%= help %></p>
            </div>
        <% } %>

        <div class="hstore-rows" style="background-color: #f9f9f9;">
            <% if(errors){ %>
                <div class="form-row field-data">
                    <div>
                        <ul class="errorlist">
                            <%= errors %>
                        </ul>
                    </div>
                </div>
            <% } %>
            <% for(key in data){ %>
                <%= _.template($('.hstore-row-template-inline').eq(0).html(), { 'key': key, 'value': data[key] }) %>
            <% } %>
        </div>

        <div class="form-row field-data hstore-textarea" style="display:none">
            <div>
                <label for="<%= id %>" class="required">{% trans 'Raw textarea' %}:</label>
                <textarea class="vLargeTextField" cols="40" id="<%= id %>" name="<%= name %>" rows="10"><%= value %></textarea>
            </div>
        </div>

        <div class="form-row add-row">
            <a href="#" class="hs-add-row" title="{% trans 'Add another row' %}">
                <img src="{{ STATIC_URL }}admin/img/icon_addlink.gif" width="10" height="10" alt="{% trans 'Add Another' %}">
                {% trans "Add row" %}
            </a>

            <a href="#" class="hstore-toggle-txtarea" title="{% trans 'toggle textarea' %}" style="float:right; padding-right: 10px">
                <img src="{{ STATIC_URL }}admin/img/icon_changelink.gif" width="10" height="10" alt="{% trans 'toggle textarea' %}">
                {% trans 'toggle textarea' %}
            </a>
        </div>
    </div>
</script>

<script>django.jQuery(function() { initDjangoHStoreWidget('{{ field_name }}') });</script>

Error when running tests

I'm getting the following error when testing apps in my project:

./manage.py test <app name>
AttributeError: 'DatabaseWrapper' object has no attribute '_version'

This appears to be caused by the django_hstore.postgresql_psycopg2 as changing back to django.db.backends.postgresql_psycopg2 does not raise the error.

Anyone else having similar issues?

Widget not working in Admin TabularInline

The form field breaks inlines in the admin.

Establish deprecation policy

We should establish some deprecation policy for django versions.
My proposition: support last two stable django versions.

Filtering by a tag not working?

I created a Way model, like this:

from django_hstore import hstore

class Way(models.Model):
tags = hstore.DictionaryField(db_index=True, null=True)

and then did this

way = Way.objects.all()[1]

print way.tags

Out[8]:
{'name': 'Northeast Butler Street',
...other data here...
}

then did this:

print Way.objects.filter(tags__icontains={'name' : way.tags['name']})

and got nothing

Out[12]: []

Am I using the filter wrong?