Comments (14)
failing test case please
from django-hstore.
We're running into a similar problem. When switching from the django.db.backends.postgresql_psycopg2
backend to django.contrib.gis.db.backends.postgis
, all of our tests relying on Django Hstore suddenly start failing. On further investigation, we found that the hstore.DictionaryField
was returning a string rather than a dictionary (in issue #79, notice the last test assertion failure is doing the exact same thing: AssertionError: '"v"=>"1", "v2"=>"3"' is not an instance of ...
). It appears that this is what @ddohler is experiencing (TypeError: string indices must be integers, not str
because thing2.data
is a str
rather than a dict
).
This leads me to believe that there are some issues surrounding using django-hstore
with PostGIS
. @nemesisdesign could you comment on what version of PostGIS
is being used in the Travis tests?
We are using:
posgresql 9.3.4
postgis 2.1.2 r12389
("POSTGIS="2.1.2 r12389" GEOS="3.4.2-CAPI-1.8.2 r3921" PROJ="Rel. 4.8.0, 6 March 2012" GDAL="GDAL 1.10.1, released 2013/08/26" LIBXML="2.9.1" LIBJSON="UNKNOWN" RASTER"
)
django 1.7.4
django-hstore 1.2.4
@ddohler Can you please post your versions?
from django-hstore.
According to http://docs.travis-ci.com/user/using-postgresql/ and https://github.com/djangonauts/django-hstore/blob/master/.travis.yml we have postgresql 9.1 and postgis 2.1.
it's correct, this started happening after 8ac21be...2e9164f before then the "CREATE EXTENSION HSTORE;" command was being called at each connection. That was cleaned up, but it caused my tests to fail at that time too.
@alukach what happens if you run: "SELECT t.oid FROM pg_type t JOIN pg_namespace ns ON typnamespace = ns.oid WHERE typname = 'hstore' and nspname = 'public';" on your database?
(the broken one I mean)
from django-hstore.
I think it has to do with some postgresql internals we are missing.
Have you also tried doing something like:
DROP EXTENSION hstore;
CREATE EXTENSION hstore;
from django-hstore.
Okay, I feel like I figured it out. It's a pretty strange one.
I was able to reproduce this bug with the two following conditions:
- Creating a DB off of a template DB that did not have
hstore
enabled. - Using the
django.contrib.gis.db.backends.postgis
backend.
If you create a django project database from a template that does not have hstore
enabled, for some reason the postgis
backend will not properly use hstore
on any test database that is created and instead return the data as a string ("prop1"=>"1", "prop2"=>"test_value"
). I had originally thought that the test database was created only from the template database at test runtime, but this does not appear to be true (EDIT: Read below, this is not the issue). If you enable hstore
on the template database and the django project database used, django-hstore
will still fail on tests if you did not originally create the django project database from a template with the hstore
extension. This bug does not occur with the django.db.backends.postgresql_psycopg2
backend.
I am still not sure why this exists but was able to reproduce the issue as shown below. Currently, my only suggestion to resolve this issue is to drop the django project database and recreate it from a template with the hstore
extension enabled. Obviously, this is extremely inconvenient to do with a production database (backing up the data, renaming the db, creating a new db, loading in the data). There is probably a better solution but, at this time, I am unsure of what that is.
Example:
$ mkvirtualenv hstore
New python executable in hstore/bin/python
Installing setuptools, pip...done.
(hstore)$ git clone https://github.com/djangonauts/django-hstore.git
(hstore)$ cd django-hstore/
(hstore)$ pip install -r requirements.txt
(hstore)$ pip install django
(hstore)$ createdb django_hstore # template1 currently has hstore enabled
(hstore)$ python runtests.py # Tests will pass
Creating test database for alias 'default'...
../home/alukach/.virtualenvs/hstore/local/lib/python2.7/site-packages/django/db/models/base.py:1029: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
if f.blank and raw_value in f.empty_values:
/home/alukach/.virtualenvs/hstore/local/lib/python2.7/site-packages/django/db/models/fields/__init__.py:458: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
if value in self.empty_values:
........................................./home/alukach/Projects/django-hstore/django_hstore/query.py:57: RemovedInDjango18Warning: 'is_managed' is deprecated.
if not transaction.is_managed(using=self.db):
...........................................................................
----------------------------------------------------------------------
Ran 118 tests in 1.915s
OK
Destroying test database for alias 'default'...
(hstore)$ psql django_hstore -c "SELECT t.oid, typarray FROM pg_type t JOIN pg_namespace ns ON typnamespace = ns.oid WHERE typname = 'hstore';"
oid | typarray
--------+----------
907772 | 907777
(1 row)
(hstore)$ dropdb django_hstore # Drop db
(hstore)$ psql template1 -c "DROP EXTENSION hstore;" # Remove hstore support on template
DROP EXTENSION
(hstore)$ createdb django_hstore # Recreate DB from template without hstore
(hstore)$ psql template1 -c "CREATE EXTENSION hstore;" # Add hstore to template (test DBs should be based off of this, right?)
CREATE EXTENSION
(hstore)$ psql django_hstore -c "CREATE EXTENSION hstore;" # Add hstore to created DB
CREATE EXTENSION
(hstore)$ python runtests.py # Tests will fail
Creating test database for alias 'default'...
.E/home/alukach/.virtualenvs/hstore/local/lib/python2.7/site-packages/django/db/models/base.py:1029: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
if f.blank and raw_value in f.empty_values:
/home/alukach/.virtualenvs/hstore/local/lib/python2.7/site-packages/django/db/models/fields/__init__.py:458: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
if value in self.empty_values:
EE..E...........E...E.......EE.E.........FE....F........EE..........EF...FE....F...F./home/alukach/Projects/django-hstore/django_hstore/query.py:57: RemovedInDjango18Warning: 'is_managed' is deprecated.
if not transaction.is_managed(using=self.db):
F.......EE.........EF......F.FF
======================================================================
ERROR: test_admin_add (tests.django_hstore_tests.tests.SchemaTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 718, in test_admin_add
self.assertEqual(d.number, 3)
File "/home/alukach/Projects/django-hstore/django_hstore/virtual.py", line 49, in __get__
return getattr(instance, self.hstore_field_name).get(self.name, self.default)
AttributeError: 'str' object has no attribute 'get'
======================================================================
ERROR: test_admin_add_utf8 (tests.django_hstore_tests.tests.SchemaTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 730, in test_admin_add_utf8
self.assertEqual(d.number, 3)
File "/home/alukach/Projects/django-hstore/django_hstore/virtual.py", line 49, in __get__
return getattr(instance, self.hstore_field_name).get(self.name, self.default)
AttributeError: 'str' object has no attribute 'get'
======================================================================
ERROR: test_admin_change (tests.django_hstore_tests.tests.SchemaTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 741, in test_admin_change
response = self.client.get(url)
File "/home/alukach/.virtualenvs/hstore/local/lib/python2.7/site-packages/django/test/client.py", line 470, in get
**extra)
File "/home/alukach/.virtualenvs/hstore/local/lib/python2.7/site-packages/django/test/client.py", line 286, in get
return self.generic('GET', path, secure=secure, **r)
File "/home/alukach/.virtualenvs/hstore/local/lib/python2.7/site-packages/django/test/client.py", line 358, in generic
return self.request(**r)
File "/home/alukach/.virtualenvs/hstore/local/lib/python2.7/site-packages/django/test/client.py", line 440, in request
six.reraise(*exc_info)
File "/home/alukach/.virtualenvs/hstore/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 111, in get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/alukach/.virtualenvs/hstore/local/lib/python2.7/site-packages/django/contrib/admin/options.py", line 583, in wrapper
return self.admin_site.admin_view(view)(*args, **kwargs)
File "/home/alukach/.virtualenvs/hstore/local/lib/python2.7/site-packages/django/utils/decorators.py", line 105, in _wrapped_view
response = view_func(request, *args, **kwargs)
File "/home/alukach/.virtualenvs/hstore/local/lib/python2.7/site-packages/django/views/decorators/cache.py", line 52, in _wrapped_view_func
response = view_func(request, *args, **kwargs)
File "/home/alukach/.virtualenvs/hstore/local/lib/python2.7/site-packages/django/contrib/admin/sites.py", line 206, in inner
return view(request, *args, **kwargs)
File "/home/alukach/.virtualenvs/hstore/local/lib/python2.7/site-packages/django/contrib/admin/options.py", line 1456, in change_view
return self.changeform_view(request, object_id, form_url, extra_context)
File "/home/alukach/.virtualenvs/hstore/local/lib/python2.7/site-packages/django/utils/decorators.py", line 29, in _wrapper
return bound_func(*args, **kwargs)
File "/home/alukach/.virtualenvs/hstore/local/lib/python2.7/site-packages/django/utils/decorators.py", line 105, in _wrapped_view
response = view_func(request, *args, **kwargs)
File "/home/alukach/.virtualenvs/hstore/local/lib/python2.7/site-packages/django/utils/decorators.py", line 25, in bound_func
return func.__get__(self, type(self))(*args2, **kwargs2)
File "/home/alukach/.virtualenvs/hstore/local/lib/python2.7/site-packages/django/db/transaction.py", line 394, in inner
return func(*args, **kwargs)
File "/home/alukach/.virtualenvs/hstore/local/lib/python2.7/site-packages/django/contrib/admin/options.py", line 1419, in changeform_view
form = ModelForm(instance=obj)
File "/home/alukach/.virtualenvs/hstore/local/lib/python2.7/site-packages/django/forms/models.py", line 319, in __init__
object_data = model_to_dict(instance, opts.fields, opts.exclude)
File "/home/alukach/.virtualenvs/hstore/local/lib/python2.7/site-packages/django/forms/models.py", line 151, in model_to_dict
data[f.name] = f.value_from_object(instance)
File "/home/alukach/.virtualenvs/hstore/local/lib/python2.7/site-packages/django/db/models/fields/__init__.py", line 848, in value_from_object
return getattr(obj, self.attname)
File "/home/alukach/Projects/django-hstore/django_hstore/virtual.py", line 49, in __get__
return getattr(instance, self.hstore_field_name).get(self.name, self.default)
AttributeError: 'str' object has no attribute 'get'
======================================================================
ERROR: test_create (tests.django_hstore_tests.tests.SchemaTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 844, in test_create
self.assertEqual(s1.number, 2)
File "/home/alukach/Projects/django-hstore/django_hstore/virtual.py", line 49, in __get__
return getattr(instance, self.hstore_field_name).get(self.name, self.default)
AttributeError: 'str' object has no attribute 'get'
======================================================================
ERROR: test_schemadatabag_save (tests.django_hstore_tests.tests.SchemaTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 680, in test_schemadatabag_save
self.assertEqual(d.number, 4)
File "/home/alukach/Projects/django-hstore/django_hstore/virtual.py", line 49, in __get__
return getattr(instance, self.hstore_field_name).get(self.name, self.default)
AttributeError: 'str' object has no attribute 'get'
======================================================================
ERROR: test_utf8 (tests.django_hstore_tests.tests.SchemaTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 821, in test_utf8
self.assertEqual(d.char, u'รจ')
File "/home/alukach/Projects/django-hstore/django_hstore/virtual.py", line 49, in __get__
return getattr(instance, self.hstore_field_name).get(self.name, self.default)
AttributeError: 'str' object has no attribute 'get'
======================================================================
ERROR: test_boolean (tests.django_hstore_tests.tests.TestDictionaryField)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 110, in test_boolean
self.assertEqual(json.loads(databag.data['boolean']), True)
TypeError: string indices must be integers, not str
======================================================================
ERROR: test_decimal (tests.django_hstore_tests.tests.TestDictionaryField)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 62, in test_decimal
self.assertEqual(databag.data['dec'], force_text(Decimal('1.01')))
TypeError: string indices must be integers, not str
======================================================================
ERROR: test_dictionary (tests.django_hstore_tests.tests.TestDictionaryField)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 101, in test_dictionary
self.assertEqual(json.loads(databag.data['dict']), {'subkey': 'subvalue'})
TypeError: string indices must be integers, not str
======================================================================
ERROR: test_hslice (tests.django_hstore_tests.tests.TestDictionaryField)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 405, in test_hslice
self.assertEqual(DataBag.objects.hslice(id=alpha.id, attr='data', keys=['v']), {'v': '1'})
File "/home/alukach/Projects/django-hstore/django_hstore/managers.py", line 33, in hslice
return self.filter(**params).hslice(attr, keys)
File "/home/alukach/Projects/django-hstore/django_hstore/query.py", line 46, in selector
return method(self, query, *args, **params)
File "/home/alukach/Projects/django-hstore/django_hstore/query.py", line 294, in hslice
return dict((key, field._value_to_python(value)) for key, value in result[0].items())
AttributeError: 'str' object has no attribute 'items'
======================================================================
ERROR: test_list (tests.django_hstore_tests.tests.TestDictionaryField)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 96, in test_list
self.assertEqual(json.loads(databag.data['list']), ['a', 'b', 'c'])
TypeError: string indices must be integers, not str
======================================================================
ERROR: test_long (tests.django_hstore_tests.tests.TestDictionaryField)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 76, in test_long
self.assertEqual(databag.data['long'], force_text(l))
TypeError: string indices must be integers, not str
======================================================================
ERROR: test_number (tests.django_hstore_tests.tests.TestDictionaryField)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 88, in test_number
self.assertEqual(databag.data['num'], '1')
TypeError: string indices must be integers, not str
======================================================================
ERROR: test_serialization_deserialization (tests.django_hstore_tests.tests.TestDictionaryField)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 430, in test_serialization_deserialization
self.assertEqual(json.loads(str(DataBag.objects.get(name='alpha').data)), json.loads(str(alpha.data)))
File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 369, in decode
raise ValueError(errmsg("Extra data", s, end, len(s)))
ValueError: Extra data: line 1 column 4 - line 1 column 20 (char 3 - 19)
======================================================================
ERROR: test_hremove (tests.django_hstore_tests.tests.TestReferencesField)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 1057, in test_hremove
self.assertEqual(RefsBag.objects.get(name='alpha').refs['0'], alpha.refs['0'])
File "/home/alukach/Projects/django-hstore/django_hstore/dict.py", line 149, in __getitem__
value = super(self.__class__, self).__getitem__(*args, **kwargs)
File "/home/alukach/Projects/django-hstore/django_hstore/dict.py", line 73, in __getitem__
value = super(HStoreDict, self).__getitem__(*args, **kwargs)
KeyError: '0'
======================================================================
ERROR: test_hslice (tests.django_hstore_tests.tests.TestReferencesField)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 1071, in test_hslice
self.assertEqual(RefsBag.objects.hslice(id=alpha.id, attr='refs', keys=['0']), {'0': refs[0]})
File "/home/alukach/Projects/django-hstore/django_hstore/managers.py", line 33, in hslice
return self.filter(**params).hslice(attr, keys)
File "/home/alukach/Projects/django-hstore/django_hstore/query.py", line 46, in selector
return method(self, query, *args, **params)
File "/home/alukach/Projects/django-hstore/django_hstore/query.py", line 294, in hslice
return dict((key, field._value_to_python(value)) for key, value in result[0].items())
AttributeError: 'str' object has no attribute 'items'
======================================================================
ERROR: test_simple_retrieval (tests.django_hstore_tests.tests.TestReferencesField)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 995, in test_simple_retrieval
self.assertEqual(Ref.objects.get(name='0'), alpha.refs['0'])
File "/home/alukach/Projects/django-hstore/django_hstore/dict.py", line 149, in __getitem__
value = super(self.__class__, self).__getitem__(*args, **kwargs)
File "/home/alukach/Projects/django-hstore/django_hstore/dict.py", line 73, in __getitem__
value = super(HStoreDict, self).__getitem__(*args, **kwargs)
KeyError: '0'
======================================================================
FAIL: test_hremove (tests.django_hstore_tests.tests.TestDictionaryField)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 395, in test_hremove
self.assertEqual(DataBag.objects.get(name='alpha').data, alpha.data)
AssertionError: '"v"=>"1", "v2"=>"3"' != {'v2': '3', 'v': '1'}
======================================================================
FAIL: test_hupdate (tests.django_hstore_tests.tests.TestDictionaryField)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 411, in test_hupdate
self.assertEqual(DataBag.objects.get(name='alpha').data, alpha.data)
AssertionError: '"v"=>"1", "v2"=>"3"' != {'v2': '3', 'v': '1'}
======================================================================
FAIL: test_properties_hstore (tests.django_hstore_tests.tests.TestDictionaryField)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 589, in test_properties_hstore
self.assertEqual(type(instance.data), HStoreDict)
AssertionError: <type 'str'> != <class 'django_hstore.dict.HStoreDict'>
======================================================================
FAIL: test_replace_full_dictionary (tests.django_hstore_tests.tests.TestDictionaryField)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 229, in test_replace_full_dictionary
self.assertEqual(replacement, DataBag.objects.get(name='foo').data)
AssertionError: {'added': 'new', 'change': 'new value'} != '"added"=>"new", "change"=>"new value"'
======================================================================
FAIL: test_unicode_processing (tests.django_hstore_tests.tests.TestDictionaryField)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 207, in test_unicode_processing
self.assertEqual(greets, DataBag.objects.get(name='multilang').data)
AssertionError: {u'en': u'hello, world', u'zh': u'\u4f60\u597d\uff0c\u4e16\u754c', u'de': u'Gr\xfc\xdfe, Welt', u'jp': u'\u3053\u3093\u306b\u3061\u306f\u3001\u4e16\u754c', u'es': u'hola, ma\xf1ana', u'he': u'\u05e9\u05dc\u05d5\u05dd, \u05e2\u05d5\u05dc\u05dd'} != '"de"=>"Gr\xc3\xbc\xc3\x9fe, Welt", "en"=>"hello, world", "es"=>"hola, ma\xc3\xb1ana", "he"=>"\xd7\xa9\xd7\x9c\xd7\x95\xd7\x9d, \xd7\xa2\xd7\x95\xd7\x9c\xd7\x9d", "jp"=>"\xe3\x81\x93\xe3\x82\x93\xe3\x81\xab\xe3\x81\xa1\xe3\x81\xaf\xe3\x80\x81\xe4\xb8\x96\xe7\x95\x8c", "zh"=>"\xe4\xbd\xa0\xe5\xa5\xbd\xef\xbc\x8c\xe4\xb8\x96\xe7\x95\x8c"'
======================================================================
FAIL: test_location_create (tests.django_hstore_tests.tests.TestDictionaryFieldPlusGIS)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 1216, in test_location_create
self.assertEqual(other_loc.data, {'prop1': '1', 'prop2': 'test_value'})
AssertionError: '"prop1"=>"1", "prop2"=>"test_value"' != {'prop1': '1', 'prop2': 'test_value'}
======================================================================
FAIL: test_location_hupdate (tests.django_hstore_tests.tests.TestDictionaryFieldPlusGIS)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 1222, in test_location_hupdate
self.assertEqual(loc.data, {'prop1': '2', 'prop2': 'test_value'})
AssertionError: '"prop1"=>"2", "prop2"=>"test_value"' != {'prop1': '2', 'prop2': 'test_value'}
======================================================================
FAIL: test_simple_retrieval_get (tests.django_hstore_tests.tests.TestReferencesField)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 1000, in test_simple_retrieval_get
self.assertEqual(Ref.objects.get(name='0'), alpha.refs.get('0'))
AssertionError: <Ref: Ref object> != None
======================================================================
FAIL: test_location_create (tests.django_hstore_tests.tests.TestReferencesFieldPlusGIS)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 1248, in test_location_create
self.assertEqual(loc_1.data, {'prop1': '1', 'prop2': 'test_value'})
AssertionError: '"prop1"=>"1", "prop2"=>"test_value"' != {'prop1': '1', 'prop2': 'test_value'}
======================================================================
FAIL: test_location_hupdate (tests.django_hstore_tests.tests.TestReferencesFieldPlusGIS)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 1256, in test_location_hupdate
self.assertEqual(loc.data, {'prop1': '2', 'prop2': 'test_value'})
AssertionError: '"prop1"=>"2", "prop2"=>"test_value"' != {'prop1': '2', 'prop2': 'test_value'}
======================================================================
FAIL: test_hstore_registring_in_transaction_block (tests.django_hstore_tests.tests.NotTransactionalTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alukach/Projects/django-hstore/tests/django_hstore_tests/tests.py", line 942, in test_hstore_registring_in_transaction_block
self.assertIsInstance(qs[0].data, HStoreDict)
AssertionError: '"v"=>"1", "v2"=>"3"' is not an instance of <class 'django_hstore.dict.HStoreDict'>
----------------------------------------------------------------------
Ran 118 tests in 1.962s
FAILED (failures=11, errors=17)
Destroying test database for alias 'default'...
(hstore)$ dropdb django_hstore
(hstore)$ createdb django_hstore # Recreate DB (from template with hstore enabled)
(hstore)$ python runtests.py # Tests will pass
Creating test database for alias 'default'...
../home/alukach/.virtualenvs/hstore/local/lib/python2.7/site-packages/django/db/models/base.py:1029: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
if f.blank and raw_value in f.empty_values:
/home/alukach/.virtualenvs/hstore/local/lib/python2.7/site-packages/django/db/models/fields/__init__.py:458: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
if value in self.empty_values:
........................................./home/alukach/Projects/django-hstore/django_hstore/query.py:57: RemovedInDjango18Warning: 'is_managed' is deprecated.
if not transaction.is_managed(using=self.db):
...........................................................................
----------------------------------------------------------------------
Ran 118 tests in 1.994s
OK
Destroying test database for alias 'default'...
(hstore)$
from django-hstore.
@nemesisdesign And to answer your question, this is the output of SELECT t.oid FROM pg_type t JOIN pg_namespace ns ON typnamespace = ns.oid WHERE typname = 'hstore' and nspname = 'public';
on my database and test database when the conditions for the bug are met:
(hstore)$ psql test_django_hstore -c "SELECT t.oid FROM pg_type t JOIN pg_namespace ns ON typnamespace = ns.oid WHERE typname = 'hstore' and nspname = 'public';"
oid
--------
913637
(1 row)
(hstore)$ psql django_hstore -c "SELECT t.oid FROM pg_type t JOIN pg_namespace ns ON typnamespace = ns.oid WHERE typname = 'hstore' and nspname = 'public';"
oid
--------
913497
(1 row)
from django-hstore.
@alukach -- Great sleuthing! That looks like it explains our problem as well, because our database setup process is similar:
- Create
template_postgis
- Create django app database from
template_postgis
- Run
CREATE EXTENSION hstore;
on django app database - Run
CREATE EXTENSION hstore;
ontemplate_postgis
In other words, template_postgis
is not hstore-enabled until after the django app database is created, which then (apparently) causes the postgis backend to erroneously believe that the test database is not hstore-enabled when in fact it is, while the postgresql_psycopg2 backend correctly detects and utilizes the hstore extension.
from django-hstore.
Okay, doing some late night research, I think I'm beginning to see what the issue is. Here are my notes so far:
When running tests with the postgis
backend:
- This function is called during setup with
settings_dict['NAME']
set as the Django db name (ex.django_hstore
) (this is set toNone
when using thepostgresql_psycopg
backend). - That opens a connection, calling
django-hstore
's connection handler. - That calls
register_hstore
frompsycopg2.extras
, which iterates through the connection'soid
s, discovering & registering thehstore
oid
for the prod db.
It isn't until later that the connection is opened for the test db (test_django_hstore
). When that connection is opened, register_hstore
is not rerun. This means that psycopg2
will be looking for hstore
at the oid
that it existed under on django_hstore
, rather than the oid
from test_django_hstore
. If they're both created from the same template, the oid
s will match and everything will work fine. However, if the django_hstore
db has the hstore
extension created later, then it's oid
will NOT match the template oid
, thus causing errors.
It appears that a possible solution could be to rerun register_hstore
when the DB changes from django_hstore
to test_django_hstore
. I haven't really looked into how to do that.
from django-hstore.
@alukach thank you for your research.
Any other suggestion of improvement?
Federico
from django-hstore.
I think I've found the problem and solution. My reproduction was different though.
Django signals are used to observe every new connection and call psycopg2's register_hstore function on it.
Django signals will, by default, store references to the receiver function as weakrefs.
In a low memory situation these can go away and future connections are not registered to use hstore features.
My suggested fix is on PR-128 - #128
from django-hstore.
Hey Guys,
I'm also facing the same issue with "HstoreField" , while running the django unit Test-Cases .
I don't understand why its throwing error Test_Database but not throwing error for actual Database creation. postgres ==9.3.10. django ==1.8.6
//////////////////////////////
Error :: "django.db.utils.ProgrammingError: can't adapt type 'dict'"
///////////////////////////
Is there a way to fixed it.????
from django-hstore.
That's not the error I was seeing, but possibly that's a 1.6-1.8 difference.
There's a fix for this issue in master - install from there and see if that resolves your issue?
from django-hstore.
Thanks @rklyne,
https://docs.djangoproject.com/en/1.8/ref/contrib/postgres/fields/
I got solution from above url.
from django-hstore.
I'm closing this, hoping it is will be finally solved with #128
from django-hstore.
Related Issues (20)
- Update HStoreWhereNode to follow django's WhereNode HOT 1
- Add settings to documentation
- ValueError when saving a regular JSON string HOT 2
- Drop support for django 1.6
- Add support for Django 1.9
- register_hstore_handler is not run during django management commands HOT 6
- icontains lookup doesn't work with dictionary params
- ProgrammingError: can't adapt type 'HStoreDict' HOT 1
- 1.4.1 regression: TypeError: Decimal is not JSON serializable HOT 3
- GrappelliAdminHStoreWidget - Deleting elements doesn't work properly inside StackedInline HOT 4
- Broken image "src" for icons in admin for Django >= 1.9 HOT 2
- 'module' object has no attribute 'subclassing' in django 1.10 HOT 9
- Documentation for old versions HOT 4
- Move documentation to readthedocs.org HOT 2
- HstoreField over escaping strings HOT 1
- Able to dumpdata with django but not loaddata HOT 1
- New maintainers needed
- Import Error?
- Security vulnerability related to underscore-min.js
- ImportError: cannot import name 'ugettext_lazy' from 'django.utils.translation' HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from django-hstore.