ahawker/django-ulid

Universally Unique Lexicographically Sortable Identifier (ULID) support in Django

django-ulid

Universally Unique Lexicographically Sortable Identifier (ULID) support in Django.

Status

This project is actively maintained.

Installation

To install django-ulid from pip:

    $ pip install django-ulid

To install ulid from source:

    $ git clone [email protected]:ahawker/django-ulid.git
    $ cd django-ulid && python setup.py install

Usage

Adding a ULID field to your Django models is straightforward. It can be a normal field or a primary key.

from django.db import models
from django_ulid.models import default, ULIDField

class Person(models.Model):
    id = ULIDField(default=default, primary_key=True, editable=False)

Passing in default to the ULIDField will automatically create a default value using the ulid.new function. If you do not want a default value, None by default, feel free to omit it.

from django.db import models
from django_ulid.models import ULIDField

class Person(models.Model):
    optional_id = ULIDField()

Adding a ULID field to your Django REST Framework serializers is also straightforward.

Simply importing the django_ulid.serializers module will automatically register the ULIDField serializer by overriding the serializer_field_mapping on the default ModelSerializer.

from django_ulid import serializers

If you are using a ULID as a primary key on a model, you need to create a custom PrimaryKeyRelatedField to automatically serialize the instance through the foreign key.

import functools
from django_ulid.serializers import ULIDField
from rest_framework import serializers

PersonPrimaryKeyRelatedField = functools.partial(serializers.PrimaryKeyRelatedField,
                                                 allow_null=True,
                                                 allow_empty=True,
                                                 pk_field=ULIDField(),
                                                 queryset=Person.objects.all())

class OrganizationSerializer(serializers.ModelSerializer):
    owner = PersonPrimaryKeyRelatedField()

Contributing

If you would like to contribute, simply fork the repository, push your changes and send a pull request. Pull requests will be brought into the master branch via a rebase and fast-forward merge with the goal of having a linear branch history with no merge commits.

License

Apache 2.0

Dependencies

• Django
• ulid-py

django-ulid's Issues

How to use ULIDField in SlugRelatedField

We can use

PersonPrimaryKeyRelatedField = functools.partial(serializers.PrimaryKeyRelatedField,
                                                 allow_null=True,
                                                 allow_empty=True,
                                                 pk_field=ULIDField(),
                                                 
                                                 queryset=ShopCategory.objects.all())

But i want to use SlugRelatedField instant PrimaryKeyRelatedField. But in slugrelatedfield i need to pass slug_field but i cannot pass slug field into functools.partial.
How can i use that function?

Django admin not working

Traceback

<h3 style="padding: 0px; margin: 1em 0px 0.5em; color: rgb(0, 0, 0); font-family: sans-serif; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;">__str__ returned non-string (type ULID)</h3>

9 | {% for field in line %}
-- | --
<div{% if not line.fields\|length_is:'1' %} class="fieldBox{% if field.field.name %} field-{{ field.field.name }}{% endif %}{% if not field.is_readonly and field.errors %} errors{% endif %}{% if field.field.is_hidden %} hidden{% endif %}"{% elif field.is_checkbox %} class="checkbox-row"{% endif %}>
{% if not line.fields\|length_is:'1' and not field.is_readonly %}{{ field.errors }}{% endif %}
{% if field.is_checkbox %}
{{ field.field }}{{ field.label_tag }}
{% else %}
{{ field.label_tag }}
{% if field.is_readonly %}
<div class="readonly">{{ field.contents }}</div>
{% else %}
{{ field.field }}
{% endif %}
{% endif %}
{% if field.field.help_text %}
<div class="help">{{ field.field.help_text\|safe }}</div>
{% endif %}
</div>
{% endfor %}
</div>
{% endfor %}
</fieldset>

Add information about DB schema to docs

The ULID field will use a UUID type if the database connection its using has one, e.g. postgres. If it does not, it will store it as text/varchar. This should be noted (with a snippet from a table schema) in the docs.

Does not work on mysql

When Django tries to load data from the database it checks that the internal_type of the field is UUIDField (https://github.com/django/django/blob/stable/3.2.x/django/db/backends/mysql/operations.py#L302-L303) and therefore assumes that it is uuid and tries to parse it which fails because the connection.features.has_native_uuid_field is False and the ulid is stored as base32.

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/django/core/handlers/wsgi.py", line 133, in __call__
    response = self.get_response(request)
  File "/usr/local/lib/python3.9/site-packages/django/core/handlers/base.py", line 130, in get_response
    response = self._middleware_chain(request)
  File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 49, in inner
    response = response_for_exception(request, exc)
  File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 114, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 149, in handle_uncaught_exception
    return debug.technical_500_response(request, *exc_info)
  File "/usr/local/lib/python3.9/site-packages/django_extensions/management/technical_response.py", line 40, in null_technical_500_response
    raise exc_value.with_traceback(tb)
  File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.9/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python3.9/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/usr/local/lib/python3.9/site-packages/django/views/generic/base.py", line 70, in view
    return self.dispatch(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/utils/decorators.py", line 43, in _wrapper
    return bound_method(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/views/decorators/debug.py", line 89, in sensitive_post_parameters_wrapper
    return view(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/allauth/account/views.py", line 149, in dispatch
    return super(LoginView, self).dispatch(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/allauth/account/views.py", line 77, in dispatch
    response = super(RedirectAuthenticatedUserMixin, self).dispatch(
  File "/usr/local/lib/python3.9/site-packages/django/views/generic/base.py", line 98, in dispatch
    return handler(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/allauth/account/views.py", line 104, in post
    if form.is_valid():
  File "/usr/local/lib/python3.9/site-packages/django/forms/forms.py", line 175, in is_valid
    return self.is_bound and not self.errors
  File "/usr/local/lib/python3.9/site-packages/django/forms/forms.py", line 170, in errors
    self.full_clean()
  File "/usr/local/lib/python3.9/site-packages/django/forms/forms.py", line 373, in full_clean
    self._clean_form()
  File "/usr/local/lib/python3.9/site-packages/django/forms/forms.py", line 400, in _clean_form
    cleaned_data = self.clean()
  File "/usr/local/lib/python3.9/site-packages/allauth/account/forms.py", line 178, in clean
    user = get_adapter(self.request).authenticate(self.request, **credentials)
  File "/usr/local/lib/python3.9/site-packages/allauth/account/adapter.py", line 587, in authenticate
    user = authenticate(request, **credentials)
  File "/usr/local/lib/python3.9/site-packages/django/views/decorators/debug.py", line 42, in sensitive_variables_wrapper
    return func(*func_args, **func_kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/contrib/auth/__init__.py", line 76, in authenticate
    user = backend.authenticate(request, **credentials)
  File "/usr/local/lib/python3.9/site-packages/django/contrib/auth/backends.py", line 42, in authenticate
    user = UserModel._default_manager.get_by_natural_key(username)
  File "/usr/local/lib/python3.9/site-packages/django/contrib/auth/base_user.py", line 45, in get_by_natural_key
    return self.get(**{self.model.USERNAME_FIELD: username})
  File "/usr/local/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 431, in get
    num = len(clone)
  File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 262, in __len__
    self._fetch_all()
  File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 1324, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 68, in __iter__
    for row in compiler.results_iter(results):
  File "/usr/local/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1122, in apply_converters
    value = converter(value, expression, connection)
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/mysql/operations.py", line 318, in convert_uuidfield_value
    value = uuid.UUID(value)
  File "/usr/local/lib/python3.9/uuid.py", line 177, in __init__
    raise ValueError('badly formed hexadecimal UUID string')
ValueError: badly formed hexadecimal UUID string

')

django-ulid/tests/test_serializers.py

Lines 12 to 14 in 897e28e

	@pytest.mark.xfail(reason='TODO')
	def test_sanity():
	assert False

---

This issue was generated by todo based on a TODO comment in 897e28e. It's been assigned to @ahawker because they committed the code.

')

django-ulid/tests/test_forms.py

Lines 12 to 14 in 897e28e

	@pytest.mark.xfail(reason='TODO')
	def test_sanity():
	assert False

---

This issue was generated by todo based on a TODO comment in 897e28e. It's been assigned to @ahawker because they committed the code.

Using ULID field for primary key causing migration to keep being recreated

I'm using ULID as the primary key for a couple of my models, as well as ULID for a couple non-PK keys. When i run python manage.py makemigrations, a migration is made attempting to redeclare the ulid fields being used as pk, but the non-pk fields aren't affected which is good. If I try to run the migration, it results with an error, reproduced below.

Not sure why django wants to keep creating this migration, as its the exact same operation as the initial migration.

models.py:

class Document(MP_Node):
    id = ULIDField(default=default, primary_key=True, editable=False)
    tree_root_pk = ULIDField(null=True, blank=True)

class Tag(models.Model):
    id = ULIDField(default=default, primary_key=True, editable=False)

Initial migration:

migrations.CreateModel(
    name='Tag',
    fields=[
        ('id', django_ulid.models.ULIDField(default=ulid.api.api.Api.new, editable=False, primary_key=True, serialize=False)),

migrations.CreateModel(
    name='Document',
    fields=[
        ('id', django_ulid.models.ULIDField(default=ulid.api.api.Api.new, editable=False, primary_key=True, serialize=False)),
        ('tree_root_pk', django_ulid.models.ULIDField(blank=True, null=True)),

Migration that keeps getting recreated every time I run python manage.py makemigrations:

operations = [
    migrations.AlterField(
        model_name='document',
        name='id',
        field=django_ulid.models.ULIDField(default=ulid.api.api.Api.new, editable=False, primary_key=True, serialize=False),
    ),
    migrations.AlterField(
        model_name='tag',
        name='id',
        field=django_ulid.models.ULIDField(default=ulid.api.api.Api.new, editable=False, primary_key=True, serialize=False),
    ),
]

The error I get if I try to run this superfluous migration:

Operations to perform:
  Apply all migrations: admin, auth, contenttypes, djstripe, documents, sessions, users
Running migrations:
  Applying documents.0003_auto_20201022_1945...Traceback (most recent call last):
  File "manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "lib/python3.7/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
    utility.execute()
  File "lib/python3.7/site-packages/django/core/management/__init__.py", line 395, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "lib/python3.7/site-packages/django/core/management/base.py", line 328, in run_from_argv
    self.execute(*args, **cmd_options)
  File "lib/python3.7/site-packages/django/core/management/base.py", line 369, in execute
    output = self.handle(*args, **options)
  File "lib/python3.7/site-packages/django/core/management/base.py", line 83, in wrapped
    res = handle_func(*args, **kwargs)
  File "lib/python3.7/site-packages/django/core/management/commands/migrate.py", line 233, in handle
    fake_initial=fake_initial,
  File "lib/python3.7/site-packages/django/db/migrations/executor.py", line 117, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "lib/python3.7/site-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "lib/python3.7/site-packages/django/db/migrations/executor.py", line 245, in apply_migration
    state = migration.apply(state, schema_editor)
  File "lib/python3.7/site-packages/django/db/migrations/migration.py", line 124, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "lib/python3.7/site-packages/django/db/migrations/operations/fields.py", line 249, in database_forwards
    schema_editor.alter_field(from_model, from_field, to_field)
  File "lib/python3.7/site-packages/django/db/backends/base/schema.py", line 565, in alter_field
    old_db_params, new_db_params, strict)
  File "lib/python3.7/site-packages/django/db/backends/postgresql/schema.py", line 154, in _alter_field
    new_db_params, strict,
  File "lib/python3.7/site-packages/django/db/backends/base/schema.py", line 678, in _alter_field
    old_default = self.effective_default(old_field)
  File "lib/python3.7/site-packages/django/db/backends/base/schema.py", line 303, in effective_default
    return field.get_db_prep_save(self._effective_default(field), self.connection)
  File "lib/python3.7/site-packages/django/db/backends/base/schema.py", line 282, in _effective_default
    default = field.get_default()
  File "lib/python3.7/site-packages/django/db/models/fields/__init__.py", line 829, in get_default
    return self._get_default()
TypeError: new() missing 1 required positional argument: 'self'

can't adapt type 'ULID' / invalid input syntax for type uuid

Hey first of all thanks for all the great work putting this together eh, both the python ulid package itself and django_ulid.

I've run into an issue where when I run a query like:

'UPDATE "documents_document" SET numchild=numchild+1 WHERE path='0001' AND tree_root_pk='01EMF1XKJB3G1BXPKFC6MA7B04'

I get an error like: "can't adapt type 'ULID'""

I realized that this is because I need to str() the ULID id, otherwise I kept getting the ULID class object in my queries

But once I ensured the ULID is a string, I get the error:

Error: invalid input syntax for type uuid: "01EMF1XKJB3G1BXPKFC6MA7B04"
LINE 1: ...ild=numchild+1 WHERE path='0001' AND tree_root_pk='01EMF1XKJ...

From what I understand, this is simply the field failing postgres's uuid type checking (e.g. it's too short of a string). When I try everything the same, but with uuid-compatible string instead of ulid string, everything works because it passes the uuid typecheck

Now of course postgres doesn't have ulid type checking (tho apparently something similar is being proposed).

So I believe best bet is to forcefully cast the postgres field to ::text, but when I try that I get:

Error: operator does not exist: uuid = text
LINE 1: ...hild=numchild+1 WHERE path='0001' AND tree_root_pk='01EMF1XK...

So I'm not quite sure how to bypass postgres' uuid field checks on the django_ulid field. Is anyone else running into this issue?

Full Traceback:
Environment:

Request Method: POST
Request URL: http://127.0.0.1:8000/document/01EMF1XKJB3G1BXPKFC6MA7B04/

Django Version: 3.0.5
Python Version: 3.7.3
Installed Applications:
['django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'django.contrib.postgres.fields',
'django.contrib.humanize',
'users',
'documents.apps.DocumentsConfig',
'crispy_forms',
'treebeard',
'djstripe',
'froala_editor',
'debug_toolbar',
'django_extensions']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'users.middleware.CustomSubscriptionPaymentMiddleware',
'debug_toolbar.middleware.DebugToolbarMiddleware']

Traceback (most recent call last):
File "/home/.virtualenvs/km_dev/lib/python3.7/site-packages/django/db/backends/utils.py", line 86, in _execute
return self.cursor.execute(sql, params)

The above exception (can't adapt type 'ULID') was the direct cause of the following exception:
File "/home/.virtualenvs/km_dev/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/home/.virtualenvs/km_dev/lib/python3.7/site-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/home/.virtualenvs/km_dev/lib/python3.7/site-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/.virtualenvs/km_dev/lib/python3.7/site-packages/django/views/generic/base.py", line 71, in view
return self.dispatch(request, *args, **kwargs)
File "/home/.virtualenvs/km_dev/lib/python3.7/site-packages/django/views/generic/base.py", line 97, in dispatch
return handler(request, *args, **kwargs)
File "/project/documents/views.py", line 1098, in post
comment = comment_form.make_new_document_object()
File "/project/documents/forms.py", line 151, in make_new_document_object
new_object = self.target_object.add_child_plus_count(**data)
File "/project/documents/models.py", line 202, in add_child_plus_count
return self.add_child(**kwargs)
File "/project/documents/mp_tree.py", line 1031, in add_child
return MP_AddChildHandler(self, **kwargs).process()
File "/project/documents/mp_tree.py", line 373, in process
'sorted-sibling', **self.kwargs)
File "/project/documents/mp_tree.py", line 1043, in add_sibling
return MP_AddSiblingHandler(self, pos, **kwargs).process()
File "/project/documents/mp_tree.py", line 464, in process
self.run_sql_stmts()
File "/project/documents/mp_tree.py", line 149, in run_sql_stmts
cursor.execute(sql, vals)
File "/home/.virtualenvs/km_dev/lib/python3.7/site-packages/debug_toolbar/panels/sql/tracking.py", line 198, in execute
return self._record(self.cursor.execute, sql, params)
File "/home/.virtualenvs/km_dev/lib/python3.7/site-packages/debug_toolbar/panels/sql/tracking.py", line 133, in _record
return method(sql, params)
File "/home/.virtualenvs/km_dev/lib/python3.7/site-packages/django/db/backends/utils.py", line 100, in execute
return super().execute(sql, params)
File "/home/.virtualenvs/km_dev/lib/python3.7/site-packages/django/db/backends/utils.py", line 68, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/home/.virtualenvs/km_dev/lib/python3.7/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/home/.virtualenvs/km_dev/lib/python3.7/site-packages/django/db/backends/utils.py", line 86, in _execute
return self.cursor.execute(sql, params)
File "/home/.virtualenvs/km_dev/lib/python3.7/site-packages/django/db/utils.py", line 90, in exit
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/home/.virtualenvs/km_dev/lib/python3.7/site-packages/django/db/backends/utils.py", line 86, in _execute
return self.cursor.execute(sql, params)

Exception Type: ProgrammingError at /document/01EMF1XKJB3G1BXPKFC6MA7B04/
Exception Value: can't adapt type 'ULID'

Foreign keys are being treated as integers in postgresql

')

django-ulid/tests/test_models.py

Lines 12 to 14 in 897e28e

	@pytest.mark.xfail(reason='TODO')
	def test_sanity():
	assert False

---

This issue was generated by todo based on a TODO comment in 897e28e. It's been assigned to @ahawker because they committed the code.

TypeError: Object of type ULID is not JSON serializable

...is a common error when using this with Django and various libraries like grappelli, allauth, .... because Python nor Django can JSON serialise your custom ULID types.

ahawker/ulid#492

Dependabot couldn't authenticate with https://pypi.python.org/simple/

Dependabot couldn't authenticate with https://pypi.python.org/simple/.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

View the update logs.