grantmcconnaughey / django-field-history Goto Github PK
View Code? Open in Web Editor NEWA Django app to track changes to model fields.
License: BSD 3-Clause "New" or "Revised" License
A Django app to track changes to model fields.
License: BSD 3-Clause "New" or "Revised" License
$ ./manage.py createinitialfieldhistory
Traceback (most recent call last):
File "src/manage.py", line 10, in
execute_from_command_line(sys.argv)
File "/home/vagrant/mdm/env/local/lib/python2.7/site-packages/django/core/management/init.py", line 399, in execute_from_command_line
utility.execute()
File "/home/vagrant/mdm/env/local/lib/python2.7/site-packages/django/core/management/init.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/vagrant/mdm/env/local/lib/python2.7/site-packages/django/core/management/init.py", line 272, in fetch_command
klass = load_command_class(app_name, subcommand)
File "/home/vagrant/mdm/env/local/lib/python2.7/site-packages/django/core/management/init.py", line 75, in load_command_class
module = import_module('%s.management.commands.%s' % (app_name, name))
File "/home/vagrant/mdm/env/local/lib/python2.7/site-packages/django/utils/importlib.py", line 40, in import_module
import(name)
File "/home/vagrant/mdm/env/local/lib/python2.7/site-packages/field_history/management/commands/createinitialfieldhistory.py", line 4, in
from django.apps import apps
ImportError: No module named apps
I use a uuid for a primary key on my objects, and as such call it uuid. When I go to create the initial field history, I get this error:
File "lib/python2.7/site-packages/field_history/management/commands/createinitialfieldhistory.py", line 35, in handle object_id=obj.id,
AttributeError: 'Order' object has no attribute 'id'
I fixed this by changing line 35 in createinitialfieldhistory.py to object_id=obj.pk
Hi.
There is a problem with Models which have primary key with default=uuid
id = models.UUIDField(default=uuid.uuid4, editable=False, primary_key=True)
The initial/first field history value is not saved to database.
It is caused by is_new_object = instance.pk is None
in tracker.py
This approach has a problem:
In the save method, self.pk will never be None when default=uuid.uuid is set.
A large number of django users (and a lot of non-official django tutorials) believe that checking self.pk in the save method is a safe way to detect and decide whether an instance of a model is new or not.
No its not. The safe way to detect and decide whether an instance of a model is new or not is to use self._state object
https://docs.djangoproject.com/en/3.2/ref/models/instances/#state
So I think you should switch from
is_new_object = instance.pk is None
to
is_new_object = instance._state.adding
InnoDB indexes in MySQL can only be created for columns with a max length of 767 bytes (see the third bullet under Maximums and Minimums), so the TextField that FieldHistory.object_id
currently uses causes migration 0001 to throw an exception. Example output:
Operations to perform:
Apply all migrations: field_history
Running migrations:
Rendering model states... DONE
Applying field_history.0001_initial...Traceback (most recent call last):
File "project/proj/manage.py", line 8, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 346, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 394, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python2.7/site-packages/raven/contrib/django/management/__init__.py", line 41, in new_execute
return original_func(self, *args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 445, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 222, in handle
executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
File "/usr/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 110, in migrate
self.apply_migration(states[migration], migration, fake=fake, fake_initial=fake_initial)
File "/usr/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 148, in apply_migration
state = migration.apply(state, schema_editor)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 91, in __exit__
self.execute(sql)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 111, in execute
cursor.execute(sql, params)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 79, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python2.7/site-packages/django/db/utils.py", line 98, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 124, in execute
return self.cursor.execute(query, args)
File "/usr/local/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
self.errorhandler(self, exc, value)
File "/usr/local/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
django.db.utils.OperationalError: (1170, "BLOB/TEXT column 'object_id' used in key specification without a key length")
The obvious solution is to change FieldHistory.object_id
to a fixed-length field, such as a CharField
.
Is there any way we can temporarily disable tracking either by disconnecting signals or using some magic settings?
Example reason for doing this would be to pull historical data (timestamp, value) from already existing source. I'm not sure this is possible at this point..
@grantmcconnaughey I'll be glad to make PR + docs, after we figure out the way to handle this.
Thanks!
This does not work, if object representation have type unicode
:
return '{} field history for {}'.format(self.field_name, self.object)
fix for python 2.7:
return u'{} field history for {}'.format(self.field_name, self.object)
https://github.com/grantmcconnaughey/django-field-history/blob/master/field_history/models.py#L31
Please add function so you can take the user's which change instance from request
I t's just my suggestion but I think, for PostgreSQL you shoud use fields like HStoreField or JSONField for serialized_data field.
Please add keyword for track all fields, like a __all__
Hi!
I have a question. Your tracker uses threading.locals. Have you ever tried to test this decisiton on a production with more or less big number of users who make changes to your models at the same time?
If you use uwsgi then Django will be just forked by some number of processes (2, 4, 5, whatever you set in your uwsgi settings). So, it means that if you have 1000 online users then, e.g., and 4 forks of Django, then 250 of your 'requests' which you store in thread will go to the same locals and will overwrite each other.
To my mind, it's not serious to tell people that they should use this middleware in their project with full confidence that it will log all changes correctrly.
I faced with the problem, when I want to track fields of parent model using django-field-history
app i'm unable to save the value of the field due to the problem of how Django serializers works. They all returns copy just of tables, but working with inherited models we need to search among parent fields too.
As result data
field doesn't contains any fields values.
Django 1.10 using MIDDLEWARE instead of MIDDLEWARE_CLASSES in settings.py. See https://docs.djangoproject.com/en/1.10/releases/1.10/#new-style-middleware for details.
I like your approach to recording model history. I've tried using django-reversion before but it broke as I migrated my models by adding or deleting fields. I wonder, have you considered how django-field-history would handle migrations?
I would like to rename the field "status" to "portal_status" in my app. Originally this is what it looks like:
I executed python manage.py renamefieldhistory --model=accounts.account --from_field=status --to_field=portal_status
and saw that 619 rows were updated.
Upon closer inspection however, I see that field_name is changed but not the field inside serialized_data.
This causes my application to crash.
Perhaps we should also parse serialized_data to and modify the keys accordingly?
Consider the following models:
class OKB_Article(models.Model):
VISUALIZATION_CHOICES = (
('is_private', 'Privado'),
('is_public', 'Publico'),
)
id_record = models.BigIntegerField()
title = models.CharField(max_length=200)
visualization = models.CharField(
max_length=50, choices=VISUALIZATION_CHOICES, default='is_private')
article_grouping = models.ForeignKey(
OKB_ArticleGrouping,
blank=True,
null=True)
author = models.ForeignKey(User, related_name='user_author')
motive_type = models.ForeignKey(OSD_Motive, blank=True, null=True)
article_type = models.ForeignKey(OKB_ArticleType, blank=True, null=True)
content = models.TextField()
approved_by = models.ForeignKey(
User, related_name='approved_by', blank=True, null=True)
created_date = models.DateTimeField(auto_now_add=True)
status_type = models.ForeignKey(OSD_Status)
history = FieldHistoryTracker([
'motive_type', 'article_grouping', 'article_type',
'visualization', 'status_type'
])
class Meta:
db_table = 'okb_article'
def __unicode__(self):
return self.title
class OSD_Status(models.Model):
name = models.CharField(max_length=50)
color = models.CharField(max_length=6, choices=COLOR_CHOICES,
default="ffce93")
description = models.CharField(max_length=200)
behavior = models.ForeignKey(OSD_Behavior)
motive = models.ManyToManyField(OSD_Motive, through='OSD_StatusMotive')
status = models.BooleanField(default=True)
class Meta:
db_table = 'osd_status'
def __unicode__(self):
return "%s - %s" % (self.name, self.behavior)
As you can see the field status_type in OKB_Article has a ManyToManyField called motive.
If I'm tracking the status_type field, the field_value function only works for that field; I can't access the field_value of any other field, it throws a DeserializationError mentioning the status_type field even though I'm not accesing it. See the screenshot where I reproduce and mimick what the field_value function does inside an ipdb debugger:
However, this only happens when I'm tracking said field. If I remove it from the FieldHistoryTracker list, I can access all the other fields without a problem.
I'm using the latest version of Django 1.9.x along with the latest Python 2.7.x on Linux.
I need to run python manage.py createinitialfieldhistory
inside provision script during launch, so every time I deploy it creates new values for already tracked fields, which I don't think is necessary.
Is it possible for this command to omit already tracked objects? Thanks.
Hello!
I am using PostgreSQL and I'm getting the following error upon python manage.py createinitialfieldhistory
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "manage.py", line 28, in <module>
execute_from_command_line(sys.argv)
File "/home/vagrant/grt/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
utility.execute()
File "/home/vagrant/grt/lib/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/vagrant/grt/lib/python3.6/site-packages/django/core/management/base.py", line 316, in run_from_argv
self.execute(*args, **cmd_options)
File "/home/vagrant/grt/lib/python3.6/site-packages/django/core/management/base.py", line 353, in execute
output = self.handle(*args, **options)
File "/home/vagrant/grt/lib/python3.6/site-packages/field_history/management/commands/createinitialfieldhistory.py", line 37, in handle
field_name=field).exists():
File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/models/query.py", line 715, in exists
return self.query.has_results(using=self.db)
File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/models/sql/query.py", line 516, in has_results
return compiler.has_results()
File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1031, in has_results
return bool(self.execute_sql(SINGLE))
File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1061, in execute_sql
cursor.execute(sql, params)
File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/backends/utils.py", line 100, in execute
return super().execute(sql, params)
File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/backends/utils.py", line 68, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
return self.cursor.execute(sql, params)
File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: relation "field_history_fieldhistory" does not exist
LINE 1: SELECT (1) AS "a" FROM "field_history_fieldhistory" WHERE ("...
my model:
class Title(models.Model):
id = models.PositiveIntegerField(primary_key=True)
name = models.CharField(max_length=255)
priority = models.PositiveIntegerField(null=True, blank=True)
field_history = FieldHistoryTracker(['priority'])
tried doing this as well: https://django-field-history.readthedocs.io/en/latest/readme.html#working-with-mysql (just in case) - no success.
What could be wrong?
edit:
you have to use python manage.py migrate your new field first
We had an issue in https://github.com/twschiller/open-synthesis/issues/214 where we'd get a database integrity error about the FieldHistory referencing users that didn't exist
We had to add the following to our base classes to reset the request context after tests where we used the Django test client with a logged in user
def tearDown(self) -> None:
FieldHistoryTracker.thread.request = None
Not a huge problem but it would be good to have the installation and quickstart sections of the docs mention that you need to run manage.py migrate
after adding 'field_history' to INSTALLED_APPS. Would save django newbies and people who don't spend a ton of time with it a minute or two of head-scratching, and is more technically complete.
In postgresql there is specific and optimized field for JSON, jsonb which can permit the direct used of the json object (and i guess that other dbb has his own).
The created table stock in serialized_ data the object. I think it could really improve the lib efficiency if the field type could set himself regarding project configuration.
Hi,
It's possible track all the fields of the model?
Thanks
missing this feature on the project, which could be very usefull. Or at last the documentation must explain how the history table looks like and his behavior. Anyway good job and good luck/
Just an issue to keep track of the discussion in #25 (comment)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.