gregmuellegger / django-autofixture Goto Github PK
View Code? Open in Web Editor NEWCan create auto-generated test data.
License: BSD 3-Clause "New" or "Revised" License
Can create auto-generated test data.
License: BSD 3-Clause "New" or "Revised" License
Unfortunately, there is no way to input a minimum and maximum value for a FloatGenerator.
Current implementation of post_process_instance
implies that user, that would override it must save manually modified instance inside method of child class.
But actually Im not sure - is it really intended by author to be as that. Old documentation documentation says that this method should return "modified instance" in "last possible moment" which might mean moment before saving.
post_process_instance
in UserFixture
illustrates what Im talking about, take a look, with this implementation any modifications done by post_process_instance
will be lost during creation, including password, because instance not being saved.
My model:
class Product(models.Model):
name = models.CharField(max_length=100)
description = models.TextField()
cost = models.FloatField(blank=True, null=True)
price = models.FloatField()
I try with:
(venv)vagrant@lucid32:~/project$ ./manage.py loadtestdata app.Product:3
Traceback (most recent call last):
File "./manage.py", line 14, in
execute_manager(settings)
File "/usr/local/venv/local/lib/python2.7/site-packages/django/core/management/init.py", line 438, in execute_manager
utility.execute()
File "/usr/local/venv/local/lib/python2.7/site-packages/django/core/management/init.py", line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/venv/local/lib/python2.7/site-packages/django/core/management/base.py", line 191, in run_from_argv
self.execute(_args, *_options.dict)
File "/usr/local/venv/local/lib/python2.7/site-packages/django/core/management/base.py", line 220, in execute
output = self.handle(_args, *_options)
File "/usr/local/venv/local/lib/python2.7/site-packages/django/db/transaction.py", line 217, in inner
res = func(_args, *_kwargs)
File "/usr/local/venv/local/lib/python2.7/site-packages/autofixture/management/commands/loadtestdata.py", line 208, in handle
autofixture.create(model, count, *_kwargs)
File "/usr/local/venv/local/lib/python2.7/site-packages/autofixture/init.py", line 92, in create
return autofixture.create(count)
File "/usr/local/venv/local/lib/python2.7/site-packages/autofixture/base.py", line 478, in create
instance = self.create_one(commit=commit)
File "/usr/local/venv/local/lib/python2.7/site-packages/autofixture/base.py", line 456, in create_one
instance.save()
File "/usr/local/venv/local/lib/python2.7/site-packages/django/db/models/base.py", line 460, in save
self.save_base(using=using, force_insert=force_insert, force_update=force_update)
File "/usr/local/venv/local/lib/python2.7/site-packages/django/db/models/base.py", line 553, in save_base
result = manager._insert(values, return_id=update_pk, using=using)
File "/usr/local/venv/local/lib/python2.7/site-packages/django/db/models/manager.py", line 195, in _insert
return insert_query(self.model, values, *_kwargs)
File "/usr/local/venv/local/lib/python2.7/site-packages/django/db/models/query.py", line 1436, in insert_query
return query.get_compiler(using=using).execute_sql(return_id)
File "/usr/local/venv/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 791, in execute_sql
cursor = super(SQLInsertCompiler, self).execute_sql(None)
File "/usr/local/venv/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 735, in execute_sql
cursor.execute(sql, params)
File "/usr/local/venv/local/lib/python2.7/site-packages/django/db/backends/util.py", line 34, in execute
return self.cursor.execute(sql, params)
File "/usr/local/venv/local/lib/python2.7/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 44, in execute
return self.cursor.execute(query, args)
django.db.utils.IntegrityError: null value in column "price" violates not-null constraint
Any tip?
I have the following model at my app
BLOG_POST_STATUS = (
('U', 'Waiting for triage'), # This is the status after we receive the blog post draft.
('R', 'Waiting to be reviewed'), # Blog post is assigned to one staff to be reviewed.
('G', 'Waiting to be proofread'), # Blog post is assigned to be proofread by the community officer.
('L', 'Waiting to be published'), # Blog post will be publish by the community officer.
('P', 'Published'), # Blog post is published and have a URL at the website.
('D', 'Declined'), # Blog post submitted by mistake.
('O', 'Out of date'), # Blog post that wait too long to be publish for any reason.
('X', 'Remove'), # When the fellow decided to remove their request.
)
class Blog(models.Model):
status = models.CharField(
choices=BLOG_POST_STATUS,
max_length=1,
default="U"
)
title = models.CharField(
max_length=MAX_CHAR_LENGTH,
null=True,
blank=True
)
published_url = models.CharField(
max_length=MAX_CHAR_LENGTH,
null=True,
blank=True
)
tweet_url = models.CharField(
max_length=MAX_CHAR_LENGTH,
null=True,
blank=True
)
added = models.DateTimeField(auto_now_add=True)
updated = models.DateTimeField(auto_now=True)
def save(self, *args, **kwargs): # pylint: disable=arguments-differ
if self.published_url:
self.status = 'P'
super(Blog, self).save(*args, **kwargs)
After I run
$ python --version
Python 3.6.0 :: Continuum Analytics, Inc.
$ python manage.py shell
Python 3.6.0 |Continuum Analytics, Inc.| (default, Dec 23 2016, 12:22:00)
Type 'copyright', 'credits' or 'license' for more information
>>> import django
>>> django.__version__
'1.11.3'
>>> import autofixture
>>> autofixture.__version__
'0.12.1'
>>> exit()
$ python manage.py loadtestdata myapp.Blog:120
I ended up with 120 Blog entries but all has a not null published_url
and status
equals to P
. Would be great to have a more random generator.
when I use sqllite as my django databases when I run this command
python manage.py loadtestdata -v0 product.TrustProduct:10
it works perfectly well
But when I use Mysql as my database;
Django throwout this error message:
django.db.utils.OperationalError: (1118, 'Row size too large (> 8126). Changing some columns to TEXT or BLOB or using ROW_FORMAT=DYNAMIC or ROW_FORMAT=COMPRESSED may help. In current row format, BLOB prefix of 768 bytes is stored inline.')
Add the ability to provide a Python regular expression string to the StringGenerator constructor to only allow strings that match the regular expression to be inserted into the database.
For example, in autofixtures.py
:
class EntryAutoFixture(AutoFixture):
field_values = {
'blog': generators.StringGenerator(
chars=string.ascii_letters + string.digits + '-',
match='^[a-zA-Z0-9][a-zA-Z0-9\-]*$',
min_length=1,
max_length=40
),
}
This would, in effect, limit strings generated to only containing alphanumeric characters and dashes and the string cannot begin with a dash.
This would be useful where there are regex validators present in Django url configurations and on Django models.
I can provide a Pull Request if this proposal is acceptable.
It would be great if it could auto integrate with faker lib so that test data is realistic w/ minimum effort.
Thanks for creating this, it saved me tons of time as you pointed out but (being new) it did take me a while to figure out how to use it. I was so happy to find something to autopopulate for me, I can't tell you how much i wanted to hug you.
I've forked and cleaned up the readme just a bit and added a bit to the examples to make them a little more clear to new devs. I'd like to submit a pull!
Thanks again!
Am I missing any steps?
CreateInstanceError: Cannot solve constraints for "supporters.Person", tried 1000 times. Please check value generators or model constraints. At least the following fields are involved: user
the model:
class Person(models.Model):
user = models.ForeignKey(User, unique=True)
or
class Person(models.Model):
user = models.OneToOneField(User)
Seems like this should be easy to handle...not sure why it is having this failure.
In init.py:
if isinstance(model_or_iterable, (list, tuple, set)):
should be:
if not isinstance(model_or_iterable, (list, tuple, set)):
def unregister(model_or_iterable, fail_silently=False):
'''
Remove one or more models from the autofixture registry.
'''
from django.db import models
if isinstance(model_or_iterable, (list, tuple, set)):
model_or_iterable = [model_or_iterable]
for model in models:
if isinstance(model, basestring):
model = models.get_model(*model.split('.', 1))
I got it on Mac OS 10.10.1
File "./manage.py", line 10, in
execute_from_command_line(sys.argv)
File "lib/python2.7/site-packages/django/core/management/init.py", line 399, in execute_from_command_line
utility.execute()
File "lib/python2.7/site-packages/django/core/management/init.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
self.execute(_args, *_options.dict)
File "lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
output = self.handle(_args, *_options)
File "lib/python2.7/site-packages/django/db/transaction.py", line 339, in inner
return func(_args, *_kwargs)
File "lib/python2.7/site-packages/autofixture/management/commands/loadtestdata.py", line 227, in handle
autofixture.create(model, count, *_kwargs)
File "lib/python2.7/site-packages/autofixture/init.py", line 131, in create
return autofixture.create(count, *_create_kwargs)
File "lib/python2.7/site-packages/autofixture/base.py", line 535, in create
instance = self.create_one(commit=commit, **kwargs)
File "lib/python2.7/site-packages/autofixture/base.py", line 482, in create_one
self.process_field(instance, field)
File "lib/python2.7/site-packages/autofixture/base.py", line 381, in process_field
value = self.get_value(field)
File "lib/python2.7/site-packages/autofixture/base.py", line 377, in get_value
value = generator()
File "lib/python2.7/site-packages/autofixture/generators.py", line 70, in call
return self.get_value()
File "lib/python2.7/site-packages/autofixture/generators.py", line 66, in get_value
value = self.generate()
File "lib/python2.7/site-packages/autofixture/generators.py", line 639, in generate
ContentFile(get_placeholder_image(width, height))
File "lib/python2.7/site-packages/autofixture/placeholder.py", line 22, in get_placeholder_image
font = ImageFont.truetype(font, size=fontsize, encoding=encoding)
File "lib/python2.7/site-packages/PIL/ImageFont.py", line 218, in truetype
return FreeTypeFont(filename, size, index, encoding)
File "lib/python2.7/site-packages/PIL/ImageFont.py", line 134, in init
self.font = core.getfont(file, size, index, encoding)
File "lib/python2.7/site-packages/PIL/ImageFont.py", line 34, in getattr
raise ImportError("The _imagingft C module is not installed")
ImportError: The _imagingft C module is not installed
For starters, amazing utility and just what I was looking for. Thank you for writing and maintaining this project.
If a model has a DateTimeField()
and settings.py
contains USE_TZ=True
then I am receiving the following error:
/home/me/.pythonbrew/venvs/Python-2.7.2/myproject/lib/python2.7/site-packages/
django/db/models/fields/__init__.py:827: RuntimeWarning: DateTimeField
received a naive datetime (2010-12-22 18:30:51.443334) while time zone
support is active. RuntimeWarning)
ValueError: astimezone() cannot be applied to a naive datetime
I've got a workaround by doing the following in generators.py
:
class DateTimeGenerator(Generator):
from django.utils.timezone import now
min_date = now() - datetime.timedelta(365 * 5)
max_date = now() + datetime.timedelta(365 * 1)
The Django version of now()
[1] returns a naive or aware datetime object according to the value of USE_TZ
.
[1] https://docs.djangoproject.com/en/dev/topics/i18n/timezones/#naive-and-aware-datetime-objects
This will override the default value for AutoFixture, esp. noticable when default is set on a subclass.
So i have a model called Usage
, with a many to many to another model called Reservation
and this reservation has a foreign key to another named Offer
How can i create test data with the fk and m2m automatically generating the new objects?
this is what i have so far:
from autofixture import AutoFixture
from .models import Usage
def run(verbose=True):
fixture = AutoFixture(
Usage, generate_fk=['card', 'reservations__offer'],
generate_m2m={'reservations': (1, 1)}
)
user_card_usages = fixture.create(2)
print ("Finished loading fixtures: ", user_card_usages)
The reservations__offer
is not being generated (i am not sure about the syntax)
Providing a max_value to the FloatGenerator fails with:
__init__() got an unexpected keyword argument 'max_value'
There simply is a typo in generators.py, I will provide a pull request shortly.
This is a great package. Thank you, @gregmuellegger, for creating this!
It would be nice to generate capitalized or titlecased titles. Maybe for fields called "title", or maybe some other rule.
No pressure, just something on my wish list. If you think this would be a good idea, I can take a stab at implementing it.
The Link
class in base.py
is an arcane relict from four years ago and hard to understand.
We should try to remove the class in incorporate the logic into the Autofixture
class.
My development environment depends on Django 1.10. Builtin tests does not cover anything for 1.10. Is it working on 1.10 or is there any plan to migrate to higher versions of Django?
Since Django 1.7 we can optionally add AppConfig
subclass path to INSTALLED_APPS
, like:
INSTALLED_APPS = [
'my_core.apps.CoreAppConfig',
# ...
]
Docs on that:
https://docs.djangoproject.com/en/1.8/ref/applications/
Support I want to generate a car
lists with random brand. But brands are from my given list.
Can I do that in your autofixture?
Hi,
I'm very new to Django and to this plugin too, so please be patient.
I wanted to report this error when using the package with Django 2. I have a minimal working example here. The error can be reproduced following these steps
git clone https://github.com/davethecipo/autofixture-django2-bug.git
cd autofixture-django2-bug
python3 -m venv venv35
## activate virtual environment, then run
cd project
pip install -r requirements.txt
./run.sh
Hello, i noticed that autofixtures are missing UUIDField support, so the any fixture of the model with UUID field raises IntegrityError in case when that uuid field on the model is required.
For now i'am monkey patching the base class and everything works fine:
import uuid
from django.db.models import fields
from autofixture import AutoFixture, generators
from autofixture.base import AutoFixtureBase
class UUIDGenerator(generators.Generator):
def generate(self):
return uuid.uuid4()
def monkey_patch_autofixture():
AutoFixtureBase.field_to_generator[fields.UUIDField] = UUIDGenerator
Ability to add default photos from custom folder path
I ran into errors when trying to loadtestdata --generate-fk=ALL
for a model that has defined a generic relation. See my test case for an example ericbuckley@47c0caf.
I got AttributeError: 'NoneType' object has no attribute '_meta' exception in process_m2m.
through = field.rel.through #returns None!
auto_created_through_model = through._meta.auto_created # 'NoneType' object has no attribute '_meta'
So I changed it for :
through = field.rel.through
auto_created_through_model = through and through._meta.auto_created
Its executing (right now ;-) ), but does it eligible solution?
I wanted to create a user with predefined password for a test so i did this:
import autofixture
class Test(TestCase):
def test(self):
user = autofixture.create_one(User, username='test', password='test')
This raises an exception : TypeError: init() got an unexpected keyword argument 'username'
To fix this i had to import autofixture.autofixures like this:
import autofixture
import autofixture.autofixtures
class Test(TestCase):
def test(self):
user = autofixture.create_one(User, username='test', password='test')
which worked
As I understood from autofixture / base.py there is no support for this.
It would be nice to have support for imagefields. The obvious problem is what images should be used? Well place a kitten is a start: http://placekitten.com/ or something like this http://placehold.it/
so the process of auto filling imagefield is not that hard:
Possible problems:
Any thoughts?
I typed the command as below:
$ python manage.py loadtestdata entry.Entry:3
but it's now working. I got this:
Unknown command: 'loadtestdata'
Type 'manage.py help' for usage.
Maybe something wrong?
I'm trying to create model instance like this:
AutoFixture(User, field_values={'date_joined': joined})
Here I'm explicitly telling AutoFixture class, that I want custom value for
date_joined field. But it still uses default value.
This can be fixed using overwrite_defaults=True:
AutoFixture(User, field_values={'date_joined': joined},
overwrite_defaults=True)
But then, all fields loose their default values.
I think, that if field_values are specified, then it must be used
overwriting defaults.
I just spent a while digging through the source to figure out how to limit the set of objects that would be set as a foreignkey. I got it to work, but I think that is such a cool feature that it should get documented in the README.
(pull request forthcoming!)
i am pretty new to this but i am running into a RuntimeError: maximum recursion depth exceeded while calling a Python object
with my foreign key
python manage.py loadtestdata tourist_card.UserCardUsage:1 --generate-m2m=reservations:1:1 --generate-fk=ALL --settings=settings_local
auth.User(pk=3744): J_X0vN_vb9g7bShriTHQmp
RuntimeError: maximum recursion depth exceeded while calling a Python object
``
What am i doing wrong?
As test for #73 shows django-autofixture isn't django 1.9 compatible.
I used this in tests, there's almost 50 unit tests now. But when running test with nosetests command, sometimes I got TransactionManagementError
. Full traceback is listed below:
======================================================================
ERROR: test suite for <class 'trades.tests.test_pingpp.TestPingpp'>
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/Cellar/numpy/1.11.0/libexec/nose/lib/python2.7/site-packages/nose/suite.py", line 209, in run
self.setUp()
File "/usr/local/Cellar/numpy/1.11.0/libexec/nose/lib/python2.7/site-packages/nose/suite.py", line 292, in setUp
self.setupContext(ancestor)
File "/usr/local/Cellar/numpy/1.11.0/libexec/nose/lib/python2.7/site-packages/nose/suite.py", line 315, in setupContext
try_run(context, names)
File "/usr/local/Cellar/numpy/1.11.0/libexec/nose/lib/python2.7/site-packages/nose/util.py", line 471, in try_run
return func()
File "/Users/windrunner/bxzz/phantasm/phantasm/tests/base.py", line 43, in setUpClass
AutoFixture(model).create(10)
File "/usr/local/lib/python2.7/site-packages/autofixture/base.py", line 550, in create
instance = self.create_one(commit=commit, **kwargs)
File "/usr/local/lib/python2.7/site-packages/autofixture/base.py", line 498, in create_one
process = self.check_constraints(instance)
File "/usr/local/lib/python2.7/site-packages/autofixture/base.py", line 454, in check_constraints
constraint(self.model, instance)
File "/usr/local/lib/python2.7/site-packages/autofixture/constraints.py", line 37, in unique_constraint
if model._default_manager.filter(**check).exists():
File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 586, in exists
return self.query.has_results(using=self.db)
File "/usr/local/lib/python2.7/site-packages/django/db/models/sql/query.py", line 484, in has_results
return compiler.has_results()
File "/usr/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 811, in has_results
return bool(self.execute_sql(SINGLE))
File "/usr/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 840, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 59, in execute
self.db.validate_no_broken_transaction()
File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 327, in validate_no_broken_transaction
"An error occurred in the current transaction. You can't "
TransactionManagementError: An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.
It seems that if a field is specified to be None in field_values, django-autofixture assumes that an incorrect value was generated for that field and therefore re-generates the instance (which will of course fail again). Is there a way to specify that a field should actually be None?
autofixture/init.py", line 62, in unregister
for model in models:
TypeError: 'module' object is not iterable
from django.contrib.auth.models import UNUSABLE_PASSWORD
no longer exists in Django 1.6, causing django-autofixture to crash.
There appears to be a new alternative
from django.contrib.auth.hashers import UNUSABLE_PASSWORD_PREFIX
There is also set_unusable_password() function on the auth model as well which is probably a better substitute.
Mat
I noticed today that this doesn't work. Nasty, because my tests now failed randomly, which was hard to debug. Is this something that can be fixed?
Is there any plan to add support to Django 2.0?
I may be watching at it wrong, but I have a problem even using generate_fx=True, because I think it will only create fk of the model being fixtured. In my example I have Equipment, with Device, With Sector, with Facility, all foreign keys, being device and sector m2m. and I would like to create an Equipment, without manually creating all the rest of the setup.
As discussed here, #58 , sometimes it would be great to recursively generate foreign key relations. maybe there could be a keyword that would specify the recursive limit or something
def get_generator(self, field):
if field.name == 'content_type':
return generators.ChoicesGenerator(
values=[ContentType.objects.get_for_model(m)
for m in [ModelOne, ModelTwo, ModelThree]])
elif field.name == 'object_id':
# where to pick right model class? ¯\(°_o)/¯
I think, for that case, instances of GeneridForeignKey
are also could be passed to get_generator()
, and being skipped in base implementation, so user will can handle them in subclass method.
Or another hook like prost_process_instance
could be defined, which must be called before instance being saved first time.
Hello,
I am would like ask why you use custom ppa to provide python 2.6.,2,7.33 and pypy.
All of them are supported by Travis http://docs.travis-ci.com/user/languages/python/ now.
If you drop ppa you can drop sudo
and move to container-infrastructure which work faster ( http://docs.travis-ci.com/user/migrating-from-legacy/ ).
Greetings,
geodjango allows you to define point fields it will be great to be able to generate data with them with auto fixtures.
thank you for your this App
I suggest add support for django-filebrowser fields. It should detect accepted extensions, connect it to supported extensions of placeholer and return right fixtures.
autofixture.generators.InstanceGenerator
fails when you defined your model with a Q
object in limit_choices_to
. InstanceGenerator
assumes that limit_choices_to
is always a dictionary.
It looks like InstanceSelector
has the same issue.
Hi,
I installed django-autofixture on Ubuntu 12.10 32-bit with Python 2.7.3 and django 1.4.3. After installation when I type the following command I get the subsequent errors:
django-admin.py loadtestdata
Error:
unknown command: 'loadtestdata'
Type 'django-admin.py help' for usage.
So I looked up django-admin help and there is no loadtestdata command. BUT there is a 'loaddata' listed. So then I tried the following command:
django-admin.py loaddata myapp.Merchant:3
This lead to the following error:
ImportError: Settings cannot be imported, because environment variable DJANGO_SETTINGS_MODULE is undefined.
I installed django-fixtures using pip in a virtualenv. Please help!
When generating an object and some of its foreign keys at once by calling e.g. AutoFixture(Book, generate_fk=['author'])
, any custom autofixture registered against Author will be ignored, and instead a default generated autofixture will be used.
This behaviour surprised me, is it a deliberate design decision or a bug?
Hello, great package.
In my opinion it lacks ability to easily set max length instead of getting it from model.
I have defined my field_values as below:
field_values = {
'name': namelist[randint(0,len(namelist)-1)][0],
}
with namelist is defined as a list of username.
But when i launch the manage.py with python manage.py loadtestdata webapp.user:50, i have 50 users with the same name...
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.