write a better utils/populate.py to load sample data from a json or yaml file

[quokkaproject]

write a better utils/populate.py to load sample data from a json or yaml file

[rochacbruno]

Use factoryboy for it.

https://github.com/rbarrois/factory_boy

http://stackoverflow.com/questions/14346177/how-do-you-use-factory-boy-to-model-a-mongoengine-embeddeddocument

[mauricioabreu]

In my honest opinion, better than use a external tool, we can take advantage of mongoimport[1].
With mongoimport (which comes with MongoDB facilities) we can just issue a command to import a json file to the database since it is just sample data.

---

1: http://docs.mongodb.org/manual/reference/program/mongoimport/

[rochacbruno]

👍

Do you how to implement it? so I think we just need to create the source files matching Quokka's database model. (is there a mongoexport tool? to generate raw Json from the current database?)

Also, we will need to write up a tutorial in a wiki page or maybe create a script to run it from manage.py populate

[rochacbruno]

amazing! http://docs.mongodb.org/manual/reference/program/mongoexport/

[mauricioabreu]

Yes and it is mongoexport that you described above.
mongoexport exports collection so I think I should iterate over all models which create collections and issue the command mongoexport using the same approach with mongoimport. "show collections" MongoDB command could do the job as well.

[mauricioabreu]

So, I have worked in this approach:
https://gist.github.com/mauricioabreu/6590333 (code is not finished, but you get the idea)

But it is going wrong for several reasons.

Let's describe them one by one:

1 - It is hard to maintain since mongoexport has no limit (actually it is comming soon, but nowadays it is not even an option). to achieve success on this it would require a workaround to generate low amount of records.

2 - It is based of files, so a lot of things can happen like (wrong file, corrupted file, no permissions, etc)

3 - It is hard to check if a class creates a physical collection (this is why I am checking if it is a subclass of DynamicDocument and Document from mongoengine)

I think factory boy is a better approach in this case.

[rochacbruno]

Thanks to @abnerpc there is now a populate from json file. I think there is no need to use mongodump for this feature.