Comments (2)
I will add some documentation on debugging tips, but here are a few things to start:
General Tips
-
Given the wide variety of use cases and failure points, Data Wizard traps most errors by default, to ensure the user can get a short, hopefully informative message rather than a generic 500 error. The trapped errors are logged via python's
logging
module. -
The threading backend (enabled by default) adds another layer of indirection when trying to identify an exception.
-
Thus, if you are writing a custom Iter or Serializer class, make sure each component works in isolation before trying to debug within the Data Wizard stack. (See examples below)
-
Once you have confirmed that itertable and the serializer are working individually, try running
data_wizard
without any web UI traffic via the CLI (./manage.py runwizard
). -
Once that is working, try running through the web UI with
./manage.py runserver
and theimmediate
backend:
DATA_WIZARD = {
"BACKEND": "data_wizard.backends.immediate"
}
Debugging File Loading/Parsing (IterTable)
To debug issues loading and parsing files, try using itertable
directly:
from itertable import load_file
for row in load_file('/path/to/file.xlsx'):
print(row)
Note that existing releases of
itertable
automatically suppress theOSError
raised when a file is inaccessible, so it doesn't even make it back to Data Wizard. For the next release, I changed this to raiseitertable.exceptions.LoadFailed
unlessrequire_existing
is explicitly set to false.
If you are writing a custom Iter class, test the class with a similar loop:
from myapp import CustomIter
for row in CustomIter(filename='/path/to/file.xlsx'):
print(row)
Debugging the Serializer (DRF)
To investigate validation issues, try instantiating the DRF serializer class directly.
from data_wizard import registry
Serializer = registry.get_serializer("My Model")
serializer = Serializer(data={"test": "data"})
serializer.is_valid(raise_exception=True)
Note that
data_wizard
traps any and all serializer errors for individual rows, saving only the error text to theRecord
table. The full stack trace is still sent to the Pythonlogging
module.
from django-data-wizard.
I wouldn't mind some debugging insights, not for the reasons you asked... but since even using "BACKEND": "data_wizard.backends.immediate"
I couldn't seem to get my IDE (Pycharm) to catch any errors from data_wizard
. Which combined with the test setup complexity, makes it harder than it needs to be to work with this library. I'm trying to fix #31 because I'm using the very common django-storages
library, and making zero progress because I'm getting no useful output from tests or debugging.
from django-data-wizard.
Related Issues (20)
- ImportError with DRF >= 3.8 HOT 2
- getting 401 Unauthorized when trying to run wizard HOT 5
- Cannot search Natural Key when use unique=True in a NaturalKeyModel HOT 3
- 'NoneType' object has no attribute 'run' HOT 1
- Upon installation admin interface error: Reverse for 'app_list' not found. 'app_list' is not a valid view function or pattern name. HOT 4
- update for wq.app 1.2.0
- Error 403 in GET /datawizard/3/status/ HOT 1
- DataError: integer out of range with Postgres HOT 1
- backend doesn't support absolute paths: Support for non local filesystem HOT 4
- Custom Serializer questions HOT 1
- Troubles importing datas when target model has many to many field HOT 1
- django.contrib.admin.sites.AlreadyRegistered: The model Run is already registered in app 'data_wizard'.
- Error Loading Template HOT 1
- ImportError: cannot import name 'MutableMapping' from 'collections' HOT 1
- Did anyone tried it with django 4.06 python 3.10 HOT 1
- No pyproject.toml (poetry) config file HOT 1
- No runtests.sh mentioned in contributing.md and `python -m django test` fails HOT 1
- Failing URL lookup HOT 7
- Empty Identifiers for nullable ForeignKeyFields HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from django-data-wizard.