translationDatabase
Goals
The goals for translationDatabase are to manage and track data for languages and the progress of getting unrestricted biblical content into every language.
For more information on the unfoldingWord project, see the About page.
Data Sources
A lot of the sources of data are pull into and managed as repo as part of the Debian project called simply, ISO Codes.
In Use
Other Potential Sources
- ISO 639-2 - LOC
- ISO 639-3 - SIL - Names Index
- ISO 639-3 - SIL - Macrolanguage Mappings
- ISO 639-3 - SIL - Retirements
- ISO 3166 - ISO
- ISO 15924 - Codes for the representation of names of scripts
- Unicode Supplemental Data
- Geo Names - Languages in their own writing systems
- Glottolog
Getting Started
To setup a new working environment of this project, several items are needed:
- Python (consult the requirements.txt for specific libraries/packages)
- Redis
- Postgres
- Node
Building Static Media
npm install
npm run watch # run a watcher on the static folder
npm run build # builds static and exits
npm run buildprod # builds for production (uglify/minification)
Initialize the Database
After installing requirements (via pip) within your environment or virtualenv:
python manage.py migratepython manage.py loaddata sitespython manage.py loaddata uw_network_seedpython manage.py loaddata uw_region_seedpython manage.py loaddata uw_title_seedpython manage.py loaddata uw_media_seedpython manage.py loaddata additional-languagespython manage.py reload_imports
At this point, the basic country and language datasets will be populated but without many optional fields or extra data.
Updating the /exports/langnames.json and /exports/langnames_short.json endpoints
When languages are added or updated, run this command to update the data locally:
python manage.py rebuild_langnamesSwitch to the master branch and run this command to update the data on the server:
ec run web python manage.py rebuild_langnamesDocker Deployments
translationDatabase was previously built using the Heroku-18 stack and deployed on Heroku dynos.
It is now being deployed using Heroku's Docker container support.
This was configured via:
heroku stack:set container -a ${HEROKU_APP_NAME:-translation-database-demo}
(and repeated using HEROKU_APP_NAME=translation-database).
Deploying via heroku.yml
This application can be deployed to Heroku via Git.
Heroku's documentation on Git / GitHub deployments can be found here:
To deploy the master branch to the translation-database-demo site:
git checkout master
heroku git:remote -a translation-database-demo
git push heroku master:main
For additional documentation, see Building Docker Images with heroku.yml
Building the Docker image manually
These instructions are provided as a convenience; the application should be deployable following Deploying via heroku.yml above.
NOTE: This assumes that you have a version of Docker installed.
- Build the production image
rm -Rf archive archive.tgz
git archive HEAD > archive.tgz
mkdir -p archive
tar -xvf archive.tgz -C archive
cd archive
docker build --platform=linux/amd64 -f Dockerfile -t td .
cd ..
rm -Rf archive- Run via
# assumes environment variables populated in
# .dev-env file
docker run --name=td --rm -d --env-file ./.dev-env -p 8000:8000 td
Push the Docker image to Heroku manually
If you wanted to deploy the pre-built image to Heroku, you would need to:
- have access to the
translation-database-demoandtranslation-databaseapps on Heroku - have authenticated with the Heroku Container Registry
- Tag and push for $APP_NAME:
docker tag td registry.heroku.com/${HEROKU_APP_NAME:-translation-database-demo}/web
docker tag td registry.heroku.com/${HEROKU_APP_NAME:-translation-database-demo}/worker
docker push registry.heroku.com/${HEROKU_APP_NAME:-translation-database-demo}/web
docker push registry.heroku.com/${HEROKU_APP_NAME:-translation-database-demo}/worker- Release to Heroku
heroku container:release web worker -a ${HEROKU_APP_NAME:-translation-database-demo}Repeat the steps above with the HEROKU_APP_NAME variable set for the production environment:
export HEROKU_APP_NAME=translation-database
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent