ulsdevteam/ccvgd-database

Contemporary Chinese Village Gazetteer Data database

Contemporary Chinese Village Gazetteer Data database

Context

Contemporary Chinese Village Gazetteer Data (CCVG Data, 数字村庄) is a project initiated by the University Library System to collect data from village gazetteers and present it in open-access datasets, supporting teaching and research on Chinese village history and development.

Purpose

This project reorganizes that raw data into a relational database and a web-based presentation of the data.

Project folders Overview

• UX_Nuoqi: Contains an AXURE RP prototyping file and HTML representation of the proposed web UI.
• day10_bootstrap : Contains an early prototype for the UI presentation.
• process_data: Contains early Jupyter notebooks for building the database.
• some codes to build database: Contains early SQLs used to restructure the database.
• pythonScript: Contains python scripts that will load the recorded 18 csv file into existing MySQL database. For details please read the README file inside pythonScript folder

Authorship and License

This is a work of the University of Pittsburgh.

Student work supervised by Daqing He, in collaboration with the Pitt ULS:

• Qi Xin
• Nuoqi Zeng
• Nianyun Qi

Original work in this repo is licensed under the MIT-license. Included works retain their respective licenses.

ccvgd-database's Issues

Clarify build process for database

@Niq995 , can you clarify some of the process for building the database?

Specifically:

• Can you provide a dataflow diagram of how the input files are processed by the python scripts into the output files?

  • I've drafted what I think this might look like here:
    

• Can you clarify what kinds of "mistakes" we might expect in the record that need to be fixed in advance per bullet 2?

• Can you suggest what the process would look like for correcting duplicate pinyins, and for determining if new categories need to be added?

pythonScript loader assumes case-insentivity

Some folder names:

ccvgd-database/pythonScript/main_process.py

Line 5 in 977fec7

output_csv_dirct = "Database Data"

And table/column names:

ccvgd-database/pythonScript/Table.py

Line 22 in 977fec7

" CONSTRAINT `village_村_ibfk_1` FOREIGN KEY (`gazetteerId_村志代码`) REFERENCES `gazetteerinformation_村志信息` (`gazetteerid_村志代码`)"

do not match the case used elsewhere in the repo.