nlp_termproject_cssm530_akpinar's Introduction

Note: The term paper uploaded on 19 May 2024 is not peer-reviewed. We will make corrections until next Sunday after we receive feedback from our instructors and classmates.
Note2 : The term paper uploaded on 06 June 2024 includes an extra Introduction. The rest of the research left untouched for re-writing with a different approach. Training sets needs upgrade from more years. Scraping code will be enhanced. The working process will soon be migrated to thesis_log, and the final project to python.exercises repositories.
v2 file requires code edits to work locally

NLP_TermProject_CSSM530_akpinar

Read Me

NLP Term Research Project Spring2024, Automated Text Processing for Social Sciences

There are three parts to the research process. Data acquisition and cleaning for creating the base datasets. Target prediction on new data using Turkish BERT. Result evaluation by checking F1 score using manually validated data, and applying topic clustering using LDA. These pipelines are provided in resmi_gazete_scrape.ipynb, BERT_predict.ipynb, and evaluate_validate.ipynb, respectively. All necessary modules are stored in base_imports.py file. All necessary documents for replication are indicated in relevant file destinations.
To access the working environment, check requirements.txt, which comprises all the packages of the environment.
For preparing, refining, and editing the code, as well as documentation for requirements, I got help from ChatGPT-4o.
I am receiving an error when I want to share the chat content. However, the chat logs can be provided if needed.

Recommend Projects