Third Assignment for Language Analytics in Cultural Data Science
The assignments is about using word embeddings for expanding search queries in a lyrics database. The code finds how many, and what proportion of a given artists' songs contain a given keyword or words semantically closely related to that keyword.
I chose a fault tolerant approach, this has the following implications:
- If the given artist can't be exactly matched, you will get a warning, but the code will continue running with the closest fuzzy match.
- If the given keyword can't be exactly matched in the embedding models' vocabulary, you will get a warning, but the code will continue running with the closest fuzzy match.
You will need to download the Spotfy Million Song dataset.
the file should be placed in a dat
directory.
- dat/
- "Spotify Million Song Dataset_exported.csv"
Install dependencies:
pip install -r requirements.txt
You can query songs by using the src/query.py
command line interface for checking how many songs of a given artist contain a keyword or closesly related word.
python3 src/query.py -a "Steely Dan" -w "cousin"
The most similar terms are the following: cousin, nephew, brother, son, uncle, eldest, grandson, daughter, father, grandfather, niece
-------------------------------------------------------------
Artist Steely Dan has 15 songs (17.05%) that contain words related to cousin.
-------------------------------------------------------------
By passing the --print_songs
flag you can also see the individual songs containing these terms.
These are:
- Cousin Dupree
- Almost Gothic
- Babylon Sisters
- Chain Lightning
- Deacon Blues
- Don't Take Me Alive
- Godwhacker
- Green Flower Street
- Kid Charlemagne
- Pixeleen
- Pretzel Logic
- Sign In Stranger
- Time Out Of Mind
- Turn That Heartbeat Over Again
- Two Against Nature
Parameter | Description | Default |
---|---|---|
-a or --artist |
Name of the artist to query the songs of. | - |
-w or --query_word |
The seed word to base the semantic query on. | - |
-k or --k_expansion |
Number of termas most similar to the seed term to include in the query. | 10 |
--print_songs |
Flag to indicate whether the names of the songs should be printed. | False |
Additionally the script will produce csv files with the CO2 emissions of the substasks in the code (
emissions/
). This is necessary for Assignment 5, and is not directly relevant to this assignment.
Note: The
emissions/emissions.csv
file should be ignored. This is due to the fact, that codecarbon can't track process and task emissions at the same time.
It would be good to know how the number of expansion terms affect results and to which extent the pipeline is sensitive to this parameter. Systematic evaluations might give us more information about the implications of setting higher or lower values.