04mayukh/youngsheldon-at-semeval-2021-task-7-hahackathon

This repository contains the code for submission made at SemEval 2021: Task 7: HaHackathon: Detecting and Rating Humor and Offense

youngsheldon-at-semeval-2021-task-7-hahackathon's Issues

slang.txt missing

It is referenced in the task 1b notebook but the file is not in this repository

Library Incompatabilities

I cannot install the required libraries due to incompatabilities in the currently available libraries.

This command fails
conda create --yes --name Haha_Mayukh nltk pandas tensorflow-gpu notebook tf-models-official --channel conda-forge --channel hcc

with a long list of libraries (mostly ones required by the above libraries) that are incompatible with each other.

pip doesn't check for incompatabilities as thoroughly as conda does, and I noticed that some libraries used in subtask 1b.ipynb weren't installed via pip commands in the notebook. So I'm not sure how to replicate this code.

Do you have any suggestions?

code references out-of-date subdirectory names

The code references directories like /content/
Actual files in data/train (or test or dev)

Task 1 a class weights

The model.fit has a class weights parameter that references an empty dictionary
class_weight=class_weight_

I think it was meant to say
class_weight=class_weight_motivation

Hardware-specific code hard-coded

Use of tensorflow.distribute modules needs to be rewritten to be hardware agnostic.
see get_strategy() in this webpage
https://www.tensorflow.org/guide/distributed_training

So I suggest using code like this

if tf.config.list_physical_devices('TPU'):
    resolver = tf.distribute.cluster_resolver.TPUClusterResolver(tpu='grpc://' + os.environ['COLAB_TPU_ADDR'])
    tf.config.experimental_connect_to_cluster(resolver)
    tf.tpu.experimental.initialize_tpu_system(resolver)
    print("All devices: ", tf.config.list_logical_devices('TPU'))
    strategy = tf.distribute.TPUStrategy(resolver)
elif tf.config.list_physical_devices('GPU'):
  strategy = tf.distribute.MirroredStrategy()
else:  # Use the Default Strategy
  strategy = tf.distribute.get_strategy()