joolsa/binary-classifier-for-online-hate-detection-in-multiple-social-media-platforms

binary-classifier-for-online-hate-detection-in-multiple-social-media-platforms's Issues

Problem reproducing the result

I was redirected from the paper Developing an online hate classifier for multiple social media platforms.

I am wondering how to reproduce the result we see in the paper such as Table 9, Table 10, etc.

Based on the code and the pdf file in the repository right now, APG-online-hate-classifier.csv is used for training and new_data.csv is used for testing.

But seems like comments contain in new_data.csv is a subset of APG-online-hate-classifier.csv...

requirements.txt file?

Do you have a requirements.txt file?

I'm trying to run this with python 3.7 and "old" versions of the modules but I got:

/usr/local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/usr/local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/usr/local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/usr/local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/usr/local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/usr/local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
/usr/local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/usr/local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/usr/local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/usr/local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/usr/local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/usr/local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
/usr/local/lib/python3.7/site-packages/sklearn/externals/joblib/__init__.py:15: DeprecationWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.
  warnings.warn(msg, category=DeprecationWarning)
WARNING:tensorflow:From /usr/local/lib/python3.7/site-packages/bert/optimization.py:87: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.

WARNING:tensorflow:From /usr/src/myapp/bert_functions.py:135: The name tf.layers.Layer is deprecated. Please use tf.compat.v1.layers.Layer instead.

WARNING:tensorflow:From predict.py:42: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

2021-08-17 22:26:03.274287: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3409955000 Hz
2021-08-17 22:26:03.274801: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x563d0ca4d320 executing computations on platform Host. Devices:
2021-08-17 22:26:03.274821: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): <undefined>, <undefined>
2021-08-17 22:27:06.626009: W tensorflow/core/graph/graph_constructor.cc:1352] Importing a graph with a lower producer version 27 into an existing graph with producer version 38. Shape inference will have run different parts of the graph with different producer versions.
WARNING:tensorflow:Entity <bound method BertLayer.call of <bert_functions.BertLayer object at 0x7f7ccccaab50>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: converting <bound method BertLayer.call of <bert_functions.BertLayer object at 0x7f7ccccaab50>>: AttributeError: module 'gast' has no attribute 'Index'
WARNING:tensorflow:Entity <bound method BertLayer.call of <bert_functions.BertLayer object at 0x7f7ccccaab50>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: converting <bound method BertLayer.call of <bert_functions.BertLayer object at 0x7f7ccccaab50>>: AttributeError: module 'gast' has no attribute 'Index'
WARNING:tensorflow:From /usr/local/lib/python3.7/site-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python3.7/site-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
Traceback (most recent call last):
  File "predict.py", line 44, in <module>
    bert_model = build_model(max_seq_length, bert_path="https://tfhub.dev/google/bert_uncased_L-12_H-768_A-12/1")
  File "/usr/src/myapp/bert_functions.py", line 195, in build_model
    model = tf.keras.models.Model(inputs=bert_inputs, outputs=pred)
  File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py", line 129, in __init__
    super(Model, self).__init__(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/engine/network.py", line 167, in __init__
    tf_utils.assert_no_legacy_layers(self.layers)
  File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/utils/tf_utils.py", line 399, in assert_no_legacy_layers
    'than tf.layers)'.format(layer_str))
TypeError: The following are legacy tf.layers.Layers:
  <bert_functions.BertLayer object at 0x7f7ccccaab50>
To use keras as a framework (for instance using the Network, Model, or Sequential classes), please use the tf.keras.layers implementation instead. (Or, if writing custom layers, subclass from tf.keras.layers rather than tf.layers)

I'm using:

bert-tensorflow==1.0.1
joblib==0.13.2        
pandas==0.24.2        
params-flow==0.2.2    
py-params==0.6.3      
scikit-learn==0.21.2  
tqdm==4.32.2          
xgboost==0.90         
tensorflow==1.14.0    
matplotlib==3.1.0     
tensorflow-hub==0.5.0