aniskywalker / sarcasmdetection Goto Github PK
View Code? Open in Web Editor NEWSarcasm detection on tweets using neural network
Home Page: https://www.aclweb.org/anthology/W/W16/W16-0425.pdf
License: GNU Lesser General Public License v3.0
Sarcasm detection on tweets using neural network
Home Page: https://www.aclweb.org/anthology/W/W16/W16-0425.pdf
License: GNU Lesser General Public License v3.0
I am running into an error that's caused by line 286 of sarcasm_detection_model_CNN_LSTM_DNN.py
. As you can see the string slice, to get the directory above the working directory, with index os.getcwd().rfind('/')
doesn't work with a path that contains backslashes instead of forward slashes (as is the case in Windows).
Traceback (most recent call last):
File "sarcasm_detection_model_CNN_LSTM_DNN.py", line 303, in
t.load_trained_model(weight_file='weights.05__.hdf5')
File "sarcasm_detection_model_CNN_LSTM_DNN.py", line 201, in load_trained_model
self.__load_model(self._model_file_path + model_file, self._model_file_path + weight_file)
File "sarcasm_detection_model_CNN_LSTM_DNN.py", line 206, in __load_model
self.model = model_from_json(open(model_path).read())
FileNotFoundError: [Errno 2] No such file or directory: 'c:\Users\Niels\Documents\Maaksels\SarcasmDetection\sr\resource/text_model/weights/model.json'
basepath = os.path.abspath(os.path.join(os.getcwd(), '..'))
Should work, and achieve the desired result, for both Windows and Unix style paths.
I'll be committing this fix to the pull request that I've already made.
Just open an issue and wanna discuss about it.
Now I have encountered this phenomenon. I have a bunch of data from daily communication, well, as you may be realized, it is completely different from what we have witnessed in the training set.
So I am wondering what should I do in order to enhance the performance. Anyone could give some ideas??
Two questions:
I have a CSV with one column that contains tweets.
https://pastebin.com/wxwbmD16 this took about 1 hour for me. I was wondering how I could use a saved model and don't start training from scratch.
Can you please upload this file as well?
[jalal@goku src]$ python sarcasm_detection_model_CNN_LSTM_DNN_word2vec.py
Using TensorFlow backend.
2018-03-03 20:58:10.558623: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2018-03-03 20:58:10.558655: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2018-03-03 20:58:10.558664: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2018-03-03 20:58:10.558671: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2018-03-03 20:58:10.558678: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
2018-03-03 20:58:10.860328: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 0 with properties:
name: GeForce GTX 1080 Ti
major: 6 minor: 1 memoryClockRate (GHz) 1.6705
pciBusID 0000:05:00.0
Total memory: 10.92GiB
Free memory: 10.17GiB
2018-03-03 20:58:11.117002: W tensorflow/stream_executor/cuda/cuda_driver.cc:523] A non-primary context 0x55a328acfc60 exists before initializing the StreamExecutor. We haven't verified StreamExecutor works with that.
2018-03-03 20:58:11.117900: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 1 with properties:
name: GeForce GTX 1080 Ti
major: 6 minor: 1 memoryClockRate (GHz) 1.6705
pciBusID 0000:06:00.0
Total memory: 10.92GiB
Free memory: 10.76GiB
2018-03-03 20:58:11.118705: I tensorflow/core/common_runtime/gpu/gpu_device.cc:976] DMA: 0 1
2018-03-03 20:58:11.118723: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 0: Y Y
2018-03-03 20:58:11.118730: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 1: Y Y
2018-03-03 20:58:11.118744: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:05:00.0)
2018-03-03 20:58:11.118752: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:1) -> (device: 1, name: GeForce GTX 1080 Ti, pci bus id: 0000:06:00.0)
Loading resource...
initializing...
test_maxlen 30
Traceback (most recent call last):
File "sarcasm_detection_model_CNN_LSTM_DNN_word2vec.py", line 285, in <module>
t.load_trained_model()
File "sarcasm_detection_model_CNN_LSTM_DNN_word2vec.py", line 187, in load_trained_model
self.__load_model(self._model_file + 'model_wv.json', self._model_file + weight_file)
File "sarcasm_detection_model_CNN_LSTM_DNN_word2vec.py", line 192, in __load_model
self.model = model_from_json(open(model_path).read())
FileNotFoundError: [Errno 2] No such file or directory: 'SarcasmDetection/resource/text_model/weights/model_wv.json'
[jalal@goku src]$ ls ../resource/text_model
text_model/ text_model_2D/
[jalal@goku src]$ ls ../resource/text_model/
TestResults.txt.analysis vocab_list.txt weights
[jalal@goku src]$ ls ../resource/text_model/weights/
model.json weights.01__.hdf5 weights.03__.hdf5 weights.05__.hdf5 weights.07__.hdf5 weights.09__.hdf5
model.json.hdf5 weights.02__.hdf5 weights.04__.hdf5 weights.06__.hdf5 weights.08__.hdf5 weights.10__.hdf5
Hi there,
Thank you to the authors for making your research, code, and results public. I'm trying to run the model from sarcasm_detection_model_CNN_LSTM_DNN.py
but keep coming across this error:
model loaded from file...
Traceback (most recent call last):
File "sarcasm_detection_model_CNN_LSTM_DNN.py", line 314, in
t.load_trained_model(model_file='model.json', weight_file='model.json.hdf5')
File "sarcasm_detection_model_CNN_LSTM_DNN.py", line 208, in load_trained_model
self.__load_model(self._model_file_path + model_file, self._model_file_path + weight_file)
File "sarcasm_detection_model_CNN_LSTM_DNN.py", line 215, in __load_model
self.model.load_weights(model_weight_path)
File "C:\Users\Me\Anaconda3\envs\sarcasmpy\lib\site-packages\keras\models.py", line 768, in load_weights
topology.load_weights_from_hdf5_group(f, layers, reshape=reshape)
File "C:\Users\Me\Anaconda3\envs\sarcasmpy\lib\site-packages\keras\engine\topology.py", line 3365, in load_weights_from_hdf5_group
str(len(filtered_layers)) + ' layers.')
ValueError: You are trying to load a weight file containing 7 layers into a model with 5 layers.
The model.json
says that it uses Keras v 2.1.6, which I'm using. I'm also working with Python 3.7.
Has anyone come across this? Thanks for your help.
I ran the following:
[jalal@goku src]$ python sarcasm_detection_model_attention.py
(commented the training part).
And got this error:
#_L_S
#_s_t
#_S_h
#_s_w
#_f_u
#_n_o
#_B_u
#_t_o
#_n_e
#_t_r
#_I_J
#_l_i
vocab loaded...
Token coverage: 0.4024455818214407
Word coverage: 0.006309334276785187
Error: Error when checking : expected embedding_1_input to have shape (None, 30) but got array with shape (2000, 50)
Do you know how should I fix it?
Thank you so much for making your code and dataset available on Github for fellow researchers.
But I see that the data files for the dataset with context information are missing from the repository. Can you please share them with me or upload them to the GitHub repo?
train_file = basepath + '/resource/train/Train_context_moods_v1.txt’
validation_file = basepath + '/resource/dev/Dev_context_moods.txt’
test_file = basepath + '/resource/test/Test_context_AW.txt'
May I know is the dataset here the original one in the paper? - Fracking sarcasm using neural network.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.