Comments (6)
The problem comes from the word nothing
. It is considered as a negating word and there is a rule that will 'flip' the valence of a token when such word is found preceding your input.
If you try the same case with a sentence slightly longer, it won't make the link between the negating word and the smiley, and you will get the expected weightage:
sentence: nothing at all for redheads :(
polarity got: {'neg': 0.367, 'neu': 0.633, 'pos': 0.0, 'compound': -0.44}
from vadersentiment.
@CodeWingX The README provides a description of the values in the lexicon:
We collected intensity ratings on each of our candidate lexical features from ten independent human raters (for a total of 90,000+ ratings). Features were rated on a scale from "[–4] Extremely Negative" to "[4] Extremely Positive", with allowance for "[0] Neutral (or Neither, N/A)".
We kept every lexical feature that had a non-zero mean rating, and whose standard deviation was less than 2.5 as determined by the aggregate of ten independent raters.
I assume based on this information that vader_lexicon.txt holds the following format:
Token | Valence | Standard Deviation | Human Ratings |
---|---|---|---|
(:< | -0.2 |
2.03961 |
[-2, -3, 1, 1, 2, -1, 2, 1, -4, 1] |
amorphous | -0.2 |
0.4 |
[0, 0, 0, 0, 0, 0, -1, 0, 0, -1] |
If you want to follow the same rigorous process as the author of the study, you should find 10 independent humans to evaluate each word you want to add to the lexicon, make sure the standard deviation doesn't exceed 2.5, and take the average rating for the valence. This will keep the file consistent.
Now if you just want to make the algorithm work on these new cases quickly, the standard deviation and human ratings are indeed not necessary. Only the token and valences are used.
from vadersentiment.
Would it be better if the negating words don't negate emoticons? Emoticons, unlike other words in a sentence, have a meaning which summarises the emotion felt while writing the text. So if they are treated as a separate sentence.
example: "nothing for redheads :(" -> "nothing for redheads. sad."
I don't mean to say, swap the emoticon with a word, but the scoring to be done this way?
from vadersentiment.
This makes sense for sure, then to say if this is better or not would require implementing the rule, measuring the accuracy of the new algorithm and comparing it to the current one. The repo contains quite a lot of human-scored sentences, so that shouldn't be an issue if you want to spend the time to look into this idea 😃
from vadersentiment.
@Hiestaa I do want to try out a few things, but I don't yet understand how to go about the values in the vader_lexicon.txt.
Going through the source code I inferred that only the word and the valence? are being taken into consideration for scoring sentences. So if I need to add more words to the list, do the other two values not matter?
from vadersentiment.
Is there a study that shows empirical effects of emoticons and emojis in negated sentences? I've seen papers showing emojis/emoticons as sentence negations themselves... e.g., "I love my job 👎 ". But I haven't (yet) found anything describing a negation effect on the emoji/emoticon... e.g., your example "nothing for redheads :(". My intuition is that the general rule (in most cases) is that sentence negations (not, isn't, nothing, ain't) don't affect emoji/emoticon, and that in most cases the emoji/emoticon is what people actually key in on for judging overall sentiment.
from vadersentiment.
Related Issues (20)
- Dictionary contains phrases like "fed up" that will never hit because of how the sentence is tokenized HOT 1
- Is NLP still required when using VADER?
- vaderSentiment data output in a different order than specified in the docs
- Wrong weight assigned for hashtags with capitals
- Doubt about threshold values used in VADER categorization
- is this thing still alive?
- incorrect result while running on large dataset HOT 2
- Demo None Type not Iterable HOT 2
- Adding Turkish words HOT 2
- help with adding values to vader_lexicon.txt HOT 1
- Download additional DATASETS AND TESTING RESOURCES mentioned in README HOT 3
- Total dataset is decreasing after processed by VADER
- VADER can't parse the word 'bad ass'? HOT 2
- Positive score always 0? HOT 2
- `SPECIAL_CASES` do not work HOT 3
- Support for new Emojis
- list index out of range error persists... HOT 2
- Supporting Aspect Based sentiment analysis
- Add Julia Fork Link To ReadMe
- the sentimentIntensityAnalyzer VADER is giving results only on emojicon logic and NOT text HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from vadersentiment.