Comments (8)
Sorry for the late reply, but yes it works now exactly as it should! Perfect, thank you!
from chat-analytics.
There's quite a few issues with that regex (unnecessary grouping, TLD limited to 6 characters, TLD required, misses many characters allowed in path). I feel like it would make more sense to use a regex actually already used by one of the supported platforms, since I'm pretty sure it's anything that the platform highlights blue which we want to exclude. I already found the regex Discord uses ^(https?:\/\/[^\s<]+[^<.,:;"')\]\s])
and it is much more simple and lenient (matching many non-URLs, which I'd much prefer over not matching many URLs). I can't remember why their regex only matches the beginning of a string, but modified for ChatAnalytics it would be https?:\/\/[^\s<]+[^<.,:;"')\]\s]
. I'll make a PR now
from chat-analytics.
Hey @UnduGT could you try generating the report again? (using the web app)
Thanks @hopperelec for the Regex!
from chat-analytics.
Hi! Thanks for the report!
I havent experienced this myself, you mean the small embed Discord includes after sending a link or the link itself?
The links itself should be tokenized as URLs and be completely excluded from the Language tab. Maybe the tokenizer is broken?
chat-analytics/pipeline/process/nlp/Tokenizer.ts
Lines 51 to 56 in de8c517
from chat-analytics.
It's the link itself, for example if I search for "MakeModelVariant1" in Discord it shows over 400 results consisting only of links from the same page, no Embeds.
from chat-analytics.
It seems the regex is missing additional URL parameters!
We should find a better regular expression
from chat-analytics.
Could just add an ampersand to the character set. Also, not sure what the purpose of that non-capturing group is, or the trailing forward slash considering that's already part of the prior character set
from chat-analytics.
Lets go for it, do you want to open a PR?
I did not write that regex, I shamelessly stole it from the internet 😅. It has to be revised at some point
from chat-analytics.
Related Issues (20)
- Telegram parse error HOT 4
- Allow spliting the data portion into multiple files. HOT 3
- Error parsing file after ~75GB of JSON HOT 3
- Everything is processed in your browser. Lock No data leaves your device. HOT 4
- Add Signal platform HOT 2
- package.json not recognized in NPX HOT 2
- Discord - Locally stored users profile pictures not working HOT 4
- Show positive/negative content/authors? HOT 3
- Words like "yes" and "no" do not appear HOT 2
- Suggestion: Sticker Tab
- website throws "Call start date is after end date" after uploading data HOT 2
- [Suggestion] Instagram chat analysis HOT 15
- Handle Unicode escape sequences for emojis and symbols HOT 1
- Emoji Sentiment scores get processed as string HOT 1
- cant install the chat analytics package HOT 3
- Names of Reaction Users are Missing HOT 1
- compile chat-analytics HOT 1
- About regular vs custom emojis HOT 5
- Language tab doesn't seem to detect french HOT 23
- [Feature] Exporting Charts HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from chat-analytics.