cgl/portuguese-nlp

Nlp work on Brazil Portuguese newswire text

portuguese-nlp

Nlp work on Brazil Portuguese newswire text

You can browse the dataset online and see annotations on drive

We have x number of newswire articles collected between years 1994-2016. After preprocessing the dataset, since the articles are in html format, we first clean the tags and rename all files such as:

folca/data/2005/01/01/19.html --> folca/parsed-data/2005_01_01_19.html

and collect them all in one folder.

• 1. Preprocessing on dataset
• 2. Crawling and Organizing the Training Set
• 3. Classification by Graphlab
• 4. Reports

More

• brat Documentation!
• Weka Documentation!
• Some parsing with Stanford Parser

portuguese-nlp's Issues

extractannotationlists file

Hello there,
as I understood from /doc/brad.md it is implied that extractannotationlists script is existing in extraction folder. However, it's not there. did I misunderstand or the script was not shared.

Thanks in Advance

Cannot Execute Binary File

I downloaded both LX Tokenizer and Stanford Parser, but when I want to run the first step (tokenization), I run into the following problem: /Users/zhengxinyong/Desktop/data/Tokenizer/Tokenizer/tokenizer: cannot execute binary file

My folder is structured as follows:

data/
    Tokenizer/
    input/
        try.txt
    tokenized/
    stanford-parser-4.0.0/

I have run the command chmod +x Tokenizer/Tokenizer/run-Tokenizer.sh as well, but when I run the following code on my terminal (cat input/try.txt | Tokenizer/Tokenizer/run-Tokenizer.sh > tokenized/try.txt;), I run into the error of not able to execute the binary file.

Dataset link is not working