lsiecker/text-mining

Natural and Technical Language Processing using Spacy, Named Entity Recognition and a custom Relationship Extraction and Labeling component

text-mining's Issues

Preprocessor class documenteren #part1

RE model trainen en optimizen #part1

Clean data #part2

Syntax transformeren voor REBEL/BERT #part1

Preprocess and export data #part2

RE model valideren #part1

Data preprocessing notebook twee datasets genereren uitwerken #part1

Nu is alleen 0.97 similarity geimplementeerd

Kies feasibile RE method #part1

Precision and Recall dataselectie uitvoeren #part1

training_data and validation_data is equal

The following code will result in the output down below and it is needed to differ.

print("Training data info item 1 \ntext:")
print(training_data[0][0])
print("Labels:")
print(*training_data[0][1]["entities"], sep = "\n")

print("\n Validation data info item 1 \ntext:")
print(validation_data[0][0])
print("Labels:")
print(*validation_data[0][1]["entities"], sep = "\n")

Output:

Training data info item 1 
text:
 Ephesus (; ; ; may ultimately derive from ) was a city in Ancient Greece on the coast of Ionia, southwest of present-day Seluk in zmir Province, Turkey.
Labels:
[1, 9, 'landmark_name']

 Validation data info item 1 
text:
 Ephesus (; ; ; may ultimately derive from ) was a city in Ancient Greece on the coast of Ionia, southwest of present-day Seluk in zmir Province, Turkey.
Labels:
[1, 8, 'landmark_name']