TopicNet library was created to assist in the task of building topic models.
It aims at automating model training routine freeing more time for artistic process of constructing a target functional for the task at hand.
Consider using TopicNet if:
- you want to explore BigARTM functionality without writing an overhead;
- you need help with rapid solution prototyping;
- you want to build a good topic model quickly (out-of-box, with default parameters);
- you have an ARTM model at hand and you want to explore it's topics.
TopicNet provides an infrastructure for your prototyping with the help of Experiment class and helps to observe results of your actions via viewers module.
And here is sample code of the TopicNet baseline experiment:
from topicnet.cooking_machine.config_parser import build_experiment_environment_from_yaml_config
from topicnet.cooking_machine.recipes import ARTM_baseline as config_string
config_string = config_string.format(
dataset_path = '/data/datasets/NIPS/dataset.csv',
modality_list = ['@word'],
main_modality = '@word',
specific_topics = [f'spc_topic_{i}' for i in range(19)],
background_topics = [f'bcg_topic_{i}' for i in range( 1)],
)
experiment, dataset = (
build_experiment_environment_from_yaml_config(
yaml_string = config_string,
experiment_id = 'sample_config',
save_path = 'sample_save_folder_path',
)
)
experiment.run(dataset)
best_model = experiment.select('PerplexityScore@all -> min')[0]Define TopicModel from an ARTM model at hand or with help from model_constructor module, where you can set models main parameters. Then create an Experiment, assigning a root position to this model and path to store your experiment. Further, you can define a set of training stages by the functionality provided by the cooking_machine.cubes module.
Further you can read documentation here.
Core library functionality is based on BigARTM library. So BigARTM should also be installed on the machine. Fortunately, the installation process should not be so difficult now. Below are the detailed explanations.
The easiest way to install everything is via pip (but currently works fine only for Linux users!)
pip install topicnetThe command also installs BigARTM library, not only TopicNet. However, BigARTM Command Line Utility will not be assembled. Pip installation makes it possible to use BigARTM only through Python Interface.
If working on Windows or Mac, you should install BigARTM by yourself first, then pip install topicnet will work just fine.
We are hoping to bring all-in-pip installation support to the mentioned systems.
However, right now you may find the following guide useful.
To avoid installing BigARTM you can use docker images with preinstalled different versions of BigARTM library:
docker pull xtonev/bigartm:v0.10.0
docker run -t -i xtonev/bigartm:v0.10.0Checking if all installed successfully:
$ python
>>> import artm
>>> artm.version()Alternatively, you can follow BigARTM installation manual. There is also a pair of tips which may provide additional help for Windows users:
- Go to the installation page for Windows and download the 7z archive in the Downloads section.
- Use Anaconda
conda installto download all the Python packages that BigARTM requires. - Path variables must be set through the GUI window of system variables, and, if the variable
PYTHONPATHis missing — add it to the system wide variables. Close the GUI window.
After setting up the environment you can fork this repository or use pip install topicnet to install the library.
One can also install the library from GitHub, which may give more flexibility in developing (for example, making one's own viewers or regularizers a part of the module as .py files)
git clone https://github.com/machine-intelligence-laboratory/TopicNet.git
cd topicnet
pip install .As Linux installation may be done solely using pip, TopicNet can be used in such online services as
Google Colab and
Kaggle Notebooks.
All you need is to run the following command in a notebook cell:
! pip install topicnetThere is also a notebook in Google Colab made by Nikolay Gerasimenko, where BigARTM is build from source. This may be useful, for example, if you want to use the BigARTM Command Line Utility.
Let's say you have a handful of raw text mined from some source and you want to perform some topic modelling on them. Where should you start?
Every ML problem starts with data preprocess step. TopicNet does not perform data preprocessing itself. Instead, it demands data being prepared by the user and loaded via Dataset class. Here is a basic example of how one can achieve that: rtl_wiki_preprocessing.
Here we can finally get on the main part: making your own, best of them all, manually crafted Topic Model
We need to load our data prepared previously with Dataset:
DATASET_PATH = '/Wiki_raw_set/wiki_data.csv'
dataset = Dataset(DATASET_PATH)In case you want to start from a fresh model we suggest you use this code:
from topicnet.cooking_machine.model_constructor import init_simple_default_model
artm_model = init_simple_default_model(
dataset=dataset,
modalities_to_use={'@lemmatized': 1.0, '@bigram':0.5},
main_modality='@lemmatized',
specific_topics=14,
background_topics=1,
)Note that here we have model with two modalities: '@lemmatized' and '@bigram'.
Further, if needed, one can define a custom score to be calculated during the model training.
from topicnet.cooking_machine.models.base_score import BaseScore
class CustomScore(BaseScore):
def __init__(self):
super().__init__()
def call(self,
model,
eps=1e-5,
n_specific_topics=14):
phi = model.get_phi().values[:,:n_specific_topics]
specific_sparsity = np.sum(phi < eps) / np.sum(phi < 1)
return specific_sparsityNow, TopicModel with custom score can be defined:
from topicnet.cooking_machine.models.topic_model import TopicModel
custom_scores = {'SpecificSparsity': CustomScore()}
topic_model = TopicModel(artm_model, model_id='Groot', custom_scores=custom_scores)For further model training and tuning Experiment is necessary:
from topicnet.cooking_machine.experiment import Experiment
experiment = Experiment(experiment_id="simple_experiment", save_path="experiments", topic_model=topic_model)Defining a next stage of the model training to select a decorrelator parameter:
from topicnet.cooking_machine.cubes import RegularizersModifierCube
my_first_cube = RegularizersModifierCube(
num_iter=5,
tracked_score_function='PerplexityScore@lemmatized',
regularizer_parameters={
'regularizer': artm.DecorrelatorPhiRegularizer(name='decorrelation_phi', tau=1),
'tau_grid': [0,1,2,3,4,5],
},
reg_search='grid',
verbose=True,
)
my_first_cube(topic_model, dataset)Selecting a model with best perplexity score:
perplexity_criterion = 'PerplexityScore@lemmatized -> min COLLECT 1'
best_model = experiment.select(perplexity_criterion)If you need a topic model now, you can use one of the code snippets we call recipes.
from topicnet.cooking_machine.recipes import BaselineRecipe
training_pipeline = BaselineRecipe()
EXPERIMENT_PATH = '/home/user/experiment/'
training_pipeline.format_recipe(dataset_path=DATASET_PATH)
experiment, dataset = training_pipeline.build_experiment_environment(save_path=EXPERIMENT_PATH,)after that you can expect a following result:
Browsing the model is easy: create a viewer and call its view() method (or view_from_jupyter() — it is advised to use it if working in Jupyter Notebook):
from topicnet.viewers import TopTokensViewer
toptok_viewer = TopTokensViewer(best_model, num_top_tokens=10, method='phi')
toptok_viewer.view_from_jupyter()More info about different viewers is available here: viewers.
It is a convention to write data designating modalities with @ sign taken by TopicNet from BigARTM.
CubeCreator helps to perform a grid search over initial model parameters. How can I do it with modalities?
Modality search space can be defined using standart library logic like:
class_ids_cube = CubeCreator(
num_iter=5,
parameters: [
name: 'class_ids',
values: {
'@text': [1, 2, 3],
'@ngrams': [4, 5, 6],
},
]
reg_search='grid',
verbose=True,
)However, for the case of modalities a couple of slightly more convenient methods are availiable:
parameters : [
{
'name' : 'class_ids@text',
'values': [1, 2, 3]
},
{
'name' : 'class_ids@ngrams',
'values': [4, 5, 6]
}
]
parameters:[
{
'class_ids@text' : [1, 2, 3],
'class_ids@ngrams': [4, 5, 6]
}
]If you find a bug, or if you would like the library to have some new features — you are welcome to contact us or create an issue or a pull request!
It also worth noting that TopicNet library is always open to improvements in several areas:
- New custom regularizers.
- New topic model scores.
- New topic models or recipes to train topic models for a particular task/with some special properties.
- New datasets (so as to make them available for everyone to download and conduct experiments with topic models).
When citing topicnet in academic papers and theses, please use this BibTeX entry:
@InProceedings{bulatov-EtAl:2020:LREC,
author = {Bulatov, Victor and Alekseev, Vasiliy and Vorontsov, Konstantin and Polyudova, Darya and Veselova, Eugenia and Goncharov, Alexey and Egorov, Evgeny},
title = {TopicNet: Making Additive Regularisation for Topic Modelling Accessible},
booktitle = {Proceedings of The 12th Language Resources and Evaluation Conference},
month = {May},
year = {2020},
address = {Marseille, France},
publisher = {European Language Resources Association},
pages = {6747--6754},
url = {https://www.aclweb.org/anthology/2020.lrec-1.833}
}
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent