jbdatascience/ecco

Tools to visuals and explore NLP language models.

Ecco is a python library for explaining Natural Language Processing models using interactive visualizations.

It provides multiple interfaces to aid the explanation and intuition of Transformer-based language models. Read: Interfaces for Explaining Transformer Language Models.

Ecco runs inside Jupyter notebooks. It is built on top of pytorch and transformers.

The library is currently an alpha release of a research project. Not production ready. You're welcome to contribute to make it better!

Installation

# Assuming you had PyTorch previously installed
pip install ecco

Documentation

To use the project:

import ecco

# Load pre-trained language model.
lm = ecco.from_pretrained('distilgpt2')

# Input text
text = "The countries of the European Union are:\n1. Austria\n2. Belgium\n3. Bulgaria\n4."

# Generate 20 tokens to complete the input text.
output = lm.generate(text, generate=20, do_sample=True)

This does the following:

1. It loads a pretrained Huggingface DistilGPT2 model. It wraps it an ecco LM object that does useful things (e.g. it calculates input saliency, can collect neuron activations).
2. We tell the model to generate 20 tokens.
3. The model returns an ecco OutputSeq object. This object holds the output sequence, but also a lot of data generated by the generation run, including the input sequence and input saliency values. If we set activations=True in from_pretrained(), then this would also contain neuron activation values.
4. output can now produce various interactive explorables. Examples include:

• output.saliency() to generate input saliency explorable [Input Saliency Colab Notebook]
• output.run_nmf() to to explore non-negative matrix factorization of neuron activations [Neuron Activation Colab Notebook]