guidelabs/infembed

Find the samples, in the test data, on which your (generative) model makes mistakes.

♾️ Embed

InfEmbed is an error detection library for machine learning models (classifiers and generative models) built in PyTorch. Given a test dataset, InfEmbed allows you to find the groups of samples in the test data on which your model makes mistakes (errors).

InfEmbed is currently under actively development, so we welcome bug reports, pull requests, and your feedback.

Installation

We recommend that you setup a conda or virtual environment for your analysis.

Installation Requirements

• Python >= 3.9
• PyTorch >= 1.12

Manual install via cloning the repo

git clone https://github.com/guidelabs/infembed.git

# now change into the infembed directory
cd infembed

# install the package
pip install .

Getting Started and Tutorial

In this section, we will go over a simple tutorial setting to demonstrate the capabilities of the InfEmbed package.

FAQ

• How is error finding different from applying an explanability method?
  • An explanability method typically identifies the most important features (or data samples) that a model is sensitive to for its output. This is different from whether the model is more likely to make a mistake on that sample.
• Do you support generative models?
  • Yes, our approach translates in a straightforward manner to generative models on images, and large language models trained on text. We will provide examples on how to apply the package for both settings in upcoming updates.
• What does is mean for a generative model to make a mistake (or error)?
  • This typically means that the model has low performance when it generates outputs with similar features to the sample under consideration. For example, it might be the case that an image generative model generates blurry outputs for Fruits. It could be that a large language model is more likely to hallucinate on a particular type of prompt.

References

Infembed implements several embedding algorithms.

• Our implementation approach is inspired by the Captum. However, we focus on error discovery.

• ArnoldiEmbedder is based on the work of Wang et. al. (2023). It uses a Arnoldi iteration to compute an approximation to the Hessian.

• FastKFACEmbedder is based on the work of Gross, Bae, & Anil et. al. (2023). It uses a KFAC approximation to the Hessian, which makes the following approximations: it computes the Gauss-Newton Hessian (GNH), which is always guaranteed to be positive semi-definitive. The GNH is assumed to be block-diagonal, where the blocks correspond to parameters from different layers.

• GradientEmbedder is based on the work of Zeng et. al. (2023). It use a PCA projected version of the gradients for determining the error clusters.

License

InfEmbed is distributed under the BSD license.

infembed's Issues

Cannot compute embeddings in the examples

Hi,

I have tried to replicate the results of your paper and for that have run the notebooks with toy and AGNews examples but failed as there are still some bugs. Could you please share the version of the code that you used in the paper: https://papers.nips.cc/paper_files/paper/2023/file/8278a2e5f9db8489cd908d20c43f1f87-Paper-Conference.pdf

Thanks and regards,
Ana

Computing train embeddings

Hi there,
I was able to successfully use the tutorial in imagenet_case_study_fast_kfac.ipynb on my own data to compute influence embeddings for my test set.

However, I would like to find slice opponents, so I am wondering how to compute influence embeddings for the training data?
I tried calling predict for my training data on the embedder that was fit to the training data, but the results seemed wrong as all the embeddings were essentially zero.
Alternatively, is there a simpler and/or built-in way to compute slice opponents?

thanks!

missing dependencies

The following libraries are not included into pyproject.toml :

dill
faiss
lightning