peikalunci/epdilutedclassifier

Diluted continuous perceptron using expectation propagation

Code to reproduce the results of the paper "Expectation propagation on the diluted classifier" by Alfredo Braunstein, Thomas Gueudré, Andrea Pagnani and Mirko Pieropan [1]. This package uses the implementation of Gaussian EP available at [2].

Installation

After dowloading the repository, open Julia, change your current directory to that of the repository and type ] to enter the Pkg REPL. From there, run activate . and then instantiate in order to install all required dependencies. These instructions can be executed automatically at Julia startup by including the following lines of code in the ~/.julia/config/startup.jl file (create the ~/.julia/config folder and the file if not already present):

using Pkg 
if isfile("Project.toml") && isfile("Manifest.toml") 
    Pkg.activate(".") 
end

Usage

The routines needed to perform the EP inference can be found in SparsePerceptron.jl and in run_instances.jl. The generative models for the weights of the teacher perceptron and for the training set of patterns are implemented in generate_instances.jl.

Tests

Some simple tests can be run by including the following scripts in Julia from the Julia REPL:

• test_gaussian_iid_noiseless.jl: sparse perceptron learning from a set of correctly labeled i.i.d. Gaussian patterns;
• test_gaussian_cor_noiseless.jl: sparse perceptron training using a set of correctly labeled patterns sampled from a multivariate Gaussian distribution;
• test_noisy_case.jl: the script runs an example of student perceptron learning from a set of i.i.d. Gaussian patterns and another one from correlated Gaussian patterns. In both cases, a small fraction of the examples are mislabeled.
• test_corr_binary_patterns_noiseless.jl: sparse perceptron learning from a set of correctly labeled binary patterns generated by a network of perceptrons according to an asynchronous Glauber dynamics at zero temperature.

Each script runs the EP based inference on a randomly generated instance and prints the teacher-student overlap and the related mean squared error in dB (both as the estimation is performed and at the end of the inference procedure).

References

[1] Alfredo Braunstein, Thomas Gueudré, Andrea Pagnani, and Mirko Pieropan. Expectation propagation on the diluted Bayesian classifier. Phys. Rev. E 103 043301 (Apr. 2021).

[2] GaussianEP package: https://github.com/abraunst/GaussianEP