jkbren/einet

Uncertainty and causal emergence in complex networks

Effective information and causal emergence in python

Python code for calculating effective information in networks. This can then be used to search for macroscale representations of a network such that the coarse grained representation has more effective information than the microscale, a phenomenon known as causal emergence. This code accompanies the recent paper:

The emergence of informative higher scales in complex networks
Brennan Klein and Erik Hoel, 2020. Complexity.
doi:10.1155/2020/8932526

---

Fig. 1: Effective information vs network size.

Fig. 2: Causal emergence vs preferential attachment.

---

Tutorial Notebooks (works in progress...)

1. Chapter 01 - Network Effective Information
2. Chapter 02 - Network Size and Effective Information
3. Chapter 03 - Determinism and Degeneracy
4. Chapter 04 - Effective Information in Real Networks
5. Chapter 05 - Causal Emergence in Preferential Attachment and SBMs
6. Chapter 06 - Causal Emergence and the Emergence of Scale
7. Chapter 07 - Estimating Causal Emergence in Real Networks
8. Chapter 08 - Miscellaneous
9. Chapter 09 - Spectral Causal Emergence

Installation and Usage

In order to use this code, first clone/download the repository. Below is a simple example usage. Please feel free to reach out if you find any bugs, have any questions, or if for some reason the code does not run.

>>> from ei_net import *
>>> import networkx as nx
>>> G = nx.karate_club_graph()
>>> print("effective_information(G) =", effective_information(G))

EI(G) = 2.3500950888734686

The tutorial notebooks are designed to walk through some of the main results from the paper above, in addition to several in-depth analyses that were not included in the original paper.

Requirements

This code is written in Python 3.x and uses the following packages:

• NetworkX
• Scipy
• Numpy
• And for replicating figures, you will need:
  • matplotlib
  • Pandas

The colormaps in the paper are from https://matplotlib.org/cmocean/ and the named colors are from https://medialab.github.io/iwanthue/.

Citation

If you use these methods and this code in your own research, please cite our paper:

Klein, B. & Hoel, E. (2020). The emergence of informative higher scales in complex networks. Complexity, no. 8932526. doi:10.1155/2020/8932526.

Bibtex:

@article{Klein2020causalemergence,
    title = {{The emergence of informative higher scales in complex networks}},
    author = {Klein, Brennan and Hoel, Erik},
    journal = {Complexity},
    year = {2020},
    pages = {1--12},
    volume = {2020},
    arxivId = {1907.03902v2},
    doi = {10.1155/2020/8932526}
}

See also:

• Hoel, E. (2017). When the map is better than the territory. Entropy. 19(5), 188; doi: 10.3390/e19050188.
  • recent work making explicit connections between causal emergence and the channel capacity of a model.
• Hoel, E., Albantakis, L., & Tononi, G. (2013). Quantifying causal emergence shows that macro can beat micro. Proceedings of the National Academy of Sciences. 110 (49) 19790-19795. doi: 10.1073/pnas.1314922110.
  • the first work to quantify causal emergence, showing how and why certain coarse-grained models can have more effective information.

einet's Issues

question - causal_emergence_spectral() for directed graphs

Hi,

first thanks for developing this new approach for network analysis. I am a big fan of the "certainty paradox" that you identified in biological systems.

As I was implementing your scripts to test some ideas, I realized that the function causal_emergence_spectral() returns a graph of Effective Information = 0 if a directed graph is given, which was unexpected since I assumed that G_macro would be exactly the same as G_micro if no better mapping had been found. Differently, causal_emergence() is able to find a macro mapping with a higher EI.

Here is the code to reproduce the issue:

import numpy as np
import networkx as nx
import einet.code.ce_net as ce_net

# generate sample graph
N = 100
G = nx.scale_free_graph(N, alpha=0.2, beta=0.55, gamma=0.25, seed=123)
# nx.draw(G, alpha=0.5)

# undirected causal emergence
ce_cg = ce_net.causal_emergence(nx.Graph(G), printt=False)
ce_spec = ce_net.causal_emergence_spectral(nx.Graph(G))

print('Undirected CE')
print('EI_micro='+str(np.round(ce_cg['EI_micro'],2))+' | EI_macro_cg='+str(np.round(ce_cg['EI_macro'],2))+' | EI_macro_spec='+str(np.round(ce_spec['EI_macro'],2)) )
print('N_micro_cg=',str(len(ce_cg['G_micro'].nodes)),'|','N_macro_cg=',str(len(ce_cg['G_macro'].nodes)))
print('N_micro_spec=',str(len(ce_spec['G_micro'].nodes)),'|','N_macro_spec=',str(len(ce_spec['G_macro'].nodes)))

# directed causal emergence
ce_cg = ce_net.causal_emergence(nx.DiGraph(G), printt=False)
ce_spec = ce_net.causal_emergence_spectral(nx.DiGraph(G))

print('Directed CE')
print('EI_micro='+str(np.round(ce_cg['EI_micro'],2))+' | EI_macro_cg='+str(np.round(ce_cg['EI_macro'],2))+' | EI_macro_spec='+str(np.round(ce_spec['EI_macro'],2)) )
print('N_micro_cg=',str(len(ce_cg['G_micro'].nodes)),'|','N_macro_cg=',str(len(ce_cg['G_macro'].nodes)))
print('N_micro_spec=',str(len(ce_spec['G_micro'].nodes)),'|','N_macro_spec=',str(len(ce_spec['G_macro'].nodes)))

[OUTPUT]
Undirected CE
EI_micro=3.9 | EI_macro_cg=4.34 | EI_macro_spec=4.32
N_micro_cg= 100 | N_macro_cg= 59
N_micro_spec= 100 | N_macro_spec= 66
Directed CE
EI_micro=3.16 | EI_macro_cg=3.88 | EI_macro_spec=0.0
N_micro_cg= 100 | N_macro_cg= 62
N_micro_spec= 100 | N_macro_spec= 1

After reading and testing the functions a bit more, I understood that the main issue was at the spectral decomposition level, since only 2 eigenvalues were higher than 0 in this case, and therefore everything was put into the same cluster / macronode afterwards.

So, my question is: how could one implement your causal_emergence_spectral() in a directed graph?

In the same line of thought of adding dist_add to the nodes that are not in the Markov blanket of each node, I tried adding dist_add to the positions of the distance matrix that correspond to the edges that appear in the undirected adjacency matrix, but not in the directed adjacency matrix. Although I could get a network with a higher EI, I think it does not make sense mathematically since I would be decomposing the undirected Wout and not the directed Wout, and therefore one may not get rid of the degenerated parts of the graph.
Have you implemented other approaches yet?

Thanks in advance!

Miquel