stadlmax/graph-posterior-network

Graph Posterior Network: Bayesian Predictive Uncertainty for Node Classification (NeurIPS 2021)

graph-posterior-network's Issues

AttributeError: Can't get attribute 'DataEdgeAttr'

Hi, I followed the environment creation and when I tried to direct run the vanilla classification with the given yml, the error happened. I spent some time but still have no clue how to fix this. Could you help to look into this and help me out? Thank you so much!

(gpn) labcluster@lambda-:/data/xueyingy/Graph-Posterior-Network$ python3 train_and_eval.py with configs/gpn/classification_gpn_10.yaml data.dataset=CoraML
WARNING:root:The OGB package is out of date. Your version is 1.3.1, while the latest version is 1.3.4.
Using backend: pytorch
2022-10-25 23:30:22.619741: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
WARNING:root:Added new config entry: "data.dataset"
WARNING:root:Added new config entry: "data.ood_flag"
WARNING:root:Added new config entry: "data.root"
WARNING:root:Added new config entry: "data.split"
WARNING:root:Added new config entry: "data.split_no"
WARNING:root:Added new config entry: "data.test_samples_per_class"
WARNING:root:Added new config entry: "data.train_samples_per_class"
WARNING:root:Added new config entry: "data.val_samples_per_class"
WARNING:root:Added new config entry: "model.K"
WARNING:root:Added new config entry: "model.add_self_loops"
WARNING:root:Added new config entry: "model.alpha_evidence_scale"
WARNING:root:Added new config entry: "model.alpha_teleport"
WARNING:root:Added new config entry: "model.approximate_reg"
WARNING:root:Added new config entry: "model.dim_hidden"
WARNING:root:Added new config entry: "model.dim_latent"
WARNING:root:Added new config entry: "model.dropout_prob"
WARNING:root:Added new config entry: "model.entropy_reg"
WARNING:root:Added new config entry: "model.flow_weight_decay"
WARNING:root:Added new config entry: "model.gaussian_layers"
WARNING:root:Added new config entry: "model.init_no"
WARNING:root:Added new config entry: "model.loss_reduction"
WARNING:root:Added new config entry: "model.maf_layers"
WARNING:root:Added new config entry: "model.model_name"
WARNING:root:Added new config entry: "model.pre_train_mode"
WARNING:root:Added new config entry: "model.radial_layers"
WARNING:root:Added new config entry: "model.seed"
WARNING:root:Added new config entry: "model.use_batched_flow"
WARNING:root:Added new config entry: "run.eval_mode"
WARNING:root:Added new config entry: "run.experiment_directory"
WARNING:root:Added new config entry: "run.experiment_name"
WARNING:root:Added new config entry: "run.gpu"
WARNING:root:Added new config entry: "run.job"
WARNING:root:Added new config entry: "run.save_model"
WARNING:root:Added new config entry: "training.epochs"
WARNING:root:Added new config entry: "training.finetune_epochs"
WARNING:root:Added new config entry: "training.lr"
WARNING:root:Added new config entry: "training.stopping_metric"
WARNING:root:Added new config entry: "training.stopping_minimize"
WARNING:root:Added new config entry: "training.stopping_mode"
WARNING:root:Added new config entry: "training.stopping_patience"
WARNING:root:Added new config entry: "training.stopping_restore_best"
WARNING:root:Added new config entry: "training.warmup_epochs"
WARNING:root:Added new config entry: "training.weight_decay"
WARNING:train_and_eval:No observers have been added to this run
ERROR:train_and_eval:Failed after 0:00:00!
Traceback (most recent calls WITHOUT Sacred internals):
  File "train_and_eval.py", line 65, in run_experiment
    results = experiment.run()
  File "/data/xueyingy/Graph-Posterior-Network/gpn/experiments/multiple_run_experiment.py", line 50, in run
    results = self.run_transductive_experiment()
  File "/data/xueyingy/Graph-Posterior-Network/gpn/experiments/multiple_run_experiment.py", line 75, in run_transductive_experiment
    experiment = TransductiveExperiment(
  File "/data/xueyingy/Graph-Posterior-Network/gpn/experiments/transductive_experiment.py", line 96, in __init__
    self.dataset = ExperimentDataset(data_cfg, to_sparse=data_cfg.to_sparse)
  File "/data/xueyingy/Graph-Posterior-Network/gpn/experiments/dataset.py", line 58, in __init__
    dataset = DatasetManager(**data_cfg.to_dict())
  File "/data/xueyingy/Graph-Posterior-Network/gpn/data/dataset_manager.py", line 124, in DatasetManager
    data = D.CitationFull(root, 'Cora_ML', default_transform, None)
  File "/home/labcluster/.local/lib/python3.8/site-packages/torch_geometric/datasets/citation_full.py", line 39, in __init__
    self.data, self.slices = torch.load(self.processed_paths[0])
  File "/home/labcluster/.local/lib/python3.8/site-packages/torch/serialization.py", line 607, in load
    return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
  File "/home/labcluster/.local/lib/python3.8/site-packages/torch/serialization.py", line 882, in _load
    result = unpickler.load()
  File "/home/labcluster/.local/lib/python3.8/site-packages/torch/serialization.py", line 875, in find_class
    return super().find_class(mod_name, name)
AttributeError: Can't get attribute 'DataEdgeAttr' on <module 'torch_geometric.data.data' from '/home/labcluster/.local/lib/python3.8/site-packages/torch_geometric/data/data.py'>

The calculation of log_q in bgcn.py

Hi,
In gpn/models/bgcn.py line 110,
I think there is a small mistake.
Shouldn't it be like self.conv_1.log_q + self.conv_2.log_q?
Thanks for reading.

ResolvePackageNotFound for installing the conda environment

Hi, I tried to install the conda environment for the repo and encountered this problem:

(base) viet@cad660901:~/work/Graph-Posterior-Network$ conda env create -f gpn_environment.yml python==3.8 --force
Collecting package metadata (repodata.json): done
Solving environment: failed

ResolvePackageNotFound: 
  - python==3.8.3=hcff3b4d_2c

I wonder if there is any change from your side that I should follow? Thank you.

Epistemic uncertainty with and without effects

Hi, I ran the experiments for ood_loc_gpn_{10,16} to reproduce the results. The metrics are logged after training as follows:

	val	test
accuracy	0.923077	0.889306
brier_score	0.280527	0.269254
ECE	0.10638	0.0702577
confidence_aleatoric_apr	0.966722	0.980251
confidence_epistemic_apr	0.95776	0.962611
confidence_structure_apr	nan	nan
confidence_aleatoric_auroc	0.706276	0.864282
confidence_epistemic_auroc	0.635031	0.772384
confidence_structure_auroc	nan	nan
CE	0.37979	0.339846
avg_prediction_confidence_aleatoric	0.816697	0.819579
avg_prediction_confidence_epistemic	14919.8	14410.5
avg_sample_confidence_aleatoric	0.816697	0.819579
avg_sample_confidence_epistemic	16556.9	15927.3
avg_sample_confidence_features	15835.7	16779.6
avg_sample_confidence_neighborhood	nan	nan
average_entropy	0.513569	0.501767
ood_detection_aleatoric_apr	0.801985	0.78415
ood_detection_aleatoric_auroc	0.833289	0.836685
ood_detection_epistemic_apr	0.80048	0.788122
ood_detection_epistemic_auroc	0.852889	0.855747
ood_detection_features_apr	0.75896	0.748484
ood_detection_features_auroc	0.81794	0.8237
ood_detection_neighborhood_apr	nan	nan
ood_detection_neighborhood_auroc	nan	nan
ood_detection_structure_apr	nan	nan
ood_detection_structure_auroc	nan	nan
ood_accuracy	0	0
ood_avg_prediction_confidence_aleatoric	0.556919	0.558297
ood_avg_prediction_confidence_epistemic	2879.92	2808.5
ood_avg_sample_confidence_aleatoric	0.556919	0.558297
ood_avg_sample_confidence_epistemic	4068.24	4109.26
ood_avg_sample_confidence_neighborhood	nan	nan
ood_avg_sample_confidence_features	2063.81	1759.3
ood_average_entropy	1.03177	1.03767
id_accuracy	0.923077	0.889306
id_avg_prediction_confidence_aleatoric	0.816697	0.819579
id_avg_prediction_confidence_epistemic	14919.8	14410.5
id_avg_sample_confidence_aleatoric	0.816697	0.819579
id_avg_sample_confidence_epistemic	16556.9	15927.3
id_avg_sample_confidence_features	15835.7	16779.6
id_average_entropy	0.513569	0.501767

Is this experiment exclusive for the With Effect case? I would like to reproduce 3 cases: aleatoric w/Net, Epist w/Net and Epist w/o Net as table 1 in the paper. Could you provide me with some guidance? Thank you.

Test set proportion in data/split.py

Upon analyzing the sample_per_class function in the code (around lines 188 and 189), I noticed that it seems to be causing a discrepancy in the size of the test set. According to the code logic, it results in the test set being approximately (1-20%) * 80% of the total data, rather than the stated 80% of the total data.