GCGL, TWP error when using higher GCN hidden units

[WeiWeic6222848]

GCGL, TWP error when using higher GCN hidden units

[QueuQ]

When the GCN hidden unit size is larger than the input size, TWP will throw the following error:

Traceback (most recent call last):
  File "/mnt/c/Users/Wei_Wei/PycharmProjects/CGLB/GCGL/train.py", line 141, in main
    AP, AF, acc_matrix,cls_matrix = main(args, valid=True)
  File "/mnt/c/Users/Wei_Wei/PycharmProjects/CGLB/GCGL/pipeline.py", line 529, in pipeline_multi_class
    train_func(train_loader, loss_criterion, tid, args)
  File "/mnt/c/Users/Wei_Wei/PycharmProjects/CGLB/./GCGL/Baselines/twp_model.py", line 188, in observe_clsIL
    eloss.backward()
  File "/home/wwei/miniconda3/envs/GNN-DL-py38/lib/python3.8/site-packages/torch/_tensor.py", line 488, in backward
    torch.autograd.backward(
  File "/home/wwei/miniconda3/envs/GNN-DL-py38/lib/python3.8/site-packages/torch/autograd/__init__.py", line 197, in backward
    Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

because of the following code snippet:

CGLB/GCGL/Backbones/graphconv.py

Lines 155 to 188 in 793a346

	if self._in_feats > self._out_feats:
	# mult W first to reduce the feature size for aggregation.
	if weight is not None:
	feat = th.matmul(feat, weight)
	graph.srcdata['h'] = feat
	#######
	graph.ndata['feat'] = feat
	graph.apply_edges(lambda edges: {'e': th.sum((th.mul(edges.src['h'], th.tanh(edges.dst['h']))),1)})
	e = self.leaky_relu(graph.edata.pop('e'))
	e_soft = edge_softmax(graph, e)
	graph.ndata.pop('feat')
	#######
	graph.update_all(fn.copy_src(src='h', out='m'),
	fn.sum(msg='m', out='h'))
	rst = graph.dstdata['h']
	else:
	# aggregate first then mult W
	graph.srcdata['h'] = feat
	#######
	graph.ndata['feat'] = feat
	graph.apply_edges(lambda edges: {'e': th.sum((th.mul(edges.src['h'], th.tanh(edges.dst['h']))),1)})
	e = self.leaky_relu(graph.edata.pop('e'))
	e_soft = edge_softmax(graph, e)
	graph.ndata.pop('feat')
	#######
	graph.update_all(fn.copy_src(src='h', out='m'),
	fn.sum(msg='m', out='h'))
	rst = graph.dstdata['h']
	if weight is not None:
	rst = th.matmul(rst, weight)

When hidden unit size is bigger than the input size, the if statement is false, causing the else statement to be executed. However, in the else statement, for the first layer of GCN, the edge weight is computed without interacting with the trainable weight, making it unoptimizable, hence the error.

Thanks a lot for pointing out this bug. I have now invalidate the else statement temporarily to avoid this issue. I tested it and it should be able to run now.