cluster with more than one wasp-node in a chain

[iotaledger]

cluster with more than one wasp-node in a chain

[dessaya]

The nodes crash with the following stack trace:

[!wasp 1] 
[!wasp 1] goroutine 165 [running]:
[!wasp 1] github.com/anthdm/hbbft.NewRBC(0x2, 0x0, 0x1, 0xc00026dd70, 0x2, 0x2, 0x0, 0x7fefa05b2c98, 0xc0000f6780, 0x0, ...)
[!wasp 1] 	/home/dessaya/go/pkg/mod/github.com/kape1395/[email protected]/rbc.go:105 +0x37c
[!wasp 1] github.com/anthdm/hbbft.NewACS(0x2, 0x0, 0x1, 0xc00026dd70, 0x2, 0x2, 0x0, 0x7fefa05b2c98, 0xc0000f6780, 0xc0001ad140)
[!wasp 1] 	/home/dessaya/go/pkg/mod/github.com/kape1395/[email protected]/acs.go:97 +0x2b0
[!wasp 1] github.com/iotaledger/wasp/packages/chain/consensus/commonsubset.NewCommonSubset(0x89d0ad466b85f6d0, 0x56ee4b0300000000, 0x2a84a47ef161cc38, 0x9b7abfb6767e2336, 0x5af4214263cf1d07, 0x6ae95fcfbd, 0x22e1b48, 0xc0002d28c0, 0x2, 0x22d5f10, ...)
[!wasp 1] 	/home/dessaya/work/iota/wasp/evm/packages/chain/consensus/commonsubset/commonsubset.go:112 +0x297
[!wasp 1] github.com/iotaledger/wasp/packages/chain/consensus/commonsubset.(*CommonSubsetCoordinator).getOrCreateCS(0xc0000f6800, 0x89d0ad466b85f6d0, 0xc000000000, 0xc0003009e0, 0x0, 0x0, 0x0)
[!wasp 1] 	/home/dessaya/work/iota/wasp/evm/packages/chain/consensus/commonsubset/commonsubsetcoordinator.go:172 +0x496
[!wasp 1] github.com/iotaledger/wasp/packages/chain/consensus/commonsubset.(*CommonSubsetCoordinator).RunACSConsensus(0xc0000f6800, 0xc0006c2000, 0xf8, 0x500, 0x89d0ad466b85f6d0, 0xc000000000, 0xc0003009e0)
[!wasp 1] 	/home/dessaya/work/iota/wasp/evm/packages/chain/consensus/commonsubset/commonsubsetcoordinator.go:101 +0x13f
[!wasp 1] github.com/iotaledger/wasp/packages/chain/committee.(*committee).RunACSConsensus(0xc0005800b0, 0xc0006c2000, 0xf8, 0x500, 0x89d0ad466b85f6d0, 0x0, 0xc0003009e0)
[!wasp 1] 	/home/dessaya/work/iota/wasp/evm/packages/chain/committee/committee.go:239 +0x6e
[!wasp 1] github.com/iotaledger/wasp/packages/chain/consensus.(*Consensus).proposeBatchIfNeeded(0xc0002d4000)
[!wasp 1] 	/home/dessaya/work/iota/wasp/evm/packages/chain/consensus/action.go:64 +0x27e
[!wasp 1] github.com/iotaledger/wasp/packages/chain/consensus.(*Consensus).takeAction(0xc0002d4000)
[!wasp 1] 	/home/dessaya/work/iota/wasp/evm/packages/chain/consensus/action.go:33 +0xeb
[!wasp 1] github.com/iotaledger/wasp/packages/chain/consensus.(*Consensus).eventTimerMsg(0xc0002d4000, 0x0)
[!wasp 1] 	/home/dessaya/work/iota/wasp/evm/packages/chain/consensus/eventproc.go:85 +0xe8
[!wasp 1] github.com/iotaledger/wasp/packages/chain/consensus.(*Consensus).recvLoop(0xc0002d4000)
[!wasp 1] 	/home/dessaya/work/iota/wasp/evm/packages/chain/consensus/consensus.go:149 +0x2f2
[!wasp 1] created by github.com/iotaledger/wasp/packages/chain/consensus.New
[!wasp 1] 	/home/dessaya/work/iota/wasp/evm/packages/chain/consensus/consensus.go:102 +0x4f6

The following test reproduces the same error (just trying to deploy a chain with N=2, Q=2:

func TestDeployChainN2Q2(t *testing.T) {
	setup(t, "test_cluster")

	counter1, err := clu.StartMessageCounter(map[string]int{
		"dismissed_chain": 0,
		"state":           2,
		"request_out":     1,
	})
	check(err, t)
	defer counter1.Close()

	committee := clu.Config.AllNodes()[:2]
	quorum := 2
	_, err = clu.DeployChainWithDKG("my chain", committee, committee, uint16(quorum))
	check(err, t)

	if !counter1.WaitUntilExpectationsMet() {
		t.Fail()
	}
}

[kape1395]

As a quick workaround, the deploy should work with 4 nodes. Then the setup should tolerate 1 faulty node.
I will look at the way to support clusters of 1<N<4.