Why does decision node holds 16 routing probabilities when there are 15 split nodes in a tree which depth is 4?

[chrischoy]

Why does decision node holds 16 routing probabilities when there are 15 split nodes in a tree which depth is 4?

[Mudwall]

" w_d_ensemble.append(init_prob_weights([625, N_LEAF], -1, 1))" in your code, why not " w_d_ensemble.append(init_prob_weights([625, N_LEAF-1], -1, 1))" for 15 decision nodes?

[zroach]

" w_d_ensemble.append(init_prob_weights([625, N_LEAF], -1, 1))" in your code, why not " w_d_ensemble.append(init_prob_weights([625, N_LEAF-1], -1, 1))" for 15 decision nodes?

Making this change gave me the following traceback (Mac 10.14.1, python 3.6.2):

Roachs-MacBook-Air:DNDF roach$ python3 model.py

.....

2018-11-21 21:22:56.537935: W tensorflow/core/framework/op_kernel.cc:1290] CtxFailure at gather_op.cc:117: Invalid argument: indices[112,1] = 3847 is not in [0, 3840)
2018-11-21 21:22:56.537931: W tensorflow/core/framework/op_kernel.cc:1290] CtxFailure at gather_op.cc:117: Invalid argument: indices[112,1] = 3847 is not in [0, 3840)
2018-11-21 21:22:56.537927: W tensorflow/core/framework/op_kernel.cc:1290] CtxFailure at gather_op.cc:117: Invalid argument: indices[112,2] = 3843 is not in [0, 3840)
2018-11-21 21:22:56.537935: W tensorflow/core/framework/op_kernel.cc:1290] CtxFailure at gather_op.cc:117: Invalid argument: indices[112,1] = 3847 is not in [0, 3840)
2018-11-21 21:22:56.538731: W tensorflow/core/framework/op_kernel.cc:1290] CtxFailure at gather_op.cc:117: Invalid argument: indices[112,4] = 3841 is not in [0, 3840)
2018-11-21 21:22:56.538734: W tensorflow/core/framework/op_kernel.cc:1290] CtxFailure at gather_op.cc:117: Invalid argument: indices[112,2] = 3843 is not in [0, 3840)
2018-11-21 21:22:56.538823: W tensorflow/core/framework/op_kernel.cc:1290] CtxFailure at gather_op.cc:117: Invalid argument: indices[112,2] = 3843 is not in [0, 3840)
2018-11-21 21:22:56.538879: W tensorflow/core/framework/op_kernel.cc:1290] CtxFailure at gather_op.cc:117: Invalid argument: indices[112,8] = 3840 is not in [0, 3840)
2018-11-21 21:22:56.538878: W tensorflow/core/framework/op_kernel.cc:1290] CtxFailure at gather_op.cc:117: Invalid argument: indices[112,4] = 3841 is not in [0, 3840)
2018-11-21 21:22:56.538934: W tensorflow/core/framework/op_kernel.cc:1290] CtxFailure at gather_op.cc:117: Invalid argument: indices[112,4] = 3841 is not in [0, 3840)
2018-11-21 21:22:56.538999: W tensorflow/core/framework/op_kernel.cc:1290] CtxFailure at gather_op.cc:117: Invalid argument: indices[112,1] = 3847 is not in [0, 3840)
2018-11-21 21:22:56.539004: W tensorflow/core/framework/op_kernel.cc:1290] CtxFailure at gather_op.cc:117: Invalid argument: indices[112,8] = 3840 is not in [0, 3840)
2018-11-21 21:22:56.539088: W tensorflow/core/framework/op_kernel.cc:1290] CtxFailure at gather_op.cc:117: Invalid argument: indices[112,8] = 3840 is not in [0, 3840)
2018-11-21 21:22:56.539176: W tensorflow/core/framework/op_kernel.cc:1290] CtxFailure at gather_op.cc:117: Invalid argument: indices[112,8] = 3840 is not in [0, 3840)
2018-11-21 21:22:56.539565: W tensorflow/core/framework/op_kernel.cc:1290] CtxFailure at gather_op.cc:117: Invalid argument: indices[112,2] = 3843 is not in [0, 3840)
2018-11-21 21:22:56.539185: W tensorflow/core/framework/op_kernel.cc:1290] CtxFailure at gather_op.cc:117: Invalid argument: indices[112,4] = 3841 is not in [0, 3840)
2018-11-21 21:22:56.544609: W tensorflow/core/framework/op_kernel.cc:1290] CtxFailure at gather_op.cc:117: Invalid argument: indices[112,1] = 3847 is not in [0, 3840)
2018-11-21 21:22:56.544609: W tensorflow/core/framework/op_kernel.cc:1290] CtxFailure at gather_op.cc:117: Invalid argument: indices[112,8] = 3840 is not in [0, 3840)
2018-11-21 21:22:56.544623: W tensorflow/core/framework/op_kernel.cc:1290] CtxFailure at gather_op.cc:117: Invalid argument: indices[112,4] = 3841 is not in [0, 3840)
2018-11-21 21:22:56.544623: W tensorflow/core/framework/op_kernel.cc:1290] CtxFailure at gather_op.cc:117: Invalid argument: indices[112,2] = 3843 is not in [0, 3840)
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1322, in _do_call
    return fn(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1307, in _run_fn
    options, feed_dict, fetch_list, target_list, run_metadata)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1409, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[112,1] = 3847 is not in [0, 3840)
	 [[Node: GatherV2_18 = GatherV2[Taxis=DT_INT32, Tindices=DT_INT32, Tparams=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](Reshape_4, Add_18, gradients/concat_6/axis)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "model.py", line 224, in <module>
    p_keep_conv: 0.8, p_keep_hidden: 0.5})
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 900, in run
    run_metadata_ptr)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1135, in _run
    feed_dict_tensor, options, run_metadata)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1316, in _do_run
    run_metadata)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1335, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[112,1] = 3847 is not in [0, 3840)
	 [[Node: GatherV2_18 = GatherV2[Taxis=DT_INT32, Tindices=DT_INT32, Tparams=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](Reshape_4, Add_18, gradients/concat_6/axis)]]

Caused by op 'GatherV2_18', defined at:
  File "model.py", line 187, in <module>
    tf.add(batch_indices, batch_complement_indices)))
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 2666, in gather
    return gen_array_ops.gather_v2(params, indices, axis, name=name)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 3141, in gather_v2
    "GatherV2", params=params, indices=indices, axis=axis, name=name)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3417, in create_op
    op_def=op_def)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1743, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): indices[112,1] = 3847 is not in [0, 3840)
	 [[Node: GatherV2_18 = GatherV2[Taxis=DT_INT32, Tindices=DT_INT32, Tparams=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](Reshape_4, Add_18, gradients/concat_6/axis)]]

So perhaps this parameter doesn't exactly represent the number of decision nodes? ... because your math is not wrong