nglehuy/sasegan

Self-Attention Generative Adversarial Network for Speech Enhancement using Tensorflow 2

sasegan's Issues

ValueError:Attempting to convert None to Tensor while executing "self._check_save_interval()"

Expected Behavior

I'm using my own training data to train a model, and it should begin to train.

Current Behavior

While executing self._check_save_interval() in _train_epoch(), it compares self.step with self.total_train_steps, which seems to be assigned to None, causing Tensorflow raises ValueError.

Possible Solution

According to the base ASR module, there's one line in asrdataset.py that calculates total_steps, but not exists in SeganDataset.py, I guess it needs some fix but I don't know how to approach.

Steps to Reproduce

1. I only changed data paths in config.yml
2. python train_sasegan.py

Context (Environment)

Tensorflow-gpu 2.30
Ubuntu 16.04
dependencies installed by setup.py

Detailed Description

   File "train_sasegan.py", line 90, in <module>
     segan_trainer.fit(train_dataset=dataset, train_bs=args.tbs)
   File "/share/homes/samtsao/miniconda3/lib/python3.8/site-packages/TensorFlowASR-0.3.1-py3.8.egg/tensorflow_asr/runners/base_runners.py", line 312, in fit
     self.run()
   File "/share/homes/samtsao/miniconda3/lib/python3.8/site-packages/sasegan-0.1.2-py3.8.egg/sasegan/runners/trainer.py", line 63, in run
   File "/share/homes/samtsao/miniconda3/lib/python3.8/site-packages/TensorFlowASR-0.3.1-py3.8.egg/tensorflow_asr/runners/base_runners.py", line 221, in _train_epoch
     self._check_save_interval()
   File "/share/homes/samtsao/miniconda3/lib/python3.8/site-packages/TensorFlowASR-0.3.1-py3.8.egg/tensorflow_asr/runners/base_runners.py", line 328, in _check_save_interval
     self.steps >= self.total_train_steps:
   File "/share/homes/samtsao/miniconda3/lib/python3.8/site-packages/tensorflow/python/ops/variables.py", line 1074, in _run_op
     return tensor_oper(a.value(), *args, **kwargs)
   File "/share/homes/samtsao/miniconda3/lib/python3.8/site-packages/tensorflow/python/ops/gen_math_ops.py", line 4062, in greater_equal
     return greater_equal_eager_fallback(
   File "/share/homes/samtsao/miniconda3/lib/python3.8/site-packages/tensorflow/python/ops/gen_math_ops.py", line 4097, in greater_equal_eager_fallback
     _attr_T, _inputs_T = _execute.args_to_matching_eager([x, y], ctx)
   File "/share/homes/samtsao/miniconda3/lib/python3.8/site-packages/tensorflow/python/eager/execute.py", line 267, in args_to_matching_eager
     ret = [ops.convert_to_tensor(t, dtype, ctx=ctx) for t in l]
   File "/share/homes/samtsao/miniconda3/lib/python3.8/site-packages/tensorflow/python/eager/execute.py", line 267, in <listcomp>
     ret = [ops.convert_to_tensor(t, dtype, ctx=ctx) for t in l]
   File "/share/homes/samtsao/miniconda3/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 1499, in convert_to_tensor
     ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
   File "/share/homes/samtsao/miniconda3/lib/python3.8/site-packages/tensorflow/python/framework/constant_op.py", line 338, in _constant_tensor_conversion_function
     return constant(v, dtype=dtype, name=name)
   File "/share/homes/samtsao/miniconda3/lib/python3.8/site-packages/tensorflow/python/framework/constant_op.py", line 263, in constant
     return _constant_impl(value, dtype, shape, name, verify_shape=False,
   File "/share/homes/samtsao/miniconda3/lib/python3.8/site-packages/tensorflow/python/framework/constant_op.py", line 275, in _constant_impl
     return _constant_eager_impl(ctx, value, dtype, shape, verify_shape)
   File "/share/homes/samtsao/miniconda3/lib/python3.8/site-packages/tensorflow/python/framework/constant_op.py", line 300, in _constant_eager_impl
     t = convert_to_eager_tensor(value, ctx, dtype)
   File "/share/homes/samtsao/miniconda3/lib/python3.8/site-packages/tensorflow/python/framework/constant_op.py", line 98, in convert_to_eager_tensor
     return ops.EagerTensor(value, ctx.device_name, dtype)
 ValueError: Attempt to convert a value (None) with an unsupported type (<class 'NoneType'>) to a Tensor.
 2020-12-04 16:24:42.566338: W tensorflow/core/kernels/data/generator_dataset_op.cc:103] Error occurred when finalizing GeneratorDataset iterator: Failed precondition: Python interpreter state is not initialized. The process may be terminated.
 	 [[{{node PyFunc}}]]

Full log for reference sasegan_TF2.log
Any advice will be appreciated.

BaseTrainer and BaseTester module not available in tensorflow_asr

Hey, thanks for sharing your code. I am not able to find tensorflow_asr.runners.base_runner function in (https://github.com/TensorSpeech/TensorFlowASR), So, how to get BaseTrainer and BaseTester module for further training the model? It would be great if you can share the function or if you have implemented your own function in tensorflow_asr, then please share it on Github. Thanks!