dthanuja/eng

A CNN based Depth, Optical Flow, Flow Uncertainty and Camera Pose Prediction pipeline

eng's Issues

The implementation of expSE3 is wrong

The results I got from this function is different from that I got in Matlab toolbox. I am investigating this

Training

Hi there,

I have stereo endoscopic videos recorded using our custom built camera.
I want to use this repository to get the depth and the pose. But I cannot find the code for the training using the new data. All I can see is about demo.

Or please let me know if I am missing training code somehow.

Regards,
Jacob

Question: Why need to normalize optical flow using focal_length_x and focal_length_y

ENG/vo_utils.py

Line 49 in 6d22690

flow_x = flow_r[:,0,:]/tf.expand_dims(k[:,0,0],-1)

Hi Thanuja,

I went through your fantastic code. I am somehow confused in this line. Could you please explain to me why flow need to normalized using focal length? Thank you

How to test the code on new test images

Hi,

I have test images whose camera pose I would like to know. How to run your network to infer that ? Currently I am facing the following error, If I just import the images and run the network to infer!
Reading csvfile: test.csv
(1, 192, 640, 128)
(1, 192, 640, 128)
2021-01-04 11:15:36.687614: I T:\src\github\tensorflow\tensorflow\core\platform\cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
Restored model parameters
Running inference : 0
Traceback (most recent call last):
File "C:\Users\Photogauge\Anaconda3\envs\eng\lib\site-packages\tensorflow\python\client\session.py", line 1322, in _do_call
return fn(*args)
File "C:\Users\Photogauge\Anaconda3\envs\eng\lib\site-packages\tensorflow\python\client\session.py", line 1307, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "C:\Users\Photogauge\Anaconda3\envs\eng\lib\site-packages\tensorflow\python\client\session.py", line 1409, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.OutOfRangeError: FIFOQueue '_1_batch/fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
[[Node: batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT, DT_INT32], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](batch/fifo_queue, batch/n)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\Photogauge\Anaconda3\envs\eng\lib\runpy.py", line 184, in _run_module_as_main
"main", mod_spec)
File "C:\Users\Photogauge\Anaconda3\envs\eng\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "D:\Work\frame_measurement\architectures\ENG\ENG\test_depth_flow_pose.py", line 230, in
infer_depth_flow_pose(args,0,1,0)
File "D:\Work\frame_measurement\architectures\ENG\ENG\test_depth_flow_pose.py", line 209, in infer_depth_flow_pose
(rgb_left, rgb_right) = sess.run([rgb_left_tensor, rgb_right_tensor])
File "C:\Users\Photogauge\Anaconda3\envs\eng\lib\site-packages\tensorflow\python\client\session.py", line 900, in run
run_metadata_ptr)
File "C:\Users\Photogauge\Anaconda3\envs\eng\lib\site-packages\tensorflow\python\client\session.py", line 1135, in _run
feed_dict_tensor, options, run_metadata)
File "C:\Users\Photogauge\Anaconda3\envs\eng\lib\site-packages\tensorflow\python\client\session.py", line 1316, in _do_run
run_metadata)
File "C:\Users\Photogauge\Anaconda3\envs\eng\lib\site-packages\tensorflow\python\client\session.py", line 1335, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.OutOfRangeError: FIFOQueue '_1_batch/fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
[[Node: batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT, DT_INT32], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](batch/fifo_queue, batch/n)]]

Caused by op 'batch', defined at:
File "C:\Users\Photogauge\Anaconda3\envs\eng\lib\runpy.py", line 184, in _run_module_as_main
"main", mod_spec)
File "C:\Users\Photogauge\Anaconda3\envs\eng\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "D:\Work\frame_measurement\architectures\ENG\ENG\test_depth_flow_pose.py", line 230, in
infer_depth_flow_pose(args,0,1,0)
File "D:\Work\frame_measurement\architectures\ENG\ENG\test_depth_flow_pose.py", line 183, in infer_depth_flow_pose
images,cams_batch,shapes_batch = dataset2.csv_inputs(args.test_csv)
File "D:\Work\frame_measurement\architectures\ENG\ENG\TestDataset.py", line 71, in csv_inputs
capacity= 3
File "C:\Users\Photogauge\Anaconda3\envs\eng\lib\site-packages\tensorflow\python\training\input.py", line 988, in batch
name=name)
File "C:\Users\Photogauge\Anaconda3\envs\eng\lib\site-packages\tensorflow\python\training\input.py", line 762, in _batch
dequeued = queue.dequeue_many(batch_size, name=name)
File "C:\Users\Photogauge\Anaconda3\envs\eng\lib\site-packages\tensorflow\python\ops\data_flow_ops.py", line 483, in dequeue_many
self._queue_ref, n=n, component_types=self._dtypes, name=name)
File "C:\Users\Photogauge\Anaconda3\envs\eng\lib\site-packages\tensorflow\python\ops\gen_data_flow_ops.py", line 3799, in queue_dequeue_many_v2
component_types=component_types, timeout_ms=timeout_ms, name=name)
File "C:\Users\Photogauge\Anaconda3\envs\eng\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "C:\Users\Photogauge\Anaconda3\envs\eng\lib\site-packages\tensorflow\python\framework\ops.py", line 3414, in create_op
op_def=op_def)
File "C:\Users\Photogauge\Anaconda3\envs\eng\lib\site-packages\tensorflow\python\framework\ops.py", line 1740, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

OutOfRangeError (see above for traceback): FIFOQueue '_1_batch/fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
[[Node: batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT, DT_INT32], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](batch/fifo_queue, batch/n)]]