chenyanglei / automatic-video-colorization Goto Github PK
View Code? Open in Web Editor NEW[CVPR2019] Fully Automatic Video Colorization with Self-Regularization and Diversity
License: MIT License
[CVPR2019] Fully Automatic Video Colorization with Self-Regularization and Diversity
License: MIT License
Hello, sorry to bother you! I've read your paper and source code recently. You mentioned that you train the colorization model firstly, and then the colorization model and refinement model are trained jointly. But the main_whole.py file may only include the joint training code?
If I want to reproduce the result, do I need to add a new file to train the colorization network firstly?
And in the main_whole.py file, the pre-trained model of colorization network may not be loaded. After the colorization network is trained, should I load the pre-trained model of colorization network when training the whole network?
Looking forward to your apply! Thank you!
Hi,
When I looking at the code, I notice that the bilateral loss seem to be annotated. Does it take part in the training?
Thanks!
Download.py is not downloaded
Even the link provided by you i.e.ckpt is also not working please help
Please tell the steps to download.
Hi!
I wanna use our code for my course project but it is not clear how data should be arrange. Is directory JPEGImages supposed to have gray frames? If so than where should i put colored (true) frames? Also it is not clear how to "generate the optical flow by PWC-Net".
Could you please clarify that?
I tried to run your model on video followed your instruction. However, it shows the error:
OSError: cannot identify image file 'test_video/girl_video.mp4'
I have tried with other videos, but still not run. Do I need to convert it to a specific video format?
Thank you so much.
Thank you for sharing your good code:)
I have a question about function named "Bilateral_NN".
def Bilateral_NN(color_image, neigh):
h,w = color_image.shape[:2]
color_image_X = np.tile(np.arange(w),[h,1])/500.
color_image_Y = np.tile(np.arange(h),[w,1]).T/500.
color_image_all = np.concatenate([color_image,color_image_X[:,:,np.newaxis], color_image_Y[:,:,np.newaxis]],axis=2)
neigh.fit(np.reshape(color_image_all, [-1, 5]))
idxs = neigh.kneighbors(np.reshape(color_image_all, [-1, 5]), 5, return_distance=False)
return idxs
Thank you
Hello, your work is so interesting. I wanna know, how can i reach the eval work as you do. whats the data for eval? And how to do for eval video data?
It seems like flow_warp.so was not compiled with tensorflow_framework.so dynamic lib. Is there any solution to this error?
Traceback (most recent call last):
File "main_whole.py", line 11, in
import flow_warp as flow_warp_op
File "/home/qianming/code/Fully-Automatic-Video-Colorization/flow_warp.py", line 4, in
tf.resource_loader.get_path_to_datafile("./lib/flow_warp.so"))
File "/home/qianming/anaconda3/envs/torch/lib/python3.6/site-packages/tensorflow/python/framework/load_library.py", line 61, in load_op_library
lib_handle = py_tf.TF_LoadLibrary(library_filename)
tensorflow.python.framework.errors_impl.NotFoundError: libcudart.so.8.0: cannot open shared object file: No such file or directory
Trying to evaluate your code.
Unfortunately the only result I've got is:
result = getattr(asarray(obj), method)(*args, **kwds)
ValueError: attempt to get argmax of an empty sequence
I tried:
python test.py --img_path demo_imgs/ILSVRC2012_val_00040251.JPEG
python test.py --img_path demo_imgs/
Result the same.
It only creates "teat_result" folder with empty " model_imagecolor" subfolder.
Greatly appreciate for your assistance.
hello, i can't connect https://docs.google.com/uc?export=download. Is it bad requests? error 400. Could you solve it? Thank you
Hi, Mr.Chenyang and Mr.Qifeng.
I have a question about in your paper "Fully-Automatic-Video-Colorization-with-Self-Regularization-and-Diversity".
In your paper, you used φ in your proposed loss function for diversity.
I wonder what this φ represents.
I checked your cited paper "Interactive_Image_Segmentation_with_Latent_Diversity", but I can't find φ.
Could you tell me, what do φ represents?
Thank you.
Hello, your work is so interesting. I wanna know, if I wanna use your mehtod to compare with my method. But my training dataset and test data are different from yours, should I use my dataset to train your model again?
When I run this code;
!python main_woflow.py --model ckpt_woflow --use_gpu 1 --test_img /content/frames
I'm getting this error;
File "main_woflow.py", line 129, in
im=np.float32(scipy.misc.imread(test_img, 'L'))/255.0
AttributeError: module 'scipy.misc' has no attribute 'imread'
Hello,thanks for your project.
I meet an error in download_models.py,line34
download_file_from_google_drive('0B_B_FOgPxgFLRjdEdE9NNTlzUWc', 'VGG_Model/imagenet-vgg-verydeep-19.mat')
FileNotFoundError: [Errno 2] No such file or directory: 'VGG_Model/imagenet-vgg-verydeep-19.mat'
I changed VGG_Model/imagenet-vgg-verydeep-19.mat
with imagenet-vgg-verydeep-19.mat
and download sucessful, then move it in VGG_Model. So I think if you can justify your code in download_models.py.
Hello, I've tried to train this whole model, but I found that the training speed is very slow because the KNN calculating. Is there any other choice? Thank you!
Is it possible to have a script for the refinement network so that I can run the colourisation output through this multiple times as mentioned in your paper to gain better temporal coherence?
When I run the code for that example data you provided in test_sample0 I can't
Hi i want to colorize my black and white video i downloaded the model and ckpt_woflow i want to ask how to use this thing. Thanks
Namespace(model='ckpt_woflow', test_dir='test_sample0', test_img='', use_gpu=1)
'grep' is not recognized as an internal or external command,
operable program or batch file.
Traceback (most recent call last):
File "main_woflow.py", line 30, in
os.environ["CUDA_VISIBLE_DEVICES"]=str(np.argmax( [int(x.split()[2]) for x in subprocess.Popen("nvidia-smi -q -d Memory | grep -A4 GPU | grep Free", shell=True, stdout=subprocess.PIPE).stdout.readlines()]))
File "<array_function internals>", line 6, in argmax
File "E:\mi\lib\site-packages\numpy\core\fromnumeric.py", line 1153, in argmax
return _wrapfunc(a, 'argmax', axis=axis, out=out)
File "E:\mi\lib\site-packages\numpy\core\fromnumeric.py", line 58, in _wrapfunc
return _wrapit(obj, method, *args, **kwds)
File "E:\mi\lib\site-packages\numpy\core\fromnumeric.py", line 47, in _wrapit
result = getattr(asarray(obj), method)(*args, **kwds)
ValueError: attempt to get argmax of an empty sequence
(base) E:\animegan\video attack\Fully-Automatic-Video-Colorization-with-Self-Regularization-and-Diversity-master>python main_woflow.py --model ckpt_woflow --use_gpu 1 --test_dir test_sample0
how to fix it
Thanks for your nice work!
Could you please provide the pretrained model of 'hyper_unet' ?
In implementation of this code:
vgg_rawnet=scipy.io.loadmat('VGG_Model/imagenet-vgg-verydeep-19.mat')
I am getting the following error:
File "/home/akshat_suwalka/Fully-Automatic-Video-Colorization-with-Self-Regularization-and-Diversity-master/utils.py", line 273, in
vgg_rawnet=scipy.io.loadmat('VGG_Model/imagenet-vgg-verydeep-19.mat')
File "/home/akshat_suwalka/anaconda3/lib/python3.7/site-packages/scipy/io/matlab/mio.py", line 218, in loadmat
matfile_dict = MR.get_variables(variable_names)
File "/home/akshat_suwalka/anaconda3/lib/python3.7/site-packages/scipy/io/matlab/mio5.py", line 292, in get_variables
res = self.read_var_array(hdr, process)
File "/home/akshat_suwalka/anaconda3/lib/python3.7/site-packages/scipy/io/matlab/mio5.py", line 252, in read_var_array
return self._matrix_reader.array_from_header(header, process)
File "mio5_utils.pyx", line 673, in scipy.io.matlab.mio5_utils.VarReader5.array_from_header
File "mio5_utils.pyx", line 719, in scipy.io.matlab.mio5_utils.VarReader5.array_from_header
File "mio5_utils.pyx", line 893, in scipy.io.matlab.mio5_utils.VarReader5.read_cells
File "mio5_utils.pyx", line 671, in scipy.io.matlab.mio5_utils.VarReader5.read_mi_matrix
File "mio5_utils.pyx", line 721, in scipy.io.matlab.mio5_utils.VarReader5.array_from_header
File "mio5_utils.pyx", line 968, in scipy.io.matlab.mio5_utils.VarReader5.read_struct
File "mio5_utils.pyx", line 671, in scipy.io.matlab.mio5_utils.VarReader5.read_mi_matrix
File "mio5_utils.pyx", line 719, in scipy.io.matlab.mio5_utils.VarReader5.array_from_header
File "mio5_utils.pyx", line 893, in scipy.io.matlab.mio5_utils.VarReader5.read_cells
File "mio5_utils.pyx", line 671, in scipy.io.matlab.mio5_utils.VarReader5.read_mi_matrix
File "mio5_utils.pyx", line 703, in scipy.io.matlab.mio5_utils.VarReader5.array_from_header
File "mio5_utils.pyx", line 777, in scipy.io.matlab.mio5_utils.VarReader5.read_real_complex
File "mio5_utils.pyx", line 450, in scipy.io.matlab.mio5_utils.VarReader5.read_numeric
File "mio5_utils.pyx", line 355, in scipy.io.matlab.mio5_utils.VarReader5.read_element
File "streams.pyx", line 200, in scipy.io.matlab.streams.ZlibInputStream.read_string
File "streams.pyx", line 193, in scipy.io.matlab.streams.ZlibInputStream.read_into
OSError: could not read bytes
Please help me in it.
And if you have implemented in tensorflow 2.0 , then can you please share it??
Successfully run single image colorization:
python test.py --img_path demo_imgs/ILSVRC2012_val_00040251.JPEG
however attempt to colorize the images in a folder by:
python test.py --img_path demo_imgs
throws an error:
IsADirectoryError: [Errno 21] Is a directory: 'demo_imgs'
The same error happened when running:
python test_div_video.py --use_gpu 1 --video_path demo_vid
Hello, Sorry to bother you, but I really can't download the weight files "VGG_Model/ imagenet-vGG -verydeep-19.mat" and "CKpt_woflow. zip". Can you send it to my mailbox if convenient? My email address is [email protected]
thank you very much.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.