foamliu / deep-image-matting-pytorch Goto Github PK
View Code? Open in Web Editor NEWDeep Image Matting implementation in PyTorch
License: MIT License
Deep Image Matting implementation in PyTorch
License: MIT License
Can you achieve the performance that the paper claims in your experiment?
Thanks for your great work!
I saw your result is much better than the paper post. Is the fc6 you dropped matters?
I try to run eval.py on my laptop that doesn't have cuda and I got the following error:
Traceback (most recent call last):
File "eval.py", line 62, in <module>
pred = model(x)
File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 547, in __call__
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/torch/nn/parallel/data_parallel.py", line 146, in forward
"them on device: {}".format(self.src_device_obj, t.device))
RuntimeError: module must have its parameters and buffers on device cuda:0 (device_ids[0]) but found one of them on device: cpu
I also modified the load state code to
checkpoint = torch.load(checkpoint, map_location=lambda storage, loc: storage)
but it doesn't fix all issues.
I have the dataset in the data folder unextracted with the same name. Though it displays file not found error while I try to run pre_process.py
fg_path_test = 'data/fg_test/'
a_path_test = 'data/mask_test/'
bg_path_test = 'data/bg_test/'
There is two stage in the paper, but i find the DIMModel only implement stage one. And the loss only include alpha_prediction loss, there is no composition loss. Is that true?
Thank you very much for making this project open source. I have a question and I hope to get your answer.
In the demo.py
We should provide two images:
Does 1_alpha.png must be obtained manually by PS?
Is there any way to get it automatically?
So we only need to upload a picture of 1.png.
When resume training using checkpoints,get error above raised by line 38 from train.py:
'model = checkpoint['model'].module'
,any suggestions?@foamliu
can you give me dataset for Adobe Deep Image Matting Dataset? [email protected]! thanks
/Users/7plus/opt/anaconda3/lib/python3.7/site-packages/torch/serialization.py:435: SourceChangeWarning: source code of class 'torch.nn.parallel.data_parallel.DataParallel' has changed. you can retrieve the original source code by accessing the object's source attribute or set torch.nn.Module.dump_patches = True
and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
/Users/7plus/opt/anaconda3/lib/python3.7/site-packages/torch/serialization.py:435: SourceChangeWarning: source code of class 'torch.nn.modules.conv.Conv2d' has changed. you can retrieve the original source code by accessing the object's source attribute or set torch.nn.Module.dump_patches = True
and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
Traceback (most recent call last):
File "demo.py", line 69, in
checkpoint = torch.load(checkpoint)
File "/Users/7plus/opt/anaconda3/lib/python3.7/site-packages/torch/serialization.py", line 368, in load
return _load(f, map_location, pickle_module)
File "/Users/7plus/opt/anaconda3/lib/python3.7/site-packages/torch/serialization.py", line 542, in _load
result = unpickler.load()
File "/Users/7plus/opt/anaconda3/lib/python3.7/site-packages/torch/serialization.py", line 505, in persistent_load
data_type(size), location)
File "/Users/7plus/opt/anaconda3/lib/python3.7/site-packages/torch/serialization.py", line 114, in default_restore_location
result = fn(storage, location)
File "/Users/7plus/opt/anaconda3/lib/python3.7/site-packages/torch/serialization.py", line 95, in _cuda_deserialize
device = validate_cuda_device(location)
File "/Users/7plus/opt/anaconda3/lib/python3.7/site-packages/torch/serialization.py", line 79, in validate_cuda_device
raise RuntimeError('Attempting to deserialize object on a CUDA '
RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location='cpu' to map your storages to the CPU.
Hi,
Thank you for sharing your PyTorch version of reimplementation. Would you like to share the PyTorch version you used to development?
I am using PyTorch 1.0.1, CUDA 9, two RTX 2080 Ti to run the 'train.py' since I see you use Data Parallel module to support multi-GPUs training. However, I encountered and the trackbacks are here:
Traceback (most recent call last):
File "train.py", line 171, in
main()
File "train.py", line 167, in main
train_net(args)
File "train.py", line 64, in train_net
logger=logger)
File "train.py", line 103, in train
alpha_out = model(img) # [N, 3, 320, 320]
File "/home/mingfu/anaconda3/envs/tensorflow_gpu/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/mingfu/anaconda3/envs/tensorflow_gpu/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 143, in forward
outputs = self.parallel_apply(replicas, inputs, kwargs)
File "/home/mingfu/anaconda3/envs/tensorflow_gpu/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 153, in parallel_apply
return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])
File "/home/mingfu/anaconda3/envs/tensorflow_gpu/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 83, in parallel_apply
raise output
File "/home/mingfu/anaconda3/envs/tensorflow_gpu/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 59, in _worker
output = module(*input, **kwargs)
File "/home/mingfu/anaconda3/envs/tensorflow_gpu/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/mingfu/Deep-Image-Matting-v2/models.py", line 127, in forward
up4 = self.up4(up5, indices_4, unpool_shape4)
File "/home/mingfu/anaconda3/envs/tensorflow_gpu/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/mingfu/Deep-Image-Matting-v2/models.py", line 87, in forward
outputs = self.conv(outputs)
File "/home/mingfu/anaconda3/envs/tensorflow_gpu/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/mingfu/Deep-Image-Matting-v2/models.py", line 43, in forward
outputs = self.cbr_unit(inputs)
File "/home/mingfu/anaconda3/envs/tensorflow_gpu/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/mingfu/anaconda3/envs/tensorflow_gpu/lib/python3.6/site-packages/torch/nn/modules/container.py", line 92, in forward
input = module(input)
File "/home/mingfu/anaconda3/envs/tensorflow_gpu/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/mingfu/anaconda3/envs/tensorflow_gpu/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 320, in forward
self.padding, self.dilation, self.groups)
RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED
I have tested the DATA PARALLELISM using the example here and it works well.
Did you observe any degradation in performance by just using alpha_loss, not total loss or compositional loss?
您好,刚发现预训练模型无法解压缩,请问这个如何解压呢?谢谢
When i run train.py, I get this error "RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED", What made this error? What should I do? The configuration environment of my computer is ‘python3.5 pytorch1.1.0 cudn9.0 cudnn7.1.3 torchvision0.3.0’
7plus@7pluss-MacBook-Pro ~ % python pre_process.py
python: can't open file 'pre_process.py': [Errno 2] No such file or directory
So how to deal with it?
Hi,
Will the pretrained model of this "Deep-Image-Matting-PyTorch" project work on the "Mobile-Image-Matting" library too?
In the Mobile-Image-Matting library,
https://github.com/foamliu/Deep-Mobile-Matting/releases/download/v1.0/BEST_checkpoint.tar
https://github.com/foamliu/Deep-Image-Matting-v2/releases/download/v1.0/BEST_checkpoint.tar
But the above points to the large checkpoint file from the main "Deep-Image-Matting-v2" project.
Trying to evaluate on this second link give following error
AttributeError: Can't get attribute 'DIMModel' on <module 'models' from '/content/gdrive/My Drive/MobileImageMatting/Mobile-Image-Matting/models/__init__.py'>
Any help?
when I try to uncompress the frozen model
it shows
tar: This does not look like a tar archive
tar: Skipping to next header
tar: Exiting with failure status due to previous errors
this means the .tar file is not complete
hello,
using your checkpoint for other objects like shoes 👠 , etc it doesnt work and the objects are fully transparent.
i want to train my custom objects from coco dataset or my own set to generate the .tar file.
how can i archive this ?
Is the evaluation progress like
input 1920 * 1080 alpha 1920 * 1080 trimap 1920 *1080
then resize input and trimap to 320 * 320
input 320 * 320 trimap 320 * 320 to the model, then get the prediction 320 * 320
prediction 320 * 320, resize to 1920 * 1080
pred 1920 * 1080 alpha 1920 * 1080
finally compare the resized prediction and the alpha(gt)
i test on the DIM-tensorflow version and cannot get the number in paper, because the sad,mse,conn and grad are effected by the image size
Thanks
whenever pre-trained weights are used for training the model using own dataset, the following error is occurring.
python3 train.py --batch-size 4 --checkpoint checkpoint/BEST_checkpoint.tar
/usr/local/lib/python3.5/dist-packages/torch/serialization.py:454: SourceChangeWarning: source code of class 'torch.nn.parallel.data_parallel.DataParallel' has changed. you can retrieve the original source code by accessing the object's source attribute or set torch.nn.Module.dump_patches = True
and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.5/dist-packages/torch/serialization.py:454: SourceChangeWarning: source code of class 'torch.nn.modules.conv.Conv2d' has changed. you can retrieve the original source code by accessing the object's source attribute or set torch.nn.Module.dump_patches = True
and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.5/dist-packages/torch/serialization.py:454: SourceChangeWarning: source code of class 'torch.nn.modules.batchnorm.BatchNorm2d' has changed. you can retrieve the original source code by accessing the object's source attribute or set torch.nn.Module.dump_patches = True
and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.5/dist-packages/torch/serialization.py:454: SourceChangeWarning: source code of class 'torch.nn.modules.activation.ReLU' has changed. you can retrieve the original source code by accessing the object's source attribute or set torch.nn.Module.dump_patches = True
and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
Traceback (most recent call last):
File "train.py", line 180, in
main()
File "train.py", line 176, in main
train_net(args)
File "train.py", line 71, in train_net
logger=logger)
File "train.py", line 112, in train
alpha_out = model(img) # [N, 3, 320, 320]
File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/parallel/data_parallel.py", line 143, in forward
if t.device != self.src_device_obj:
File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 539, in getattr
type(self).name, name))
AttributeError: 'DataParallel' object has no attribute 'src_device_obj'
I am looking for a deep learning library/sdk which can be used to remove the background from any image automatically (with quality as good as www.remove.bg).
I tried some image segmentation SDKs with pre-trained models such as Tensorflow Lite & Fritz AI, but the accuracy of the cutout mask was very low, amongst other issues.
Criteria :-
If the foreground consists of person holding a balloon, sittting on a chair, with a pet on his side, then I want all of this to get extracted. Not just the human cutout. The segmentation SDKs I tried are only extracting humans (the chair gets vanished), that too with a very low quality mask (hair gets cut, parts of ear gets cut, etc).
I want even the finer details like the hair, delicate clothes, etc to be extracted perfectly.
I want to use this technology on mobile phones (in an Android app) which should ideally work even in an offline environment. If this option is difficult to achieve, then plan B would be install the technoloy on our server.
I have been reading a lot and I am currently lost in the sea of various technologies out there (OpenCV, Deep Matting, Mask RCNN, Instance Segmentation, Detectron2, Tensorflow, Pytorch, etc). I wonder what magic is happening behind the curtains of www.remove.bg
Would this library help me to achieve what I am looking for? Any help you could provide would be awesome.
Thanks a ton!
After i run pre_process.py successfully, i run train.py but failed. The error shows me that 'FileNotFoundError: [Errno 2] No such file or directory: 'train_names.txt''. i have no this file. What's worse, i do not know anything about this file 'train_names.txt'. What should i do?
File "demo.py", line 84, in
new_bgs = random.sample(new_bgs, 10)
File "C:\Users\15432\AppData\Local\conda\conda\envs\python34\lib\random.py", line 324, in sample
raise ValueError("Sample larger than population")
ValueError: Sample larger than population
Can I do inference with original image size instead of 320x320?
可以共享一下deep image matting数据集的百度网盘链接吗?
Can you give me a pretrained model that I can use?
I noticed 3 insteresting changes in this version compared to v1:
I saw you used dilation/erosion to generate trimaps during testing. However, Adobe provided trimaps which should be used for testing for fair comparison. Correct me if I am wrong. thanks.
Now, I got a model, I want to use it but I can't, because I have not the Trimaps of my pictures. Are there the script of code to build the Trimaps? How can i get the Trimaps of my pictures?
The training code occupies three blocks of GPU memory, so that my classmates can't run. I would like to ask how many repochs are needed for training?
When I used on some cars it didn't give good results.
i've downloaded the model but both 7z and tar reports unknown format.
any suggestions what to check?
i'm on windows, if it matters
您好好@foamliu:
MATTING效果非常棒!
我的任务是吧前景提取出来,
我想问下,如何通过您的这个MATTING库提取出前景图像呢?我想应该比较简单,但我对matting不是很了解所以不太会
能否做一个简单的说明,非常感谢!
Hi, thank you for the code.
I tried to download the pretrained model and extract it but it dosnt work.
tar xvf BEST_checkpoint.tar BEST_checkpoint
results in
tar: Ceci ne ressemble pas à une archive de type « tar »
tar: On saute à l'en-tête suivant
tar: BEST_checkpoint : non trouvé dans l'archive
tar: Arrêt avec code d'échec à cause des erreurs précédentes
anything i'm doing the wrong way ? or the provided tar is not valid ?
kind reards
The pretrainde model not found , could you commit it again?
can you give me dataset for Adobe Deep Image Matting Dataset? [email protected] ! thanks
Hi,thanks for your excellent work.Now i prepare my own datasets.This datasets are consists of thounds of high resolution image(average 4000*4000).They are all full human body images.When i
process these images,i meet a questions:
When i crop the trimap(generated from alpha),often crop some places which are not include hair.Such as foot,leg.Is it ok to input these images into model?@foamliu
how to automate the process of creating mask ,as I see demo rely on masks ? Mask creates using DeepLab are not as refined as in demo.
When i download the file "BEST_checkpoint.tar" successfully, i can't unpack it. Actually, when i try to unpack 'BEST_checkpoint.tar', it make an error. Is it my fault , or, Is the file mistaken?
Hi
I have question regarding alpha needed during testing phase. Per my understanding, paper states that for testing/inference only original image & corresponding trimap are needed and encoder/decoder framework will predict the alpha. On the other hand Ground truth alpha (generated by photoshop) is required for training.
I am aware that originally authors calculated alpha manually for all 431 objects and after composing distributed among training and testing sets. So training and testing both gets high quality alpha produced manually. This is fine is testing images of authors are used.
However for the case if I chose another image from internet
Why alpha being passed during testing phase to generate trimap? and how do I get that alpha? Is it the case that mask generated by any segmentation framework can be used as initial/rough alpha as input to generate_trimap and by running through the network will produce the relatively accurate alpha?
Thank you.
Hi,
Thanks for you work!
its looking awesome output.
I want to integrate your demo into android project. Is it possible to integrate model into android Project? If it possible, then How can i integrate this model into android project?
Can you please give some suggestions?
Thanks in advance.
how can i get "The Composition-1k testing dataset"
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.