zhouyanzhao / prm Goto Github PK
View Code? Open in Web Editor NEWWeakly Supervised Instance Segmentation using Class Peak Response, in CVPR 2018 (Spotlight)
Weakly Supervised Instance Segmentation using Class Peak Response, in CVPR 2018 (Spotlight)
When I run demo/main.ipynb
state = torch.load('./snapshots/model_latest.pt')
---------------------------------------------------------------------------
FileNotFoundError Traceback (most recent call last)
<ipython-input-13-d7aa072378d3> in <module>()
1 # loaded pre-trained weights
2 model = nn.DataParallel(model)
----> 3 state = torch.load('./snapshots/model_latest.pt')
4 model.load_state_dict(state['model'])
5 model = model.module.cuda()
/opt/conda/lib/python3.6/site-packages/torch/serialization.py in load(f, map_location, pickle_module)
354 (sys.version_info[0] == 3 and isinstance(f, pathlib.Path)):
355 new_fd = True
--> 356 f = open(f, 'rb')
357 try:
358 return _load(f, map_location, pickle_module)
FileNotFoundError: [Errno 2] No such file or directory: './snapshots/model_latest.pt'
In the demo file, when I try to run : run_tasks('./config.yml'). I am getting the following error.
KeyError: 'Could not find a Nest module matches query "*fc_resnet50".'
Could you please help me solve this.
Hi,
I love the work you have done. But how can we run this for our own set of images/videos?
Thanks
Dear Yanzhao,
Sorry to bother but I have some issues when running the code. In the paper, there is a description as "We first upsample the class response maps to the size of the image via bilinear interpolation." However, in the code , the size of upsampling class response map is 112*112. It doesn't match the above description for the input image is of size 448 * 448. And I don't understand the meaning of the parameter sub_pixel_locating_factor.
Hope for your reply.
Hi, can anyone help me, how to get the json files in the data folder?
Will te be possible to run in python 3.5.2
Will the implementation be open sourced soon? Amazing work btw.
Do anyone known how to train my dataset by using this repo?
Hi authors,
Can you please explain the peak_threshold and how to find a suitable value for a different dataset.
PRM/install/modules/peak_response_mapping.py
Line 173 in 1bf2068
Pankesh
Hi,
Very cool work! However I run into some issues trying to experiment with it.
How much GPU/CPU memory is recommended for training and inference?
I have a pretty powerful machine (GPU RTX 24 GB RAM) but I could only train with a batch size of 1.
Also at inference time I get stack overflow at the line:
proposals = list(map(modules.rle_decode, json.load(f)))
Any advice on how to fix it would be appreciated.
Thanks
Hi @ZhouYanzhao
May I ask how did you implement Peak Stimulation (local max) ?
Did you build a custom layer or by existing pytorch function ?
I'm trying to build one, but still haven't come up with a solution which makes it differentiable yet.
Hi Yanzhao,
Thanks for the code! Could you explain how to set up the nest modules? I tried 'nest module list' without prm filter, but it still only gives the three modules under prm instead of all 25 modules. So when I try to run the demo, it gives this error: 'Could not find a Nest module matches query "*pascal_voc_classification".' How to set it up properly?
Thanks!
=====
Reinstalled and solved! thanks
EOFError Traceback (most recent call last)
in
----> 1 run_tasks('./config.yml')
~/.conda/envs/prm/lib/python3.6/site-packages/nest/parser.py in run_tasks(config_file, param_file, verbose)
135 logger.info('Finished (%s).' % (U.format_elapse(seconds=(end_time - param_start_time).total_seconds())))
136 else:
--> 137 resolved_config = parse_config(config, env_vars=env_vars)
138 check_all_resolved(resolved_config)
139
~/.conda/envs/prm/lib/python3.6/site-packages/nest/parser.py in parse_config(config, env_vars, global_vars)
58 elif isinstance(val, dict):
59 config[key] = parse_config(
---> 60 val, env_vars=env_vars, global_vars=global_vars)
61 if key == '_var':
62 U.merge_dict(global_vars, config[key], union=True)
~/.conda/envs/prm/lib/python3.6/site-packages/nest/parser.py in parse_config(config, env_vars, global_vars)
58 elif isinstance(val, dict):
59 config[key] = parse_config(
---> 60 val, env_vars=env_vars, global_vars=global_vars)
61 if key == '_var':
62 U.merge_dict(global_vars, config[key], union=True)
~/.conda/envs/prm/lib/python3.6/site-packages/nest/parser.py in parse_config(config, env_vars, global_vars)
68 return nest_module(**config)
69 else:
---> 70 return nest_module(**config, delay_resolve=True)
71
72 return config
~/.conda/envs/prm/lib/python3.6/site-packages/nest/modules.py in call(self, *args, **kwargs)
160 try:
161 self._check_params(resolved_params)
--> 162 returns = self.func(**resolved_params)
163 except KeyError as exc_info:
164 if 'Nest module' in str(exc_info):
~/PRM-pytorch/install/prm.py in fc_resnet50(num_classes, pretrained)
17 """FC ResNet50.
18 """
---> 19 model = FC_ResNet(models.resnet50(pretrained), num_classes)
20 return model
21
~/.conda/envs/prm/lib/python3.6/site-packages/torchvision-0.2.1-py3.6.egg/torchvision/models/resnet.py in resnet50(pretrained, **kwargs)
186 model = ResNet(Bottleneck, [3, 4, 6, 3], **kwargs)
187 if pretrained:
--> 188 model.load_state_dict(model_zoo.load_url(model_urls['resnet50']))
189 return model
190
~/.conda/envs/prm/lib/python3.6/site-packages/torch/utils/model_zoo.py in load_url(url, model_dir, map_location, progress)
64 hash_prefix = HASH_REGEX.search(filename).group(1)
65 _download_url_to_file(url, cached_file, hash_prefix, progress=progress)
---> 66 return torch.load(cached_file, map_location=map_location)
67
68
~/.conda/envs/prm/lib/python3.6/site-packages/torch/serialization.py in load(f, map_location, pickle_module)
301 f = open(f, 'rb')
302 try:
--> 303 return _load(f, map_location, pickle_module)
304 finally:
305 if new_fd:
~/.conda/envs/prm/lib/python3.6/site-packages/torch/serialization.py in _load(f, map_location, pickle_module)
457 f.seek(0)
458
--> 459 magic_number = pickle_module.load(f)
460 if magic_number != MAGIC_NUMBER:
461 raise RuntimeError("Invalid magic number; corrupt file?")
EOFError: Ran out of input
Your works is amazing!
Still, I'm puzzled about some details of the peak back-propagation process in your paper.
In the paper, a few demo results of your model show a promising performance that PRM could work well on most cases where objects in the image are not so dense. I'm wondering that what if the input image is crowded with objects, for example a group of people or a traffic jam. The reason I propose this issue is that as far as I'm concerned, there will be many problems when it encountering this kind of situations.
First, since the model extracts peaks of the feature map (which called Peak stimulation), if many objects are overlapping or very close to each other, then it is very likely of your model being not able to locate precise peaks of each of them, because their features are probably messed up with each other.
Second, I've noticed that in paper, some results of peak back-propagation are shown, where different colors are used and making it fancy. My question is, according to the results, peak back-propagation could only recover part of objects, no matter what the object is, in paper, there are two kids playing with a car, and none of them are fully recovered after peak back-propagation. So, how do they turn into the final result?
I know you are very busy, sorry for this long issue. My last question is, what is the speed (frame per second) of the whole model during inference?
Hi,
I ran into this problem
TypeError: The param "transform" of Nest module "fetch_data" should be type of "Union[Callable, NoneType]". Got
image_transform(
[✓] image_size: Union[int, List[int]],
augmentation: dict,
[✓] mean: List[float] = [0.485, 0.456, 0.406],
[✓] std: List[float] = [0.229, 0.224, 0.225]) -> Callable
Please check if some important params of Nest module "image_transform" have been forgotten in use.
Anyone of you know how to solve this? Or alternativelty anyone of you has the training code without Nest?
Thanks
Hi, I fine-tuned the model with my own dataset. I just fine-tuned the classification layer freezing the feature layers, with a very small dataset of 48 images. I have only 6 classes.
However, while testing I am always getting "No class peak response detected". Can anyone please suggest me the probable reason for that.
I am trying to load ./snapshots/model_latest.pt
to run the main.ipynb
notebook. Is there a link to the latest model?
Thank you.
In demo/main.ipynb, In[11] line 6, proposals = list(map(modules.rle_decode, json.load(f)))
. I can not find rle_decode in config.yml. Does any one know where can I find it? Thanks.
My server can't connect to the external network, I have no way to nest, I have downloaded the zip to the local and uploaded it to the server, how should I install PRM?
普通pytorch code并不是很复杂,要是有能不依赖nest工具的运行版本就好了。毕竟为了运行一个code去熟悉另外一套封装,加大了复杂度,与其它code也不易整合。
Thanks for the code!
What scripts did you use to compute the mAP and the average best overlapping (ABO) metrics?
Thanks!
hi,@ZhouYanzhao,I am confused that How to prepare the "pre-computed object proposals".
Hope to receive your reply!Tks
Hi,
This is an interesting work.
But we can't achieve the performance reported in PRM (mAP50: 26.8 with MCG proposals).
We can only get 11.5 mAP50 with MCG proposals downloaded from https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/mcg/
and 21.5 mAP50 with COB proposals downloaded from http://www.vision.ee.ethz.ch/~cvlsegmentation/cob/code.html.
We use the default parameters of PRM (https://github.com/ZhouYanzhao/PRM/blob/pytorch/demo/config.yml) to train the classification network, (change the train_splits from trainval to trianaug, of course). But we notice that both the quality of the peaks and the instance masks are worse than those reported in the paper,
So we wonder if you use other hyper-parameter settings in your experiments?
Besides, according to our observations, the MCG proposals from https://data.vision.ee.ethz.ch/jpont/mcg/MCG-Pascal-Segmentation_trainvaltest_2012-proposals.tgz are much worse than the those shown in the paper and the supplement material. So do we need to retrain MCG with PASCAL train set to generate better proposals?
Would you please point out the differences between my experiments and yours that may results in the gap? or could you give us some advice to boost the performance?
Thanks a lot.
Hi @ZhouYanzhao !
I have a problem when I run this "state = torch.load('./snapshots/model_latest.pt')"
problem: "No such file or directory: './snapshots/model_latest.pt' ".?
How can I get this file?
Execute " nest module list --filter prm" and get the following result.(Ubuntu16)
(pt-py3) root@ebe535e9a1a8:~# nest module list --filter prm
Traceback (most recent call last):
File "/root/Envs/pt-py3/lib/python3.5/site-packages/nest/modules.py", line 346, in _import_nest_modules_from_file
spec.loader.exec_module(py_module)
File "", line 665, in exec_module
File "", line 222, in _call_with_frames_removed
File "/root/PRM/install/prm.py", line 30, in
filter_type: Union[str, int, float] = 'median') -> nn.Module:
File "/root/Envs/pt-py3/lib/python3.5/site-packages/nest/modules.py", line 262, in _register
return create_module(args[0])
File "/root/Envs/pt-py3/lib/python3.5/site-packages/nest/modules.py", line 259, in create_module
return type('NestModule', (NestModule,), dict(slots=(), doc=doc))(func, nest_meta)
File "/root/Envs/pt-py3/lib/python3.5/site-packages/nest/modules.py", line 69, in init
self._check_definition()
File "/root/Envs/pt-py3/lib/python3.5/site-packages/nest/modules.py", line 80, in _check_definition
if v.default is not inspect.Parameter.empty and not U.is_annotation_matched(v.default, v.annotation):
File "/root/Envs/pt-py3/lib/python3.5/site-packages/nest/logger.py", line 59, in wrapper
res = func(*args, **kwargs)
File "/root/Envs/pt-py3/lib/python3.5/site-packages/nest/utils.py", line 236, in is_annotation_matched
sub_annotation = annotation.args
AttributeError: type object 'Union' has no attribute 'args'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/root/Envs/pt-py3/bin/nest", line 11, in
sys.exit(main())
File "/root/Envs/pt-py3/lib/python3.5/site-packages/nest/main.py", line 4, in main
CLI()
File "/root/Envs/pt-py3/lib/python3.5/site-packages/nest/cli.py", line 71, in init
getattr(self, 'cmd_' + args.command)('nest ' + args.command, sys.argv[2:])
File "/root/Envs/pt-py3/lib/python3.5/site-packages/nest/cli.py", line 162, in cmd_module
module_info = ['%s (%s)' % (k, v.meta.get('version', 'version')) for k, v in module_manager]
File "/root/Envs/pt-py3/lib/python3.5/site-packages/nest/modules.py", line 702, in iter
self._update_modules()
File "/root/Envs/pt-py3/lib/python3.5/site-packages/nest/modules.py", line 692, in _update_modules
ModuleManager._import_nest_modules_from_dir(meta['module_path'], namespace, self.py_modules, self.nest_modules, meta)
File "/root/Envs/pt-py3/lib/python3.5/site-packages/nest/modules.py", line 407, in _import_nest_modules_from_dir
ModuleManager._import_nest_modules_from_file(file_path, namespace, py_modules, nest_modules, meta)
File "/root/Envs/pt-py3/lib/python3.5/site-packages/nest/modules.py", line 356, in _import_nest_modules_from_file
if (type(exc_info) is ModuleNotFoundError or type(exc_info) is ImportError) and exc_info.name is not None:
NameError: name 'ModuleNotFoundError' is not defined
from nest import modules, run_tasks works well for me,
while when executing, I got this error:
AttributeError: type object 'tqdm' has no attribute '_lock'
Is there anyone who ever met this error?
Hi Yanzhao,Does this PRM project support training?
I want to training on VOC 2007. But I did not find the training codes.
TypeError occurs when I run the cell " run_tasks('./config.yml') ".The details are as follows.Everything is OK when I install the nest and PRM modules.Hope someone could help me solve this problem.Please~
帮帮我~
TypeError: The param "transform" of Nest module "fetch_data" should be type of "Union[Callable, NoneType]". Got
image_transform(
[✓] image_size: Union[int, List[int]],
augmentation: dict,
[✓] mean: List[float] = [0.485, 0.456, 0.406],
[✓] std: List[float] = [0.229, 0.224, 0.225]) -> Callable
Please check if some important params of Nest module "image_transform" have been forgotten in use.
RuntimeError Traceback (most recent call last)
in
----> 1 run_tasks('./config.yml')
~/.local/lib/python3.6/site-packages/nest/parser.py in run_tasks(config_file, param_file, verbose)
135 logger.info('Finished (%s).' % (U.format_elapse(seconds=(end_time - param_start_time).total_seconds())))
136 else:
--> 137 resolved_config = parse_config(config, env_vars=env_vars)
138 check_all_resolved(resolved_config)
139
~/.local/lib/python3.6/site-packages/nest/parser.py in parse_config(config, env_vars, global_vars)
68 return nest_module(**config)
69 else:
---> 70 return nest_module(**config, delay_resolve=True)
71
72 return config
~/.local/lib/python3.6/site-packages/nest/modules.py in call(self, *args, **kwargs)
160 try:
161 self._check_params(resolved_params)
--> 162 returns = self.func(**resolved_params)
163 except KeyError as exc_info:
164 if 'Nest module' in str(exc_info):
~/Nest-pytorch/install/trainer.py in network_trainer(data_loaders, model, criterion, optimizer, parameter, meters, hooks, max_epoch, test_interval, resume, log_path, device, use_data_parallel, use_cudnn_benchmark, random_seed)
81
82 # setup model
---> 83 model = model.to(device)
84
85 # multi-gpu support
~/.local/lib/python3.6/site-packages/torch/nn/modules/module.py in to(self, *args, **kwargs)
423 return t.to(device, dtype if t.is_floating_point() else None, non_blocking)
424
--> 425 return self._apply(convert)
426
427 def register_backward_hook(self, hook):
~/.local/lib/python3.6/site-packages/torch/nn/modules/module.py in _apply(self, fn)
199 def _apply(self, fn):
200 for module in self.children():
--> 201 module._apply(fn)
202
203 def compute_should_use_set_data(tensor, tensor_applied):
~/.local/lib/python3.6/site-packages/torch/nn/modules/module.py in _apply(self, fn)
199 def _apply(self, fn):
200 for module in self.children():
--> 201 module._apply(fn)
202
203 def compute_should_use_set_data(tensor, tensor_applied):
~/.local/lib/python3.6/site-packages/torch/nn/modules/module.py in _apply(self, fn)
199 def _apply(self, fn):
200 for module in self.children():
--> 201 module._apply(fn)
202
203 def compute_should_use_set_data(tensor, tensor_applied):
~/.local/lib/python3.6/site-packages/torch/nn/modules/module.py in _apply(self, fn)
221 # with torch.no_grad():
222 with torch.no_grad():
--> 223 param_applied = fn(param)
224 should_use_set_data = compute_should_use_set_data(param, param_applied)
225 if should_use_set_data:
~/.local/lib/python3.6/site-packages/torch/nn/modules/module.py in convert(t)
421
422 def convert(t):
--> 423 return t.to(device, dtype if t.is_floating_point() else None, non_blocking)
424
425 return self._apply(convert)
RuntimeError: CUDA error: out of memory
I have one GPU: GeForce RTX 2060
python :3.6.9
It is really weird it has out of memory before training. could you please give me some suggestions?
作者您好,我看你的介绍时说可以使用cpu训练,但速度很慢,现在我受条件限制只能用cpu,在运行demo时报了这个错:
~\Nest-pytorch\install\trainer.py in network_trainer(data_loaders, model, criterion, optimizer, parameter, meters, hooks, max_epoch, test_interval, resume, log_path, device, use_data_parallel, use_cudnn_benchmark, random_seed)
74 device = torch.device(device)
75 if device.type == 'cuda':
---> 76 assert torch.cuda.is_available(), 'CUDA is not available.'
77 torch.backends.cudnn.benchmark = use_cudnn_benchmark
很明显这是没装cuda的结果,但nest作为一个网上基本查不到资料的工具,我也不知道该怎么改才能继续在cpu状态训练,希望您能回答我,谢谢。
when running this commans, hook modules such as 'print_state' are not installing!
nest module install github@ZhouYanzhao/Nest:pytorch pytorch
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.