mahyarnajibi / sniper Goto Github PK
View Code? Open in Web Editor NEWSNIPER / AutoFocus is an efficient multi-scale object detection training / inference algorithm
License: Other
SNIPER / AutoFocus is an efficient multi-scale object detection training / inference algorithm
License: Other
Hi,
First of all, thanks for providing this nicely organized repo.
I have successfully run SNIPER demo without any problem.
However, for R-FCN 3K, I am not able to run the demo. There is strange error, which originates from asnumpy() function.
The error message I got is
SNIPER/CRCNN-mxnet/3rdparty/nnvm/include/nnvm/tuple.h:438: Check failed: dim == static_cast(ndim()) (2 vs. 1) dimension do not match target dimension 2 vs 1
I am using CUDA 9.1 with cuDNN 7.02 on a P3.x16 machine.
I have tried diffrent configures on the cmake, e.g., with/withou mkl, but I still cannot figure out what is the problem.
I can easily convert other NDArray to numpy array without any problem. I just cannot convert the output of the R-FCN network.
Please advise.
Hi, I got an index out of bound error when training my own model.
loading annotations into memory...
Done (t=0.35s)
creating index...
index created!
num_images 11500
COCO_train2014 gt roidb loaded from ./data/cache/COCO_train2014_gt_roidb.pkl
appending ground truth annotations
Reading cached proposals after NMS* from data/proposals/COCO_train2014_rpn_after_nms.pkl
Done!
append flipped images to roidb
filtered 7928 roidb entries: 23000 -> 15072
add bounding box regression targets
bbox target means:
[[0. 0. 0. 0.]
[0. 0. 0. 0.]]
[0. 0. 0. 0.]
bbox target stdevs:
[[0.1 0.1 0.2 0.2]
[0.1 0.1 0.2 0.2]]
[0.1 0.1 0.2 0.2]
Creating Iterator with 15072 Images
Total number of extracted chips: 72201
Done!
The Iterator has 72201 samples!
Initializing the model...
Optimizer params: {'wd': 0.01, 'lr_scheduler': <train_utils.lr_scheduler.WarmupMultiBatchScheduler object at 0x7fee7fce4610>, 'multi_precision': True, 'learning_rate': 0.00015, 'rescale_grad': 1.0, 'clip_gradient': None, 'momentum': 0.9}
aaa
[00:32:22] src/operator/nn/./cudnn/./cudnn_algoreg-inl.h:107: Running performance tests to find the best convolution algorithm, this can take a while... (setting env variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable)
Exception in thread Thread-8:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "lib/iterators/PrefetchingIter.py", line 61, in prefetch_func
self.next_batch[i] = self.iters[i].next()
File "lib/iterators/MNIteratorBase.py", line 90, in next
if self.iter_next():
File "lib/iterators/MNIteratorBase.py", line 87, in iter_next
return self.get_batch()
File "lib/iterators/MNIteratorE2E.py", line 99, in get_batch
self.batch = self._get_batch()
File "lib/iterators/MNIteratorE2E.py", line 136, in _get_batch
nids = processed_roidb[i]['props_in_chips'][cropid]
IndexError: index 12 is out of bounds for axis 0 with size 12
How can I fix this?
I've noticed that you replace the last few rois with gt_boxes in MultiProposalTarget Operater, which means that there are accurate proposals for each RoI batch.
Hi, I got a problem which really confused me for several days, and I cannot solve. If you may help, that will be great.
The error message shows bellow:
loading annotations into memory...
Done (t=4.05s)
creating index...
index created!
num_images 40504
COCO_val2014 gt roidb loaded from ./data/cache/COCO_val2014_gt_roidb.pkl
filtered 367 roidb entries: 40504 -> 40137
add bounding box regression targets
bbox target means:
[[0. 0. 0. 0.]
[0. 0. 0. 0.]]
[0. 0. 0. 0.]
bbox target stdevs:
[[0.1 0.1 0.2 0.2]
[0.1 0.1 0.2 0.2]]
[0.1 0.1 0.2 0.2]
Creating Iterator with 40137 Images
Total number of extracted chips: 155974
Done!
Traceback (most recent call last):
File "/home/arias/Projects/Python/SNIPER/main_train.py", line 74, in <module>
threads=config.TRAIN.NUM_THREAD, pad_rois_to=400)
File "lib/iterators/MNIteratorE2E.py", line 31, in __init__
super(MNIteratorE2E, self).__init__(roidb, config, batch_size, threads, nGPUs, pad_rois_to, False)
File "lib/iterators/MNIteratorBase.py", line 33, in __init__
self.get_batch()
File "lib/iterators/MNIteratorE2E.py", line 99, in get_batch
self.batch = self._get_batch()
File "lib/iterators/MNIteratorE2E.py", line 171, in _get_batch
all_labels = self.pool.map(self.anchor_worker.worker, worker_data)
File "/home/arias/anaconda3/envs/python2/lib/python2.7/multiprocessing/pool.py", line 253, in map
return self.map_async(func, iterable, chunksize).get()
File "/home/arias/anaconda3/envs/python2/lib/python2.7/multiprocessing/pool.py", line 572, in get
raise self._value
TypeError: Cannot cast ufunc subtract output from dtype('float64') to dtype('uint16') with casting rule 'same_kind'
The first line of the README contains one typo: "instance-level segmentataion" instead of "segmentation"
Hi;
I noticed ms < (self.chip_size - self.chip_stride - 1) / im_scale
is served as a constraint to choose the valid gt_boxes id(please see below), which however is unmentioned in you paper. Could you please explain the idea behind this constraint? Thank you!
elif i == 0:
# The finest scale (but not the only scale)
ids = np.where((area < self.valid_ranges[i][1]) &
(ms < (self.chip_size - self.chip_stride - 1) / im_scale) & (ws >= 2) & (hs >= 2))[0]
else:
# An intermediate scale
ids = np.where((area >= self.valid_ranges[i][0]) & (area < self.valid_ranges[i][1])
& (ms < (self.chip_size - self.chip_stride - 1) / im_scale))[0]
Hi, I am trying to clone the openimage branch by using command:
git clone -b openimages2 --recursive https://github.com/mahyarnajibi/SNIPER.git
and I got a fatal error stating that I cannot read from SNIPER-mxnet remote module. The complete error message is as follow:
Cloning into 'SNIPER'...
remote: Counting objects: 1663, done.
remote: Compressing objects: 100% (89/89), done.
remote: Total 1663 (delta 45), reused 96 (delta 34), pack-reused 1540
Receiving objects: 100% (1663/1663), 22.00 MiB | 9.45 MiB/s, done.
Resolving deltas: 100% (1009/1009), done.
Checking connectivity... done.
Submodule 'SNIPER-mxnet' ([email protected]:mahyarnajibi/SNIPER-mxnet.git) registered for path 'SNIPER-mxnet'
Cloning into 'SNIPER-mxnet'...
The authenticity of host 'github.com (192.30.255.112)' can't be established.
RSA key fingerprint is SHA256:nThbg********************************6E5SY8.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'github.com,192.30.255.112' (RSA) to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.Please make sure you have the correct access rights
and the repository exists.
fatal: clone of '[email protected]:mahyarnajibi/SNIPER-mxnet.git' into submodule path 'SNIPER-mxnet' failed
Hi, what is the cuda version? python version
Hi , I tried to use SNIPER on a new dataset but the perfomance is not satisfying. I think the problem is that my own neg-chips are not as good as yours in coco dataset. Could you please release the SNIP code so that I can train my dataset on SNIP and compare their performances?
Thanks a lot
when i train use main_train.py, appear the following question.
Optimizer params: {'wd': 0.01, 'lr_scheduler': <train_utils.lr_scheduler.WarmupMultiBatchScheduler object at 0x7fe7702edd90>, 'multi_precision': True, 'learning_rate': 0.00015, 'rescale_grad': 1.0, 'clip_gradient': None, 'momentum': 0.9}
aaa
[00:33:49] /home/czn/scc/SNIPER/SNIPER-mxnet/src/operator/nn/./cudnn/./cudnn_algoreg-inl.h:107: Running performance tests to find the best convolution algorithm, this can take a while... (setting env variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable)
CUDA error: too many resources requested for launch
Hello, @mahyarnajibi, I am new to mxnet and when I run the cmake command, it turns out:
-- The C compiler identification is GNU 4.8.4
-- The CXX compiler identification is GNU 4.8.4
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- CMake version '3.11.4' using generator 'Unix Makefiles'
-- The CUDA compiler identification is NVIDIA 8.0.26
-- Check for working CUDA compiler: /usr/local/cuda-8.0/bin/nvcc
-- Check for working CUDA compiler: /usr/local/cuda-8.0/bin/nvcc -- works
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Performing Test SUPPORT_CXX11
-- Performing Test SUPPORT_CXX11 - Success
-- Performing Test SUPPORT_CXX0X
-- Performing Test SUPPORT_CXX0X - Success
-- Performing Test SUPPORT_MSSE2
-- Performing Test SUPPORT_MSSE2 - Success
CMake Deprecation Warning at 3rdparty/mkldnn/CMakeLists.txt:21 (cmake_policy):
The OLD behavior for policy CMP0048 will be removed from a future version
of CMake.
The cmake-policies(7) manual explains that the OLD behaviors of all
policies are deprecated and that a policy should be set to OLD only under
specific short-term circumstances. Projects should be ported to the NEW
behavior and not rely on setting a policy to OLD.
CMake Deprecation Warning at 3rdparty/mkldnn/CMakeLists.txt:22 (cmake_policy):
The OLD behavior for policy CMP0054 will be removed from a future version
of CMake.
The cmake-policies(7) manual explains that the OLD behaviors of all
policies are deprecated and that a policy should be set to OLD only under
specific short-term circumstances. Projects should be ported to the NEW
behavior and not rely on setting a policy to OLD.
-- CMAKE_BUILD_TYPE is unset, defaulting to Release
-- Detecting Intel(R) MKL: trying mklml_intel
-- Detecting Intel(R) MKL: trying mklml
-- Detecting Intel(R) MKL: trying mkl_rt
CMake Warning at 3rdparty/mkldnn/cmake/MKL.cmake:178 (message):
Intel(R) MKL not found. Some performance features may not be available.
Please run scripts/prepare_mkl.sh to download a minimal set of libraries or
get a full version from https://software.intel.com/en-us/intel-mkl
Call Stack (most recent call first):
3rdparty/mkldnn/cmake/OpenMP.cmake:25 (include)
3rdparty/mkldnn/CMakeLists.txt:57 (include)
-- Found OpenMP_C: -fopenmp (found version "3.1")
-- Found OpenMP_CXX: -fopenmp (found version "3.1")
-- Found OpenMP: TRUE (found version "3.1")
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
-- VTune profiling environment is unset
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - found
-- Found Threads: TRUE
-- Could NOT find MKL (missing: MKL_INCLUDE_DIR MKLML_GNU_LIBRARY MKLDNN_LIBRARY)
-- MKL not found
-- Found CUDA: /usr/local/cuda-8.0 (found version "8.0")
-- Could NOT find MKL (missing: MKL_INCLUDE_DIR MKLML_GNU_LIBRARY MKLDNN_LIBRARY)
-- Found OpenBLAS libraries: /usr/lib/libopenblas.so
-- Found OpenBLAS include: /usr/include
-- Could NOT find Gperftools (missing: GPERFTOOLS_LIBRARIES GPERFTOOLS_INCLUDE_DIR)
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.26")
-- Could NOT find Jemalloc (missing: JEMALLOC_LIBRARY JEMALLOC_INCLUDE_DIR)
-- OpenCV_LIBS=opencv_core;opencv_highgui;opencv_imgproc
-- OpenCV found (/usr/share/OpenCV)
-- Found OpenMP_C: -fopenmp
-- Found OpenMP_CXX: -fopenmp
-- Found OpenMP: TRUE
CMake Error at SNIPER/SNIPER-mxnet/build/3rdparty/openmp/runtime/CMakeFiles/CMakeTmp/CMakeLists.txt:3 (project):
Generator
Unix Makefiles
does not support toolset specification, but toolset
cuda=,host=x64
was specified.
CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
CMake Error at 3rdparty/openmp/runtime/cmake/LibompGetArchitecture.cmake:57 (try_run):
Failed to configure test project build system.
Call Stack (most recent call first):
3rdparty/openmp/runtime/CMakeLists.txt:40 (libomp_get_architecture)
-- Configuring incomplete, errors occurred!
Would you please be so kind as to help me out? Thanks.
Hi,
I am currently trying to improve my performance for a certain dataset, and I am not using negative chips to train my algorithms. From the visualization it looks fine, however there are some false positives that I do want to avoid. I am pretty sure that is because of the fact that I don't do negative chip training.
Can you specify more on how to get the negative chips? From the paper it says:
To this end, for negative chip mining, we first train RPN for a couple of epochs. No negative chips are used for training this network. The task of this network is to roughly guide us in selecting regions which are likely to contain false positives, so it is not necessary for it to be very accurate. This RPN is used to generate proposals over the entire training set. We assume that if no proposals are generated in a major portion of the image by RPN, then it is unlikely to contain an object instance.
And in the code, I found that there's a setting called TEST.EXTRACT_PROPOSALS
- but reading thru the code, I am not sure if it is doing what's described in the paper.
Any pointers will be appreciated! Thanks!
Great work. I think it would be really great if there is an inference example script for running openimagev4 pretrained model just similar the current demo.py (COCO).
What are the CRCNN-0007 weights trained on?
For context, it fails to detect people in images.
Hi, I have a quick question regarding model inference. Let's say I've trained a model using SNIPER, for a regular object detection task, can the model be served:
The reason is that for some reason, we need to inference the model on CPU. I guess it might be possible since the NN is still standard resnet + faster RCNN, but I am not sure about this (since I do see a few updates in the SNIPER-mxnet module which might be relevant to accomplish the goal above).
Thanks!
Thanks for open sourcing the code, it looks great!
One question I have is how to train new proposals? Since I want to use SNIPER in a very different dataset, I assume the best way for me is to re-calculate the proposals? In that case, how do I re-train the proposal network?
Thanks!
Hi,I am training my model in openimages dataset. I see in your openimage config file that you used a pretrained model called CRCNN-1epoch.But I download your pretrain model with bash scripts/download_pretrained_model.sh ,but there is no file named CRCNN-0001.Can you tell me where is this file to download.
HI.
Thinks for provide this nice project.
When I compile this repo,I met some problems, and I solve it with compile OpenBLAS and lapcak,link lapack/liblapack.a to /usr/lib/liblapack.a. And then,i run the demo.py,But it not response after print two line 'aaa' and no any other message.Then i change the default config to fit my double 1080ti GPU.I just change the gpus from '0,1,2,3,4,5,6,7' to '0,1',and change the lr from '0.015' to '0.04'.Download pretrained_model and proposals,run the main_train.py script.But the process exhaust all my 32G memory and the computer not responsed.I don't know how to solve this problem.
Env: ubuntu 16.04.04 (kernel 4.4), CUDA9.0/cudnn 7.1.4, double Nvidia 1080 ti,python 2.7
Can you give me some abvise.
Hi authors, Great work! And I know that you're migrating caffe SSH to this repo which will be re-write by MXnet. I have done such work these days but can not achieve the same accuracy as it was by Caffe. For example, single scale 79.1% mAP on widerface-hard, vs reported 81.4% from caffe SSH.
Do you have any tips on it?
Thanks.
when FLIP is set to true, running the repository against coco will produce some error like below:
wrote gt roidb to /data/cache/COCO_train2014_gt_roidb.pkl
append flipped images to roidb
Traceback (most recent call last):
File "main_train.py", line 59, in <module>
for image_set in image_sets]
File "lib/data_utils/load_data.py", line 36, in load_proposal_roidb
roidb = imdb.append_flipped_images(roidb)
File "lib/dataset/imdb.py", line 249, in append_flipped_images
'proposal_scores': roidb[i]['proposal_scores']}
KeyError: 'proposal_scores'
And changing FLIP to false will eliminate the error. I suspect there's something wrong with the flipping logic (didn't dig deep though).
Hi,
Which specific model did you use for Resnet 50? Can you provide the link to download the pre-trained resnet50 on imagenet? Looks like it's not available in the mxnet model zoo...
Thanks!
Hi, I use SNIPER to train my own dataset. When training, I got nan loss as bellow,
Iter[0] Batch[100]
Train-RPNAcc=nan
Train-RPNLogLoss=nan
Train-RPNL1Loss=nan
Train-RCNNAcc=nan
Train-RCNNLogLoss=nan
Train-RCNNL1LossCRCNN=nan
It just occurs every several batches. My batch size is 4, and every 100 batches, it occurs. Other time everything just looks fine like bellow,
Iter[0] Batch[195]
Train-RPNAcc=0.782257
Train-RPNLogLoss=0.652749
Train-RPNL1Loss=0.023370
Train-RCNNAcc=0.979158
Train-RCNNLogLoss=2.133237
Train-RCNNL1LossCRCNN=0.046825
I trying to decrease lr
to 0.0005, but not work.
I'm using a custom dataset. But it's asking for RPN weights, is there any way to bypass that. If I load the weights, it failing with the following error:
Loading data/proposals/voc_2007_trainval_rpn.pkl....
Done!
Applying NMS...
Traceback (most recent call last):
File "main_train.py", line 59, in <module>
for image_set in image_sets]
File "lib/data_utils/load_data.py", line 32, in load_proposal_roidb
roidb = eval('imdb.' + proposal + '_roidb')(roidb, append_gt,proposal_path=proposal_path)
File "lib/dataset/imdb.py", line 139, in rpn_roidb
rpn_roidb = self.load_rpn_roidb(gt_roidb,proposal_path)
File "lib/dataset/imdb.py", line 126, in load_rpn_roidb
box_list, mapping_list = self.load_rpn_data(proposal_path)
File "lib/dataset/imdb.py", line 109, in load_rpn_data
keeps = p.map(nmsp, ttboxes)
File "/usr/lib/python2.7/multiprocessing/pool.py", line 251, in map
return self.map_async(func, iterable, chunksize).get()
File "/usr/lib/python2.7/multiprocessing/pool.py", line 567, in get
raise self._value
IndexError: too many indices for array
Thank You for sharing code.
fyi cannot perform git clone --recursive on SNIPER:
fatal: repository 'https://github.com/mahyarnajibi/SNIPER-mxnet/' not found
fatal: clone of 'https://github.com/mahyarnajibi/SNIPER-mxnet' into submodule path 'SNIPER-mxnet' failed
i run the demo.py with mobilenetv2 model on the P40 ,about 150ms,the speed is slow,something wrong?
i never change the config.
Tester: 1/1, Detection: 0.1813s, Post Processing: 0.001738s
Tester: 1/1, Detection: 0.1461s, Post Processing: 0.001268s
Tester: 1/1, Detection: 0.1313s, Post Processing: 0.001222s
Hi, @mahyarnajibi ,, I am new to mxnet and when I run the command,:python demo.py. it turns out:
mxnet.base.MXNetError: [15:56:34] /home/chase/SNIPER/SNIPER-mxnet/src/operator/nn/batch_norm.cu:527: Check failed: err == cudaSuccess (7 vs. 0) Name: BatchNormalizationUpdateOutput ErrStr:too many resources requested for launch
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/libmxnet.so(dmlc::StackTraceabi:cxx11+0x1bc) [0x7f8cdb156b5c]
[bt] (1) /usr/local/lib/libmxnet.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x28) [0x7f8cdb157f08]
[bt] (2) /usr/local/lib/libmxnet.so(void mxnet::op::BatchNormForwardImpl<mshadow::gpu, mshadow::half::half_t, float>(mshadow::Streammshadow::gpu, mxnet::OpContext const&, mxnet::op::BatchNormParam const&, std::vector<mxnet::TBlob, std::allocatormxnet::TBlob > const&, std::vector<mxnet::OpReqType, std::allocatormxnet::OpReqType > const&, std::vector<mxnet::TBlob, std::allocatormxnet::TBlob > const&, std::vector<mxnet::TBlob, std::allocatormxnet::TBlob > const&)+0x31e) [0x7f8cdcd2a05e]
[bt] (3) /usr/local/lib/libmxnet.so(void mxnet::op::BatchNormForward<mshadow::gpu, mshadow::half::half_t, float>(mxnet::OpContext const&, mxnet::op::BatchNormParam const&, std::vector<mxnet::TBlob, std::allocatormxnet::TBlob > const&, std::vector<mxnet::OpReqType, std::allocatormxnet::OpReqType > const&, std::vector<mxnet::TBlob, std::allocatormxnet::TBlob > const&, std::vector<mxnet::TBlob, std::allocatormxnet::TBlob > const&)+0x301) [0x7f8cdcd2a841]
[bt] (4) /usr/local/lib/libmxnet.so(void mxnet::op::BatchNormComputemshadow::gpu(nnvm::NodeAttrs const&, mxnet::OpContext const&, std::vector<mxnet::TBlob, std::allocatormxnet::TBlob > const&, std::vector<mxnet::OpReqType, std::allocatormxnet::OpReqType > const&, std::vector<mxnet::TBlob, std::allocatormxnet::TBlob > const&)+0x852) [0x7f8cdcd1fcc2]
[bt] (5) /usr/local/lib/libmxnet.so(mxnet::exec::FComputeExecutor::Run(mxnet::RunContext, bool)+0x50) [0x7f8cdb2a0660]
[bt] (6) /usr/local/lib/libmxnet.so(+0xc55ba8) [0x7f8cdb2a3ba8]
[bt] (7) /usr/local/lib/libmxnet.so(mxnet::engine::ThreadedEngine::ExecuteOprBlock(mxnet::RunContext, mxnet::engine::OprBlock)+0x429) [0x7f8cdcb149c9]
[bt] (8) /usr/local/lib/libmxnet.so(void mxnet::engine::ThreadedEnginePerDevice::GPUWorker<(dmlc::ConcurrentQueueType)0>(mxnet::Context, bool, mxnet::engine::ThreadedEnginePerDevice::ThreadWorkerBlock<(dmlc::ConcurrentQueueType)0>, std::shared_ptrdmlc::ManualEvent const&)+0xeb) [0x7f8cdcb18c4b]
[bt] (9) /usr/local/lib/libmxnet.so(std::_Function_handler<void (std::shared_ptrdmlc::ManualEvent), mxnet::engine::ThreadedEnginePerDevice::PushToExecute(mxnet::engine::OprBlock, bool)::{lambda()#3}::operator()() const::{lambda(std::shared_ptrdmlc::ManualEvent)#1}>::_M_invoke(std::_Any_data const&, std::shared_ptrdmlc::ManualEvent&&)+0x4e) [0x7f8cdcb18ece]
Would you please to help me?Thank you.
Hello,
First of all thank you for putting the code here on github so we can all use and experiment with it.
I am trying to train on my own dataset,but I am having a little bit of trouble.
More specifically, I have this error:
File "lib/iterators/MNIteratorE2E.py", line 194, in _get_batch
processed_list = processed_list.get()
File "/home/radud/anaconda3/envs/mxnetpy2/lib/python2.7/multiprocessing/pool.py", line 572, in get
raise self._value
TypeError: 'NoneType' object has no attribute 'getitem'
Any help would be appreciated.
Thank you.
Thanks for your code very much.
Could you report the human AP on the coco test-dev dataset ?
Hi, I try to train SNIPER on my own dataset, but the performance always keep low as bellow,
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.004
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.015
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.002
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.005
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.067
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.125
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.134
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.011
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.176
Loss and accuracy seem just fine as bellow,
Train-RPNAcc=0.965601
Train-RPNLogLoss=0.114530
Train-RPNL1Loss=0.009350
Train-RCNNAcc=0.963693
Train-RCNNLogLoss=0.225078
Train-RCNNL1LossC
RCNN=0.085674
Here is the yaml config,
# --------------------------------------------------------------
# SNIPER: Efficient Multi-Scale Training
# Licensed under The Apache-2.0 License [see LICENSE for details]
# by Mahyar Najibi, Bharat Singh
# --------------------------------------------------------------
---
MXNET_VERSION: "mxnet"
output_path: "./output/sniper_res101_bn"
symbol: resnet_mx_101_e2e
gpus: '0'
CLASS_AGNOSTIC: true
default:
kvstore: device
network:
pretrained: "./data/pretrained_model/resnet_mx_101_open"
pretrained_epoch: 0
PIXEL_MEANS:
- 103.939
- 116.779
- 123.68
RPN_FEAT_STRIDE: 16
FIXED_PARAMS:
- conv0
- bn0
- stage1
ANCHOR_RATIOS:
- 0.25
- 0.5
- 1
- 2
- 4
ANCHOR_SCALES:
- 2
- 4
- 7
- 10
- 13
- 16
- 24
NUM_ANCHORS: 35
dataset:
NUM_CLASSES: 5
dataset: coco
dataset_path: "./data/n_classes"
image_set: train2014
root_path: "./data"
test_image_set: val2014
proposal: rpn
TRAIN:
## CHIP GENERATION PARAMS
# Whether to use C++ or python code for chip generation
CPP_CHIPS: true
# How many parts the dataset should be divided to for parallel chip generation
# This is used to keep the memory limited
CHIPS_DB_PARTS: 20
# Multi-processing params
# These parameters are used for parallel chip generation, NMS, etc.
# Please consider adjusting them for your system
NUM_PROCESS: 8
NUM_THREAD: 8
# Whether to train with segmentation mask
WITH_MASK: false
# Training scales
# The last scale (or the only scale) should be the desired max resolution in pixels
# Other scales should be scaling coefficients
SCALES:
- 0.5
- 0.8
- 2
- 3.0
- 1.667
- 512.0
# Valid ranges in each scale
VALID_RANGES:
- !!python/tuple [-1,-1]
- !!python/tuple [-1,-1]
- !!python/tuple [-1,-1]
- !!python/tuple [-1,-1]
- !!python/tuple [-1,-1]
- !!python/tuple [-1,-1]
lr: 0.0001 #0.002 #0.0005
lr_step: '5.33'
warmup: true
fp16: true
warmup_lr: 0.0005 #0.00005
wd: 0.0001
scale: 100.0
warmup_step: 1000 #4000 #1000
begin_epoch: 0
end_epoch: 100
# whether flip image
FLIP: false
# whether shuffle image
SHUFFLE: true
# whether use OHEM
ENABLE_OHEM: true
# size of images for each device, 2 for rcnn, 1 for rpn and e2e
BATCH_IMAGES: 4
# e2e changes behavior of anchor loader and metric
END2END: true
# R-CNN
# rcnn rois batch size
BATCH_ROIS: -1
BATCH_ROIS_OHEM: 256
# rcnn rois sampling params
FG_FRACTION: 0.25
FG_THRESH: 0.5
BG_THRESH_HI: 0.5
BG_THRESH_LO: 0.0
# rcnn bounding box regression params
BBOX_REGRESSION_THRESH: 0.5
BBOX_WEIGHTS:
- 1.0
- 1.0
- 1.0
- 1.0
# RPN anchor loader
# rpn anchors batch size
RPN_BATCH_SIZE: 256
# rpn anchors sampling params
RPN_FG_FRACTION: 0.5
RPN_POSITIVE_OVERLAP: 0.5
RPN_NEGATIVE_OVERLAP: 0.4
RPN_CLOBBER_POSITIVES: false
# rpn bounding box regression params
RPN_BBOX_WEIGHTS:
- 1.0
- 1.0
- 1.0
- 1.0
RPN_POSITIVE_WEIGHT: -1.0
# used for end2end training
# RPN proposal
CXX_PROPOSAL: false
RPN_NMS_THRESH: 0.7
RPN_PRE_NMS_TOP_N: 6000
RPN_POST_NMS_TOP_N: 300
RPN_MIN_SIZE: 0
# approximate bounding box regression
BBOX_NORMALIZATION_PRECOMPUTED: true
BBOX_MEANS:
- 0.0
- 0.0
- 0.0
- 0.0
BBOX_STDS:
- 0.1
- 0.1
- 0.2
- 0.2
USE_NEG_CHIPS: false
TEST:
# Maximum number of detections per image
# Set to -1 to disable
MAX_PER_IMAGE: 50
# Whether to do multi-scale inference
SCALES:
# - !!python/tuple [1400, 2000]
- !!python/tuple [800, 1280]
- !!python/tuple [480, 512]
# Number of images per gpu for each scale
BATCH_IMAGES:
- 1
- 1
- 1
# Number of concurrent jobs used for inference
# if greater than 1, the roidb is distributed over
# concurrent jobs to increase throughput
CONCURRENT_JOBS: 2
# Ranges to specify valid proposal length
# in each of the test scale, square area
# would be computed based on the lengths
# to invalidate, -1 means unbounded, use
# -1 everywhere if you want to have all proposals
VALID_RANGES:
# - !!python/tuple [-1,90]
# - !!python/tuple [32,180]
# - !!python/tuple [75,-1]
- !!python/tuple [-1, -1]
- !!python/tuple [-1, -1]
# Use rpn to generate proposal
HAS_RPN: true
# RPN Parameters
RPN_NMS_THRESH: 0.7
RPN_PRE_NMS_TOP_N: 6000
RPN_POST_NMS_TOP_N: 300
RPN_MIN_SIZE: 0
PROPOSAL_NMS_THRESH: 0.7
PROPOSAL_PRE_NMS_TOP_N: 20000
PROPOSAL_POST_NMS_TOP_N: 2000
PROPOSAL_MIN_SIZE: 0
# NMS Parameters
# Whether to apply NMS based on threshold or sigma
NMS: -1 #0.45
NMS_SIGMA: 0.55
# Which epoch of the training be used for testing
TEST_EPOCH: 100
# VISUALIZATION CONFIG
VISUALIZATION_PATH: './debug/visualization'
# Whether to visualize all intermediate scales
# before aggregation (when doing multi-scale inference)
# If False, only final detections are saved to
# VISUALIZATION_PATH
VISUALIZE_INTERMEDIATE_SCALES: false
# PROPOSAL EXTRACTION FLAGS
# If true only would extract proposals
EXTRACT_PROPOSALS: false
# The folder path to be used for saving proposals
PROPOSAL_SAVE_PATH: 'output/proposals'
# Number of proposals extracted per scale
# SCALES and BATCH_IMAGES above would be used to
# Specify scales and number of images per batch for
# each scale, no valid ranges would be applied for
# aggregating proposals
N_PROPOSAL_PER_SCALE: 300
I have struggled for several days but couldn't figure it out.
Hope you can give some advice.
Thanks :)
Hey,
Sniper is an awesome work. However ,FPN is a standard component of object detection for scales with slightly computation increased. Have you tried to combine Sniper with FPN ?
/usr/local/lib/python2.7/dist-packages/h5py/init.py:36: FutureWarning: Conversion of the second argument of issubdtype from float
to np.floating
is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type
.
from ._conv import register_converters as _register_converters
Traceback (most recent call last):
File "demo.py", line 123, in
main()
File "demo.py", line 87, in main
convert=True, process=True)
File "lib/train_utils/utils.py", line 90, in load_param
arg_params, aux_params = load_checkpoint(prefix, epoch)
File "lib/train_utils/utils.py", line 56, in load_checkpoint
save_dict = mx.nd.load('%s-%04d.params' % (prefix, epoch))
File "/home/czn/scc/SNIPER/SNIPER-mxnet/python/mxnet/ndarray/utils.py", line 175, in load
ctypes.byref(names)))
File "/home/czn/scc/SNIPER/SNIPER-mxnet/python/mxnet/base.py", line 149, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [04:49:05] /home/czn/scc/SNIPER/SNIPER-mxnet/src/ndarray/ndarray.cc:1770: Check failed: fi->Read(data) Invalid NDArray file format
Stack trace returned 10 entries:
[bt] (0) /home/czn/scc/SNIPER/SNIPER-mxnet/python/mxnet/../../build/libmxnet.so(dmlc::StackTraceabi:cxx11+0x56) [0x7ff817364586]
[bt] (1) /home/czn/scc/SNIPER/SNIPER-mxnet/python/mxnet/../../build/libmxnet.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x28) [0x7ff817364d38]
[bt] (2) /home/czn/scc/SNIPER/SNIPER-mxnet/python/mxnet/../../build/libmxnet.so(mxnet::NDArray::Load(dmlc::Stream*, std::vector<mxnet::NDArray, std::allocatormxnet::NDArray >, std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > >)+0x204) [0x7ff817586e44]
[bt] (3) /home/czn/scc/SNIPER/SNIPER-mxnet/python/mxnet/../../build/libmxnet.so(MXNDArrayLoad+0xc1) [0x7ff81754c471]
[bt] (4) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7ff82fa2ee40]
[bt] (5) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7ff82fa2e8ab]
[bt] (6) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7ff82fc3e3df]
[bt] (7) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(+0x11d82) [0x7ff82fc42d82]
[bt] (8) python(PyEval_EvalFrameEx+0x578f) [0x4c15bf]
[bt] (9) python(PyEval_EvalCodeEx+0x306) [0x4b9ab6]
What would be the best way to setup a python script to get a detection from a single image from a trained R-FCN-3k model? Thanks!
Hi,In openimages.py ,i want to know the reason why you make gt_roidb's max_overlaps and max_classes only equal boxes' class but not include crowed' class.
Really awesome work!
Do you plan to release an inference script for the openimages2
branch, as well as a pretrained model?
Installation and all compilations went smoothly. However, when run "python demo.py", encountered following error:
*** invalid %N$ use detected ***
Aborted (core dumped)
Further inspection shows a simple test command "import mxnet" gives same error.
Env: ubuntu 16.04.04 (kernel 4.4), CUDA9.0/cudnn 7.0, Nvidia 1080 ti.
Tried another machine with ubuntu 16.04.04 (kernel 4.13) with different CPUs and builds, has exact same error.
Wondering anyone sees same error?
Thanks.
ps: same env runs several mxnet projects without any issue, including Deformable-ConvNet project.
Hello, @mahyarnajibi ! It is possible to test your pretrained model to detect 600 classes? I check branch openimages2
and in config see pretrained: "./data/pretrained_model/CRCNN"
row, but bash scripts/download_pretrained_model.sh
doesn't download anything, but only resnet_mx_101_open-0000
(and imagenet ofc). This is what we need, this model is pretraied for 600 classes? and we need only move(or replace config's row with path) model to CRCNN folder and run python demo.py
with custom config?
Hi, I see that in openimages2 branch, you train for 14 epochs. What are the results?
i test the coco pretrained model, the result is really very good, as openimage v4 has 600 classes, can this pretrained model can share?
hi, I've tried to run the 'demo.py' file, then met the error like:
mxnet.base.MXNetError: [22:56:26] /home/vis/xuxiaqing/code/code_my/video_detection/SNIPER/SNIPER-mxnet/src/operator/nn/batch_norm.cu:527: Check failed: err == cudaSuccess (7 vs. 0) Name: BatchNormalizationUpdateOutput ErrStr:too many resources requested for launch
My cuda version is 8.0, the cudnn version is 5.0. Are there any tips for solving this problem? many thanks~
Traceback (most recent call last):
File "main_train.py", line 64, in
for image_set in image_sets]
File "lib/data_utils/load_data.py", line 29, in load_proposal_roidb
roidb = imdb.gt_roidb()
File "lib/dataset/coco.py", line 135, in gt_roidb
roientry,flag = self._load_coco_annotation(index)
File "lib/dataset/coco.py", line 192, in _load_coco_annotation
if obj['area'] > 0 and x2 >= x1 and y2 >= y1:
KeyError: 'area'
Who have met the same question?Please help me, thanks!!
When I try bash scripts/compile.sh
Here are bugs:
x86_64-linux-gnu-gcc: error: unrecognized command line option ‘-Wdate-time’
x86_64-linux-gnu-gcc: error: unrecognized command line option ‘-fstack-protector-strong’
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
My solution is as follows:
sudo rm -r /usr/bin/gcc
sudo ln -s /usr/bin/gcc-5 /usr/bin/gcc
sudo ln -s /usr/bin/gcc-5 /usr/bin/gcc
sudo rm /usr/bin/x86_64-linux-gnu-gcc
sudo ln -s /usr/bin/gcc-5 /usr/bin/x86_64-linux-gnu-gcc
Hi team,
Another question I have is how to avoid resizing the output image (i.e. output the same image resolution as the input)? One particular example would be for the image below (some random image on Internet), the input resolution is 1023 * 968, however the output resolution is 640 * 480, plus additional white edges. BTW I tried to modify the TEST.SCALES option in the yaml file to a large number but with no luck.
SCALES:
Hi,
Thank you for your repo and paper. This is an awesome work!
I want to ask that in your paper An Analysis of Scale Invariance in Object Detection – SNIP
You mentioned an ensemble operation. You said that:
Since proposals are shared across all networks, we
average the scores and box-predictions for each RoI. During
flipping we average the detection scores and bounding
box predictions.
How to average box predictions for each RoI?
How did you implemented in your code?
Thank you!
loading annotations into memory... Done (t=0.11s) creating index... index created! num_images 8100 COCO_train2014 gt roidb loaded from ./data/cache/COCO_train2014_gt_roidb.pkl append flipped images to roidb loading annotations into memory... Done (t=0.01s) creating index... index created! num_images 900 COCO_val2014 gt roidb loaded from ./data/cache/COCO_val2014_gt_roidb.pkl append flipped images to roidb filtered 0 roidb entries: 18000 -> 18000 add bounding box regression targets bbox target means: [[0. 0. 0. 0.] [0. 0. 0. 0.]] [0. 0. 0. 0.] bbox target stdevs: [[0.1 0.1 0.2 0.2] [0.1 0.1 0.2 0.2]] [0.1 0.1 0.2 0.2] Creating Iterator with 18000 Images Total number of extracted chips: 30025 Done! Traceback (most recent call last): File "main_train.py", line 78, in <module> threads=config.TRAIN.NUM_THREAD, pad_rois_to=400) File "lib/iterators/MNIteratorE2E.py", line 32, in __init__ super(MNIteratorE2E, self).__init__(roidb, config, batch_size, threads, nGPUs, pad_rois_to, False) File "lib/iterators/MNIteratorBase.py", line 33, in __init__ self.get_batch() File "lib/iterators/MNIteratorE2E.py", line 100, in get_batch self.batch = self._get_batch() File "lib/iterators/MNIteratorE2E.py", line 195, in _get_batch processed_list = processed_list.get() File "/usr/lib/python2.7/multiprocessing/pool.py", line 567, in get raise self._value TypeError: 'NoneType' object has no attribute '__getitem__'
could you help me to fix it?
Hello,
I have read the SNIPER paper and something is not clear to me.
Is this training/inference method compatible only with 2 stage detectors like Faster RCNN?
Or can it also be applied to SSD?
Thank you
If I change rpn_post_nms_top_n, the metric will get wrong at
pred_label = pred.asnumpy().reshape(-1, last_dim).argmax(axis=1).astype('int32')
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.