researchmm / stark Goto Github PK
View Code? Open in Web Editor NEW[ICCV'21] Learning Spatio-Temporal Transformer for Visual Tracking
License: MIT License
[ICCV'21] Learning Spatio-Temporal Transformer for Visual Tracking
License: MIT License
Hi, thanks for your work! Could you pleae provide the Hz value of GOT-10K when evaluating the Stark?
hi, I Haven't used lmdb to read datasets before, could you tell me how to generate lmdb files? thanks
Hi, Thanks for your wonderful work. I notice that Transformer Tracking use the model trained with all datasets(LaSOT, GOT10K, COCO, TrackingNet) to get the evaluation result on GOT-10k test set, and the result is much better than the model trained with GOT10K only.
However, when I use the STARK-S50 pre-trained model(trained on all datasets) in your model zoo to evaluate the GOT-10k test set, I find that the AO is 0.688, which only gains small improvement compared with 0.672
I am confused with this phenomenon. Have you ever tried to evaluate the model trained with all datasets on GOT-10k test set? Or can you kindly explain the reason why there is just little performance gain to use the model trained on all datasets?
thank you for your great work. but I wonder: could you share the code for visualizing attention maps in your paper?
I'm looking forward from you.
非常感谢您公布成果,对我的学习提供了很大的帮助,很感激您的工作,我目前遇到了一些问题,我在执行
python tracking/train.py --script stark_s --config baseline --save_dir . --mode multiple --nproc_per_node 4 时,出现了错误,显示ValueError : Caught ValueError in DataLoader worker process 0
由于设备有限,训练集,我没有加上trackingnet,是否是我没有修改一些配置。望您有空帮我解答,再次非常感谢您。
Thanks for your good job! I have tried to recurring your final results, but I failed。
when i use
python tracking/train.py --script stark_s --config baseline --save_dir . --mode multiple --nproc_per_node 4
There was an error
ValueError : Caught ValueError in DataLoader worker process 0
Due to limited equipment, I did not add trackingnet in training set, whether I did not modify some configuration.I hope you have time to help me out. Thank you very much again.
Hi
I was wondering why you use FrozenBatchNorm2d
in the ResNet backbone for all BatchNorm2d
layers during training instead of using the normal BatchNorm2d
layers.
Did training get unstable? Like an increasing validation loss after some time due to probably to small batch size such that batch wise mean/std are too different to the running mean/std?
Is there a reason to use FrozenBatchNorm2d instead of turning the BN layers in eval mode?
Thanks in advance and nice work.
Thanks for your excellent work, could you please show me how to run video_demo.py with pretrained model, thanks a lot!
Hi! I have one question. When getting the result plots like "Comparisons on LaSOT test set" with other papers' models, we need raw reaults of their. So how to get it?
I have pulled the docker image from docker hub when I run a docker container the container starts and exit after one second.
please include some documentation about the docker running process if I should pass arguments when running the container.
In docker ps-a the container appears as an old container.
if you could also include full documentation about dockerized STARK training, and testing. this will be great.
nice work!could you provide your rawresults about STARK testing on these datasets?UAV,got,lasot,otb
Thanks for your work! I trained the model and want to evaluate it on the GOT10k-Val dataset to see its performance, but only see 'LaSOT', 'otb', 'nfs', 'uav', 'tc128ce' datasets, so how to evaluate on the GOT10k-Val? By the way, what's the difference between analysis_results
and analysis_results_ITP
files?
Hi! When I debug tranining with a single GPU, there is an error that no such file or directory: 'got10k/list.txt'. So what is this?
Hi.
I found that the training process will randomly crashed with RuntimeError: DataLoader worker (pid(s) 36469) exited unexpectedly
, is that normal?
I use the following training command.
python tracking/train.py --script stark_s --config baseline_got10k_only --save_dir . --mode multiple --nproc_per_node 8
thanks!
Hi,I use 8 32g tesla V100 and want to train your model. but the fps is lower than yours. Do you know how many cores/threads do you use every task/PID/ nproc? Thanks!
I am very interested in your work and thank you for your contribution. I tested Stark on vot19, and the configuration file used baseline_R101, but the result was very poor, with an EAO of 0.2. Is this result normal? Have you tested the result of vot2019?
您好,关于您训练好的模型,可以用邮箱分享一份给我吗?我在您github上放的下载链接上下载不了。我的邮箱是:[email protected]
需要的模型有:stark_S50/baseline_got10k_only/STARKS_ep0500.pth.tar,stark_ST50/baseline_got10k_only/STARKST_ep0050.pth.tar
Hi, thanks for sharing the work. I saw that in the model zoo readme file you are mentioning raw results but I couldn't find a link referring to them. Could you give me a hand? Thanks in advance.
Hello!About the implementation of Ao evaluation index of got10k val dataset, which code file is it implemented in?
Hi,
Thanks for your good job!What is the effect of replacing transformer with swin transformer?
Looking forward to your reply~
Hello when I use
python tracking/test.py stark_st baseline_got10k_only --dataset got10k_test --threads 32
python lib/test/utils/transform_got10k.py --tracker_name stark_st --cfg_name baseline_got10k_only
Tested on the got10k data set, the result is less than 20 Sequences, and the oom error occurs.
My hardware platform is 3090
Hi!
You said that "In the encoder-decoder attention module, the target query can attend to all positions on the template and the search region features, thus learning robust representations for the final bounding box prediction." in your paper. How to understand that? It's really abstract for me.
Thanks for your reply!
Hi! How long did the traning take using 8 16gb V100? I use 4 11gb 2080ti to train the model for 500 epoch and it will take 20 days.
hi,
I run several times about vot (python version), but still got the problem:
from lib.test.vot20.stark_vot20 import run_vot_exp
ModuleNotFoundError: No module named 'lib'
It seems not finding the stark project path, though I export it as:
export PYTHONPATH=/home/xxxx/projects/transformer/Stark-main:$PYTHONPATH.
Expected to solve it by inspiring from any of your answers.
Thanks!
作者您好,最近我在用模型验证nfs数据集,nfs数据集使用时的格式是怎样的呢,需要自己编写脚本将原本的zip解压吗,因为在输入命令执行之后提示路径错误,没有anno文件夹。
Hi,
Thanks for your good job! I have tried to recurring your final results, but I failed. Can you send me your checkpoint file of the training results of the first stage (my email address [email protected])? This will help me to find my problems.
Thanks.
It seems that STARK doesn't mention anything about a dynamically updated template (DUT for short) during training procedure, is it a deliberate design or am I missing something?
I reckon that the DUT is actually something like a short-term memory, and it should not be treated equally as a normal template from the first frame by the transformer, so the DUT should be explicitly included in training. However, this is not how STARK has been implemented.
So I'm curious what's the intuition or reasoning behind STARK's current training protocol of dismissing the DUT?
您好,下载的权重都是提示no module named lib.train.admin.local
Hi, thanks for your work. I find from your codes that "shuffle = True" when setting dataloaders. So if the input data is not sequential, how to update template every 200 frames? thanks!
hello
i have follow your steps to convert baseline stark_st2 to onnx successfully,
now i want to convert the onnx to tensorrt model,can you give me some advice?
if possible, please list your relevant environment (tensorrt/cuda/etc .)
thank you sincerely!
Hi, can you provide a tutorial for using lmdb format?
Hi,
Thanks for your great job!I'm recurring the works on recorded video. But there is no progress. Can you give a tutorial how to recurring your job in a video. Thanks a lot.
Hello,
Thank you for your contribution ^^,
I need to know if all the pretrained models here are working on single object tracking,
So I wounder if there's any model working as MOT
你好,我训练时,一直提醒No matching checkpoint file found,请问有没有解决办法。
Hello, I noticed some problems with the implementation of the code that detects the version of torchvision
in utils.misc.py
import torchvision
if float(torchvision.__version__[:3]) < 0.7:
from torchvision.ops import _new_empty_tensor
from torchvision.ops.misc import _output_size
This will have a fatal issue for torchvision>=0.10
python -c "import torchvision; print(torchvision.__version__[:3])"
# output equals 0.1
NVIDIA GeForce RTX 3060 Laptop GPU with CUDA capability sm_86 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_61 sm_70 sm_75 compute_37.
If you want to use the NVIDIA GeForce RTX 3060 Laptop GPU GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/
warnings.warn(incompatible_device_warn.format(device_name, capability, " ".join(arch_list), device_name))
WARNING: You are using tensorboardX instead sis you have a too old pytorch version.
Traceback (most recent call last):
File "tracking/../lib/train/admin/tensorboard.py", line 4, in <module>
from torch.utils.tensorboard import SummaryWriter
File "/home/richard/miniconda3/envs/torch1.7/lib/python3.6/site-packages/torch/__init__.py", line 189, in <module>
_load_global_deps()
File "/home/richard/miniconda3/envs/torch1.7/lib/python3.6/site-packages/torch/__init__.py", line 142, in _load_global_deps
ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL)
File "/home/richard/miniconda3/envs/torch1.7/lib/python3.6/ctypes/__init__.py", line 348, in __init__
self._handle = _dlopen(self._name, mode)
OSError: /home/richard/miniconda3/envs/torch1.7/lib/python3.6/site-packages/torch/lib/../../../../libcublas.so.11: symbol free_gemm_select version libcublasLt.so.11 not defined in file libcublasLt.so.11 with link time reference
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "tracking/video_demo.py", line 9, in <module>
from lib.test.evaluation import Tracker
File "tracking/../lib/test/evaluation/__init__.py", line 1, in <module>
from .data import Sequence
File "tracking/../lib/test/evaluation/data.py", line 3, in <module>
from lib.train.data.image_loader import imread_indexed
File "tracking/../lib/train/__init__.py", line 1, in <module>
from .admin.multigpu import MultiGPU
File "tracking/../lib/train/admin/__init__.py", line 3, in <module>
from .tensorboard import TensorboardWriter
File "tracking/../lib/train/admin/tensorboard.py", line 7, in <module>
from tensorboardX import SummaryWriter
ModuleNotFoundError: No module named 'tensorboardX'
WARNING: You are using tensorboardX instead sis you have a too old pytorch version.
Traceback (most recent call last):
File "tracking/../lib/train/admin/tensorboard.py", line 4, in <module>
from torch.utils.tensorboard import SummaryWriter
File "/home/richard/miniconda3/envs/torch1.7/lib/python3.6/site-packages/torch/__init__.py", line 189, in <module>
_load_global_deps()
File "/home/richard/miniconda3/envs/torch1.7/lib/python3.6/site-packages/torch/__init__.py", line 142, in _load_global_deps
ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL)
File "/home/richard/miniconda3/envs/torch1.7/lib/python3.6/ctypes/__init__.py", line 348, in __init__
self._handle = _dlopen(self._name, mode)
OSError: /home/richard/miniconda3/envs/torch1.7/lib/python3.6/site-packages/torch/lib/../../../../libcublas.so.11: symbol free_gemm_select version libcublasLt.so.11 not defined in file libcublasLt.so.11 with link time reference
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "tracking/video_demo.py", line 9, in <module>
from lib.test.evaluation import Tracker
File "tracking/../lib/test/evaluation/__init__.py", line 1, in <module>
from .data import Sequence
File "tracking/../lib/test/evaluation/data.py", line 3, in <module>
from lib.train.data.image_loader import imread_indexed
File "tracking/../lib/train/__init__.py", line 1, in <module>
from .admin.multigpu import MultiGPU
File "tracking/../lib/train/admin/__init__.py", line 3, in <module>
from .tensorboard import TensorboardWriter
File "tracking/../lib/train/admin/tensorboard.py", line 7, in <module>
from tensorboardX import SummaryWriter
File "/home/richard/miniconda3/envs/torch1.7/lib/python3.6/site-packages/tensorboardX/__init__.py", line 5, in <module>
from .torchvis import TorchVis
File "/home/richard/miniconda3/envs/torch1.7/lib/python3.6/site-packages/tensorboardX/torchvis.py", line 11, in <module>
from .writer import SummaryWriter
File "/home/richard/miniconda3/envs/torch1.7/lib/python3.6/site-packages/tensorboardX/writer.py", line 34, in <module>
import torch
File "/home/richard/miniconda3/envs/torch1.7/lib/python3.6/site-packages/torch/__init__.py", line 189, in <module>
_load_global_deps()
File "/home/richard/miniconda3/envs/torch1.7/lib/python3.6/site-packages/torch/__init__.py", line 142, in _load_global_deps
ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL)
File "/home/richard/miniconda3/envs/torch1.7/lib/python3.6/ctypes/__init__.py", line 348, in __init__
self._handle = _dlopen(self._name, mode)
OSError: /home/richard/miniconda3/envs/torch1.7/lib/python3.6/site-packages/torch/lib/../../../../libcublas.so.11: symbol free_gemm_select version libcublasLt.so.11 not defined in file libcublasLt.so.11 with link time reference
您好!您上传的包有问题,无法解压!
https://drive.google.com/file/d/1qOQRfaRMbQ2nmgX1NFjoQHfXOAn609QM/view
Hi,
I notice you are calculating MACS in your profile_model.py. Did you report the MACS value as the GFLOPs?
The reported GFLOPs in your paper does not match my calculation.
I tried to replace the backbone or bottleneck with other convolutional layers with BN and ReLU. The training result is good but the validation and testing results are very low.
I have already set the network.eval() while testing.
Will BN and ReLU affect the results of model training and testing?
Should I change BN to LN, even after convolutional layers?
Or it is possible to use only the convolutional layer without BN and RELU?
Hi,
Which license this repository uses?
Thanks,
Omri.
Hi! Could you please tell what is the meaning of "lmdb" in class EnvironmentSettings "self.lasot_lmdb_dir"? I guess it is the dir of val dataset of lasot?
首先感谢您优秀的工作
请问在哪里能下载swin_base_patch4_window12_384_22k.pth呢?我没有能找到任何链接。
Hi! Is it possible to train Stark on 4 11GB RTX 2080Ti gpus?
Thanks for sharing! I have some questions around the choice of template. From the paper you cropped 2^2 times the ground truth bounding box, rather than just the actual target bounding box resized to square image. My questions are:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.