ymzis69/hybridsort

[AAAI2024]Hybrid-SORT: Weak Cues Matter for Online Multi-Object Tracking

hybridsort's Issues

The parameters are not used?

yolox.core.trainer:before_train:124 - args: Namespace(experiment_name='yolox_x_ablation', name=None, dist_backend='nccl', dist_url=None, batch_size=1, devices=1, local_rank=0, exp_file='exps/example/mot/yolox_x_ablation.py', resume=False, ckpt=None, start_epoch=21, num_machines=1, machine_rank=0, fp16=True, occupy=False, opts=['fp16', '-o', '-c', 'pretrained/yolox_x.pth', '--resume', '/home/ubuntu/HybridSORT/YOLOX_outputs/yolox_x_ablation/last_epoch_ckpt.pth.tar'])
this is log, the parameters in opts do not work?

Hello, could you please let me know whether the value of "Hiou" is greater than one or less than one? Is there a potential error in Formula 6 as presented in the paper? Does it conflict with the description in Figure Four?

Hello, could you please let me know whether the value of "Hiou" is greater than one or less than one? Is there a potential error in Formula 6 as presented in the paper? Does it conflict with the description in Figure Four?

What does train/val/test_segmap mean?

Thank you for your work. Could tell me what does the test_segmap or val_segmap mean in DanceTrackDataset, which not appear in OC-Sort.

MOT20-test

This error occurs when I am testing MOT20, tools/run_hybrid_sort_dance.py. May I ask what is the problem?

Weighted HMIoU

Thanks for your work! The idea of using weak cues helps me a lot with my work in practice. Not limited to your modifications, I can find my own weak cues in my scenario, inspired by your idea, or modify your weak cues according to my dataset.

For example, insteading multiplying HIoU and IoU to get HMIoU, I use weighted average. This is because I observe that Weighted HMIoU is quite consistent and I can find a certain threshold easily between Weighted HMIoU within real tracklets & among false postitive cases.

Furthermore, I have implemented the idea of changing the weights according to the velocity direction, i.e. if a track is moving horizontally, weight of HIoU is quite large; while if he is moving vertically (e.g. towards the camera), the weight of HIoU should be set lower.

Hope that my comment is helpful!

hybrid_sort.py

Hi there. I really appreciate your great job.

While i'm read your code, I find some trouble in your code.

HybridSORT/trackers/hybrid_sort_tracker/hybrid_sort.py

Line 341 in 396f8d3

trk[:] = [pos[0][0], pos[0][1], pos[0][2], pos[0][3], kalman_score, simple_score[0]]

I think this line need to be change to

trk[:] = [pos[0][0], pos[0][1], pos[0][2], pos[0][3], kalman_score[0], simple_score[0]]

Because trackers.predict function returns List of kalman scores.

@ymzis69

Problem with onnxruntime==1.8.0 when running repository on Colab

Running your repository on Colab runs into an error when requirements are being installed:

ERROR: Could not find a version that satisfies the requirement onnxruntime==1.8.0 (from versions: 1.12.0, 1.12.1, 1.13.1, 1.14.0, 1.14.1, 1.15.0, 1.15.1)
ERROR: No matching distribution found for onnxruntime==1.8.0

Is it possible to use another version of onnxruntime?

How can we use official YOLOX as the detector?

I trained a weight from the official YOLOX (https://github.com/Megvii-BaseDetection/YOLOX), and put it in the self.ckpt. However, when I run the evaluation code using run_hybrid_sort_dance.py, the generated detection results are empty. It works well when I used the trained weight using tools/train.py from HybridSORT.

Can we use the trained weight from official YOLOX? Or some other modifications should be made.

Reid can not work

When I try to infer demo with reid by:
python3 tools/demo_track.py --demo_type image -f exps/example/mot/yolox_dancetrack_val_hybird_sort_reid.py -c pretrained/bytetrack_dance_model.pth.tar --fp16 --fuse --save_result

I foud a wrong information:
Skip loading parameter 'heads.weight' to the model due to incompatible shapes: (12353, 2048) in the checkpoint but (0, 2048) in the model! You might want to double check if this is expected.

I have already download reid model and yolox model, and I found the reid features always be [0. ,0. ,0. ... 0.]

What is the running speed of the method described in this article? Can it be deployed on Jetson NX hardware for real-time tracking? What might be the final FPS (Frames Per Second)?thanks

Hello, can this method be integrated into FairMOT?

Mismatched Paths for "val_seqmap.txt" in README.md and "run_hybrid_sort_dance.py"

Mismatched paths for val_seqmap.txt.

The path for val_seqmap.txt in README.md is datasets/dancetrack/val_seqmap.txt.

However, val_seqmap.txt in run_hybird_sort_dance.py is datasets/dancetrack/val/val_seqmap.txt.

The usage of Reid module

Is there any thing need to notice when use Reid module?
I try to do tracking in my own datasets, and I found the performance for HybirdSort are always a little better than HybirdSort+reid. In my own datasets, person are wear with different clothes so I cannot figure out why it happened.

Some questions about paper and code

I have recently read your paper and code, thanks for your excellent work. May I ask some questions?

1. In the paper, it is written: "we extend the fixed time interval of 3 frames to the stack of multiple intervals ranging from 1 to 3." However, In the code, I found that only the velocity direction of the trajectory is accumulated in here, the velocity direction of the previous observation to new detection is just still fixed time interval of 3 frames in here. Is it specifically designed like this or a mistake?
2. I found the code for simple linear prediction in here, the returned value simple_score is only used for the second stage association by BYTE. The predicted score in first stage association is still using the Kalman filter. Can I understand it this way: the score predicted by Kalman is used for the first stage association, and the score predicted by simple linear prediction is used for the second stage association.

Looking forward to your reply, thank you!

demo_track

hello,there is a bug in demo_track.py when i ran "python3 tools/demo_track.py --demo_type video -f exps/example/mot/yolox_dancetrack_val_hybrid_sort_reid.py -c pretrained/ocsort_dance_model.pth.tar --path xxxx --fp16 --fuse --save_result"

File "tools/demo_track.py", line 243, in imageflow_demo
outputs, img_info = predictor.inference(frame, timer)
ValueError: too many values to unpack (expected 2)
how can i solve it

"Hybird" or "Hybrid"?

Which name is correct "Hybird-SORT" or "Hybrid-SORT"?

The lap module requires numpy version 1.19 or lower.

The python module lap for linear assignment problem is required numpy-1.19 or lower, but current installation will be install probably latest of numpy.
So had better fix how to install numpy.

And also, current installation for pandas will be remove older numpy (such as 1.19) and install newer numpy, so should also fix it.

Question about the publication of the paper

How soon will Hybrid-SORT be published in AAAI?

CX2, CY2 = tracks[:,0], tracks[:,1] IndexError: index 1 is out of bounds for axis 1 with size 1

I am feeding Yolov8 detections to HybridSORT but I get the following error:

IndexError: index 1 is out of bounds for axis 1 with size 1

here:

https://github.com/ymzis69/HybirdSORT/blob/6101e1b225cb4b860f914049f9cccc0cd4538075/trackers/hybird_sort_tracker/association.py#L433

Am I missing something obvious?

When running demo_track.py, an error is reported:ModuleNotFoundError: No module named 'yolox_dancetrack_val_Hybrid_sort'

The detailed error is reported as:
Traceback (most recent call last):
File "d:\mot_projects\hybirdsort\yolox\exp\build.py", line 13, in get_exp_by_file
current_exp = importlib.import_module(os.path.basename(exp_file).split(".")[0])
File "C:\Users\liguo\miniconda3\envs\deepsort\lib\importlib_init_.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'yolox_dancetrack_val_Hybrid_sort'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "tools/demo_track.py", line 351, in
exp = get_exp(args.exp_file, args.name)
File "d:\mot_projects\hybirdsort\yolox\exp\build.py", line 51, in get_exp
return get_exp_by_file(exp_file)
File "d:\mot_projects\hybirdsort\yolox\exp\build.py", line 16, in get_exp_by_file
raise ImportError("{} doesn't contains class named 'Exp'".format(exp_file))
ImportError: exps/example/mot/yolox_dancetrack_val_Hybrid_sort.py doesn't contains class named 'Exp'

Hopefully someone can help with this, thanks！！！

The problem still exists. How long will it take for this bug to be fixed？

          Sorry, this was our mistake because we uploaded the wrong version of README.md. Please run the code according to this version of the README.md: https://github.com/ymzis69/HybirdSORT/tree/6101e1b225cb4b860f914049f9cccc0cd4538075, as ‘yolox_dancetrack_val_Hybrid_sort.py’ is not present in our exp file. We will have this bug fixed within the next two days. We apologize again for any inconvenience this may have caused you.

Originally posted by @ymzis69 in https://github.com/ymzis69/HybirdSORT/issues/13#issuecomment-1702393530

关于用自己的数据集训练detector的问题

您好，我想请教一下iou_loss和cls_loss的计算代码是在什么位置呀？我目前使用自己的数据集执行training时这两个数据始终为0.并且best AP也为0。非常感谢

Retrieving lost tracklet information

Hey, great work. I wanted some information regarding the lost tracks.
Referring to BoTSORT implementation, here you can see that lost tracklet information can also be used if required.
Coming to HybridSORT, I see here you are removing the dead tracklets. Do these two things mean the same, if so, can I use the dead tracklet information assuming it is the lost tracklet as in BOT-SORT's case?

I made HybridSORT available as a pluggable component for object detectors

Thank for your amazing work. Feel free to check it out here: https://github.com/mikel-brostrom/yolo_tracking

Skip loading parameter 'heads.veight' to the model

Hello, when I was trying to run '--demo_type image -f G:/pycharmprojects/pythonProject/HybridSORT-master/exps/example/mot/yolox_x_mix_mot20_ch_hybrid_sort_reid.py -c G:/pycharmprojects/pythonProject/HybridSORT-master/pretrained/ocsort_x_mot20.pth.tar --path
E:/datasets/MOT20/test/MOT20-04/img1 --fuse --save_result'
a warning occured.

I don't know why this warning occurs, how to solve it.

self-trained detector

If I want to use a self-trained detector, what should I modify?

Hey! May I ask in which journal this article has been published?

Possibility to use YoloV7 and for multi-class setting?

Is it possible to use YoloV7 (official repository: https://github.com/WongKinYiu/yolov7/tree/main) instead of YoloX?

Furthermore, (irrespective of using YoloX or YoloV7) I have a custom-trained object detector with two classes, and I would like to run the tracking on each class individually (multi-class tracking) as it avoids label switching between classes. Is there a possibility to run HybridSORT in a multi-class setting?

Adjusting max_age parameter

Hey, we had a discussion last time in this issue.
I doubted the max_age parameter. Just wanted to know the value 30 is 30 frames if I am not wrong. If so how can I adjust the final removal of lost tracks according to the FPS of the incoming stream? I have another value of FPS for the incoming stream.

segmentation fault (core dumped)

首先对大佬的工作点赞加拜服！科研小菜鸡求问大佬，我没有训练dectector，直接进行tracking，但是为什么总是报segmentation fault (core dumped)错误啊，搜了很多解决方案都不太可，望大佬空闲时能指点指点。谢谢大佬啦！

为什么ReID的提升效果这么好？

你好，请问ReID的提升效果好是因为使用了更先进的ReID还是什么原因呢。我想加入到我的方法中。谢谢

How can I use it on Android?

Suppose I have detector running on Android already, how can I run HybridSORT on top of it? Any suggestion on approaches?

About the use of detectors

hello, this is a wonderful work! I read the code for the trace process in detail, but ran into a problem. what should I do if I want to use the ckt_file variable in the image below? What detector did you use to get the data? Looking forward to your reply.
Best wishes.