Comments (5)
Hi @luminousking, this is not supported at the moment. In the current implementation we suppose that we have access to the full video.
from dot.
Hi @Soooooda69, you can get considerable speed-ups by playing with the number of initial tracks extracted with CoTracker (this is the time bottleneck in most applications). The flag --num_tracks
sets the total number of initial tracks and --sim_tracks
sets the number of simultaneous tracks. Here are a few examples of the time you can get on the varanus data:
method | num_tracks | sim_tracks | time for initial tracks | time for refinement | total time |
---|---|---|---|---|---|
default | 8192 | 2048 | 1'30 | 0'13 | 1'43 |
parallelize | 8192 | 8192 | 1'13 | 0'13 | 1'26 |
fewer tracks | 2048 | 2048 | 0'25 | 0'13 | 0'38 |
" | 1024 | 1024 | 0'18 | 0'13 | 0'31 |
" | 512 | 512 | 0'13 | 0'13 | 0'26 |
Reducing the number of tracks may affect performance, but it does not seem to be the case for this video:
varanus_512_tracks.mp4
Further speed-ups may be obtained by parallelizing the refinement steps, but it has not been implemented yet.
Online tracking is an interesting feature. I have not implemented it yet either. I don't think it would help with speed much though, but it would allow to do inference on long videos.
from dot.
Thanks for the reply! So, based on the examples, I wonder if it is the case that with even fewer num of tracks, e.g. 200, the bottle neck for total running time would lies in 'time for refinement'. Seems like it takes 13 seconds no matter the time for initial tracks. Then Cotracker might be faster than Dot for smaller tracks case, or do I understand this correct?
from dot.
The goal is to predict a dense field of N tracks from a sparse set of K tracks (extracted with CoTracker). In the example above, N is fixed (N=480*856) which is why the refinement stage always takes the same time (no matter K). Reducing K allows to speed up the first step. However, it should be noted that directly producing the N tracks with CoTracker would take much longer (see paper for details).
As mentioned in my response above, the 'time for refinement' could be reduced by parallelizing this loop:
dot/dot/models/dense_optical_tracking.py
Line 128 in e32c6f7
from dot.
Nice work! I was wondering if Dot supports point tracking for streaming unknown length video?
from dot.
Related Issues (17)
- Model and code HOT 6
- _pickle.UnpicklingError: invalid load key, '<'. HOT 17
- question about the batch size HOT 10
- Failed to save the result HOT 2
- Input format for `tracks_for_queries` mode in the model HOT 4
- inference HOT 11
- Can I add some datasets to train. HOT 10
- AssertionError: Torch not compiled with CUDA enabled HOT 2
- inference with diff video size HOT 2
- The performerance of sparse point tracker HOT 1
- Reproducing the training results HOT 1
- Inference for low gpu and less number of points HOT 5
- Training code for optical flow and point tracking HOT 2
- c/c++ support HOT 1
- Choose Checkpoint HOT 2
- Optical Flow in VFX HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dot.