Comments (3)
Kubric uses the same format as the other datasets: a set of query_points in t,y,x format (shape [batch, num_points, 3]), a set of target_points in x,y format (shape [batch, num_points, num_frames, 2]), and an binary occlusion flag (1 if occluded, 0 otherwise; shape [batch, num_points, num_frames]), and the video (scaled between -1 and 1 shape [batch, num_frames, height, width, 3]). To train on something else, you'd need to edit experiment.py to add a new dataset_constructor which will return a new python generator that generates dicts containing the above fields. Then you need to edit the config file to add the desired dataset name and its kwargs to datasets. That is, once you've written the generator, it should only be a few lines of code.
Note that the code is set up for multi-dataset training, meaning that the training class will receive a dict keyed by dataset name, with an example from each dataset. You may need to change the input_key in supervised_point_prediction.py to get it to use the correct dataset.
from tapnet.
Thanks for the reply, that worked well for me. What config parameters did you set for finetuning, such that I could finetune the existing model on the new dataset I added? Right now I'm using the checkpoint.npy
file that is available in the README
, but that already initializes to having run 100k steps. In my config, I have to set the number of training_steps to be something >100k to get it to run. In the paper you mention that for finetuning you ran 5000 steps with 100 warmup steps and a learning rate of 1e-5, how do I set those parameters to be compatible with the existing checkpoint.npy
file? Did you change the weight decay parameters at all?
from tapnet.
Can I use my personal dataset that is not related to any of the ones mentioned above?
from tapnet.
Related Issues (20)
- Re: architecture of ResNet used for track initialization HOT 2
- _pickle.UnpicklingError: Failed to interpret file '../checkpoint/checkpoint.npy' as a pickle HOT 2
- is there an open version of control code of robotap HOT 1
- OnnxExporterError: Unsupported: ONNX export of operator GridSample with 5D volumetric input. HOT 5
- ValueError: All `hk.Module`s must be initialized inside an `hk.transform` HOT 12
- ValueError: converting frame count is not supported. HOT 8
- robotap's query points selection question HOT 5
- Torchscript compatibility HOT 6
- Has anyone implemented it with tensorrt? HOT 1
- None of the algorithms provided by cuDNN heuristics worked; trying fallback algorithms HOT 2
- Pytorch <2.1.0 can't load the checkpoints correctly HOT 1
- Pretrained Weights for Pytorch Version of Online Tapir/BootsTapir HOT 2
- IndexError: boolean index did not match indexed array along dimension 1; dimension is 256 but corresponding boolean dimension is 990 HOT 2
- Training TAPIR PyTorch version script? HOT 7
- BootsTAP Training Dataset HOT 1
- `plot_tracks_v2` has bug when plotting with `trackgroup` argument. HOT 2
- KeyError: 'global_step' When I load the weight of TAPIR HOT 5
- CUDA out of memory issue when using PyTorch weights instead of JAX weights. HOT 2
- pytorch version TAPIR 's training file HOT 1
- Annotation Tool for TAP-VID HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tapnet.