nvlabs/dancing2music

dancing2music's Issues

Unicode Decode Error with input audio file - preprocess module

Hi,

I'm trying to execute the demo.py according to your specifications, but I'm getting the following warning and errors from preprocess module:

preprocess: warn: defaulting content type for 'demo/demo.wav' to 'Text'

File "demo.py", line 70, in
p.preprocess(infile, outfile)
File "/.local/lib/python3.6/site-packages/preprocess.py", line 417, in preprocess
lines = fin.readlines()
File "/usr/lib/python3.6/codecs.py", line 321, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xaf in position 5: invalid start byte

Is the audio file supposed to be transformed or at least internally considered as text? Is this transformation related to the UnicodeDecodeError? @HsinYingLee

What version are you use preprocess package

Very interesting job!
What version are you use preprocess package？
I'm don't install preprocess1.1.0

when to release the code?

Hey there, your work is so interesting! So when shall you release the source code?

How did you achieve Photo-Realisitc？

Very interesting job!
And could you tell me how did you achieve Photo-Realisitc? Any related projects? Thanks a lot.

Output video length not match the input audio length

Hi,
After modifying the output feature number to 28 in mfcc_rev.py, which is not the correct way to get the feature according to the paper, I can successfully execute demo.py.
I run demo.py and use a 4:02 .wav audio file as input. But the program outputs 72096 frames, which makes the output video length to be 1:20:xx under fps=15 setting.
Why is the length of output video much longer than the length of input audio?

Any thought on this will be helpful. Thanks!

Plan to release Pretrained model and dataset?

FileNotFoundError: [WinError 3] The system cannot find the path specified: './logs\\Decomp' & demo.py: error: argument --thr: invalid int value: 'THR'

When Run any of the following codes , I encounter errors .
Help fix the error ...

python train_decomp.py --name Decomp

FileNotFoundError: [WinError 3] The system cannot find the path specified: './logs\Decomp'

&

python train_comp.py --name Decomp --decomp_snapshot DECOMP_SNAPSHOT

FileNotFoundError: [WinError 3] The system cannot find the path specified: './logs\Decomp'

&

python demo.py --decomp_snapshot DECOMP_SNAPSHOT --comp_snapshot COMP_SNAPSHOT --aud_path AUD_PATH --out_file OUT_FILE --out_dir OUT_DIR --thr THR

demo.py: error: argument --thr: invalid int value: 'THR'

Demo.py unrecognized arguments

when I run demo.py, it shows the problem:
demo.py: error: unrecognized arguments: --comp_snapshot snapshot/Stage2.ckpt

evaluation code

Hi, thank you so much for your impressive work.

Your work is very impressive. Is there any possibility that you could make the evluation code public especially for the rhythm hit part.

Thank you so much.

Help! How do I run the code?

I'm not very fluent in python but want to use this.

3D points

Is it possible to extract the 3D coordinates of each joints ?

How to detect kinematic beats from Ddiff?

How to detect kinematic beats from Ddiff?

error with trian_decomp.py

demo.py can work well by using the pre-trained mode, but when I tried train_decomp.py I met with:

[Errno 2] No such file or directory: '../data\zumba/zfwOESZ8CkQ/19/unit/1.npy'

How to detect kinematic beats?

@tmbdev @bkhailany @mjgarland @dumerrill @sdalton1
Hi，thx to share such an outstanding work. I m interested in kinematic beats detection, can you share this work/code, Thank you a lot

What is the description of the data structure?

I have extracted data.zip but I don't understand the data structure.
What is the difference between the ballet_unit, ballet_unitseq3 and ballet_unitseq4 lists?
Are you able to share the original video sources rather than the numpy arrays?

aud_3cls.ckpt is needed！！

I got IOError: [Errno 2] No such file or directory: './data/stats/aud_3cls.ckpt', while @HsinYingLee said that, The ckpt is under the downloaded "stats" folder.
Is there somebody can share the aud_3cls.ckpt?
Thanks.

Address failure https://www.dropbox.com/s/1c7s7rn7z3pvhp9/Model_MY.zip

Address failure https://www.dropbox.com/s/1c7s7rn7z3pvhp9/Model_MY.zip

Re upload thank you

Which ckpt file should I use？

train create three hundred ckpt files。
Which ckpt file should I use？

preprocess mudule error

Hi！
I am a student of ZJU. I want to show your paper in a course. But the preprocess mudule doesn't work even though I tried many times. Could you share the preprocess module with me? Thanks a lot!

Details about FID evaluation

Hi, your work is very interesting. I'd like to know how you evaluate FID on generated dances.
In table 1, you evaluated FID on real dances and generated dances.
For real dances, do you evaluate FID on test dance data and training dance data?
For generated dances, do you also evaluate FID on generated dances and training dance data?

where is the dataset?

Hello,

The paper says: Finally, we provide a large-scale paired music and dance dataset, which is available along with the source code and models at our website. Website links to this github. But I can't find any dataset link in the repo. Ideally I'd hope to see the link in the README.md

Thanks in advance for clarification

Missing aud_3cls.ckpt

IOError: [Errno 2] No such file or directory: './data/stats/aud_3cls.ckpt'

Training data structure

Hi @HsinYingLee

Could you share that how the audio and video data are encoded into the training set provided?
I went through the data.py, but this file is just for reading the data which has already been encoded.
If I would like to make my own custom dataset, what format should I follow?
Also, what is the relation among indexes of .npy filenames in these folders: data////aud, data////unit, data////unitseq3, data////unitseq4?

Thank you!

FileNotFoundError in Dancing2Application

Hi,

I have Cloned the Dancing2Music application and Installed all the needed Dependency packages also but, I am Facing this below Error:-

Can I know, how I can resolve this Issue?

chaithra@chaithra-desktop:~/Dancing2Music$ python3 train_decomp.py --name Decomp
2022-11-22 18:19:21.235734: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.2
WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
Traceback (most recent call last):
File “train_decomp.py”, line 41, in
os.mkdir(args.log_dir)
FileNotFoundError: [Errno 2] No such file or directory: ‘./logs/Decomp’

Is that souces has some issue ?

or

If there is a proper documentation to install the application

Operands could not be broadcast together

I tried executing the command in README and got the following error:

Traceback (most recent call last):
File "demo.py", line 128, in
aud[j] = (aud[j] - mean_aud) / std_aud
ValueError: operands could not be broadcast together with shapes (15,) (28,)