sony/nnabla-examples

Neural Network Libraries https://nnabla.org/ - Examples

nnabla-examples's Issues

Output Result

Ehi :) how can i get only final video generate as output instead of grid with source image and input video? Thx

mnist classification raises runtime error

When running mnist classification, "modifying data is prohibited" error occurs in v1.20.0 and later.

nnabla-examples/mnist-collection$ python classification.py --context=cudnn
2021-08-04 09:00:00,652 [nnabla][INFO]: Initializing CPU extension...
2021-08-04 09:00:00,974 [nnabla][INFO]: Running in cudnn
2021-08-04 09:00:00,028 [nnabla][INFO]: Initializing CUDA extension...
2021-08-04 09:00:00,063 [nnabla][INFO]: Initializing cuDNN extension...
Traceback (most recent call last):
  File "classification.py", line 238, in <module>
    train()
  File "classification.py", line 154, in train
    pred = mnist_cnn_prediction(image, test=False, aug=args.augment_train)
  File "classification.py", line 59, in mnist_lenet_prediction
    c1 = F.relu(F.max_pooling(c1, (2, 2)), inplace=True)
  File "<relu>", line 3, in relu
  File "~/envs/nnabla-20/lib/python3.6/site-packages/nnabla/function_bases.py", line 999, in relu
    return F.ReLU(ctx, inplace)(x, n_outputs=n_outputs, auto_forward=get_auto_forward(), outputs=outputs)
  File "function.pyx", line 317, in nnabla.function.Function.__call__
  File "function.pyx", line 295, in nnabla.function.Function._cg_call
RuntimeError: value error in check_data_inplace
nnabla/src/nbla/computation_graph/function.cpp:83
Failed `input->allow_modify_data()`: Modifying data is prohibited by the parent function of the 0-th input data of 'ReLUCudaCudnn' (depth=3). (Parent is 'MaxPoolingCudaCudnn').

FlowNet

Hello,
is there an example of FlowNet (Correlation) implementation with nnabla ? Or another simple example for optical flow estimation ?

Regards
Armin

Query: StyleGAN2 latent_seed

Respected sir
Thank you for sharing the projects and the pre-trained models. I have been trying to use the StyleGAN2 notebook. I want to generate multiple random samples, but the latent_seed command is limiting the results. I wanted to ask how to generate a grid of random results.

I am sharing the link to the cell I want to refer to here.

ONNX Operators Implementation

• Please redirect me to the kernel level code (Implementation) of ONNX Operators by nnabla.
• Please help me in understanding at nnabla from developer perspective.

TecoGAN

What were the datasets used to train the pretrained model used in Colab?

Error running FOM colab, animated.py

2023-10-17 10:57:05,909 [nnabla][INFO]: Initializing CPU extension...
usage: animate.py [-h] [--config CONFIG] [--params PARAMS] [--source SOURCE] [--driving DRIVING]
[--out-dir OUT_DIR] [--context {cudnn,cpu}] [--output-png] [--fps FPS]
[--only-generated] [--detailed] [--full] [--adapt-movement-scale]
[--unuse-relative-movement] [--unuse-relative-jacobian]
animate.py: error: unrecognized arguments: \

---

NameError Traceback (most recent call last)

in <cell line: 2>()
1 get_ipython().system('python animate.py --source imgs/sample_src.png --driving imgs/sample_drv.mp4 --adapt-movement-scale --fps 24 \')
----> 2 --detailed --full

NameError: name 'detailed' is not defined

Could find no file with path

when i run:

!python generate.py --model tecogan_model.h5 --input-dir-lr frames/input_video/ --output-dir results/user
!ffmpeg -i results/user/output_frame_%04d.png -r 24/1 -y user_video_hr.mp4

I get the following error:

2022-10-11 05:55:28,867 [nnabla][INFO]: Initializing CPU extension...
/content/nnabla-examples/utils
2022-10-11 05:55:29,335 [nnabla][ERROR]: Extension cudnn does not exist.
Traceback (most recent call last):
File "generate.py", line 41, in
ctx = get_extension_context('cudnn')
File "/usr/local/lib/python3.7/dist-packages/nnabla/ext_utils.py", line 97, in get_extension_context
mod = import_extension_module(ext_name)
File "/usr/local/lib/python3.7/dist-packages/nnabla/ext_utils.py", line 50, in import_extension_module
raise e
File "/usr/local/lib/python3.7/dist-packages/nnabla/ext_utils.py", line 46, in import_extension_module
return importlib.import_module('.' + ext_name, 'nnabla_ext')
File "/usr/lib/python3.7/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1006, in _gcd_import
File "", line 983, in _find_and_load
File "", line 967, in _find_and_load_unlocked
File "", line 677, in _load_unlocked
File "", line 728, in exec_module
File "", line 219, in _call_with_frames_removed
File "/usr/local/lib/python3.7/dist-packages/nnabla_ext/cudnn/init.py", line 19, in
import nnabla_ext.cuda
File "/usr/local/lib/python3.7/dist-packages/nnabla_ext/cuda/init.py", line 131, in
load_shared_from_error(err)
File "/usr/local/lib/python3.7/dist-packages/nnabla_ext/cuda/init.py", line 67, in load_shared_from_error
raise err
File "/usr/local/lib/python3.7/dist-packages/nnabla_ext/cuda/init.py", line 122, in
from .init import (
ImportError: libcudart.so.10.0: cannot open shared object file: No such file or directory
ffmpeg version 3.4.11-0ubuntu0.1 Copyright (c) 2000-2022 the FFmpeg developers
built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)
configuration: --prefix=/usr --extra-version=0ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
libavutil 55. 78.100 / 55. 78.100
libavcodec 57.107.100 / 57.107.100
libavformat 57. 83.100 / 57. 83.100
libavdevice 57. 10.100 / 57. 10.100
libavfilter 6.107.100 / 6.107.100
libavresample 3. 7. 0 / 3. 7. 0
libswscale 4. 8.100 / 4. 8.100
libswresample 2. 9.100 / 2. 9.100
libpostproc 54. 7.100 / 54. 7.100

[image2 @ 0x555654e34000] Could find no file with path 'results/user/output_frame_%04d.png' and index in the range 0-4
results/user/output_frame_%04d.png: No such file or directory

ValueError: numpy.ndarray size changed

Dear experts,

I've tried to execute the demo of StyleGAN2 on Google Colab according to the following link.

https://arxiv.org/abs/1912.04958

However, I've faced the following error in the step "Get the pretrained weights".

ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

Do you have any idea how to solve the above error.
Thank you very much for your kind support in advance.

Best regards,

Record Motion Reference Video part

Hi ! when i record the video from my own computer（about 5 seconds）
then i run !python animate.py --source $input_img
--driving video.mp4
--adapt-movement-scale --fps 24
to realize the function of first order motion model.
After that , i find The video has been extended to 140 seconds , do you know how to fix it?

Failed to train using gpu

a ModuleNotFoundError was thrown when performing the following commands:

cd cifar10-100-collection
python classification.py -n cifar100_resnet23

2020-05-02 06:43:17,572 [nnabla][INFO]: Initializing CPU extension...
2020-05-02 06:43:17,866 [nnabla][ERROR]: Extension `cudnn` does not exist.
Traceback (most recent call last):
  File "classification.py", line 158, in <module>
    train()
  File "classification.py", line 60, in train
    extension_module, device_id=args.device_id, type_config=args.type_config)
  File "/home/tester/anaconda3/envs/nnabla/lib/python3.7/site-packages/nnabla/ext_utils.py", line 97, in get_extension_context
    mod = import_extension_module(ext_name)
  File "/home/tester/anaconda3/envs/nnabla/lib/python3.7/site-packages/nnabla/ext_utils.py", line 50, in import_extension_module
    raise e
  File "/home/tester/anaconda3/envs/nnabla/lib/python3.7/site-packages/nnabla/ext_utils.py", line 46, in import_extension_module
    return importlib.import_module('.' + ext_name, 'nnabla_ext')
  File "/home/tester/anaconda3/envs/nnabla/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'nnabla_ext.cudnn'

CenterNet mixed-precision training cannot work well with specific cuDNN versions

CenterNet mixed-precision training cannot work well with specific cuDNN versions.

How to reproduce

• Revision: 859d9d4
• Branch: master
• GPUs: V100
• Use nnabla/nnabla-ext-cuda-multi-gpu:py310-cuda110-mpi3.1.6-v1.34.0 as the base image and install the necessary packages. (see https://github.com/sony/nnabla-examples/blob/master/object-detection/centernet/requirements.txt)
• The CUDA/cuDNN version of the above image is cuda=11.0.3, CUDNN_VERSION=8.0.5.39
• Run the following command:

python src/main.py ctdet --config_file=cfg/resnet_18_coco_mp.yaml --data_dir path_to_coco_dataset

Error messages

2023-03-02 06:18:26,839 [nnabla][INFO]: Using DataIterator
2023-03-02 06:18:26,865 [nnabla][INFO]: Creating model...
2023-03-02 06:18:26,865 [nnabla][INFO]: {'hm': 80, 'wh': 2, 'reg': 2}
2023-03-02 06:18:26,865 [nnabla][INFO]: batch size per gpu: 24
[Train] epoch:0/140||loss: -0.0000, hm_loss:245.3517, wh_loss: 28.8467, off_loss: 28.8467, lr:1.00e-04, scale:4.00e+00:   0%|  
[Train] epoch:0/140||loss: -0.0000, hm_loss:245.3517, wh_loss: 28.8467, off_loss: 28.8467, lr:1.00e-04, scale:4.00e+00:   0%|  
[Train] epoch:0/140||loss:299.5544, hm_loss:296.1249, wh_loss: 29.4914, off_loss: 29.4914, lr:1.00e-04, scale:4.00e+00:   0%|  
[Train] epoch:0/140||loss:299.5544, hm_loss:296.1249, wh_loss: 29.4914, off_loss: 29.4914, lr:1.00e-04, scale:4.00e+00:   0%|  
[Train] epoch:0/140||loss:     nan, hm_loss:     nan, wh_loss: 30.1704, off_loss: 30.1704, lr:1.00e-04, scale:4.00e+00:   0%|  
[Train] epoch:0/140||loss:     nan, hm_loss:     nan, wh_loss: 30.1704, off_loss: 30.1704, lr:1.00e-04, scale:4.00e+00:   0%|  
[Train] epoch:0/140||loss:     nan, hm_loss:     nan, wh_loss: 21.1151, off_loss: 21.1151, lr:1.00e-04, scale:4.00e+00:   0%|  
[Train] epoch:0/140||loss:     nan, hm_loss:     nan, wh_loss: 21.1151, off_loss: 21.1151, lr:1.00e-04, scale:4.00e+00:   0%|  
[Train] epoch:0/140||loss:     nan, hm_loss:     nan, wh_loss: 24.2714, off_loss: 24.2714, lr:1.00e-04, scale:4.00e+00:   0%|  
[Train] epoch:0/140||loss:     nan, hm_loss:     nan, wh_loss: 24.2714, off_loss: 24.2714, lr:1.00e-04, scale:4.00e+00:   0%|  
[Train] epoch:0/140||loss:     nan, hm_loss:     nan, wh_loss: 21.7357, off_loss: 21.7357, lr:1.00e-04, scale:4.00e+00:   0%|  
[Train] epoch:0/140||loss:     nan, hm_loss:     nan, wh_loss: 21.7357, off_loss: 21.7357, lr:1.00e-04, scale:4.00e+00:   0%|          | 6/4929 [00:06<1:33:43,  1.14s/it]^C

or

2023-03-02 05:47:38,953 [nnabla][INFO]: Using DataIterator
2023-03-02 05:47:38,959 [nnabla][INFO]: Creating model...
2023-03-02 05:47:38,959 [nnabla][INFO]: {'hm': 80, 'reg': 2, 'wh': 2}
2023-03-02 05:47:38,964 [nnabla][INFO]: batch size per gpu: 32
^M  0%|          | 0/3697 [00:00<?, ?it/s]^M  0%|          | 0/3697 [00:04<?, ?it/s]
Traceback (most recent call last):
  File "nnabla-examples/object-detection/centernet/src/main.py", line 147, in <module>
    main(opt)
  File "nnabla-examples/object-detection/centernet/src/main.py", line 112, in main
    _ = trainer.update(epoch)
  File "nnabla-examples/object-detection/centernet/src/lib/trains/ctdet.py", line 191, in update
    total_loss, hm_loss, wh_loss, off_loss = self.compute_gradient(
  File "nnabla-examples/object-detection/centernet/src/lib/trains/ctdet.py", line 178, in compute_gradient
    return self.compute_gradient(data)
  File "nnabla-examples/object-detection/centernet/src/lib/trains/ctdet.py", line 178, in compute_gradient
    return self.compute_gradient(data)
  File "nnabla-examples/object-detection/centernet/src/lib/trains/ctdet.py", line 178, in compute_gradient
    return self.compute_gradient(data)
  [Previous line repeated 7 more times]
  File "nnabla-examples/object-detection/centernet/src/lib/trains/ctdet.py", line 175, in compute_gradient
    raise RuntimeError(
RuntimeError: Something went wrong with gradient calculations.
--------------------------------------------------------------------------

How to solve

Using a newer cuDNN version solved this issue.

• Use nnabla/nnabla-ext-cuda-multi-gpu:py310-cuda116-mpi3.1.6-v1.34.0 as the base image and install the necessary packages. (see https://github.com/sony/nnabla-examples/blob/master/object-detection/centernet/requirements.txt)
• The CUDA/cuDNN version of the above image is cuda=11.6.0, CUDNN_VERSION=8.4.0.27

checkpoint of SLEGAN models

Is implemented the way to save the chekpoints of the models in SLEGAN? The train.py need a --model-load-path but dont work with the GenIter.h5 file generated. There is a solution?

I would like to ask why my reasoning results are all black graphs?

python generate.py generate.config=F:\herong\diffusion-models\config_cifar10_32x32.yaml generate.h5=F:\herong\diffusion-models\params_cifar.h5
Hi, thank you very much for your work, I would like to ask why my reasoning results are all black graphs?

First order Motion Model: Voxceleb_trained_info.yaml could not be found

Hi,
I tried your colab for the first order motion model and stuck at "Image Animation using arbitrary source images" because of this error. Could you please fix this.
Thanks
tjess78

Traceback (most recent call last):
File "animate.py", line 337, in
main()
File "animate.py", line 333, in main
animate(args)
File "animate.py", line 113, in animate
"https://nnabla.org/pretrained-models/nnabla-examples/GANs/first-order-model/voxceleb_trained_info.yaml")
File "animate.py", line 51, in download_provided_file
download(url, filepath, False)
File "/usr/local/lib/python3.7/dist-packages/nnabla/utils/download.py", line 57, in download
r = request.urlopen(url)
File "/usr/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.7/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.7/urllib/request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python3.7/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/usr/lib/python3.7/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found

load_minst() - HTTP Error 503: Service Unavailable

I am not sure why executing python classification.py -c cudnn -d 0 inside a container created by executing docker_run_user --gpus 0 nnabla/nnabla-ext-cuda-multi-gpu:py38-cuda102-mpi3.1.6-v1.18.0 throws the HTTP Error 503: Service Unavailable error.

However, occasionally docker_run_user --gpus 0 nnabla/nnabla-ext-cuda-multi-gpu:py38-cuda102-mpi3.1.6-v1.18.0 python classification.py -c cudnn -d 0 seems to work correctly.
The word "occasionally" is used as I get a 503 error when I tried at a later time.

minor error in train_esrgan.py and train_psnr_rrdb.py

avg_psnr = avg_psnr / idx
should be :
avg_psnr = avg_psnr / val_samples

DALI 0.18.0 compatibility

imagenet-classification training script does not work with DALI 0.18.0 due to DALI's API changes. I made a quick update to the script. This requires changes to both nnabla-examples and nnabla-ext-cuda

PRs
nnabla-examples: #129
nnabla-ext-cuda: sony/nnabla-ext-cuda#207

SinGAN implemenation

Hi, @TE-AkioHayakawa san , @TE-TakuyaNarihira san, @TE-KazukiYoshiyama san.

I've made SinGAN with NNabla. If you consider this to be good to this repository, I'll clean up codes and add more experiments.

https://github.com/takuseno/singan-nnabla/

https://arxiv.org/abs/1905.01164

OOM for TECO GAN

Seems like using even a height of 360 (whicle maintaining aspect ratio) for tecogan gives runtime OOM errors; whats the largest size possible that I can use to try to upscale to 4k? I imagine if I want to upscale to 4k, I would use 1080p as the resolution for my input but its too big for the GPU to handle; if there a way to use only CPU for this?

First Order Motion Model ERROR at play_video

Run First Order Motion Model at Google Colab.
When run "play_video('result/arbitrary/sample_src.png_by_sample_drv.mp4')",
export error below.

---

FileNotFoundError Traceback (most recent call last)
in <cell line: 1>()
----> 1 play_video('result/arbitrary/sample_src.png_by_sample_drv.mp4')

in play_video(filename, height, width)
4
5 def play_video(filename, height=512, width=512):
----> 6 mp4 = open(filename, 'rb').read()
7 data_url = "data:video/mp4;base64," + b64encode(mp4).decode()
8 return HTML(f"""

FileNotFoundError: [Errno 2] No such file or directory: 'result/arbitrary/sample_src.png_by_sample_drv.mp4'

Please teach me how to result this problem.

there's a minor error in mnist-collection/dcgan.py

monitor_fake = M.MonitorImageTile(
        "Fake images", monitor, normalize_method=lambda x: x + 1 / 2.)

should be

monitor_fake = M.MonitorImageTile(
        "Fake images", monitor, normalize_method=lambda x: (x + 1) / 2.)

run word_embedding.py error

I just cloned nnabla-examples from github and run word_embedding.py as below:

[root@ word-embedding]# python word_embedding.py
2018-07-11 14:30:30,850 [nnabla][INFO]: Initializing CPU extension...
2018-07-11 14:30:34,554 [nnabla][INFO]: > /root/nnabla_data/ptb.train.txt already exists.
2018-07-11 14:30:34,554 [nnabla][INFO]: > If you have any issue when using this file,
2018-07-11 14:30:34,554 [nnabla][INFO]: > manually remove the file and try download again.
2018-07-11 14:30:34,839 [nnabla][INFO]: Running in None
Traceback (most recent call last):
File "word_embedding.py", line 408, in
main()
File "word_embedding.py", line 310, in main
args.context, device_id=args.device_id, type_config=args.type_config)
File "/root/anaconda3/lib/python3.6/site-packages/nnabla/ext_utils.py", line 97, in get_extension_context
mod = import_extension_module(ext_name)
File "/root/anaconda3/lib/python3.6/site-packages/nnabla/ext_utils.py", line 46, in import_extension_module
return importlib.import_module('.' + ext_name, 'nnabla_ext')
TypeError: must be str, not NoneType
[root@ word-embedding]#

Runtime errors in colab Stylegan2

I tried to run the colab of stylegan2 in https://colab.research.google.com/github/sony/nnabla-examples/blob/master/interactive-demos/stylegan2.ipynb
But I got runtime error in nn.load_parameters("styleGAN2_G_params.h5")

Is there any solution?

train yolov2 error

train_graph.py", line 63, in setup_impl
tcoord, mcoord, tconf, mconf, tcls, mcls = outputs
ValueError: not enough values to unpack (expected 6, got 1)

[Colab Demo] Inference for a set of images by CenterNet

We got a question in CenterNet Colab demo tutorial in Youtube.

Hi, thanks for the video. This really good.
How do I run an inference for a set of images and get the prediction results saved?

Imagenet Dataset link

I cannot figure out how to download dataset from the provided link: https://imagenet.herokuapp.com/
There is nothing on that page when I open the link in chrome browser. Can you please provide some help on how to download the dataset.

Pretrained Weights links failed

Pretrained Weights of first-order-model links failed! pls check it

Can facial images change with age?

Can this code also be used to create age-related facial images that can be created on the NVIDIA official page?

Add data partition on multi-device mode

As far as I understand that nnabla-examples/distributed/cifar10-100/multi_device_multi_process_classification.py and nnabla-examples/imagenet-classification/multi_device_multi_process_classification.py does not partition the data based on rank and size.

Consequently, some samples may gain extra weight during training, while the selection of these samples is random.

Were I correct, it would be better if data partition based on rank&size is added.