OSError: Caught OSError in DataLoader worker process 0 / image file is truncated (1 bytes not processed)

[lucidrains]

OSError: Caught OSError in DataLoader worker process 0 / image file is truncated (1 bytes not processed)

[josephrocca]

Thanks! I ran a variation of the code from the link you posted (pasted below) and it didn't detect any corrupt images. I'll look into this a bit more tomorrow and update you if I work out the cause.

from PIL import Image
from pathlib import Path
for p in Path(f'./images').glob('*.jpg'):
  try:
    im = Image.open(p)
    im2 = im.convert('RGB')
  except OSError:
    print("Cannot load : {}".format(p))

[josephrocca]

Okay, I think I see what happened here. I was testing the images on my host OS, and it wasn't finding any corruption, but the images I'm training on are inside the docker container, so there must have been a corruption during the docker cp process. I've started a new container and am up to 49k and no errors. Thanks for your help! This is a lot of fun!

[josephrocca]

Okay this time it errored at 5599/90000 with the same error message.

Also, the stylegan2_pytorch --generate command throws an error for me:

Traceback (most recent call last):
  File "/home/user/miniconda/envs/py36/bin/stylegan2_pytorch", line 53, in <module>
    fire.Fire(train_from_folder)
  File "/home/user/miniconda/envs/py36/lib/python3.6/site-packages/fire/core.py", line 138, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/home/user/miniconda/envs/py36/lib/python3.6/site-packages/fire/core.py", line 471, in _Fire
    target=component.__name__)
  File "/home/user/miniconda/envs/py36/lib/python3.6/site-packages/fire/core.py", line 675, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "/home/user/miniconda/envs/py36/bin/stylegan2_pytorch", line 31, in train_from_folder
    lr = learning_rate
  File "/home/user/miniconda/envs/py36/lib/python3.6/site-packages/stylegan2_pytorch/stylegan2_pytorch.py", line 398, in __init__
    self.dataset = Dataset(folder, image_size)
  File "/home/user/miniconda/envs/py36/lib/python3.6/site-packages/stylegan2_pytorch/stylegan2_pytorch.py", line 115, in __init__
    raise Exception(f'no images found at {folder}')
Exception: no images found at ./data

In case it's useful info: The ./models/default folder contains: model_0.pt and model_1.pt, and it looks like the ./results/default folder contains one N.jpg image per thousand training images processed (along with N-ema.jpg and N-me.jpg).

[lucidrains]

Hi Joseph! Thanks for your interest in this project! I've uploaded a fix so that it auto resizes all images in the folder to at least the target size being trained. You can also set image sizes to smaller than 128 with the --image-size flag. I've also fix the problem with --generate, it should work now.

I did some googling on your issue with the truncated error and it seems likely that one of your images may be corrupted, even if it passes the imagemagick tests. https://discuss.pytorch.org/t/oserror-image-file-is-truncated-28-bytes-not-processed-during-learning/36815/21 Could you go through that forum and try some of the suggestions there?