catalyst-team/neuro

Catalyst.Neuro

neuro's Issues

AttributeError: Can't pickle local object 'BrainDataset.__init__..' when running the Neuro_Demo notebook

Hi. I have encountered the AttributionError when running the cell in the Neuro_Demo notebook in the example folder.
The cell is

n_classes = 31
n_epochs = 2
meshnet = MeshNet(n_channels=1, n_classes=n_classes)

logdir = "logs/meshnet_mindboggle"

optimizer = torch.optim.Adam(meshnet.parameters(), lr=0.02)


scheduler = OneCycleLR(optimizer, max_lr=.02,
                 epochs=n_epochs, steps_per_epoch=len(train_loaders['train']))

runner = CustomRunner()
runner.train(
    model=meshnet,
    optimizer=optimizer,
    scheduler=scheduler,
    loaders=train_loaders,
    num_epochs=n_epochs,
    logdir=logdir,
    callbacks=[CheckpointCallback(logdir=logdir)],
    verbose=True
)

The error message is detailed as

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[46], line 14
     10 scheduler = OneCycleLR(optimizer, max_lr=.02,
     11                  epochs=n_epochs, steps_per_epoch=len(train_loaders['train']))
     13 runner = CustomRunner()
---> 14 runner.train(
     15     model=meshnet,
     16     optimizer=optimizer,
     17     scheduler=scheduler,
     18     loaders=train_loaders,
     19     num_epochs=n_epochs,
     20     logdir=logdir,
     21     callbacks=[CheckpointCallback(logdir=logdir)],
     22     verbose=True
     23 )

File [~/anaconda3/lib/python3.10/site-packages/catalyst/runners/runner.py:320](https://file+.vscode-resource.vscode-cdn.net/Users/M295878/Projects/neuro/examples/~/anaconda3/lib/python3.10/site-packages/catalyst/runners/runner.py:320), in Runner.train(self, loaders, model, engine, trial, criterion, optimizer, scheduler, callbacks, loggers, seed, hparams, num_epochs, logdir, valid_loader, valid_metric, minimize_valid_metric, verbose, timeit, check, overfit, load_best_on_end, fp16, amp, apex, ddp)
    318 self._load_best_on_end = load_best_on_end
    319 # run
--> 320 self.run()

File [~/anaconda3/lib/python3.10/site-packages/catalyst/core/runner.py:678](https://file+.vscode-resource.vscode-cdn.net/Users/M295878/Projects/neuro/examples/~/anaconda3/lib/python3.10/site-packages/catalyst/core/runner.py:678), in IRunner.run(self)
    676 except (Exception, KeyboardInterrupt) as ex:
    677     self.exception = ex
...
     58 def dump(obj, file, protocol=None):
     59     '''Replacement for pickle.dump() using ForkingPickler.'''
---> 60     ForkingPickler(file, protocol).dump(obj)

AttributeError: "Can't pickle local object 'BrainDataset.__init__.<locals>.<lambda>''

Does anyone have idea on how to fix it? Thanks!

GMWM_Prediction_and_Visualization.ipynb

download_dataset.sh google drive outdated

When running download_dataset.sh the download-gdrive command fails. If you check the URL generated from the below message you'll see it is invalid.

bash bin/download_dataset.sh --2020-07-31 22:01:34-- https://docs.google.com/uc?export=download&confirm=&id=1M1yYoFU-XlbrP07O01ZKOeP1gZCAl6pd Resolving docs.google.com (docs.google.com)... 142.250.64.110, 2607:f8b0:4006:807::200e Connecting to docs.google.com (docs.google.com)|142.250.64.110|:443... connected. HTTP request sent, awaiting response... 404 Not Found 2020-07-31 22:01:34 ERROR 404: Not Found.

Error If launch experiment

If I run following experiment from this repository https://github.com/Bekovmi/neuro-1, I will receive following error:
File "/home/Bekovmi/.conda/envs/py37/lib/python3.7/site-packages/catalyst/dl/main.py", line 41, in main
COMMANDS[args.command].main(args, uargs)
File "/home/Bekovmi/.conda/envs/py37/lib/python3.7/site-packages/catalyst/dl/scripts/run.py", line 117, in main
distributed_run(args.distributed, main_worker, args, unknown_args)
File "/home/Bekovmi/.conda/envs/py37/lib/python3.7/site-packages/catalyst/utils/distributed.py", line 188, in distributed_run
worker_fn(*args, **kwargs)
File "/home/Bekovmi/.conda/envs/py37/lib/python3.7/site-packages/catalyst/dl/scripts/run.py", line 112, in main_worker
runner.run_experiment(experiment)
File "/home/Bekovmi/.conda/envs/py37/lib/python3.7/site-packages/catalyst/core/runner.py", line 374, in run_experiment
self._run_event("on_exception")
File "/home/Bekovmi/.conda/envs/py37/lib/python3.7/site-packages/catalyst/core/runner.py", line 242, in _run_event
getattr(callback, event)(self.state)
File "/home/Bekovmi/.conda/envs/py37/lib/python3.7/site-packages/catalyst/core/callbacks/exception.py", line 17, in on_exception
raise exception
File "/home/Bekovmi/.conda/envs/py37/lib/python3.7/site-packages/catalyst/core/runner.py", line 361, in run_experiment
self._run_stage(stage)
File "/home/Bekovmi/.conda/envs/py37/lib/python3.7/site-packages/catalyst/core/runner.py", line 342, in _run_stage
self._run_epoch(stage=stage, epoch=state.epoch)
File "/home/Bekovmi/.conda/envs/py37/lib/python3.7/site-packages/catalyst/core/runner.py", line 329, in _run_epoch
self._run_loader(loader)
File "/home/Bekovmi/.conda/envs/py37/lib/python3.7/site-packages/catalyst/core/runner.py", line 289, in _run_loader
for i, batch in enumerate(loader):
File "/home/Bekovmi/.conda/envs/py37/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 345, in next
data = self._next_data()
File "/home/Bekovmi/.conda/envs/py37/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 856, in _next_data
return self._process_data(data)
File "/home/Bekovmi/.conda/envs/py37/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 881, in _process_data
data.reraise()
File "/home/Bekovmi/.conda/envs/py37/lib/python3.7/site-packages/torch/_utils.py", line 394, in reraise
raise self.exc_type(msg)
KeyError: Caught KeyError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/Bekovmi/.conda/envs/py37/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2646, in get_loc
return self._engine.get_loc(key)
File "pandas/_libs/index.pyx", line 111, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 138, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1618, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1626, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 17
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/Bekovmi/.conda/envs/py37/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
data = fetcher.fetch(index)
File "/home/Bekovmi/.conda/envs/py37/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/Bekovmi/.conda/envs/py37/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/Bekovmi/.conda/envs/py37/lib/python3.7/site-packages/catalyst/data/dataset.py", line 52, in getitem
item = self.data[index]
File "/home/Bekovmi/.conda/envs/py37/lib/python3.7/site-packages/pandas/core/frame.py", line 2800, in getitem
indexer = self.columns.get_loc(key)
File "/home/Bekovmi/.conda/envs/py37/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2648, in get_loc
return self._engine.get_loc(self._maybe_cast_indexer(key))
File "pandas/_libs/index.pyx", line 111, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 138, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1618, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1626, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 17

Google drive outdated

When running Mindboggle_Prediction_and_Visualization.ipynb for pertained models, the download-gdrive fails. The url is also invalid. The error message is listed below.

--2023-04-04 07:58:03-- https://docs.google.com/uc?export=download&confirm=&id=11i-gPKoLzEUVqVJ0UNCjG30HFuMXODkG
Resolving docs.google.com (docs.google.com)... 142.250.159.101, 142.250.159.102, 142.250.159.138, ...
Connecting to docs.google.com (docs.google.com)|142.250.159.101|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2023-04-04 07:58:03 ERROR 404: Not Found.

--2023-04-04 07:58:03-- https://docs.google.com/uc?export=download&confirm=&id=1gVjj1gYoPLk8BjZHXsznS6fbbWJaN3Gb
Resolving docs.google.com (docs.google.com)... 142.250.159.101, 142.250.159.102, 142.250.159.138, ...
Connecting to docs.google.com (docs.google.com)|142.250.159.101|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2023-04-04 07:58:03 ERROR 404: Not Found.