Comments (2)
Is there a way to better debug the core issue perhaps by utilizing the logs? I believe I have downloaded the whole LUNA dataset and extracted all the files in the proper folder. I was able to run chapter 10 code but for some reason am getting this error in chapter 11.
When I run from the Jupyter Notebook I get:
2020-10-21 07:39:18,522 INFO pid:37584 nb:004:run Running: p2ch11.prepcache.LunaPrepCacheApp(['--num-workers=4']).main()
2020-10-21 07:39:18,523 INFO pid:37584 p2ch11.prepcache:043:main Starting LunaPrepCacheApp, Namespace(batch_size=1024, num_workers=4)
2020-10-21 07:39:18,847 INFO pid:37584 p2ch11.dsets:185:init <p2ch11.dsets.LunaDataset object at 0x000001EBCBE32688>: 548723 training samples
2020-10-21 07:39:18,847 WARNING pid:37584 util.util:221:enumerateWithEstimate Stuffing cache ----/536, starting
UnpicklingError: Caught UnpicklingError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "..\lib\site-packages\torch\utils\data_utils\worker.py", line 178, in _worker_loop
data = fetcher.fetch(index)
File "..\lib\site-packages\torch\utils\data_utils\fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "..\lib\site-packages\torch\utils\data_utils\fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "E:\DLwPytorch\dlwpt-code\p2ch11\dsets.py", line 198, in getitem
width_irc,
File "..\lib\site-packages\diskcache\core.py", line 1856, in wrapper
result = self.get(key, default=ENOVAL, retry=True)
File "..\lib\site-packages\diskcache\fanout.py", line 246, in get
return shard.get(key, default, read, expire_time, tag, retry)
File "..\lib\site-packages\diskcache\core.py", line 1172, in get
value = self._disk.fetch(mode, filename, db_value, read)
File "E:\DLwPytorch\dlwpt-code\util\disk.py", line 67, in fetch
value = super(GzipDisk, self).fetch(mode, filename, value, read)
File "..\lib\site-packages\diskcache\core.py", line 274, in fetch
return pickle.load(reader)
_pickle.UnpicklingError: invalid load key, '\x00'.
from dlwpt-code.
Looks like the issue had to do with downloading the data. I had initially downloaded the files through Zenodo, but had some errors when extracting subset1 and subset9. I didn't think much of it at the time but then I ran into this error with pickle. I then downloaded subset1 and subset9 through the academic torrent, extracted, and ran this script. Seems to run fine now.
from dlwpt-code.
Related Issues (20)
- p2ch11: label_g[:,1] as target label in computeBatchLoss() ? HOT 1
- P2ch10 error: only integers, slices (`:`)...
- Post URL of book errata
- Read labels from the module, not from some random file
- Didn't set model to eval mode when validating in chapter 8
- get'IndexError: list index out of range' when run the specific series_uid in p2ch10_explore_data.ipynb
- typo
- p2ch11/training.py causes an TypeError HOT 1
- p1ch3, id(points.storage()) == id(points_t.storage()) out:False on Colab HOT 3
- findPositiveSamples() output no data, in p2ch10 (kaggle)
- Unable to visualize the images in part2ch10 code (p2ch10_explore_data.ipynb notebook)
- Fig. 7.11. in the book
- Unable to Run training.py ch11. CandidateInfo_List assertion error. HOT 5
- p2ch11: 13,5 hours to train for one epoch
- p2ch14: TypeError: linspace() missing 1 required positional arguments: "steps" HOT 2
- p2ch12 the memory explosion when i train the balanced model with the same config in p2ch11 HOT 2
- p HOT 2
- xyz2irc() is not inverse of irc2xyz()
- p2ch11 我尝试运行training这个文件,并对LunaModel 略做修改,两个结果有些不同,我不知道哪一个结果更好?
- Next version enhancement
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dlwpt-code.