nitmir/btdht-crawler

A bittorrent dht crawler

btdht-crawler's Issues

feed.py problem

File "btdht/utils.pyx", line 206, in btdht.utils._bencode (btdht/utils.c:6787)
EnvironmentError: Can only encode int, str, list or dict, not long

Bdecode fails to import?

Traceback (most recent call last):
  File "./feed.py", line 27, in <module>
    from crawler import get_id, HashToIgnore
  File "/home/btdht-crawler/crawler.py", line 19, in <module>
    import torrent
  File "/home/btdht-crawler/torrent.py", line 11, in <module>
    from btdht.utils import bencode, bdecode, _bdecode, BcodeError, ID
ImportError: cannot import name _bdecode

mongo branch fails to run

It appears that a few differences in the most recent btdht cause this to fail running. I guessed on newest btdht due to the appearance of Scheduler as an import.

This issue I resolved by just removing the bdecode_rest import:

(btdht-crawler) [brandon@elpuerto btdht-crawler]$ ./crawler.py   
Traceback (most recent call last):
  File "./crawler.py", line 20, in <module>
    import torrent
  File "/home/stor/btdht-crawler/torrent.py", line 11, in <module>
    from btdht.utils import bencode, bdecode, bdecode_rest, BcodeError, ID
ImportError: cannot import name bdecode_rest

This I resolved by having the get_id function use id = ID().value instead of id = ID() in get_id

(btdht-crawler) [brandon@elpuerto btdht-crawler]$ ./badcrawl.py 
W1:lauch data/crawler1.id
generating new id
Process Process-1:
Traceback (most recent call last):
  File "/usr/lib64/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "./badcrawl.py", line 324, in lauch
    id_base = get_id(id_file)
  File "./badcrawl.py", line 316, in get_id
    f.write(str(id))
  File "btdht/utils.pyx", line 306, in btdht.utils.ID.__str__ (btdht/utils.c:5264)
    raise NotImplementedError()
NotImplementedError