nitmir / btdht Goto Github PK
View Code? Open in Web Editor NEWA python implementation of the Bittorrent DHT (Distributed Hash Table)
License: GNU General Public License v3.0
A python implementation of the Bittorrent DHT (Distributed Hash Table)
License: GNU General Public License v3.0
how to download torrents file using this lib?
Thanks.
def search_dht(dht, info_hash):
peers = []
for i in range(0, 5):
peers = dht.get_peers(binascii.a2b_hex(info_hash))
time.sleep(1.5)
return peers
if __name__ == '__main__':
dht = btdht.DHT()
dht.start() # now wait at least 15s for the dht to boostrap
print("Initializing DHT", flush=True)
time.sleep(20)
print(search_dht(dht, '756D9FA81E9328A97A03EE134EBCA8C2CCE3B670'))
The output I get is
init socket for b'0c66c99cafff75350028b3ec10845835c75921b5'
Initializing DHT
Bootstraping
0 nodes, 0 goods, 0 bads | in: 0, out: 3 en 1s
Bootstraping
0 nodes, 0 goods, 0 bads | in: 0, out: 3 en 1s
Bootstraping
0 nodes, 0 goods, 0 bads | in: 0, out: 3 en 1s
Bootstraping
0 nodes, 0 goods, 0 bads | in: 0, out: 3 en 1s
Bootstraping
0 nodes, 0 goods, 0 bads | in: 0, out: 3 en 1s
None
I can't really find what "0 nodes, 0 goods, 0 bads | in: 0, out: 3 en 1s" mean in the documentation. Why would I be getting this "error" and not the expected (IP, PORT) outcome?
I'm getting this error frequently with Python 2.7.9 and btdht 0.3.1:
init socket for f289d6ad5d78a5dbbf4605ddafb6bfdbe38aa7ab
Bootstraping
Exception in thread scheduler:io_loop:
Traceback (most recent call last):
File "C:\Python27\lib\threading.py", line 810, in __bootstrap_inner
self.run()
File "C:\Python27\lib\threading.py", line 763, in run
self.__target(*self.__args, **self.__kwargs)
File "btdht/utils.pyx", line 1145, in btdht.utils.Scheduler._io_loop (btdht/utils.c:23139)
dht._process_incoming_message()
File "btdht/dht.pyx", line 979, in btdht.dht.DHT_BASE._process_incoming_message (btdht/dht.c:21040)
self.debug(0, "send:%r : (%r, %r)" % (e, data, addr))
UnboundLocalError: local variable 'data' referenced before assignment
Using the example from the readme:
import binascii
import btdht
import time
dht = btdht.DHT()
dht.start()
time.sleep(30)
print(dht.get_peers(binascii.a2b_hex("0403fb4728bd788fbcb67e87d6feb241ef38c75a")))
I'm getting this error with Python 3.5.2 and btdht 0.3.1:
Traceback (most recent call last):
...
File "...", line 6, in <module>
dht.start()
File "btdht/dht.pyx", line 379, in btdht.dht.DHT_BASE.start (btdht/dht.c:7806)
self._scheduler.add_dht(self)
File "btdht/utils.pyx", line 917, in btdht.utils.Scheduler.add_dht (btdht/utils.c:17508)
self.add_thread(name, function, user=user)
File "btdht/utils.pyx", line 846, in btdht.utils.Scheduler.add_thread (btdht/utils.c:15984)
typ = iterator.next()
AttributeError: 'generator' object has no attribute 'next'
Process finished with exit code 1
Script is nearly empty:
#!/usr/bin/env python3
import btdht
import binascii
dht = btdht.DHT()
dht.start()
Add ability to use DHT as k-v strorage
https://www.bittorrent.org/beps/bep_0044.html
hi
i gotten error, when doing make on the latest code, can you help?
Error compiling Cython file:
------------------------------------------------------------
...
else:
self.addr_addr_2 = addr[0]
self.addr_port = addr[1]
def __del__(self):
self.addr_addr = None
self.addr_port = None
^
------------------------------------------------------------
btdht/krcp.pyx:1472:29: Cannot assign None to int
Compiling btdht/dht.pyx because it changed.
Traceback (most recent call last):
File "setup.py", line 28, in <module>
Compiling btdht/krcp.pyx because it changed.
Compiling btdht/utils.pyx because it changed.
[1/3] Cythonizing btdht/dht.pyx
[2/3] Cythonizing btdht/krcp.pyx
ext_modules = cythonize("btdht/*.pyx") if has_cython else c_extensions,
File "/usr/lib64/python2.7/site-packages/Cython/Build/Dependencies.py", line 1026, in cythonize
cythonize_one(*args)
File "/usr/lib64/python2.7/site-packages/Cython/Build/Dependencies.py", line 1146, in cythonize_one
raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: btdht/krcp.pyx
make: *** [build] Error 1
pip install btdht
fails with
gcc -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -fPIC -Ivenv/include -I/usr/include/python3.11 -c btdht/dht.c -o build/temp.linux-x86_64-cpython-311/btdht/dht.o
btdht/dht.c:209:12: fatal error: longintrepr.h: No such file or directory
209 | #include "longintrepr.h"
| ^~~~~~~~~~~~~~~
compilation terminated.
error: command '/usr/bin/gcc' failed with exit code 1
at the same time i'm able to successfully build after installing build deps, either directly in a git checkout or by using pip install git+https://github.com/nitmir/btdht.git
Hi,
I have been trying to use enumerate_ids
but it seems to not work with Python3.
I have been doing some executions on Python2 to see how it works and all goes fine since bytes in python2 are handled as strings.
The problem resides in nset
function. I tried to convert bytes to strings to handle them there. Like this:
if i == 1:
return s[:n//8] + chr(s[n//8] | _NSET_BIT1[n % 8]).encode() + s[n//8+1:]
elif i == 0:
return s[:n//8] + chr(s[n//8] & _NSET_BIT0[n % 8]).encode() + s[n//8+1:]
else:
raise ValueError("i must be 0 or 1")
The problem with this is that the second element of the output of ``enumerate_ids``` is 21 long char. And some executions start with the first element as a 22 long so the second one is 23 long, third 22...
I have been trying to solve this but I have run out of ideas. Any hint would be appreciated.
Hi,
Im playing around with this library and have tried asking it for a small bunch of magnet hashes, in the start it goes without any issues at all. But after a random number of get_peer requests for some random hashes for popular torrents, the CPU usage spikes to 100% and stays there until i kill it.
I have tried to run only one request per 10 second, running it both in blocking and non-blocking mode, increasing the debug level, moving the query limit between 10 and 1000. No matter what changes i have done the issue appears.
I tried to reproduce the issue by making a script that simply polled the ubuntu example hash (and some more hashes based on that hash), but that did not seem to fail.
It appears that the last thing that happens before the lockup is that the library receives a couple of errors:
2984 nodes, 1607 goods, 59 bads | in: 51, out: 60 en 14s
ERROR:202:b'Server Error' pour (None, 1520126208.2239816, b'd1:ad2:id20:\x04Ogj^1+\xd3*Q\x0c\xa8\xdf\xcf\xe2\xea\xc2\x12?\xeae1:q4:ping1:t6:=\x08,\x902M1:y1:qe')
obj of type b'e'
8 nodes added to routing table
5 nodes added to routing table
ERROR:202:b'Server Error' pour (None, 1520126208.227913, b'd1:ad2:id20:\x04Ogj^1+\xd3*Q\x0c\xa8\xdf\xcf\xe2\xea\xc2\x12?\xeae1:q4:ping1:t6:\xd6\x92N\xd8 B1:y1:qe')
obj of type b'e'
8 nodes added to routing table
0 nodes added to routing table
7 nodes added to routing table
4 nodes added to routing table
ERROR:202:b'Server Error' pour (None, 1520126208.266789, b'd1:ad2:id20:\x04Ogj^1+\xd3*Q\x0c\xa8\xdf\xcf\xe2\xea\xc2\x12?\xeae1:q4:ping1:t6:d/ X\xaf\x911:y1:qe')
obj of type b'e'
8 nodes added to routing table
3 nodes added to routing table
8 nodes added to routing table
ERROR:202:b'Server Error' pour (None, 1520126208.3035867, b'd1:ad2:id20:\x04Ogj^1+\xd3*Q\x0c\xa8\xdf\xcf\xe2\xea\xc2\x12?\xeae1:q4:ping1:t6:1yV\x97\xc7F1:y1:qe')
obj of type b'e'
ERROR:202:b'Server Error' pour (None, 1520126208.2280884, b'd1:ad2:id20:\x04Ogj^1+\xd3*Q\x0c\xa8\xdf\xcf\xe2\xea\xc2\x12?\xeae1:q4:ping1:t6:\x8a,\n\x1f\xac\xa21:y1:qe')
obj of type b'e'
ERROR:202:b'Server Error' pour (None, 1520126208.2672093, b'd1:ad2:id20:\x04Ogj^1+\xd3*Q\x0c\xa8\xdf\xcf\xe2\xea\xc2\x12?\xeae1:q4:ping1:t6:x\xb7\xffe\x1c\xc81:y1:qe')
obj of type b'e'
ERROR:202:b'Server Error' pour (None, 1520126208.2678056, b"d1:ad2:id20:\x04Ogj^1+\xd3*Q\x0c\xa8\xdf\xcf\xe2\xea\xc2\x12?\xea6:target20:\x87\xd0\x0f\xdc\xdd'\x01,~(\xe0\x9f\xd4\x1c\\jz\xc6\x12be1:q9:find_node1:t6:\x10\xc5\x9e\xd7\x97'1:y1:qe")
obj of type b'e'
0 nodes added to routing table
I tried some profiling with Yappi, it says that these two calls were the most used, not sure if helpful but..
name ncall tsub ttot tavg
...5/queue.py:90 PollableQueue.empty 187.. 21.10275 42.51144 0.000023
..dist-packages/six.py:580 iteritems 187.. 23.93679 37.79221 0.000020
Any calls for get_peers afterwards will simply yield "no peers or nodes found"
I have not seen any special IO-activity while this happens,
This has been tested on the latest release
What could this be, and how can it be avoided?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.