BEP-9: Extension for Peers to Send Metadata Files

Bt

^{A full-featured BitTorrent implementation in Java 8

peer exchange | magnet links | DHT | encryption | LSD | private trackers | extended protocol | partial downloads | port forwarding}

Supported BEPs and extensions

Resources

HOME – website with documentation and tutorials
RELEASE NOTES – list of features, bugfixes and improments for each version
UPGRADE INSTRUCTIONS – version migration guide
FORUM – Google group for support and feedback
TROUBLESHOOTING - solutions for some common problems
LICENSE – licensed under Apache License 2.0

Runnable apps and demos

CLI – command-line downloader
PEER TRACKER – tracking of swarm statistics via events
YOURIP MESSENGER – usage of custom messages

Media

HABRAHABR.RU: Пишем свой BitTorrent клиент на базе библиотеки Bt - Introductory article and demonstration of basic capabilities (in Russian)
SMARTSPATE.COM: How To Write Your Own BitTorrent Client By Using Bt Library - English translation of the above article

Prerequisites

^{Currently, all peer connections are established via encryption negotation protocol (also called MSE handshake). If you're using Oracle JDK (pre 8u152), in order to be able to connect to peers you must install Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy. The reason for this requirement is that the MSE RC4 cipher uses 160 bit keys, while default Java installation allows at most 128 bit keys.}

Usage

Most recent version available in Maven Central is 1.10.

Declare the following dependencies in your project’s pom.xml:

<dependency>
    <groupId>com.github.atomashpolskiy</groupId>
    <artifactId>bt-core</artifactId>
    <version>${bt-version}</version>
</dependency>
<dependency>
    <groupId>com.github.atomashpolskiy</groupId>
    <artifactId>bt-http-tracker-client</artifactId>
    <version>${bt-version}</version>
</dependency>
<dependency>
    <groupId>com.github.atomashpolskiy</groupId>
    <artifactId>bt-dht</artifactId>
    <version>${bt-version}</version>
</dependency>
<dependency>
    <groupId>com.github.atomashpolskiy</groupId>
    <artifactId>bt-upnp</artifactId>
    <version>${bt-version}</version>
</dependency>

Building from source

git clone https://github.com/atomashpolskiy/bt.git
cd bt
mvn clean install -DskipTests

Code sample

Download a torrent from a magnet link

// enable multithreaded verification of torrent data
Config config = new Config() {
    @Override
    public int getNumOfHashingThreads() {
        return Runtime.getRuntime().availableProcessors() * 2;
    }
};

// enable bootstrapping from public routers
Module dhtModule = new DHTModule(new DHTConfig() {
    @Override
    public boolean shouldUseRouterBootstrap() {
        return true;
    }
});

// get download directory
Path targetDirectory = Paths.get(System.getProperty("user.home"), "Downloads");

// create file system based backend for torrent data
Storage storage = new FileSystemStorage(targetDirectory);

// create client with a private runtime
BtClient client = Bt.client()
        .config(config)
        .storage(storage)
        .magnet("magnet:?xt=urn:btih:af0d9aa01a9ae123a73802cfa58ccaf355eb19f1")
        .autoLoadModules()
        .module(dhtModule)
        .stopWhenDownloaded()
        .build();

// launch
client.startAsync().join();

Create a torrent

Path torrentRoot = Paths.get("/home/torrents/mytorrent");
Path file1 = Paths.get("/home/torrents/mytorrent/file1.bin");
Path file2 = Paths.get("/home/torrents/mytorrent/file2.bin");
Path dirToAdd = Paths.get("/home/torrents/mytorrent/dir_with_files");
byte[] torrentBytes = new TorrentBuilder()
        .rootPath(torrentRoot)
        .addFiles(file1, file2, dirToAdd)
        .announce("http://example.com/announce")
        .build();
Files.write(Paths.get("/home/torrents/mytorrent.torrent"), torrentBytes);

What makes Bt stand out from the crowd

Flexibility

Being built around the Guice DI, Bt provides many options for tailoring the system for your specific needs. If something is a part of Bt, then it can be modified or substituted for your custom code.

Custom backends

Bt is shipped with a standard file-system based backend (i.e. you can download the torrent file to a storage device). However, the backend details are abstracted from the message-level code. This means that you can use your own backend by providing a storage unit implementation.

Protocol extensions

One notable customization scenario is extending the standard BitTorrent protocol with your own messages. BitTorrent's BEP-10 provides a native support for protocol extensions, and implementation of this standard is already included in Bt. Contribute your own Messages, byte manipulating MessageHandlers, message consumers and producers; supply any additional info in ExtendedHandshake.

Test infrastructure

To allow you test the changes that you've made to the core, Bt ships with a specialized framework for integration tests. Create an arbitrary-sized swarm of peers inside a simple JUnit test, set the number of seeders and leechers and start a real torrent session on your localhost. E.g. create one seeder and many leechers to stress test the network overhead; use a really large file and multiple peers to stress test your newest laptop's expensive SSD storage; or just launch the whole swarm in no-files mode and test your protocol extensions.

Parallel downloads

Bt has out-of-the-box support for multiple simultaneous torrent sessions with minimal system overhead.

Partial downloads

Bt has an API for selecting only a subset of torrent files to download. See the bt.TorrentClientBuilder.fileSelector(TorrentFileSelector) client builder method. File selection works for both .torrent file-based and magnet link downloads.

Java 8 CompletableFuture

Client API leverages the asynchronous java.util.concurrent.CompletableFuture to provide the most natural way for co-ordinating multiple torrent sessions. E.g. use CompletableFuture.allOf(client1.startAsync(...), client2.startAsync(...), ...).join(). Or create a more sophisticated processing pipeline.

And much more...

check out Release Notes for details!

Troubleshooting

Can't connect to peers, everything else seems to work

If you're using an Oracle JDK, make sure that you have installed Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy.

Other BitTorrent clients can't connect to a Bt client / No incoming connections when seeding

a) If you're behind a firewall and/or a NAT (e.g. a router), make sure they are configured to allow incoming TCP and UDP connections on the ports used by Bt. Default Bt ports are 6891 and 49001 for BitTorrent and DHT respectively. NAT must additionally be configured to forward all incoming traffic on these ports to the host, that Bt is running on.

b) Many popular BitTorrent clients use UPnP and NAT-PMP to automatically configure port forwarding on NATs. Since 1.8 Bt does this as well, so make sure to include bt-upnp module in the list of dependencies.

There are exceptions in the build log (but the build completes successfully)

This is perfectly fine. Some of the tests verify that the exceptions are thrown in certain cases, hence the exception messages.

Can't run the CLI on Windows XP (java.io.IOException: Cannot run program "/bin/stty")

CLI GUI indeed does not work on Windows XP. Run in headless mode by using -H flag.

Can't connect to peers on Windows 7/8/10

There seem to be some issues with dual-stack networking in Windows JDK (e.g. see this question on SO), with Java trying to use IPv6 address, when it's not really available in the system. The simplest solution is to force Java to use IPv4 by setting java.net.preferIPv4Stack property to true.

Support and feedback

Any thoughts, ideas, criticism, etc. are welcome, as well as votes for new features and BEPs to be added. You have the following options to share your ideas, receive help or report bugs:

open a new GitHub issue
post your question on the Bt forum

	private TrackerResponse tryForAllTrackers(Function<Tracker, TrackerResponse> func) {

	List<TrackerResponse> responses = new ArrayList<>();

	for (List<Tracker> trackerTier : trackerTiers) {

	TrackerResponse response;
	Tracker currentTracker;

	for (int i = 0; i < trackerTier.size(); i++) {
	currentTracker = trackerTier.get(i);
	response = func.apply(currentTracker);
	responses.add(response);

	if (response.isSuccess()) {
	if (trackerTier.size() > 1) {
	trackerTier.remove(i);
	trackerTier.add(0, currentTracker);
	}
	return response;
	} else if (response.getError().isPresent()) {
	Throwable e = response.getError().get();
	LOGGER.warn("Unexpected error during interaction with the tracker: " + currentTracker, e);
	} else {
	LOGGER.warn("Unexpected error during interaction with the tracker: " + currentTracker +
	"; message: " + response.getErrorMessage());
	}
	}
	}

	throw new BtException("All trackers failed; responses (in chrono order): " + responses);
	}

	// bitfield: <len=0001+X><id=5><bitfield>
	private static int decodeBitfield(DecodingContext context, ByteBuffer buffer, int length) {

	int consumed = 0;

	if (buffer.remaining() >= length) {
	byte[] bitfield = new byte[length];
	buffer.get(bitfield);
	context.setMessage(new Bitfield(bitfield));
	consumed = length;
	}

	return consumed;
	}

	this.inboundBuffer = inboundBuffer;
	// process existing data immediately (e.g. there might be leftovers from MSE handshake)
	fireDataReceived();

	this.outboundBuffer = outboundBuffer;

	this.decoders = decoders;
	this.encoders = encoders;
	this.inboundQueue = new LinkedBlockingQueue<>();
	}

	@Override
	public Message decode() {
	checkHandlerIsBound();

	return inboundQueue.poll();
	}

	private void fireDataReceived() {
	ByteBuffer buffer = inboundBuffer.lockAndGet();
	try {
	processInboundData(buffer);
	} finally {
	inboundBuffer.unlock();
	}
	}

	private void processInboundData(ByteBuffer buffer) {
	int undecodedDataLimit = buffer.position();
	if (undecodedDataOffset < undecodedDataLimit) {
	buffer.flip();
	decoders.forEach(mutator -> {

	public void addBitfield(Peer peer, Bitfield bitfield) {
	validateBitfieldLength(bitfield);
	peerBitfields.put(peer, bitfield);

	for (int i = 0; i < pieceTotals.length; i++) {
	if (bitfield.getPieceStatus(i) == PieceStatus.COMPLETE_VERIFIED) {
	incrementPieceTotal(i);
	}
	}
	}

	if (acquiredConnection == null \|\| throwable != null) {
	if (LOGGER.isDebugEnabled()) {
	LOGGER.debug("Peer is unreachable: {}. Will prevent further attempts to establish connection.", peer);
	}
	unreachablePeers.putIfAbsent(peer, System.currentTimeMillis());
	}

atomashpolskiy / bt Goto Github PK

bt's Introduction