IRC: #zif on Freenode
A distributed torrent sharing and indexing network.
Zif makes it easy to share files, without having any bandwidth or storage costs. It uses BitTorrent for file transfers, but provides a decentralised index for discovering new content.
Check out the develop branch for the latest work!
an example of the HTTP API interface
an example of the Electron UI, which can be found here
- a queryable database of torrents and metadata
- a p2p network for the distribution of torrent info hashes and metadata
Peer discovery is done via an implementation of Kademlia, a DHT similar to the one Bittorrent uses. Peers are assigned addresses similar to Bitcoin addresses, except using Ed25519 and SHA3.
Zif allows users to discover file indexes. Searches can then be performed on the remote peer for files that they are indexing, and file data is sent as a response. This includes a BitTorrent info-hash, which can then be used to download the file.
Zif also allows users to mirror the index of a peer. This massively enhances search speed, and allows anyone to take a complete backup of an index.
Zif can also be routed through any SOCKS proxy, and can create a Tor onion address automatically - this aids privacy and traverses the NAT, at the cost of performance.
Now! It will likely have some bugs, but is mostly working.
You can download Zif here
Make sure that you have make installed. You will also need git, and, of course, go.
git clone https://github.com/zif/zif
cd zif
make
The resulting binary will be in zif/bin.
The Makefile doesn't seem to work so well with Windows, so you'll need to have Go properly setup and installed, this may require setting $GOPATH
go get github.com/zif/zif
The resulting "zifd" binary should be automatically installed into your $GOPATH.
Zif presents a HTTP interface for usage, so you can interact with it using curl. There is a command program available here, which makes interaction easier. siv is also downloadable from the release page.
There is also a user interface being developed here, with in-built streaming and downloading, all without needing a torrent client.
Presently I am running several Zif nodes, you can bootstrap from this network as follows (assuming you have Zifd running and listening on port 8080).
First you need Tor running: to do this, cd into the tor directory, and run tor -f torrc. You will also need to edit zifd.toml and change tor.enabled and socks.enabled to be true. The permissions on the Zif directory may well need to be 700 for Tor to be happy.
To get started, simply run zifd. The output will contain your Zif address, which will look something like this: ZncGWimPZHWxjTMj51QNKg25PTCXphtLbh
In order to connect to the rest of the network, you will need to bootstrap. This can either be done using the below API, or using siv.
curl localhost:8080/self/bootstrap/x4yknq5x7iijrmgy.onion/
curl localhost:8080/self/explore/
By default, Zif listens on localhost:8080. This is configurable in zifd.toml.
These routes affect the local peer, ie the client running on your machine. They're generally used to interact with your own database, or change settings, etc.
This is used to add a post to your database, a post is essentially a torrent infohash and some metadata. The POST body requires a parameter of data and index.
The former is JSON, and is specified as such:
InfoHash string - the torrent infohash
Title string - a name for the post
Size int - the size of the torrent in bytes
FileCount int - number of files the torrent has
Seeders int - number of seeders the torrent has
Leechers int - number of leechers the torrent has
UploadDate int - Unix timestamp in seconds
Tags string - a comma-separated list of alphanumeric tags
Meta string - a JSON-encoded object
The other parameter, index, should be either "true" or "false". This indicates whether or not Zif should add the post to the full text search index. If this is true, then the Title field will be indexed and the post will show up in search results.
This performs a full text search index on all posts that have an id greater than {since}.
This resolves a Zif address into a JSON entry. Entries are specified as such:
address Address
name string
desc string
publicAddress string
publicKey []byte
postCount int
updated uint64
signature []byte
collectionHash []byte
port int
seeds [][]byte
seeding [][]byte
seen int
Bootstraps the Zif node from the given address. This address must be a non-Zif address - for instance, a domain name, IP address, onion address, or anything else. Note that Zif can be configured to use a SOCKS proxy, see zifd.toml.
Perform a full text search on the local database.
This takes the parameters of query and page, where query is the search term and page is the page of results we want - this starts at 0.
Gets the most recent posts. The page is given as the {page} parameter.
Gets the most popular posts. The page is given as the {page} parameter.
Returns a list of peers.
Begin network exploration. This should happen automatically at start if you have peers in your routing table, otherwise it needs to be ran manually.
This is used to set various settings for the node. Here are possible values for {name}:
- name: This sets the name field of the entry and can be used to identify your node
- desc: A short description of your entry
- public: set the public address for your entry, this is what it's Zif address will resolve to
Gets values, much like set - supports the same values. Also supports:
- zif: gets the Zif address
- postcount: the number of posts this node has
- entry: the full Zif DHT entry
These routes allow you to query a remote peer. The {address} parameter refers to the encoded Zif address of the peer, like the example above.
Pings the peer.
Announces your own entry to the peer.
Performs a remote search on the peer.
Download a local copy of the peer's post database, which can then be indexed and searched.
Search the local copy of the peer's database, this only works after a successful mirror.
Get the {page} of most recent posts for the given peer.
Performs a remote search on the peer.
Add all posts with an id larger than {since} to the FTS index.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent