fbdtemme/torrenttools

Commandline tool for inspecting, creating and editing BitTorrent metafiles.

A commandline tool for creating, inspecting and modifying bittorrent metafiles.

Features | Documentation | Binary releases | Building | License

Features

• Creating bittorrent metafiles.
• Inspecting bittorrent metafiles.
• Verifying bittorrent metafiles against local data.
• Editing existing bittorrent metafiles.
• Support for the new v2 and hybrid protocols .
• Support for tracker abbreviations.
• Support for announce substitution parameters.
• Fast multi-buffer hashing with Intel ISA-L.

Example

Status

This project is under development. The commandline interface can change at any release prior to 1.0.0.

Performance

Following test were performed on a in in-memory filesystem with 1 MiB piece size and as target a file filed with random data totalling 15.0 GiB: The tested CPU is an Intel i7-7700HQ in a Dell XPS 15-9560 machine.

Documentation

Documentation is hosted on Github Pages.

Binary releases

Contents

• Windows
• macOS
• Linux
  • Fedora
  • CentOS
  • Ubuntu
  • Debian
  • Ubuntu
  • OpenSUSE
  • SUSE Linux Enterprise Server
  • Arch
  • AppImage
• Docker

Windows

An .msi installer is available as a release asset.

IMPORTANT: Please use a modern terminal that supports most ANSI escape codes, like Windows Terminal (not cmd or PowerShell).

macOS

A .pkg installer is available as a release asset.

Linux

Fedora

Binary and source packages for Fedora 32, Fedora 33, Fedora 34 and Fedora Rawhide and CentOS stream are available in a COPR repo.

sudo dnf copr enable fbdtemme/torrenttools
sudo dnf install torrenttools

CentOS/RHEL

Binary and source packages for CentOS8/RHEL8 and CentOS stream are available in a COPR repo.

sudo dnf copr enable fbdtemme/torrenttools
sudo dnf install torrenttools

Ubuntu

Binary and source packages for Ubuntu 20.04, Ubuntu 20.10, Ubuntu 21.04 are available as a PPA via launchpad.

sudo add-apt-repository ppa:fbdtemme/torrenttools
sudo apt-get update
sudo apt install torrenttools

Debian

A binary package is available for Debian Sid. Older debian distributions should use the AppImage.

echo 'deb http://download.opensuse.org/repositories/home:/fbdtemme/Debian_Unstable/ /' | sudo tee /etc/apt/sources.list.d/home:fbdtemme.list
curl -fsSL https://download.opensuse.org/repositories/home:fbdtemme/Debian_Unstable/Release.key | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/home_fbdtemme.gpg > /dev/null
sudo apt update
sudo apt install torrenttools

openSUSE

For openSUSE Tumbleweed run the following as root:

zypper addrepo https://download.opensuse.org/repositories/home:fbdtemme/openSUSE_Tumbleweed/home:fbdtemme.repo
zypper refresh
zypper install torrenttools

For openSUSE Leap 15.2 run the following as root:

zypper addrepo https://download.opensuse.org/repositories/home:fbdtemme/openSUSE_Leap_15.2/home:fbdtemme.repo
zypper refresh
zypper install torrenttools

For openSUSE Leap 15.3 run the following as root:

zypper addrepo https://download.opensuse.org/repositories/home:fbdtemme/openSUSE_Leap_15.3/home:fbdtemme.repo
zypper refresh
zypper install torrenttools

SUSE Linux Enterprise Server 15

For SLE 15 SP2 run the following as root:

zypper addrepo https://download.opensuse.org/repositories/home:fbdtemme/SLE_15_SP2/home:fbdtemme.repo
zypper refresh
zypper install torrenttools

For SLE 15 SP3 run the following as root:

zypper addrepo https://download.opensuse.org/repositories/home:fbdtemme/SLE_15_SP3/home:fbdtemme.repo
zypper refresh
zypper install torrenttools

Arch

A source package for Arch linux is available on AUR.

git clone https://aur.archlinux.org/torrenttools.git
cd torrenttools
makepkg -is

AppImage

Distributions that have no package yet can use the AppImage that is available for download as a release asset.

Docker

A docker image is available on dockerhub.

docker pull fbdtemme/torrenttools

Building

This library depends on following projects:

• CLI11
• Catch2
• CTRE
• gsl-lite
• RE2
• expected-lite
• fmt
• nlohmann/json
• yaml-cpp
• bencode
• date
• OpenSSL
• Optional: ISA-L Crypto

Almost all dependencies can be fetched from github during configure time or can be installed manually. OpenSSL has to be installed on the system in advance.

Installing build dependencies

Ubuntu

sudo apt install build-essential git cmake g++-10 libssl-dev 

Debian

sudo apt install build-essential git cmake g++-10 libssl-dev libtbb-dev

Fedora/RHEL/CentOS

sudo dnf install cmake make g++ git openssl-devel automake autoconf

Configuration

Option	Type	Description
TORRENTTOOLS_BUILD_TESTS	Bool	Build tests.
TORRENTTOOLS_BUILD_DOCS	Bool	Build documentation.
TORRENTTOOLS_INSTALL	Bool	Generate an install target.
DOTTORRENT_MB_CRYPTO_LIB	String	Pass "isal" for fast multibuffer hashing

Building

This project requires C++20. Currently only GCC 10 or later is supported.

This project can be build as every other project which makes use of the CMake build system.

mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
cmake --build . --target torrenttools

Installation

Installing the project:

sudo cmake --install . --component torrentttools

License

Distributed under the MIT license. See LICENSE for more information.

torrenttools's Issues

Add simple progress option

Add an options to disable the progress bar.
Simple progress reporting is already implemented when the output is piped to a file but there is currently no option to enable it manually.

torrenttools info: segfault on large torrent files

$ curl -s -o test.torrent https://annas-archive.gs/dyn/small_file/torrents/managed_by_aa/annas_archive_data__aacid/annas_archive_data__aacid__upload_files_trantor__20240510T044430Z--20240510T044431Z.torrent

$ du -b test.torrent
2943036 test.torrent

$ du -h test.torrent
2.9M    test.torrent

$ time torrenttools show files test.torrent | wc -l
30000
real    0m0.299s
user    0m0.197s
sys     0m0.106s

$ $(which time) -v torrenttools info test.torrent
Segmentation fault (core dumped)
Command terminated by signal 11
        Command being timed: "torrenttools info test.torrent"
        User time (seconds): 0.42
        System time (seconds): 16.89
        Percent of CPU this job got: 15%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 1:51.79
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 41596
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 13149
        Voluntary context switches: 20292
        Involuntary context switches: 31216
        Swaps: 0
        File system inputs: 5768
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0

`torrenttools show infohash --protocol=2` should not show v1 hash

torrenttools v0.6.2 always returns the v1 hash

$ torrenttools show infohash --protocol=1 test.torrent
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

$ torrenttools show infohash --protocol=2 test.torrent
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

$ torrenttools show infohash --protocol=3 test.torrent
Error: Invalid bittorrent protocol: 3

expected:
torrenttools show infohash --protocol=2 should print nothing on v1-only torrents

Build fails because of fmt

Hi, I tried building following the instructions but it fails:
torrenttools/src/tracker_database.cpp:7:10: fatal error: fmt/args.h: No such file or directory

I am on Arch Linux, and fmt is installed: fmt 7.1.3-1

Docker always restarting

I set up a docker container with docker-compose but the docker shuts down immediately since he is not receiving a command. I would like him to idle.

when I solely run docker run -it --rm fbdtemme/torrenttools this appears and the container exits:

A subcommand is required
torrenttools v0.5.0
Tools for inspecting, creating and modifying bittorrent metafiles.

Usage: torrenttools [OPTIONS] SUBCOMMAND

Options:
  -h,--help                        Print this help message and exit
  --checksum-algorithms            List the supported hash functions for per file checksums.
  -V,--version                     Show program version and exit.
  --config <path>                  Path to custom location for the config.yml file.
  --trackers-config <path>         Path to custom location for the trackers.json file.

Subcommands:
  info                             General information about BitTorrent metafiles.
  create                           Create BitTorrent metafiles.
  verify                           Verify local data against BitTorrent metafiles.
  show                             Show specific fields of BitTorrent metafiles.
  edit                             Edit BitTorrent metafiles.
  magnet                           Generate a magnet URI for BitTorrent metafiles.
  pad                              Generate padding files for a BitTorrent metafile.

But I have not even entered the container.

Here my docker-compose.yml:

version: "3.8"
services:
  torrenttools:
    image: fbdtemme/torrenttools
    container_name: torrenttools
    stdin_open: true
    tty: true 
    volumes:
      - "REDACTED:REDACTED"
    restart: unless-stopped

Same if I want to start the container with docker-compose file.

Seesm to me like the container expects an immediate command, but nothing is written in the documentary. The docker setup is not even mentioned there, nor anything additional on DockerHub.

Please make the docker container persistent, to I can launch him. And then exec into him with:

docker exec -it torrenttools /bin/bash

Or something like that, and then I can do my thing there.

Announce URL Hijacking

When you want to specify a custom announce URL but some part of the URL can be matched to an existing entry in trackers.json, then trackers.json will take precedence.

Using:
torrenttools create -a 'http://please.passthepopcorn.me:2710/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/announce' -p -l 21 -v '1' -o ~/somewhere/ ~/somewhere/something.iso

Result:
Announces:
tier 1 - http://please.passthepopcorn.me:2710/01qwqwdlc922_sample_pid_d93fqd6fji9/announce

The reason for this is that the "passthepopcorn" portion of the URL matches with the name of an entry in trackers.json. This shouldn't happen.

Changing the URL to "please.passthepopcornxxx.me" ... Result:
Announces:
tier 1 - http://please.passthepopcornxxx.me:2710/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/announce

Of course, I can remove the entry from trackers.json (since I won't be using it), but it would nice if they can co-exist

I would think that, checking if the announce value starts with http, and if yes, then do not check in trackers.json

Option for custom source tag in trackers.json

Some trackers (such as HDB) use the full name there, while others (such as PTP) use the abbreviation. It would be useful to be able to match it exactly without specifying -s each time, this way there's no need to redownload the torrent to start seeding.

ISA-L Crypto support

Hi

I have installed "sudo apt install torrenttools" and then installed ISA-L Crypto from their github. does that mean i have torrenttools with ISA-L Crypto support? or am i still using openssl

wrong infohash

[snk-tv.pro].t150.torrent
torrenttools-0.6.0-1.fc33.x86_64.rpm

torrenttools info [snk-tv.pro].t150.torrent
Metafile: /var/data/Upload/[snk-tv.pro].t150.torrent
Protocol version: v1
Infohash: 14570758501b25e736a1b29ad8b08dfbeb16fe5c
Piece size: 1 MiB (1048576 bytes)
Piece count: 802
Created by: uTorrent/3.4.5
Created on: 2021-08-05 15:16:48 UTC
Private: false
Name: King Gary s2 1080p SNK-TV
...

transmission-show [snk-tv.pro].t150.torrent
Name: King Gary s2 1080p SNK-TV
File: [snk-tv.pro].t150.torrent

GENERAL

Name: King Gary s2 1080p SNK-TV
Hash: 79cdf0937d3a6e35200aac218a36a6abb8e4fa33
Created by: uTorrent/3.4.5
Created on: Thu Aug 5 22:16:48 2021
Comment: http://snk-tv.pro/viewtopic.php?t=150
Piece Count: 802
Piece Size: 1.00 MiB
Total Size: 840.0 MB
Privacy: Public torrent
...

Single Announce URL not getting added

Using:
torrenttools create -a "https://blah1.com" -p -l 21 -v '1' -o ~/somewhere/ ~/somewhere/something

Results in no announce URL being added to the metafile

Announces:

---

Using:
torrenttools create -a "https://blah1.com" "https://blah2.com" -p -l 21 -v '1' -o ~/somewhere/ ~/somewhere/something

Results in both URLs being added to the metafile

Announces:
tier 1 - https://blah1.com
tier 2 - https://blah2.com

Publishing an official Docker image?

Thanks for what looks to be a fantastic tool.

There are some cases where having a Docker image to easily run torrenttools might be more convenient / desirable than the existing COPR / launchpad repos.

Would you be willing to consider publishing an official Docker image that is updated as part of releases?

torrenttools edit (--announce) crashing on Windows

On the Windows version, I am using the command torrenttools edit -a "http://newtracker.url/announce.php" "A.Release.by.a-GROUP.torrent" , which crashes, producing no output. Afterwards, using torrenttools info "A.Release.by.a-GROUP.torrent" shows that the announce URL has not been edited. I have also tried using the -o option, as well as using a named tracker (which doesn't update any of the relevant parts of the torrent).

On Linux, this command works as intended.

Event Viewer log:

Faulting application name: torrenttools.exe, version: 0.0.0.0, time stamp: 0x610e32a1
Faulting module name: unknown, version: 0.0.0.0, time stamp: 0x00000000
Exception code: 0xc0000005
Fault offset: 0x00007ff7ab3c0000
Faulting process ID: 0x3680
Faulting application start time: 0x01d797908bc46f20
Faulting application path: C:\Program Files\torrenttools\torrenttools.exe
Faulting module path: unknown
Report ID: 541fa805-e161-4d41-8cea-92aeb9bce5e1
Faulting package full name: 
Faulting package-relative application ID: 

Error: std::bad_alloc

Hey! The project looked extremely promising so I had to give it a go. Unfortunately I have run into an error on my first torrent creator.

torrenttools create "E:\Docker\Deluge\Data\linux.iso" --piece-size 4M --announce "XXX" -p on -o "E:\Docker\Deluge\Data\linux.torrent"

16.00 EiB in 0 directories, 1 files

Error: std::bad_alloc

Support -x,--cross-seed option

This option allows users to easily cross-seed torrents without having to manually define a source flag.
A unique id made from the hash of all announce-urls is embedded inside the info dict.
Behaviour is almost the same as the --source options except that the user does not care what the value of the unique value is exactly.

Apparently there are trackers that strip the source field so a different flag is needed.

This option is implemented by pyrocore with the x_cross_seed field.

Infinite loop parsing LS_COLORS with 256 colors

Hi,
I 'm using torrenttools-0.3.2 and it works fine on my centos 7 server.
But when i upgrade it to torrenttools-0.5.1 it is not working any more.
It stuck on this

Scanning target directory: 0 files processed
Sorting file list... Done.
Adding files to metafile... Done.

ldd /usr/bin/torrenttools

linux-vdso.so.1 =>  (0x00007ffeb21f0000)
libtbb.so.2 => /lib64/libtbb.so.2 (0x00007f596e813000)
libcrypto.so.1.1 => /usr/local/openssl/lib/libcrypto.so.1.1 (0x00007f596e34d000)
libstdc++.so.6 => /usr/local/gcc8/lib64/libstdc++.so.6 (0x00007f596dfc9000)
libm.so.6 => /lib64/libm.so.6 (0x00007f596dcc7000)
libgcc_s.so.1 => /usr/local/gcc8/lib64/libgcc_s.so.1 (0x00007f596daaf000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f596d893000)
libc.so.6 => /lib64/libc.so.6 (0x00007f596d4c5000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f596d2c1000)
librt.so.1 => /lib64/librt.so.1 (0x00007f596d0b9000)
libz.so.1 => /lib64/libz.so.1 (0x00007f596cea3000)
/lib64/ld-linux-x86-64.so.2 (0x00007f596ea55000)

I attached strace.log please take a look.
Please any advise, thank you.
strace.log

torrenttools changing windows cmd behaviour

After executing torrenttools, the arrow keys no longer move the cursor left and right, but produce the escape sequences ("^[[D^" and the like). Backspace is also affected, now deleting entire words rather than single characters.

Extract the SHA-1 hashes from the torrent and write them into a sha1sum file.

This is related to this comment in a Reddit post.

Core dumped error

When I try to create a .torrent on Fedora 37 with last version, I get following error:
/usr/include/c++/11/string_view:233: constexpr const value_type& std::basic_string_view<_CharT, _Traits>::operator[](std::basic_string_view<_CharT, _Traits>::size_type) const [with _CharT = char; _Traits = std::char_traits<char>; std::basic_string_view<_CharT, _Traits>::const_reference = const char&; std::basic_string_view<_CharT, _Traits>::size_type = long unsigned int]: Assertion '__pos < this->_M_len' failed. Aborted (core dumped)

I’ve tried different files, but no luck. Other functions seem to be working. Any help?

Implement BEP-17: http-seeding

Currently there is support for BEP-19: WebSeed - HTTP/FTP Seeding (GetRight style), but not for the other HTTP Seeding mechanism specified in BEP-17 which adds the "url-list" key.

BEP-17 HTTP seeding is superseeded by BEP-19 and should not be used in new torrents, but it is still usefull to support this for editing and inspecting older torrent files.

config.yml mentions Windows and Linux but not macOS

macOS is left out of the following

# This is an example configuration file.
# It should be copied to the suitable directory for your OS and modified there.
# Linux : $HOME/.config/torrenttools/config.yml
# Windows : %AppData%/torrenttools/config.yml

CLI11 branch name

torrenttools/external/CLI11.cmake

Line 27 in a47cb69

GIT_TAG master

this needs to be main, currently torrenttools doest build because of this

honestly you should either copy 3rd party libs into the project or add it as a git submodule fixed at a specific tag to avoid stuff like this in the future

Build on debian 11: missing TBB

I followed instructions how to build on Ubuntu (I use Debian 11) this command:
cmake --build . --target torrenttools

Gave me this error:
-- Using cryptographic library: OpenSSL -- Could NOT find TBB (missing: TBB_DIR) CMake Error at cmake/FindTBB.cmake:142 (message): Required library TBB not found. Call Stack (most recent call first): cmake/FindTBB.cmake:378 (findpkg_finish) CMakeLists.txt:54 (find_package)

Sign various packages

All binary releases should be signed to allow users to validate the contents of packages.
Currently only deb packages are signed since it is required for building on launchpad.

source tag isnt getting added

https://github.com/fbdtemme/torrenttools/blob/main/src/create.cpp#L392
is that a typo?

Using:

--comment "this is a comment"  \
--source "testsource" 

Ouput:

Source:           
Comment:          this is a comment

Using only:
--source "testsource"

Ouput:

Source:           
Comment:          this is a comment

Error: parse error: invalid character read: value limit exceeded reading newly created hybrid torrent

I used torrenttools to create a hybrid torrent on about 1TiB of data. The resulting file is about 20MB and can't be opened by qBittorrent ("the v1 and v2 metadata does not match"), but actually even torrenttools info out.torrent fails with an exception:

Catchpoint 1 (exception thrown), 0x00007ffff76a91b1 in __cxxabiv1::__cxa_throw (obj=0x555555989dd0, tinfo=0x55555593f598 <typeinfo for bencode::parsing_error>, dest=0x55555568b080 <bencode::parsing_error::~parsing_error()>) at ../../../../src/libstdc++-v3/libsupc++/eh_throw.cc:81
81      ../../../../src/libstdc++-v3/libsupc++/eh_throw.cc: No such file or directory.
(gdb) bt
#0  0x00007ffff76a91b1 in __cxxabiv1::__cxa_throw (obj=0x555555989dd0, tinfo=0x55555593f598 <typeinfo for bencode::parsing_error>, dest=0x55555568b080 <bencode::parsing_error::~parsing_error()>) at ../../../../src/libstdc++-v3/libsupc++/eh_throw.cc:81
#1  0x000055555574f93e in bencode::decode_view (view="d8:announce34:udp://tracker.coppersurfer.tk:696913:announce-listll34:udp://tracker.coppersurfer.tk:6969el40:udp://tracker.leechers-paradise.org:6969el28:udp://exodus.desync.com:6969el42:udp://tracker."...)
    at /code/build/torrenttools/build/_deps/bencode-src/include/bencode/detail/decode_view.hpp:24
#2  0x000055555574d1ca in dottorrent::read_metafile (view="d8:announce34:udp://tracker.coppersurfer.tk:696913:announce-listll34:udp://tracker.coppersurfer.tk:6969el40:udp://tracker.leechers-paradise.org:6969el28:udp://exodus.desync.com:6969el42:udp://tracker."...)
    at /code/build/torrenttools/build/_deps/dottorrent-src/src/metafile.cpp:262
#3  0x000055555574d7d6 in dottorrent::load_metafile (path=filesystem::path "/some/redacted/path/out.torrent" = {...}) at /code/build/torrenttools/build/_deps/dottorrent-src/src/metafile.cpp:324
#4  0x0000555555674291 in run_info_app (main_options=..., options=...) at /code/build/torrenttools/src/info.cpp:52
#5  0x000055555568dd9c in main (argc=3, argv=0x7fffffffd408) at /code/build/torrenttools/src/main.cpp:71

The exception message is Error: parse error: invalid character read at position 9302219: value limit exceeded.
Here is the (lightly redacted) start of the log, during the torrent creation:

Scanning target directory: 56509 files processed
Sorting file list... Done.
Adding files to metafile... Done.
Metafile:          /some/redacted/path/out.torrent
Protocol version:  v1 + v2 (hybrid)
Piece size:        4 MiB (4194304 bytes)
Piece count:       263162
Created by:        torrenttools/0.6.2
Created on:        2022-07-18 19:46:21 UTC
Private:           false
Name:              test
Source:            
Comment:           
Announce-urls:     tier 1  - udp://tracker.coppersurfer.tk:6969
                   tier 2  - udp://tracker.leechers-paradise.org:6969
                   tier 3  - udp://exodus.desync.com:6969
                   tier 4  - udp://tracker.opentrackr.org:1337/announce
                   tier 5  - udp://tracker.uw0.xyz:6969/announce
DHT nodes:         
Web seeds:         
HTTP seeds:        
Similar torrents:  
Collections:       Test collection
Other info fields: 

Files:

Metafile contains more than 1000 files: skipping file tree ...

  1 TiB in 6516 directories, 57385 files


Hashing files...
[progress snipped]

I can provide the 20MB output torrent privately if necessary.

No jammy release in launchpad repo

It seems there isn't a release of torrenttools for the Ubuntu jammy in the launchpad repo, and as such causes an error upon attempting to update repos

aur package won't build

Commands to reproduce: yay -S torrenttools
Errors:

/usr/include/fmt/core.h:1757:7: note: ‘formattable’ evaluates to false
make[3]: *** [external/dottorrent/CMakeFiles/dottorrent.dir/build.make:132: external/dottorrent/CMakeFiles/dottorrent.dir/src/chunk_reader.cpp.o] Error 1
make[3]: *** [external/dottorrent/CMakeFiles/dottorrent.dir/build.make:104: external/dottorrent/CMakeFiles/dottorrent.dir/src/chunk_hasher_single_buffer.cpp.o] Error 1
make[3]: *** [external/dottorrent/CMakeFiles/dottorrent.dir/build.make:118: external/dottorrent/CMakeFiles/dottorrent.dir/src/chunk_processor_base.cpp.o] Error 1
make[2]: *** [CMakeFiles/Makefile2:1258: external/dottorrent/CMakeFiles/dottorrent.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:1109: CMakeFiles/torrenttools.dir/rule] Error 2
make: *** [Makefile:577: torrenttools] Error 2
==> ERROR: A failure occurred in build().
    Aborting...
 -> error making: torrenttools

Aur package version: torrenttools-0.6.0-1

`Error: wstring_convert::from_bytes`

# torrenttools create --config ~/config.yml --trackers-config ~/trackers.json -t 1 -o ~/torrenttools-ftp.torrent -a udp://tracker.opentrackr.org:1337/announce 03.Audio\ Courses,\ Textbooks/
Scanning target directory: 0 files processed
Sorting file list... Done.
Adding files to metafile... Done.
Metafile:          /root/torrenttools-ftp.torrent
Protocol version:  v1
Piece size:        16 MiB (16777216 bytes)
Piece count:       567
Created by:        torrenttools/0.5.0
Created on:        2023-01-15 02:52:50 UTC
Private:           false
Name:              03.Audio Courses, Textbooks
Source:
Comment:
Announce-urls:     tier 1  - udp://tracker.opentrackr.org:1337/announce
DHT nodes:
Web seeds:
HTTP seeds:
Similar torrents:
Collections:
Error: wstring_convert::from_bytes

Happens both on Alpine container and with AppImage download.

Silent torrent creation

I would love a silent option for torrent creation that displays nothing.
Like the other OP asking for silent mentioned. It is crazy stupid fast dude. I love it. Saves me a TON of work.

Chinese characters are garbled

Whether I use utf-8 or gb2312, it produces garbled characters.

Implement BEP 38: Finding Local Data Via Torrent File Hints

More trackers in default trackers.json

It would be great if more private trackers are included in the default trackers configuration file.
If you want to have a tracker added you can post the information here or send a pull request.

Could not build on Debian Sid/testing

The current build fail at many steps.

1. I assume that Bencode recently changed it's master branch to main. But the torrentools makefile still search for a master tag in external/bencode.cmake: GIT_TAG master

   • Solution: by changing it to "main".

2. Then is fails because upstream errors with bencode compilation. I opened an issue about this upstream : Bencode: Build errors with deps and instructions

   • Current solution: by compiling and installing the deps manually.

3. Then it fails because it does not find nlohmann/json.

   • Current solution: by compiling and installing manually.

[ 81%] Building CXX object CMakeFiles/torrenttools.dir/src/common.cpp.o
In file included from /home/tomasz/Builds/torrenttools/include/common.hpp:8,
                 from /home/tomasz/Builds/torrenttools/src/common.cpp:4:
/home/tomasz/Builds/torrenttools/include/tracker_database.hpp:11:10: fatal error: nlohmann/json.hpp: Aucun fichier ou dossier de ce type
   11 | #include <nlohmann/json.hpp>
      |          ^~~~~~~~~~~~~~~~~~~
compilation terminated.

4. Some compilation errors with a pattern in lines 50 & 65 of include/file_matcher.hpp :
   • Unresolved: I stopped here because I'm not good enough with C++

[ 81%] Building CXX object CMakeFiles/torrenttools.dir/src/create.cpp.o                                                                                                                      
In file included from /home/tomasz/Builds/torrenttools/src/create.cpp:22:                                                                                                                    
/home/tomasz/Builds/torrenttools/include/file_matcher.hpp: In member function ‘void torrenttools::file_matcher::allow_extension(std::string_view)’:                                          
/home/tomasz/Builds/torrenttools/include/file_matcher.hpp:50:31: error: unable to find string literal operator ‘operator""_format’ with ‘const char [7]’, ‘long unsigned int’ arguments      
   50 |         std::string pattern = ".*.{}$"_format(extension);                                                                                                                            
      |                               ^~~~~~~~~~~~~~~                                                                                                                                        
/home/tomasz/Builds/torrenttools/include/file_matcher.hpp: In member function ‘void torrenttools::file_matcher::block_extension(std::string_view)’:                                          
/home/tomasz/Builds/torrenttools/include/file_matcher.hpp:65:31: error: unable to find string literal operator ‘operator""_format’ with ‘const char [8]’, ‘long unsigned int’ arguments      
   65 |         std::string pattern = ".*\\.{}$"_format(extension);                                                                                                                          
      |                               ^~~~~~~~~~~~~~~~~

5. It's not an error but you should add this dependancy in the Readme : termcontrol

Improve interactiveness during filesystem scan

When creating a torrent file the time to collect all files that match the filters can take a long time if the input directory contains many files. torrenttools should run this step in a separate thread instead of on the main thread and provide simple reporting of how many files are process. This will avoid giving the impression of a freeze if this steps takes longer then a few seconds.

Consider using a random value for cross-seeding rather than a hash of the announce URL

Using the hash of the announce URL as entropy for cross-seeding can lead to a potential privacy violation. Consider an anonymous uploader on a private tracker. If they use torrenttools's cross-seed feature, it becomes possible to tell if two anonymous uploads came from the same person, unless the uploader regularly rotates their passkey (which is usually a painful process). Please consider just using a random value for this like pyrocore does.

Reached max memory?

I use the include filter to select 1700 files from a folder with many more. A 700 file filter is no problem. I get the error Error: re2 compiler out of memory. I have 5GB available so I think I have enough for the regexp, but perhaps there's a configured cap somewhere?

What about giving an option of increasing the max memory usage for re2?

cat /proc/meminfo
MemTotal:        8040328 kB
MemFree:          134448 kB
MemAvailable:    5704240 kB

Flag for Silent output of "hashing Files"

is it possible to add flag for this. I want to avoid spam on terminal for folders with many files.

Thanks for tool. it's really fast.

[Error] 'torrenttools show size' displays wrong filesize

'torrenttools show size' displays wrong filesize if the torrent contains a single file above 4GB. This torrent has a piece length of 4KB. Need a solution. Thank you in advance.

How to make two files into one bittorrent seed

For example like this

root@ubuntu:/home/ubuntu# torrenttools create "/var/filebrowser/srv/VMware-workstation-full-16.2.1-18811642.exe /var/filebrowser/srv/VMware-workstation-full-16.2.1-18811642.exe.md5" -a "https://tracker.tamersunion.org/announce" "udp://opentor.org:2710/announce" -d "router.utorrent.com:6881" "router.bittorrent.com:6881" "dht.transmissionbt.com:6881" "router.bitcomet.com:6881" "dht.aelitis.com:6881" -o "VMware-workstation-full-16.2.1-18811642.exe.torrent"
Error: Path does not exist: /var/filebrowser/srv/VMware-workstation-full-16.2.1-18811642.exe /var/filebrowser/srv/VMware-workstation-full-16.2.1-18811642.exe.md5

I want to package and distribute the file with its md5 file, but it looks like it cannot be created at this stage

Unknown parsing error on Win10

Hi,

Regardless of whether I use the default config.yml, or some configured one, I am getting this error.

Error: [json.exception.parse_error.101] parse error at line 10, column 15: syntax error while parsing object - unexpected string literal; expected '}'

If I edit line 10 of the config.yml to enclose the {pid}, like so
pid: {example_pid}

Then I receive this error.
Error: yaml-cpp: error at line 10, column 10: bad conversion

how to double-check verify?

I'm having a problem verifying some large torrents that a (doubled, tripled) verified by transmission but have either their unique large file or the last of their files listed as 99% by torrenttools.

I was wondering if I had a way to get detailed output and/or instructions to debug this ?

[I then picked torf (from your performance chart) and verified one torrent. It agrees with transmission so I'm inclined to think I found a bug. How can I help?]

macOS build fails with dyld: lazy symbol binding failed: Symbol not found

macOS fails with the following error

xxx@xxx-Mac-mini ~/Downloads> /usr/local/bin/torrenttools
dyld: lazy symbol binding failed: Symbol not found: ___stack_logging_early_finished
  Referenced from: /Library/torrenttools/bin/../lib/libSystem.B.dylib
  Expected in: /Library/torrenttools/bin/../lib/libSystem.B.dylib

dyld: Symbol not found: ___stack_logging_early_finished
  Referenced from: /Library/torrenttools/bin/../lib/libSystem.B.dylib
  Expected in: /Library/torrenttools/bin/../lib/libSystem.B.dylib

fish: Job 1, '/usr/local/bin/torrenttools' terminated by signal SIGABRT (Abort)

This is on an M1 mac with Rosetta installed

Excessive memory usage for creating huge torrents (> 2 TiB)

Trying to create a torrents with ~200,000 files, 2 TiB in total, and the memory usage surged up to 12 GB when hashing. The progress isn't moving and stuck at 0 B/s. Any ideas for optimizing this?

Edit: The progress began moving after a few minutes, at a reasonable rate. But the memory usage is still high.

Edit2: No. The progress is actually broken. It first went extremely fast to 100%, and freezes. But the network activity shows the progress is still running.

Thanks in advance.

Can't create torrent using docker image

Hi,

I'm not sure what I'm doing wrong, but I just can't get it to work :

$ docker run -v /media:/media --name torrenttools --rm fbdtemme/torrenttools create -v hybrid -a udp://... udp://... -o /media/ '/media/shows/...'
Configuration error: no configuration was found
Warning: could not find config.yml file: not all features will be available.
$ 

It just exits without generating a torrent.
I've tried mapping an empty file using -v ./empty:/etc/torrenttools/config.yml but it makes no difference, exact same error.

Windows 10 stats overwritten

https://ptpimg.me/k07nar.png

DEB files download ?

Hello,

@fbdtemme mentioned DEB packages in #11.

Since the PPA cannot be used in Ubuntu 21.10, where can I find those DEB files ?

I looked at launchpad.net/torrenttools but :

torrenttools does not have any download files registered with Launchpad.

Thanks

Installation Fails on Mac OS 13.1 ventura with Apple Silicon

Hi, while installing torrenttools-0.6.2-macos-x86_64.pkg on Mac OS 13.1, the following error message occurs:

There is no other information provided.

Disable cross-seed in profile config

There's no option like that in profile config.