syncthing/syncthing

Open Source Continuous File Synchronization

---

Goals

Syncthing is a continuous file synchronization program. It synchronizes files between two or more computers. We strive to fulfill the goals below. The goals are listed in order of importance, the most important ones first. This is the summary version of the goal list - for more commentary, see the full Goals document.

Syncthing should be:

1. Safe From Data Loss

   Protecting the user's data is paramount. We take every reasonable precaution to avoid corrupting the user's files.

2. Secure Against Attackers

   Again, protecting the user's data is paramount. Regardless of our other goals, we must never allow the user's data to be susceptible to eavesdropping or modification by unauthorized parties.

3. Easy to Use

   Syncthing should be approachable, understandable, and inclusive.

4. Automatic

   User interaction should be required only when absolutely necessary.

5. Universally Available

   Syncthing should run on every common computer. We are mindful that the latest technology is not always available to every individual.

6. For Individuals

   Syncthing is primarily about empowering the individual user with safe, secure, and easy to use file synchronization.

7. Everything Else

   There are many things we care about that don't make it on to the list. It is fine to optimize for these values, as long as they are not in conflict with the stated goals above.

Getting Started

Take a look at the getting started guide.

There are a few examples for keeping Syncthing running in the background on your system in the etc directory. There are also several GUI implementations for Windows, Mac, and Linux.

Docker

To run Syncthing in Docker, see the Docker README.

Getting in Touch

The first and best point of contact is the Forum. If you've found something that is clearly a bug, feel free to report it in the GitHub issue tracker.

If you believe that you’ve found a Syncthing-related security vulnerability, please report it by emailing [email protected]. Do not report it in the Forum or issue tracker.

Building

Building Syncthing from source is easy. After extracting the source bundle from a release or checking out git, you just need to run go run build.go and the binaries are created in ./bin. There's a guide with more details on the build process.

Signed Releases

As of v0.10.15 and onwards, release binaries are GPG signed with the key D26E6ED000654A3E, available from https://syncthing.net/security/ and most key servers.

There is also a built-in automatic upgrade mechanism (disabled in some distribution channels) which uses a compiled in ECDSA signature. macOS binaries are also properly code signed.

Documentation

Please see the Syncthing documentation site [source].

All code is licensed under the MPLv2 License.

syncthing's Issues

Distributed Hash Table for discovery instead of central server

I'm not sure how to implement it, or if it even makes sense, but I'm under the impression that this is what BitTorrent Sync does.

Running out of fd:s under Linux

01:22:41 model_puller.go:176: WARNING: open /data/syncer/sync/Foton/!Incoming/.syncthing.20131229-065755-03623.dng.1388331458: too many open files
01:22:41 model_puller.go:176: WARNING: open /data/syncer/sync/Foton/!Incoming/.syncthing.20131229-065858-03633.dng.1388331459: too many open files
01:22:41 model_puller.go:176: WARNING: open /data/syncer/sync/Foton/Florida 2013/.syncthing.20131226-141931-03497.dng.1388196370: too many open files
01:22:41 model_puller.go:176: WARNING: open /data/syncer/sync/Foton/Hemma/.syncthing.20131112-200205-7133.dng.1387820823: too many open files
01:22:42 model_puller.go:176: WARNING: open /data/syncer/sync/Foton/!Incoming/.syncthing.20131229-065858-03633.dng.1388331459: too many open files
01:22:42 model_puller.go:176: WARNING: open /data/syncer/sync/Foton/Florida 2013/.syncthing.20131226-141931-03497.dng.1388196370: too many open files
01:22:42 model_puller.go:176: WARNING: open /data/syncer/sync/Foton/Hemma/.syncthing.20131112-200205-7133.dng.1387820823: too many open files
01:22:42 model_puller.go:176: WARNING: open /data/syncer/sync/Foton/Florida 2013/.syncthing.20131226-113234-03420.dng.1388196381: too many open files
01:22:42 model_puller.go:176: WARNING: open /data/syncer/sync/Foton/Florida 2013/.syncthing.20131226-115145-03432.dng.1388196308: too many open files
01:22:42 model_puller.go:176: WARNING: open /data/syncer/sync/Foton/Florida 2013/.syncthing.20131222-143434-00006.mts.1387722874: too many open files
01:22:42 model_puller.go:176: WARNING: open /data/syncer/sync/Foton/!Incoming/.syncthing.20131229-065758-03625.dng.1388331458: too many open files

Ignored files should not be hashed

Currentlt the ignore filter takes effect too late. It should filter before hashing, not after.

Support daemonisation

Should syncthing start in daemonised mode, or have an option to start/not start in daemonised mode?

Keep temporaries for future sync

If sync of a large file is interrupted, it would be better to keep the temporary around. Any blocks in that file that pass the hash check could then be reused when we retry synchronization later on. Out of date temporaries should still be deleted.

--delete by default?

I think the expected behaviour is that if you delete a file in a synchronised directory, then that change should be made everywhere else, the same for if you change a file.

mygo/src/github.com/calmh/syncthing/model/model.go:371: undefined: sha1.Sum

$ go version
go version go1.1.2 linux/amd64
$ go get github.com/calmh/syncthing
# github.com/calmh/syncthing/model
mygo/src/github.com/calmh/syncthing/model/model.go:371: undefined: sha1.Sum

Is this because go 1.2 bundles crypto/sha1?

Windows releases lack binary

The Windows releases don't seem to have a binary in them.

Incremental model updates

Don't recompute the full model on index/local updates.

tar gz releases contain weird files

From OS X I think:

$ find syncthing-linux-amd64/
syncthing-linux-amd64/
syncthing-linux-amd64/._README.md   <---
syncthing-linux-amd64/._syncthing   <---
syncthing-linux-amd64/LICENSE
syncthing-linux-amd64/._LICENSE   <---
syncthing-linux-amd64/syncthing
syncthing-linux-amd64/README.md

Zero length files are not synchronized

GUI should show how much unique stuff we have, that other nodes need to synchronize

Preserve mode of files

The octal mode of files and directories is currently not preserved.

For example:

$ touch file1 file2
$ chmod 600 file1
$ chmod 400 file2

These files will not have the modes specified above on the other nodes in the cluster.

(I'm not going to request that user and group ownership be preserved, I don't think that's useful at the moment.)

Mode and modification times on directories should be preserved

If I sync a file, the metadata on the target is different from on the source. So far I have only checked output from stat, not xattrs.

Support "~" for dir in ~/.syncthing/syncthing.ini

Would be good if this worked:

$ head ~/.syncthing/syncthing.ini 

[repository]
dir = ~/share

.stignore not replicated to other nodes in the cluster

If this is intended, should it be documented?

Inotify support

To notice changes more quickly.

Will syncthing do the right thing for leap seconds, etc.?

syncthing's model for resolving file conflicts is newest change wins. Will syncthing do the right thing during leap seconds, time zone changes, etc.?

Clarify and improve change suppression mechanism

Still troubleshooting this. I think I am seeing an issue in 0.4 where I create an empty file, then later add content to it, and the content never gets added to the other nodes.

I'll update this issue in an hour or two with proper info.

Suggestion: more verbose documentation

If the documentation explicitly stated what to do on each separate node (i.e. that the ID is not shared), and that the security comes from having an unguessable node id, then it could reach a stage where a copy and paste setup would be possible.

This might mean that the node id generation part would need a separate --option, like --gen-node-id or similar.

Ability to ignore files with glob patterns

Perhaps a .stignore file that works like .gitignore?

Provide linux arm builds of releases

I filed another issue for having problems build, but what would be really useful is to have linux armhf builds (think raspberry pi, ubuntu phone and others) to run syncthing on.

Back off on ever changing files

For example a VM image for a running machine, gets rehashed every minute and causes high CPU usage. Better to leave it alone for a wile when we see that it changes every scan.

Max out TLS security

Since the cryptography in syncthing is entirely self-enclosed and you don't need to worry about breaking external stuff, there is a good opportunity to make TLS as secure as possible.

The 30c3 talk about better cryptography might be interesting https://bettercrypto.org/

Local repository seems to get rescanned for too long at startup

Is the cache not fully doing it's thing?

Renames should be handled efficiently

Currently, seen as a delete and add of a new file to resync. We know all the blocks are identical and could easily handle it as a rename.

Add -v --version options

I can't seem to find where the version number of syncthing is stored in the source, so didn't attempt a patch.

Would be useful to have an option to view the version from the binary.

--ro mode should be advertised in INFO

Running the following command:

$ syncthing --ro
....

Does not mention in the output that syncthing is being run in readonly mode.

Missing build instructions

I grabbed syncthing 0.4.0 and am attempting to build on an arm device. There's no documentation that I could find for building it, and the build.sh script fails.

http://paste.ubuntu.com/6720643/

Add more documentation in the code?

Hello, I recently decided to write a simple open source BitTorrent Sync clone and I thought the best language for it was Go, so I took it as an opportunity to learn it.
Surprise! It has already been done :)

The code looks very well written but I don't get much because it's not very commented. If you would take some time to add a little documentation I could understand it faster and help improving it better.

Keep up the good work!

Tunneling over HTTP(S)

Might be a nice feature to have the option to proxy the protocol over http.

tooltip for status symbols in gui

Perhaps the tooltip for "not connected" (the minus sign) could be made clearer with a tooltip.

Delete of file not propagating

(Deleted content. I was mistaken.)

Warning:%!(EXTRA *errors.errorString=EOF)

I have added a third node to config.xml and get the following error:

Warning:%!(EXTRA *errors.errorString=EOF)

Running the file through xmllint makes the following change:

<?xml version="1.0"?>

but syncthing still shows the error.

I'll troubleshoot some more then post my config.xml

web interface: local node is missing if not listed in the config file

I think I have this right.

The local node is always implicitly part of every cluster, but this is not reflected in the web interface.

info output should advertise address of gui

Something like:

16:05:59 INFO: Gui started at http://127.0.0.1:8080

multiple guiAddress and listenAddress not supported

It'd be useful it multiple guiAddress and listenAddress values were supported.

e.g. say I have three network cards, and I'd like to listen on localhost and the internal device.

The info string would also need to support this.

Show version of each node in web interface

Might be useful to spot old nodes.

Improve settings handing in GUI

• Try to avoid requiring a restart at all for most settings.
• If settings are changed that require a restart, indicate those as "pending" somehow.

Auto-create syncthing.ini too

syncthing creates the directory

~/.syncthing

on its first run.

If there is no

~/.syncthing/syncthing.ini

file perhaps a bare config file with the nodeid could be created.

panic: runtime error: slice bounds out of range

[YUFJO] 2014/02/01 07:33:03 INFO:   2559 files,   1.82 GiB in local repo
[YUFJO] 2014/02/01 07:33:03 INFO:     66 files, 213.06 MiB to synchronize
panic: runtime error: slice bounds out of range

goroutine 34 [running]:
runtime.panic(0x7561e0, 0xc3136a)
        /home/jb/go-sunos/src/pkg/runtime/panic.c:264 +0xb6
github.com/calmh/syncthing/model.(*FileQueue).Get(0xc2100e0280, 0xc21068c540, 0x34, 0x0, 0x0, ...)
        /home/jb/src/github.com/calmh/syncthing/model/filequeue.go:121 +0x48e
github.com/calmh/syncthing/model.func·004()
        /home/jb/src/github.com/calmh/syncthing/model/model.go:511 +0x45f
created by github.com/calmh/syncthing/model.(*Model).AddConnection
        /home/jb/src/github.com/calmh/syncthing/model/model.go:522 +0x266

Manage syncthing on a Mac?

What do you recommend to manage syncthing on Mac? Is there some kind of tray app that lets you start things at login and restart them, click to view status, etc?

GUI needs to tell user when connection to the local node has died

Now that syncthing runs with the gui by default, the user needs to be told somehow if syncthing exits.

Ability to add a comment or description to nodes

To set a friendly name like "home-nas" to be shown instead of a long node ID.

Rethink / rework config file vs CLI options

The following command lists lots of options:

$ syncthing --help
....

It would be helpful (and keep the documentation simple) if there was a one-liner that mentioned how to add these options to the config file.

-LATEST release url?

Does GitHub provide a way of pointing a -LATEST url at the latest release under https://github.com/calmh/syncthing/releases?

Current file pull mechanism is inefficient for clusters with >2 members

The current puller launches a bunch of parallell requests to all connected nodes per each file it synchronizes. For files that are only a few blocks in size, this means we are effectively getting one block from each peer and end up being limited by the slowest of the bunch. (For large files, we might have time to get 100 blocks from one peer while we get only one from another peer, so that's fine.)

This needs to be rewritten to not handle files but a single block queue with a puller per peer node each pulling blocks from the queue as fast as they can.

Security section: breaking out of repo dir?

The README.md has a security section, but it doesn't mention if there is protection against a malicious node in a cluster, e.g. requesting files outside of the repo dir (../../../../etc/passwd or similar).

Maybe too much detail for this stage, but just a suggestion.

Allow a node to join multiple clusters

I can't find the issue for this.

Rather than having lots of instances of syncthing running on various different ports, a single syncthing instance which supports membership to multiple clusters would be nice.

Use case:

I have a PC that has the following structure:

~/Work
~/Documents
~/Photos

I share each of these directories with different people and PCs.

Error in walk truncates local repo model

19:21:22 model.go:94: INFO: 6NUTDWOOROOVAS3O5RLDZCUETCRPQBS4: 385 B/s in, 380.5 KB/s out
19:22:22 model.go:94: INFO: 6NUTDWOOROOVAS3O5RLDZCUETCRPQBS4: 368 B/s in, 341.7 KB/s out
19:22:47 model.go:202: INFO: Disconnected from node 6NUTDWOOROOVAS3O5RLDZCUETCRPQBS4
19:23:22 model.go:104: INFO:  18293 files,  24.26 GB in cluster
19:24:20 walk.go:123: WARNING: open /Users/jb/Sync/Foton/!Incoming/Temporary Folder 3A359EA8-796F-4C64-8EBE-8C04B0C2D13E: no such file or directory
19:25:22 model.go:104: INFO:  18332 files,   8.06 GB in cluster
19:25:40 walk.go:123: WARNING: open /Users/jb/Sync/Foton/!Incoming/Temporary Folder 63854B6C-85DE-454F-B429-64F7DC20B7F2: no such file or directory
19:26:22 model.go:104: INFO:  18364 files,   8.39 GB in cluster
19:26:28 main.go:290: INFO: Connected to node 6NUTDWOOROOVAS3O5RLDZCUETCRPQBS4 (out)
19:27:22 model.go:94: INFO: 6NUTDWOOROOVAS3O5RLDZCUETCRPQBS4: 38.7 KB/s in, 105.1 KB/s out
19:28:22 model.go:94: INFO: 6NUTDWOOROOVAS3O5RLDZCUETCRPQBS4: 33.1 KB/s in, 0 B/s out
19:29:22 model.go:94: INFO: 6NUTDWOOROOVAS3O5RLDZCUETCRPQBS4: 35.0 KB/s in, 0 B/s out