schollz/peerdiscovery

Pure-Go library for cross-platform local peer discovery using UDP multicast :woman: :repeat: :woman:

peerdiscovery

  

Pure-go library for cross-platform thread-safe local peer discovery using UDP multicast. I needed to use peer discovery for croc and everything I tried had problems, so I made another one.

Install

Make sure you have Go 1.5+.

go get -u github.com/schollz/peerdiscovery

Usage

The following is a code to find the first peer on the local network and print it out.

discoveries, _ := peerdiscovery.Discover(peerdiscovery.Settings{Limit: 1})
for _, d := range discoveries {
    fmt.Printf("discovered '%s'\n", d.Address)
}

Here's the output when running on two computers. (Run these gifs in sync by hitting Ctl + F5).

Computer 1:

Computer 2:

For more examples, see the scanning examples (ipv4 and ipv6) or the docs.

Testing

To test the peer discovery with just one host, one can launch multiple containers. The provided Dockerfile will run the example code. Please make sure to enable Docker's IPv6 support if you are using IPv6 for peer discovery.

# Build the container, named peertest
$ docker build -t peertest .

# Execute the following command in multiple terminals
$ docker run -t --rm peertest
Scanning for 10 seconds to find LAN peers
 100% |████████████████████████████████████████|  [9s:0s]Found 1 other computers
0) '172.17.0.2' with payload 'zqrecHipCO'

Contributing

Pull requests are welcome. Feel free to...

• Revise documentation
• Add new features
• Fix bugs
• Suggest improvements

Thanks

Thanks @geistesk for adding IPv6 support and a Notify func, and helping maintain! Thanks @Kunde21 for providing a bug fix and massively refactoring the code in a much better way. Thanks @robpre for finding and fixing bugs. Thanks @shvydky for adding dynamic payloads.

License

MIT

peerdiscovery's Issues

Add possibility to generate payload dynamically

In my case, it is required to send different payloads in every broadcast.

I propose to have an optional function pointer in Settings that will return payload on demand. In case when this pointer is nil the implementation should fallback to Payload field.

I will try to implement this in the fork and create PR.

@schollz, please let me know what do you think about my idea?

AllowSelf can skip 0.0.0.0 and connect to itself

peerdiscovery/listener.go

Line 127 in 70a1537

srcHost, _, _ := net.SplitHostPort(src.String())

For example, if the computer has lost access to the Internet, the IP 0.0.0.0.0 will appear in Notify

Can't compile using gcc-go

I've tested 2 gcc-go versions on Arch Linux 10.2.0 and 10.3.0
go version go1.14.4 gccgo (GCC) 10.2.0 linux/amd64
go version go1.14.6 gccgo (GCC) 10.3.0 linux/amd64

I've also tested on Debian testing, just to see if it was maybe Arch issue, but got same error
gccgo-go on Debian testing reports
go version go1.14.6 gccgo (Debian 10.2.1-6) 10.2.1 20210110 linux/amd64
the error is

go get -u github.com/schollz/peerdiscovery
# github.com/schollz/peerdiscovery
../../go/src/github.com/schollz/peerdiscovery/peerdiscovery.go:381:6: error: incompatible type for receiver (cannot use type PacketConn4 as type socket.Conn)
  381 | type PacketConn4 struct {
      |      ^
../../go/src/github.com/schollz/peerdiscovery/peerdiscovery.go:381:6: error: incompatible type for receiver (cannot use type PacketConn4 as type socket.Conn)
../../go/src/github.com/schollz/peerdiscovery/peerdiscovery.go:381:6: error: incompatible type for receiver (cannot use type PacketConn4 as type socket.Conn)
../../go/src/github.com/schollz/peerdiscovery/peerdiscovery.go:381:6: error: incompatible type for receiver (cannot use type PacketConn4 as type ipv4.genericOpt)
../../go/src/github.com/schollz/peerdiscovery/peerdiscovery.go:381:6: error: incompatible type for receiver (cannot use type PacketConn4 as type socket.Conn)
../../go/src/github.com/schollz/peerdiscovery/peerdiscovery.go:381:6: error: incompatible type for receiver (cannot use type PacketConn4 as type socket.Conn)
../../go/src/github.com/schollz/peerdiscovery/peerdiscovery.go:381:6: error: incompatible type for receiver (cannot use type PacketConn4 as type socket.Conn)
../../go/src/github.com/schollz/peerdiscovery/peerdiscovery.go:381:6: error: incompatible type for receiver (cannot use type PacketConn4 as type socket.Conn)
../../go/src/github.com/schollz/peerdiscovery/peerdiscovery.go:381:6: error: incompatible type for receiver (cannot use type PacketConn4 as type socket.Conn)
../../go/src/github.com/schollz/peerdiscovery/peerdiscovery.go:396:6: error: incompatible type for receiver (cannot use type PacketConn6 as type socket.Conn)
  396 | type PacketConn6 struct {
      |      ^
../../go/src/github.com/schollz/peerdiscovery/peerdiscovery.go:396:6: error: incompatible type for receiver (cannot use type PacketConn6 as type ipv6.genericOpt)
../../go/src/github.com/schollz/peerdiscovery/peerdiscovery.go:396:6: error: incompatible type for receiver (cannot use type PacketConn6 as type socket.Conn)
../../go/src/github.com/schollz/peerdiscovery/peerdiscovery.go:396:6: error: incompatible type for receiver (cannot use type PacketConn6 as type socket.Conn)
../../go/src/github.com/schollz/peerdiscovery/peerdiscovery.go:396:6: error: incompatible type for receiver (cannot use type PacketConn6 as type socket.Conn)
../../go/src/github.com/schollz/peerdiscovery/peerdiscovery.go:396:6: error: incompatible type for receiver (cannot use type PacketConn6 as type socket.Conn)
../../go/src/github.com/schollz/peerdiscovery/peerdiscovery.go:396:6: error: incompatible type for receiver (cannot use type PacketConn6 as type socket.Conn)
../../go/src/github.com/schollz/peerdiscovery/peerdiscovery.go:396:6: error: incompatible type for receiver (cannot use type PacketConn6 as type socket.Conn)
../../go/src/github.com/schollz/peerdiscovery/peerdiscovery.go:396:6: error: incompatible type for receiver (cannot use type PacketConn6 as type socket.Conn)

tagging schollz/croc#360

Images

Images

can't find each other

discoveries, _ := peerdiscovery.Discover(peerdiscovery.Settings{Limit: -1,TimeLimit: -1})
for _, d := range discoveries {
    fmt.Printf("discovered '%s'\n", d.Address)
}

run on a, b and c that are independent mac.
result:
a can discover b and c
b can discover c
c can't discover a or b

go 1.14
mac 10.15.3

possible dart version for flutter android?

As I deeply wanted to build an android app that integrate seamlessly with obs-teleport using flutter but I am having trouble connecting.

I am looking for a way to let my app get discovered by peerdiscovery in windows but as of now current packages in flutter cannot get discovered by peerdiscovery.

so now I'm trying to find a way to match the peer criteria of peerdiscovery.

Any suggestions?

thanks!

Listen socket keeps opened indefinitely when stopped via StopChan

When using StopChan to stop the discovery the listen() function never returns and therefore keeps the socket opened indefinitely.

I was able to quickly hack it in a way so that the local exit variable was part of the peerDiscovery type and listen() would check if that variable is set and quits.

Not sure if that's nice though. Also, in case no more peer discoveries are happening that function would still sit there forever with its socket listening for valid data.

Doesn't work with AllowSelf on windows

I've tried the IPV4 example with AllowSelf set to true on windows, without any other changes but It doesn't discover other local peers.

But it works on my android 10 device android/arm64 cross-compiled on windows run via termux. Windows can't discover the android peer but android can discover another instance of itself and also windows (in the same wifi).

Notify should only notify once per address+payload

Port doesn't get closed upon discovery complete

I've wrapped the Discover function like this:

func Discover() {
	info := make(map[string]string)
	info["name"] = "device"
	info["id"] = "1"
	info["address"] = ip

	b, _ := json.Marshal(&info)

	for {
		_, err := peerdiscovery.Discover(peerdiscovery.Settings{
			Limit:   -1,
			Payload: b,
			Notify: func(d peerdiscovery.Discovered) {
				// save the discovered nodes into memory
			},
		})
		if err != nil {
			log.Errorf("Error trying to discover peers: %s", err)
		}
		time.Sleep(5 * time.Second)
	}
}

If I'm understanding the Settings correctly, this should discover for 10 secs (default) for unlimited nodes, exits, and restart in 5 secs.

But what I'm seeing here is that after the discovery completes, there's a lot of listeners on port 9999 (default):

$ sudo lsof -i:9999 -n -P | wc -l
1041

Is there a way to close these listeners after the function is finished?

EDIT: The reason I'm running this in a loop is that if I set the time limit to -1, it works for some time; if the network changes, it will just stop discovering/being discovered altogether. So I figured maybe I'll just let it run periodically for 10 secs.

Ignoring configured address family can lead to deadlock on MacOS

After getting the list of interfaces, interfaces not having at least one address of the configured address family are not skipped.

Sending a multicast with an IPv4 connection on an IPv6-only interface results (at least on MacOS) in a deadlock.

Additionally, you should check the interface flags if is up and if it supports multicast at all.

This issue might be the root cause of schollz/croc#313

Memory leak

I have been using peerdiscovery in an app for work and it is fantastic. It makes finding other machines and making a distributed cluster quick and easy. Good work!!! I have noticed that when using it in a loop for continuous discovery and building/rebuilding a client list, the application memory usage was increasing. Using the example and running pprof it seems that there is a memory leak with the for loop which scans the socket made. This is running in goroutine which from what I can see doesn't exit.

Type: inuse_space
Time: Feb 10, 2020 at 7:40pm (EST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top20
Showing nodes accounting for 1.61MB, 100% of 1.61MB total
      flat  flat%   sum%        cum   cum%
    1.61MB   100%   100%     1.61MB   100%  github.com/schollz/peerdiscovery.(*peerDiscovery).listen
(pprof) list listen
Total: 1.61MB
ROUTINE ======================== github.com/schollz/peerdiscovery.(*peerDiscovery).listen in D:\goprogs\src\github.com\schollz\peerdiscovery\peerdiscovery.go
    1.61MB     1.61MB (flat, cum)   100% of Total
         .          .    322:           }
         .          .    323:   }
         .          .    324:
         .          .    325:   // Loop forever reading from the socket
         .          .    326:   for {
    1.61MB     1.61MB    327:           buffer := make([]byte, maxDatagramSize)
         .          .    328:           var (
         .          .    329:                   n       int
         .          .    330:                   src     net.Addr
         .          .    331:                   errRead error
         .          .    332:           )
(pprof)`

The above heap profile doesn't show a great deal of memory usage, however over the long run it all adds up and I had it crash the app due to memory restrictions. I resolved this by adding a bool variable into the peerdiscovery struct which I set to true at the end of the Discover method. In the socket for loop I run a check of the bool variable and break the loop if true.

Crude solution but maybe something to consider if making more updates

osc server discovery

I'm looking for something that can discover if there is a osc server (open sound control, udp) running on the same computer or in the local network.

Let's say there is a osc server with certain functionality and a tool is used to find if such a server is running (and on which port). A application might then start a server if none is running, give the user the option to connect to a running server, or let the user choose which server it wants to use.

Could peerdiscovery be helpful for this? I saw this project is using bonjour/avahi.
https://github.com/rbdannenberg/o2

I'm just orientating at the moment for possible solutions and stumbled upon this project.

Index out of range error in some rare cases

Sometimes panic occurs while discovery in progress:

panic: runtime error: index out of range [6] with length 6

goroutine 19 [running]:
github.com/schollz/peerdiscovery.Discover(0xc0002e5e50, 0x1, 0x1, 0xc00048a000, 0x6, 0x6, 0x0, 0x0)
	c:/projects/go/src/github.com/schollz/peerdiscovery/peerdiscovery.go:228 +0x1194

DisableBroadcast property could be dynamic

Hi,
We started using the framework to quickly span multiple nodes for some MPC tests, and we really really like it. It allows us to mostly forget about the network setup. Great work!
What would be nice is a little of dynamic management, especially Broadcast and Interval. Or, alternatively, a slightly better restart capability (some go routines like listen() are tricky ). Would you accept such a PR for a review?
Cheers!

Does this discover peers that are not also running this software?

Does this discover peers that are not also running this software, or do all peers need to be running this in some sort of "listening" mode?

cant discover others when wake from hibernate

how to fix it?
cant discover others when a win10 host waked from hibernate, others can discover it.
if reboot or switch the lan of win10, then all gone well.

go 1.14
mac 10.15.3
win10 professional

Discovery hangs if executed in a loop

I tried executing peerdiscovery.Discover() function in a loop and it stucks randomly after some time.
After a little debugging, my hunch is, this is due to connection not being closed in listen() function.
Please let me know if this is the actual reason or it is because of something else.