anacrolix/utp

Use anacrolix/go-libutp instead

utp

Package utp implements uTP, the micro transport protocol as used with Bittorrent. It opts for simplicity and reliability over strict adherence to the (poor) spec.

Supported

• Multiple uTP connections switched on a single PacketConn, including those initiated locally.
• Raw access to the PacketConn for non-uTP purposes, like sharing the PacketConn with a DHT implementation.

Implementation characteristics

• There is no MTU path discovery.
• A fixed 64 slot selective ack window is used in both sending and receiving.

Patches welcomed.

utp's Issues

Does utp implements congestion control ?

here: https://www.bittorrent.org/beps/bep_0029.html

Starts stalling under load

Using this benchmark:

import (
    "net"
    "testing"

    "github.com/anacrolix/utp"
)

func getTCPConnectionPair() (net.Conn, net.Conn, error) {
    lst, err := net.Listen("tcp", "127.0.0.1:0")
    if err != nil {
        return nil, nil, err
    }

    var conn0 net.Conn
    var err0 error
    done := make(chan struct{})
    go func() {
        conn0, err0 = lst.Accept()
        close(done)
    }()

    conn1, err := net.Dial("tcp", lst.Addr().String())
    if err != nil {
        return nil, nil, err
    }

    <-done
    if err0 != nil {
        return nil, nil, err0
    }
    return conn0, conn1, nil
}

func getUTPConnectionPair() (net.Conn, net.Conn, error) {
    lst, err := utp.NewSocket("udp", "127.0.0.1:0")
    if err != nil {
        return nil, nil, err
    }

    var conn0 net.Conn
    var err0 error
    done := make(chan struct{})
    go func() {
        conn0, err0 = lst.Accept()
        close(done)
    }()

    conn1, err := utp.Dial(lst.Addr().String())
    if err != nil {
        return nil, nil, err
    }

    <-done
    if err0 != nil {
        return nil, nil, err0
    }

    return conn0, conn1, nil
}


func benchConnPair(b *testing.B, c0, c1 net.Conn) {
    b.ReportAllocs()
    b.SetBytes(128 << 10)
    b.ResetTimer()

    request := make([]byte, 52)
    response := make([]byte, (128<<10)+8)

    pair := []net.Conn{c0, c1}
    for i := 0; i < b.N; i++ {
        if i%2 == 0 {
            pair[0] = c0
            pair[1] = c1
        } else {
            pair[0] = c1
            pair[1] = c0
        }

        if _, err := pair[0].Write(request); err != nil {
            b.Fatal(err)
        }

        if _, err := pair[1].Read(request[:8]); err != nil {
            b.Fatal(err)
        }
        if _, err := pair[1].Read(request[8:]); err != nil {
            b.Fatal(err)
        }
        if _, err := pair[1].Write(response); err != nil {
            b.Fatal(err)
        }
        if _, err := pair[0].Read(response[:8]); err != nil {
            b.Fatal(err)
        }
        if _, err := pair[0].Read(response[8:]); err != nil {
            b.Fatal(err)
        }
    }
}

func BenchmarkTCP(b *testing.B) {
    conn0, conn1, err := getTCPConnectionPair()
    if err != nil {
        b.Fatal(err)
    }

    defer conn0.Close()
    defer conn1.Close()

    benchConnPair(b, conn0, conn1)
}

func BenchmarkUTP(b *testing.B) {
    conn0, conn1, err := getUTPConnectionPair()
    if err != nil {
        b.Fatal(err)
    }

    defer conn0.Close()
    defer conn1.Close()

    benchConnPair(b, conn0, conn1)
}

I get the following:

PASS
BenchmarkTCP-4      5000        263246 ns/op     497.91 MB/s          27 B/op          0 allocs/op
BenchmarkUTP-4  2016/05/26 23:21:51 error resending packet: write udp [::]:56345->127.0.0.1:56344: use of closed network connection
2016/05/26 23:21:51 error resending packet: write udp [::]:56345->127.0.0.1:56344: use of closed network connection
2016/05/26 23:21:51 error resending packet: write udp [::]:56345->127.0.0.1:56344: use of closed network connection
     100     134251556 ns/op       0.98 MB/s     1195857 B/op       4016 allocs/op

If you tweak the numbers a bit, you might get it to 0.2MB/s

I did manage to get it to a point where it just mostly never completes and fails with "connection closed", but that was using syncthings protocol wrappers which do async read/writes. In cases where it did complete it was around 0.3MB/s

net.Listener interface

If i want to listen on utp packets, what i need to do?
As i see package does not have Listen method, so i need to listen udp specific port and use NewSocketFromPacketConn?
Does does it possible to create Listen method?

Conn `Addr` methods return udp

I'm not sure if this is actually an issue, but the LocalAddr and RemoteAddr methods on a Conn return an address whose network identifies as udp. I think these should return utp{4,6} instead, as its more accurate.

I currently wrap the utp conn from your package to change this functionality, but if you could add it upstream that would be very helpful.

panic in jitter

Found a panic thrown in utp: https://gist.githubusercontent.com/dignifiedquire/33c5e9b5af4305491e37/raw/7c0c8731916bf8b21f723a8c1a32c2e1276bb993/fail.log

throughput is 5 MB/s on loopback

Hi

I tested with go 1.5 on Mac Air:

$ go get github.com/anacrolix/utp
$ go get github.com/anacrolix/utp/cmd/ucat
$ ./bin/ucat -l -p 9876 >/dev/null &
$ dd if=/dev/zero bs=1m count=100 | ./bin/ucat 127.0.0.1 9876
100+0 records in
100+0 records out
104857600 bytes transferred in 18.542269 secs ( *** 5655058 bytes/sec *** )
2015/09/07 03:51:28 wrote 104857600 bytes
2015/09/07 03:51:28 received 0 bytes
2015/09/07 03:51:28 received 104857600 bytes

This is compared to native build of ucat from https://github.com/bittorrent/libutp:

$ ./build/Release/ucat -l -p 9876 >/dev/null &
$ dd if=/dev/zero bs=1m count=1000 | ./build/Release/ucat -B $((256*1024)) 127.0.0.1 9876
1000+0 records in
1000+0 records out
1048576000 bytes transferred in 11.888499 secs ( *** 88200875 bytes/sec *** )

That's a big drop..
Cheers

Deprecated

This repository currently deprecated in favour of https://github.com/anacrolix/go-libutp. Issues, PRs and comments are still accepted.

Ability to pass in a net.UDPConn

So that we could do the STUN dance or whatever is required before wrapping it in UTP.
Similarly how you can wrap and unwrap a TCP connection with tls.Server or tls.Client.

listener sockets closed when accepted connections close

If i create a socket for accepting new connections on, I want to keep that open even if all the connections that have been made to me get closed, That expectation is violated by this line: https://github.com/anacrolix/utp/blob/master/utp.go#L1252

I think that line only needs to be executed if the connections drop to zero AND the socket was created through a 'dial' (or even then probably not). Its probably the case that the right thing to do is to only close the actual socket when 'Close' is called.

Socket does not implement utpSocket (missing DialContext)

Hey @anacrolix,

FYI when trying to use anacrolix/torrent with cgo disabled, the following error pops up:

# github.com/securityscorecard/tm-torrents-discovery/vendor/github.com/anacrolix/torrent
vendor/github.com/anacrolix/torrent/utp_go.go:10: cannot use *utp.Socket as type utpSocket in return argument:
        *utp.Socket does not implement utpSocket (missing DialContext method)

spec question

the readme says:

It opts for simplicity and reliability over strict adherence to the (poor) spec

Could you please note somewhere which things you're talking about specifically? That way we can make sure to watch out for them. (I've found many bugs in transport protocol implementations, and usually notes like these helped find them).

---

By the way, uTP should be LEDBAT (RFC 6817), so it should be well specced out. Wikipedia notes:

but the details of the µTP implementation are different from those of the draft.[19][broken citation]

I haven't dived deeply into either to know what's different, but would also be relevant to document somewhere.

I suspect that Apple's LEDBAT extensions to the TCP stack (used for software updates) may also be helpful: https://opensource.apple.com/source/xnu/xnu-1699.32.7/bsd/netinet/tcp_ledbat.c

Reasoning behind this panic

I have been consistently hitting this panic in my application, and curious as to the reasoning of it, and how I may be triggering it?

https://github.com/anacrolix/utp/blob/master/socket.go#L484

The stack trace goes back to this call:

https://github.com/anacrolix/torrent/blob/master/client.go#L383

My application is handling tens of thousands of torrents, and it works very well, but eventually (<hr) hits this panic. I am not quite sure how to handle this. I am unable to produce sample code to trigger this, as although I am reliably producing it, it is infrequent and hard to track down which action is causing it.

Edit: I also am getting this output fair frequently when dealing with this load.
resetting conflicting syn

Dial leaks file descriptors

calling Dial opens a new socket, then calls dial on that socket. closing the returned connection doesnt appear to close the opened socket, leading to a file descriptor leak.

panic: listen udp :50007: bind: address already in use

Not sure what is the reason you do not call SO_REUSEADDR or lazyDestroy() does the job but I unable to restart application. Especially It hurts on Android.

Can you make Socket.destroy() exported if it helps? Or how to wait until socket will be destoryed?

Needs a license

As with the torrent package, this needs a license if it's to be usable by anyone else. :)

Setting deadlines on a connection panics

I guess this relies on the socket having had at least a single connection?

package main

import(
    "time"

    "github.com/anacrolix/utp"
)

func main() {
    s, err := utp.NewSocket("udp", ":0")
    if err == nil {
        s.SetReadDeadline(time.Now().Add(time.Second))
    }
}

panic: deadline callback is nil

goroutine 1 [running]:
panic(0x6f3120, 0xc08200af40)
    C:/Go/src/runtime/panic.go:464 +0x3f4
github.com/anacrolix/utp.(*deadlineCallback).updateTimer(0xc0820ae078)
    C:/Go/bin/src/github.com/anacrolix/utp/utp.go:89 +0xde
github.com/anacrolix/utp.(*deadlineCallback).setDeadline(0xc0820ae078, 0xeceae9880, 0x24eb8a24, 0x9dd8a0)
    C:/Go/bin/src/github.com/anacrolix/utp/utp.go:96 +0x4e
github.com/anacrolix/utp.(*connDeadlines).SetReadDeadline(0xc0820ae078, 0xeceae9880, 0x24eb8a24, 0x9dd8a0, 0x0, 0x0)
    C:/Go/bin/src/github.com/anacrolix/utp/utp.go:116 +0x57
main.main()
    C:/Users/Audrius/AppData/Local/liteide/goplay.go:12 +0x14a
exit status 2

Setting deadlines on sockets does not work

    skt, err := utp.NewSocket("udp", ":0")
    skt.SetReadDeadline(time.Now().Add(time.Second))
    _, _, err = skt.ReadFrom(nil)

blocks forever.

Lots of duplicate packets are being send

I am trying to hunt down an issue I have with this library where I notice that the uTP transfer does not back-off when there is TCP background traffic.

Tracing this issue I noticed that an extensive number of packets are being resend. I noticed this was primarily due to this call to resend which is caused by this call to ackSkipped. Using wireshark I noticed that lots of ACK packets do have the Selective ACKs extension enabled but do not have an extension bitmask:

According to the specification the length should be:

Note that the len field of extensions refer to bytes, which in this extension must be at least 4, and in multiples of 4.

There are also ACK packets that do have this set correctly:

Now as a quick hack I've tried not calling ackSkipped when there are no bytes in the extension:

		switch ext.Type {
		case extensionTypeSelectiveAck:
			if len(ext.Bytes) > 0 {
				c.ackSkipped(h.AckNr + 1)
				bitmask := selectiveAckBitmask{ext.Bytes}
				for i := 0; i < bitmask.NumBits(); i++ {
					if bitmask.BitIsSet(i) {
						nr := h.AckNr + 2 + uint16(i)
						c.ack(nr)
					} else {
						c.ackSkipped(h.AckNr + 2 + uint16(i))
					}
				}
			}
              }

and I've also changed ackSkipped to not resend the packets that often:

	switch send.acksSkipped {
	case 60, 120: // was 3, 60
		if logLevel >= 1 {
			log.Printf("acksSkipped = %d", send.acksSkipped)
		}
		ackSkippedResends.Add(1)
		send.resend()
		send.resendTimer.Reset(c.resendTimeout() * time.Duration(send.numResends))
	default:
	}

For me this is dramatically decreasing the number of duplicates that are being send.

However it does not fix my issue with uTP traffic throttling back when there is TCP background traffic.

Does this make sense?

memory usage a bit high

I'm having some of my ipfs nodes die of OOM, and looking at the heap profile I see that utp was using 100MB (of my machines 256MB of RAM). It all appears to have been allocated by the sendBufferPool's allocation function. That seems a bit much, and i'm sure some of this is user error on my part. Any ideas how to debug the issue?