bifurcation/mint

A Minimal TLS 1.3 Implementation in Go

mint - A Minimal TLS 1.3 stack

This project is primarily a learning effort for me to understand the TLS 1.3 protocol. The goal is to arrive at a pretty complete implementation of TLS 1.3, with minimal, elegant code that demonstrates how things work. Testing is a priority to ensure correctness, but otherwise, the quality of the software engineering might not be at a level where it makes sense to integrate this with other libraries. Backward compatibility is not an objective.

We borrow liberally from the Go TLS library, especially where TLS 1.3 aligns with earlier TLS versions. However, unnecessary parts will be ruthlessly cut off.

DTLS Support

Mint has partial support for DTLS, but that support is not yet complete and may still contain serious defects.

Quickstart

Installation is the same as for any other Go package:

go get github.com/bifurcation/mint

The API is pretty much the same as for the TLS module, with Dial and Listen methods wrapping the underlying socket APIs.

conn, err := mint.Dial("tcp", "localhost:4430", &mint.Config{...})
...
listener, err := mint.Listen("tcp", "localhost:4430", &mint.Config{...})

Documentation is available on godoc.org

Interoperability testing

The mint-client and mint-server executables are included to make it easy to do basic interoperability tests with other TLS 1.3 implementations. The steps for testing against NSS are as follows.

# Install mint
go get github.com/bifurcation/mint

# Environment for NSS (you'll probably want a new directory)
NSS_ROOT=<whereever you want to put NSS>
mkdir $NSS_ROOT
cd $NSS_ROOT
export USE_64=1
export ENABLE_TLS_1_3=1
export HOST=localhost
export DOMSUF=localhost

# Build NSS
hg clone https://hg.mozilla.org/projects/nss
hg clone https://hg.mozilla.org/projects/nspr
cd nss
make nss_build_all

export PLATFORM=`cat $NSS_ROOT/dist/latest`
export DYLD_LIBRARY_PATH=$NSS_ROOT/dist/$PLATFORM/lib
export LD_LIBRARY_PATH=$NSS_ROOT/dist/$PLATFORM/lib

# Run NSS tests (this creates data for the server to use)
cd tests/ssl_gtests
./ssl_gtests.sh

# Test with client=mint server=NSS
cd $NSS_ROOT
./dist/$PLATFORM/bin/selfserv -d tests_results/security/$HOST.1/ssl_gtests/ -n rsa -p 4430
# if you get `NSS_Init failed.`, check the path above, particularly around $HOST
# ...
go run $GOPATH/src/github.com/bifurcation/mint/bin/mint-client/main.go

# Test with client=NSS server=mint
go run $GOPATH/src/github.com/bifurcation/mint/bin/mint-server/main.go
# ...
cd $NSS_ROOT
dist/$PLATFORM/bin/tstclnt -d tests_results/security/$HOST/ssl_gtests/ -V tls1.3:tls1.3 -h 127.0.0.1 -p 4430 -o

mint's Issues

Use config-provided randomness throughout

Enable indirection of PSK/ticket storage (ClientSessionCache)

NSS is off-spec sending the ticket anyway, Tris won't.

Support client authentication

certificate validation not performed by default

In order to verify the certificate sent by the peer, it is necessary to set the mint.Config.AuthCertificate callback. This is insecure and unexpected (the TLS implementation in the standard library verifies the certificate by default), and undocumented as well.

What is the reason that mint behaves this way? Can we fix this?

Check all known extensions for where they are permitted

Apparently, if the server sends a signature_algorithms extension, a mint client will not object.

This can probably be systematic: check extensions by message and object if they are not allowed.

Support DHE-based resumption

TLS_DHE_PSK_WITH_AES_128_GCM_SHA256 and TLS_DHE_PSK_WITH_AES_256_GCM_SHA384

Implement PSK / resumption handshake

Add additional testcases for resumption

Resumption needs to cover the following:

• Attempt to resume a connection when the server has expired the session ticket
• Attempt to resume a connection when the server does not support the specific cipher suite (should fall back to regular handshake)

Enforce matching of cipher suite with previous PSK in 0-RTT

The PSK cipher suite in early data must match the previously established symmetric cipher.

From the latest spec (12):

6.3.2.5. Early Data Indication
The client specifies the cryptographic configuration for the 0-RTT data using the “configuration_id”, “cipher_suite”, and “extensions” values. For configurations received in-band (in a previous TLS connection) the client MUST:
Send the same cryptographic determining parameters (Section Section 6.3.2.5.1) with the previous connection. If a 0-RTT handshake is being used with a PSK that was negotiated via a non-PSK handshake, then the client MUST use the same symmetric cipher parameters as were negotiated on that handshake but with a PSK cipher suite.

Improve PSKKeyExchangeModes support

• Don't send PSKKeyExchangeModes when there's no PSK
• Add tests for correct behavior by offered mode

Enforce handshake message ordering

Handshake messages are required to arrive in a particular order (with some messages optional): ClientHello, ServerHello, EncryptedExtensions?, etc. Right now, we will read the servers messages in any order. We should enforce that they come in the proper order.

Add support for ChaCha20/Poly1305

Now that golang.org/x/crypto/chacha20poly1305 exists.

Support DHE by default

No reason not to include this, well, except for the fact that the ClientHello will bloat.

Pre-Shared Key extension incorrect in ServerHello

I have a question regarding the encoding of the PSK identity in the ServerHello in the Mint implementation.

I received the following Pre-Shared Key extension in a ServerHello:

00 09 00 29 00 05 00 03 61 62 63
...)....abc

The PSK identity in this case is ‘abc’ and it is 3 bytes long.
The Pre-Shared Key extension has the value 41 (or 0x29).

The format is

struct {
   ExtensionType extension_type;
   opaque extension_data<0..2^16-1>;
  } Extension;



 opaque psk_identity<0..2^16-1>;


  struct {

   select (Role) {
       case client:
           psk_identity identities<2..2^16-1>;

       case server:
           uint16 selected_identity;
   }
  } PreSharedKeyExtension;

I would have expected to see

00 29 00 02 00 00

In case that the selected identity from the ClientHello was the first one.

The format of the Pre-Shared Key extension encoding has changed recently but wouldn’t the encoding for the old format still be something like;

00 29 00 05 00 03 61 62 63
...)....abc

Add ticket_early_data_info to session tickets for 0-RTT

In order to use a session ticket for 0-RTT it has to have the ticket_early_data_info set accordingly. This currently prevents mint from doing 0-RTT with NSS.

Read blocks until buffer is full

Problems with NSS on OSX

I realize these are problems with NSS more than this repo, but perhaps folks trying to use OSX will find these helpful.

DOMSUFering

./ssl_gtests.sh gave error:

init.sh: Fatal DOMSUF env. variable is not defined.

Instead, I used:

DOMSUF=localhost ./ssl_gtests.sh

Pflatform

In my environment, I had to change platform from Darwin15.6_x86_64_cc_glibc_PTH_64_DBG.OBJ to:

PLATFORM=Darwin15.6.0_64_DBG.OBJ

distOPIA

Command dist/$PLATFORM/bin/selfserv -d tests_results/security/$HOST/ssl_gtests/ ... gave error:

dyld: Library not loaded: @executable_path/libssl3.dylib

Instead I had to set:

export DYLD_LIBRARY_PATH="$NSS_ROOT/dist/$PLATFORM/lib"

mint client will return incorrect buffer on 0 read

If a Read fails on EOF with 0 byte in line 25, it will append the previous buffer again.
Solution would be to use
response += string(buffer[:read])

Improve test coverage

We should aspire to have 100% coverage at all times. This goal fell by the way-side a bit as we pushed toward getting compat with NSS working.

To track gaps in coverage:

go test -coverprofile=coverage.out && go tool cover -html=coverage.out

Properly handle alerts and post-handshake handshake messages

Currently, non-ApplicationData messages after the handshake are ignored. We need to do something with them.

can not connect to real tls1.3 server

my server use tls1.3 draft 18， i change supportedVersion to 0x7f12 ， then test ，always get AEAD decrypt failed error , when i watch cloudflare/tls-tris， i found some difference:
when define CipherSuiteParams
your code ：

cipherSuiteMap = map[CipherSuite]CipherSuiteParams{
		TLS_AES_128_GCM_SHA256: {
			Suite:  TLS_AES_128_GCM_SHA256,
			Cipher: newAESGCM,
			Hash:   crypto.SHA256,
			KeyLen: 16,
			IvLen:  12,
		},
		TLS_AES_256_GCM_SHA384: {
			Suite:  TLS_AES_256_GCM_SHA384,
			Cipher: newAESGCM,
			Hash:   crypto.SHA384,
			KeyLen: 32,
			IvLen:  12,
		},

cloudflare define :

type cipherSuite struct {
	id uint16
	// the lengths, in bytes, of the key material needed for each component.
	keyLen int
	macLen int
	ivLen  int
	ka     func(version uint16) keyAgreement
	// flags is a bitmask of the suite* values, above.
	flags  int
	cipher func(key, iv []byte, isRead bool) interface{}
	mac    func(version uint16, macKey []byte) macFunction
	aead   func(key, fixedNonce []byte) cipher.AEAD
}

var cipherSuites = []*cipherSuite{
	// TLS 1.3 ciphersuites specify only the AEAD and the HKDF hash.
	{TLS_CHACHA20_POLY1305_SHA256, 32, 0, 12, nil, suiteTLS13, nil, nil, aeadChaCha20Poly1305},
	{TLS_AES_128_GCM_SHA256, 16, 0, 4, nil, suiteTLS13, nil, nil, aeadAESGCM13},
	{TLS_AES_256_GCM_SHA384, 32, 0, 4, nil, suiteTLS13 | suiteSHA384, nil, nil, aeadAESGCM13},

TLS_AES_128_GCM_SHA256 and TLS_AES_256_GCM_SHA384 ivLen is 4。is your ivLen error？

Check for missing extensions incorrect

The code in conn.go incorrectly requires various extensions to be present. This causes problems when using a PSK-based ciphersuite since neither supported groups nor signature algorithm extensions need to be present (or are even useful). Even the SNI extension is not necessary.

This check should be much later in the code when it is clear what ciphersuite will be selected.

The relevant code is here:

if !gotServerName || !gotSupportedGroups || !gotSignatureAlgorithms {
logf(logTypeHandshake, "[server] Insufficient extensions")
return fmt.Errorf("tls.server: Missing extension in ClientHello (%v %v %v %v)",
gotServerName, gotSupportedGroups, gotSignatureAlgorithms, gotKeyShares)
}

In non-blocking mode, do not automatically advance state

Right now, the server state machine automatically advances in a few cases:

https://github.com/bifurcation/mint/blob/master/server-state-machine.go#L309
https://github.com/bifurcation/mint/blob/master/server-state-machine.go#L609
https://github.com/bifurcation/mint/blob/master/server-state-machine.go#L656

This is apparently causing problems for some QUIC stacks that are using mint for the TLS 1.3 handshake. The same QUICK stacks are also the only current consumers of non-blocking mode*. So we should not auto-advance the state machine in non-blocking mode.

• In the future, we might want a separate flag

Sample server crashes

Because it wants a certificate in the config.

Can't open a pull request (see previous issue), so here's a diff to mint-server/main.go that reads a key/certificate from files.

diff --git a/bin/mint-server/main.go b/bin/mint-server/main.go
index b82a373..9f86ab0 100644
--- a/bin/mint-server/main.go
+++ b/bin/mint-server/main.go
@@ -4,12 +4,52 @@ import (
    "github.com/bifurcation/mint"
    "log"
    "net"
+   "crypto/x509"
+   "os"
+   "io/ioutil"
+   "encoding/pem"
 )

+func readConfig() *mint.Config {
+   serverName := os.Args[1]
+   serverCertFile := os.Args[2]
+   serverKeyFile := os.Args[3]
+   serverKeyBytes, err := ioutil.ReadFile(serverKeyFile)
+   if err != nil {
+       log.Fatalf("Cannot read key: %s", serverKeyFile)
+   }
+   serverCertBytes, err := ioutil.ReadFile(serverCertFile)
+   if err != nil {
+       log.Fatalf("Cannot read cert: %s", serverCertFile)
+   }
+   serverKeyPEM, _ := pem.Decode(serverKeyBytes)
+   serverKeyDER := serverKeyPEM.Bytes
+   serverCertPEM, _ := pem.Decode(serverCertBytes)
+   serverCertDER := serverCertPEM.Bytes
+   serverCert, _    := x509.ParseCertificate(serverCertDER)
+   serverKey, _    := x509.ParsePKCS1PrivateKey(serverKeyDER)
+   certificates := []*mint.Certificate{
+       &mint.Certificate{
+           Chain:      []*x509.Certificate{serverCert},
+           PrivateKey: serverKey,
+       },
+   }
+   config := &mint.Config{
+       ServerName: serverName,
+       Certificates: certificates,
+   }
+   return config
+}
+
 func main() {

    service := "0.0.0.0:4430"
-   listener, err := mint.Listen("tcp", service, &mint.Config{})
+   if len(os.Args) < 4 || os.Args[0] == "--help"  {
+       log.Printf("Usage: %s server-name cert-file private-key-file", os.Args[0])
+       os.Exit(1)
+   }
+   config := readConfig()
+   listener, err := mint.Listen("tcp", service, config)
    if err != nil {
        log.Fatalf("server: listen: %s", err)
    }

Make EndOfEarlyData a warning alert.

Currently mint sends EndOfEarlyData at the Fatal alert level instead of the Warning level for draft 18.

FYI: Notes on working with https://tls13.cloudflare.com/

As a FYI for newbies like me:

With GOPATH set to ~/gocode

go get github.com/bifurcation/mint
cd ~/gocode/src/github.com/bifurcation/mint/bin/mint-client-https

Then
go run main.go -url https://tls13.cloudflare.com/
or let lynx parse that HTML output from cloudflare:
go run main.go -url https://tls13.cloudflare.com/ | lynx -stdin

HTH

Repo cannot be forked

Because of hardcoded references to the specific GitHub repo, e.g. in mint-client/main.go, to "github.com/bifurcation/mint".

Frankly I don't know how Go solves this issue.

Thanks!

application-defined session tickets

Hello @ekr, @bifurcation,

I'm planning to implement QUIC session resumption in quic-go soon. We'll need a few changes to mint for that, so I wanted to use this issue to outline my plan for that.

At the moment, mint sends a random value for the ticket. We will need a way for the application to provide an arbitrary resumption token here. I suggest that we

• add a ResumptionTokenHandler somewhat analogous to the CookieHandler

The ResumptionTokenHandler would be set by the application, and when requested provide a resumption token (and a ticket lifetime, and the maximum early data size) to mint. This token would then be encrypted using the CookieSource implemented in #141. When mint receives a PskIdentity, it would ask the ResumptionTokenHandler to validate the provided token, and decide whether to continue with the resumption or to fall back to the normal handshake.

Currently, mint sends one NewSessionTicket after the handshake completes. In general, a server can send an arbitrary number of session tickets, so we should

• add a Conn.SendSessionTicket() to trigger sending of a new session ticket

We should probably keep sending one session ticket right after the handshake completes, since this provides a useful default for users who don't need fine-grained control.

I just read the corresponding sections of the TLS 1.3 spec, so I might have missed something important. What do you think?

Get `go vet` and `golint` passing

Add syntax support for embedded pointers

Right now, the syntax submodule does not support marshal or unmarshal of structs with field of pointer type.

// type A struct { f Foo }   <-- OK
// type B struct { f *Foo }  <-- Not OK

We should do as encoding/json does, and dereference on marshal / allocate on unmarshal. Note, however, that unlike the JSON case, we cannot handle nil values in the TLS encoding, so all pointers will have to be non-nil.

Enable server to enforce its own preferences

Correctly reset record layer after receiving HRR

If you are sending early data and you receive HRR, you need to send CH in the clear, but right now we don't reset the record layer keys.

Implement obfuscated_ticket_age

Implement certificate selection algorithm

Once we have the ability to configure certificates, we will need to implement the certificate selection algorithm defined in the spec, using SNI, signature_algorithms, and supported_groups.

Rename non-alerts within Alert namespace

Because we're using Alert more generally than the standard namespace, we've ended up with some things in the enum that aren't really alerts, like AlertNoAlert and AlertStatelessRetry. We should rename those (or add some other safeguards) to avoid accidentally sending them on the wire.

Implement HelloRetryRequest

When the client doesn't send a key share from a supported group, the server should send a HelloRetryRequest message. If the client hasn't generated key shares for all supported groups, it should generate one and retry the ClientHello.

Add finite-field DH

Write tests for hackathon code

Some code was merged without thorough testing, add full test coverage for these commits.

Key Share tests failing

The test TestNewKeyShare sometimes fails here (maybe once every 50 test runs or so): https://github.com/bifurcation/mint/blob/master/crypto_test.go#L71, because the length is 1 byte less than expected.

Furthermore, in https://github.com/bifurcation/mint/blob/master/crypto_test.go#L68, priv and pub seem to be interchanged, so maybe that's the reason this test is failing and all we need to do to fix this is swap the variables.

I don't know enough about the EC primitives here to see what the right fix is.

Support OCSP Stapling

Panic in server

Got this connecting with Firefox..

2017/01/24 12:58:51 Listening on port 4430
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x8 pc=0x1171e4]

goroutine 1 [running]:
panic(0x3e4260, 0xc82000a0f0)
/usr/local/Cellar/go/1.6/libexec/src/runtime/panic.go:464 +0x3e6
github.com/bifurcation/mint.(*ServerHandshake).HandleClientHello(0xc820436000, 0xc820427520, 0x661174, 0x2, 0x2, 0x6612a0, 0x4, 0x4, 0x6613f8, 0x6, ...)
/Users/ekr/dev/go/src/github.com/bifurcation/mint/handshake.go:849 +0x2634
github.com/bifurcation/mint.(*Conn).serverHandshake(0xc82040b950, 0x0, 0x0)
/Users/ekr/dev/go/src/github.com/bifurcation/mint/conn.go:671 +0x53c
github.com/bifurcation/mint.(*Conn).Handshake(0xc82040b950, 0x0, 0x0)
/Users/ekr/dev/go/src/github.com/bifurcation/mint/conn.go:469 +0x321
github.com/bifurcation/mint.(*Listener).Accept(0xc820427440, 0x7a9a60, 0xc820028050, 0x0, 0x0)
/Users/ekr/dev/go/src/github.com/bifurcation/mint/tls.go:42 +0xbf
net/http.(*Server).Serve(0xc820413480, 0x7a9a00, 0xc820427440, 0x0, 0x0)
/usr/local/Cellar/go/1.6/libexec/src/net/http/server.go:2117 +0x129
main.main()
/Users/ekr/dev/go/src/github.com/bifurcation/mint/bin/mint-server-https/main.go:123 +0x1006
exit status 2

Add connection state

Right now, there's a bunch of information about the connection that is not exposed to calling code. crypto/tls exposes this information through the ConnectionState struct (link). We should do something similar.

Merge ConnectionParams and ConnectionState

See #135

Add configuration of client/server

We should allow calling code to configure the crypto and credentials used by the client and server. We won't need as many options as crypto/tls (e.g., because we punt X.509 stuff to the calling code), but we need more than we have.

Add support for certificate validation in client

Add `GetCertificate` to `mind.Config`

Not sure it's possible, but I'd like to have GetCertificate from tls.Config in mind.Config, so I can use https://godoc.org/rsc.io/letsencrypt.

Running only the Mint Server

Hey, is it possible to only run the Mint server, instead of using NSS as client and mint as server or vice versa?

I have a program to test out tls servers by sending messages to them, so it would be coul if I could test Mint as well

Review public API and add documentation

Right now, most of the API is private (starts with a lower-case letter). We should consider whether it would be useful to expose more lower-layer primitives, e.g., to facilitate testing of other stacks (à la FlexTLS). We should make sure that all public types and methods are documented.

implement client auth types

The Go standard library offers multi different modes for verifying client certificates: https://golang.org/pkg/crypto/tls/#ClientAuthType.
Currently, by setting mint.Config.RequireClientAuth, mint requires a certificate, but doesn't verify it, i.e. only the type tls.RequireAnyClientCert.

Support NPN / ALPN