google/exposure-notifications-server

Exposure Notification Reference Server | Covid-19 Exposure Notifications

Archive status announcement

The Exposure Notifications Server projects are scheduled to move into archive status on July 1, 2023.

v1.17.0 is the last planned regular release. Maintenance releases may be published before the public archive date if necessary to address issues that may arise.

The Exposure Notifications Authors want to thank everyone who contributed to this project either directly or indirectly

Exposure Notification Reference Key Server

COVID‑19 Exposure Notifications API

Exposure Notification Reference Key Server Documentation

In our continued effort to help governments and health authorities during the COVID-19 pandemic, we have authored an open source reference implementation of an Exposure Notification Key Server.

The server reference in this repository implements the Exposure Notifications API and provides reference code for working with Android and iOS apps that are built by public health authorities. The reference server source code is available on GitHub and can be deployed on any infrastructure or cloud provider selected by a public health authority.

Our hope is by making this privacy-preserving server implementation available to health authorities, we can enable their developers to use the open source code to get started quickly.

Overview

The server is responsible for the following functions:

• Accepting the temporary exposure keys of affected users from mobile devices.

• Validating the temporary exposure keys using the device attestation API.

• Storing the temporary exposure keys in a database.

• Periodically generating incremental files that will be downloaded by mobile devices to perform the key matching algorithm on the mobile device.

• Sending a public key to devices, and digitally signing the incremental files with a private key.

• Periodically deleting old temporary exposure keys. After 14 days, or configured time period, the exposure keys can no longer be matched to a device.

Tutorials and reference documentation

You can read tutorials on deploying and using the reference Exposure Notification Key Server here:

• Overview
• Deployment Guide
• Key Server Migration Guide
• Reference Documentation
• TEK Validation Guide
• Server Functional Requirements
• Server Deployment Options
• API Definitions

Issues and Questions

If you have a question about Exposure Notifications in your region, please contact your local public health authority directly.

You can open a GitHub Issue. Please be sure to include as much detail as you can to help aid in addressing your concern. If you wish to reach out privately, you can send an e-mail [email protected].

Contributing to this project

Contributions to this project are welcomed. For more information about contributing to this project, see the contribution guidelines.

exposure-notifications-server's Issues

bucket name should be part of ExportConfig

The bucket name is currently an environment variable, but should be based on each export config.

/kind bug

put dependencies behind interfaces

In particular

• Secret Manager
• KMS
• GCS

Add 'platform' field to APIConfig table

Add platform field for mobile device platform.

android will indicate to use the SafetyNet device attestation
ios will indicate to use the iOS Device Check attestation

do we need EnforceApkDigest

I'm not sure why we need this option. Is this for testing only? If so, please provide a debug flag to disable it on production code.

In production mode, we want to enforce it and we should always verify APKDigestSHA256 is set to a good value

Is my understanding correct?

cleanup: add unit tests

Invalid token is logged but does not return an error

In safetynet.go:

if !token.Valid { logger.Errorf("invalid JWS attestation passed.") }

It' not obvious this function exits. Please confirm this is the case.

Can we add a comment? Maybe also a return nil,nil for peace of mind.

Are there side effects of this exit()? Does the server use a a fork-and-exec on each request? Or will the exit() also kill other requests being processed?

Standardize API responses

As discussed briefly in #195
cc @sethvargo

use uint32 for things that are never negative

in exposure.go, interval_number and intervalCount are defined as signed integers but they cannot take negative values.

terraform: ensure all service accounts can read secrets

Harden the computation of Nonce

Location
nonce.go

Problem
The nonce (besides not being randomly generated server-side) does not use a unique encoding. Currently it's calculated as

cleartext := n.appPackageName + strings.Join(n.ttKeysBase64, "") + strings.Join(n.regions, "")

So if an app is called com.app and sends N keys, the nonce will be equivalent to an app called com.app.key0 with the following N-1 keys. I think this should be detected during SafetyNet verification, since the packagename and its hash/publickey will be reported in the attestation.

But for additional protection - defense in depth - it would be great to update to something like:

cleartext := b64(n.appPackageName) + "." + b64(strings.Join(n.ttKeysBase64, "")) + "." + b64(strings.Join(n.regions, ""))

This is similar to jwt encoding.

support database migration

There should be a way to change the DB schema without dropping and recreating it. We at pkg.go.dev have been happy with https://github.com/golang-migrate/migrate.

Add specific field in publish API for roaming

Two options -

• leave as is, each country would whitelist it's app to publish for "all regions"
• separate out the home region that the app is publishing for and roaming regions that are part of the request

I'm inclined to leave it as is. A server should only be configured to accept regions that the sever has roaming agreements with

test: create load test for exposure key ingest

wipeout naming is misleading

wipeout.go and similar methods make it sound like data will be securely deleted from the server, but it's not.

Not sure what the name should be though. Note that we don't care about secure deletion, since data stored is public anyway.

validate data before storing in db

in exposure.go

There is already a TODO. This issue is only for tracking purposes

lock protocol should check unique ID on unlock

In the lock protocol (internal/database/lock.go): if a process times out while holding a lock, another process steals the lock, and then the first finishes, it will delete the second's valid lock. Maybe add a timestamp column to the table which can be checked on delete.

implement optional export encryption

/kind enhancement

add signing for export files

needs

• add signing keys reference to ExportConfig
• sha256 of each ExportFile - will be the ecdsa signature of the sha256 of the export contents.

verify apkCertificateDigestSha256

currently not verified in safetynet.go.

There is already a TODO(). I'm opening this issue just for tracking purposes.

reject attestation that don't contain a nonce

see safetynet.go

I think this should return an error, rather than a warning. I think it's probably just for testing. If we want testing, maybe add a special flag to make sure debug code not accumulate and it's easy to turn it off.

Create installer tool

Create an installer CLI tool to build, install and configure the components end to end.

• Verify presence of gcloud and cloud_sql_proxy (needed for DB schema)
• verify project ID to setup
• verify permissions on user
• Configuration questionnaire - get some setup parameters
• Enable APIs
  • Cloud Run
  • KMS
  • Secret Manager
  • Cloud Storage
  • Cloud Scheduler
  • CloudSQL
  • Cloud Build
• create service accounts
  • publish server, needs DB access
  • export server, needs DB access, GCS, access, keystore access
  • federation receiver - needs DB access and KMS (signing JWTs)
  • federation puller - needs DB access and Cloud Secrets (PKs for verifying JWTs)
  • wipeout - needs DB access
  • export wipeout - needs DB and GCS access
  • scheduler SA to invoke wipeout
  • scheduler SA to invoke export
• create CloudSQL instance
  • create database and database user w/ random password stored in secret manager
  • make SSL work between the cloud run services and the DB
• build the binaries
• create the cloud run services
• create the cloud scheduler for wipeout

denormalize signing key info in export tables

In a world where we might have multiple signatures on an export (whether from a key rolling, or an external authority cosigning on the exports) we would want to populate multiple SignatureInfo's on the export. Currently our ExportConfig and ExportBatch assume one signing key (and don't allow for additional info like key version or identifier).

cleanup: load test

Should verify how long a run takes at scale. Also verify it doesn't adversely impact table performance during cleanup.

SafetyNet nonce is not randomly-generated by server

Location
internal/android/nonce.go

Problem
The nonce is generated based on values selected by the client app.
Without being random, the attestation may be replayed to the server. This seems consistent with SafetyNet doc:obtain-a-nonce

Proposed Fix
I think we need the client to call the server to get a nonce, then reply with the same nonce.
This will introduce some statefulness on server. Or we could generate PRF(K, expiry_time, other_metadata) so that we need not store the nonce server-side: this gives the attacker a window for replay, but I think it is acceptable.

The nonce could look like expiry_time||other_metadata||PRF(...)

Note: there are options opts.MinValidTime and opts.MaxValidTime in the code; and they that may be also be to mitigate replays to some extent. I'm not sure what happens if the nonce is re-used and the timestamp is validated: safetynet page warns attackers could gather attestations blob.

Note also that the use of opts.MinValidTime and opts.MaxValidTime is currently optional in the code.

Refactor duplicate code in mains

Create a common main and eliminate copy/paste-is code.

/assign

ensure keys are consecutive on upload

No gaps and no overlap on intervals

/assign
/kind bug

add decoy records and/or increase batchWindow

in publish.go

We add to the database each set of keys uploaded by users, with a 15mn timestamp granularity as defined in exposure.go.

It's dangerous to give such granularity, as these records will be made public. 15mn could give an attacker additional information to de-anonymize users, in combination with other sources of info. How can we mitigate these risks?

We can increase the granularity to something larger, eg 1hr? This begs the question: how often will these records be made available for download? If the records are available for download right away, then someone could simply query the database every minute and look for new records, even if the timestamp has a larger granularity. So the publication time must be in sync with the batchWindow time.

Note that the server may log the exact time requests are received, so it's easy for the server admin to link which database records correspond to which time in the log. We may need to also align it to the batchWindow, unless this is cumbersome for troubleshooting problems...

In addition, do we want to add fake records to the database? This is to ensure an attacker cannot correlate key-upload times with actual patients. I think this will be taken care of in the phone app, by sending decoy key-uploads requests, as discussed in go/apollo-network-disclosures.

Write client/server interop spec

/cc @crwilcox @gurayAlsac @peetsasaki

split apart internal/api package

Each piece of the API should have it's own package

/assign @mikehelmick

change to 200 response codes for most requests

Add terraform for federation services

Two federation services on Cloud Run: federationin and federationout

federationin is invoked on a schedule (e.g., hourly) and must be accessible only to the scheduler (e.g., Cloud Scheduler)

federationout is a gRPC endpoint that can be called by external parties. It requires an OIDC token, so it should be "self-guarded" already.

Separate packages, include internal/model to minimize exported fields.

@crwilcox plan

• Move all packages under api/ to internal/package-name
• Combine database and model -> model to avoid dependency cycles
• don't extend DB, instead pass in for use by model specific methods

ensure logs based metrics appear on all binaries

/kind enhancements
/assign

Create single entrypoint binary

Create a single binary that has all the handlers installed and can demo a complete e2e working system locally.

rename parseAttestion()

the function parseAttestation() is misleading because it not only parses, but also validates the signature.

Can we change the name?

don't store data in /tmp

see env.go

the file contains a key/secret from SM so it should be kept outside /tmp which is word readable. There's also a risk that the file concatenation is

fName := "/tmp/secret-%v" + envVar with envVar = "../../whatever"

This is not exploitable in our use case, but let's avoid doing this since the code will be copy/pasted in other contexts.

We could define a 'working directory' and save everything inside it; after verifying the file we write is inside it?

In addition, we may make the file non-readable to avoid file read vuln (eg using untrusted data from GET requests); and make the file readable only when it needs to be read.

Support multi-tenancy for iOS DeviceCheck

The initial implementation of DeviceCheck only allows a single app per server. We need to support a per-app configuration. Users will likely need to store a reference to a secret in the database and then the app looks up that secret at runtime on a per-app basis.

• Refs #153
• Pending on #154

enforce maximum number of keys per uploads

In publish.go, we're not verifying the number of keys submitted by a device.

I think we want to force devices to submit the maximum number of keys (144). If we don't do that, it leaks when the infected device installed the app, which may be used as additional data to de-anonymize users.

e2e: create an automated e2e test

We have done some manual runs, but it would be great to have a test that can do an automated e2e walkthrough of exposure keys getting uploaded and exposure keys getting downloaded

reject attestation without time validation

See safetynet.go.
This is also true for the opts checks (CTSProfileMatch, BasicIntegrity).

Currently those checks just log a warning (logger.Warnf) if they fail, but will not return an error. It should return an error. Related to the bug here: #95

remove bypass_safetynet or provide a debug flag

This is an option in the database and config. I think it's only used for testing? If so, please provide a debug flag that can be easily turned off.

This is used, for example in verify.go. We need to gate these thru a global debug flag.

do we need time as parameters to TransformPublish()?

in file exposure.go we're passing a time as parameter.

I don't think the caller should need to pass a time, and the call to Time.Now() should happen within TransformPublish().

support multiple APKHashes per app package

For support dev and release keys builds

/kind enhancement

why do we need correctIntervalCount()

in exposure.go, we're 'fixing' the intervalCount sent by devices.

I don't really understand why. Does this not indicate a problem client side that should be fixed in client code?

allow DB password to be read from a secret store

cleanup: cloud run setup

make server more robust to padded and non-padded base64 encodings

For a publish - as long as the client consistently pads or doesn't pad this should work.

/kind bug

Implement iOS device check

/kind enhancement

Simplify sharing configuration between packages

Right now a new configuration has to be plumbed through quite a few structs and its tedious to add new fields (see ac0fc4e as an example).

Some values come from the database, some from Secret Manager, and some from the environment. Since most things are internal/, it would be great if we had a single unified way to pass configuration between packages.

Sending exposure key with `--app=com.example.ios.app` results in nil exception

Repro steps:

# Run the steps on https://github.com/google/exposure-notifications-server/blob/master/CONTRIBUTING.md#running-locally

# In another pane
$ go run tools/exposure-client/main.go --regions=US --app=com.example.ios.app

Stack trace:

{"severity":"info","ts":1589421052.7978437,"logger":"default","caller":"dbapiconfig/apiconfig.go:113","message":"loaded new APIConfig values"}
2020/05/14 01:50:52 http: panic serving [::1]:55936: runtime error: invalid memory address or nil pointer dereference
goroutine 81 [running]:
net/http.(*conn).serve.func1(0xc0005a3ae0)
        /usr/lib/google-golang/src/net/http/server.go:1772 +0x139
panic(0xd43580, 0x15cb260)
        /usr/lib/google-golang/src/runtime/panic.go:975 +0x3e3
crypto/ecdsa.boringPrivateKey(0x0, 0xf68418, 0x7ff232465c28, 0x0)
        /usr/lib/google-golang/src/crypto/ecdsa/boring.go:59 +0x37
crypto/ecdsa.Sign(0xf837c0, 0xf68418, 0x0, 0xc000362280, 0x20, 0x20, 0x20, 0xc0005180c0, 0x55, 0x55)
        /usr/lib/google-golang/src/crypto/ecdsa/ecdsa.go:196 +0x8de
github.com/dgrijalva/jwt-go.(*SigningMethodECDSA).Sign(0xc00019adb0, 0xc0005180c0, 0x55, 0xdd0400, 0x0, 0x80f15b7a, 0x731f559f7f33099a, 0xc0001be5a0, 0x7300000000e1efa0)
        /usr/local/google/home/kgood/go/pkg/mod/github.com/dgrijalva/[email protected]+incompatible/ecdsa.go:119 +0x1c9
github.com/dgrijalva/jwt-go.(*Token).SignedString(0xc0001456a8, 0xdd0400, 0x0, 0x3, 0xc0001be648, 0x0, 0x1b)
        /usr/local/google/home/kgood/go/pkg/mod/github.com/dgrijalva/[email protected]+incompatible/token.go:55 +0x8d
github.com/google/exposure-notifications-server/internal/ios.newSignedJWT(0x0, 0x0, 0x0, 0x0, 0x0, 0x203000, 0x203000, 0x203000, 0xc000145778)
        /usr/local/google/home/kgood/exposure-notifications/internal/ios/devicecheck.go:125 +0x41c
github.com/google/exposure-notifications-server/internal/ios.ValidateDeviceToken(0xf9a9e0, 0xc000368f00, 0xc0002edf40, 0x11, 0xc0005b81e0, 0x0, 0x0)
        /usr/local/google/home/kgood/exposure-notifications/internal/ios/devicecheck.go:52 +0x85
github.com/google/exposure-notifications-server/internal/verification.VerifyDeviceCheck(0xf9a9e0, 0xc000368f00, 0xc000344480, 0xc0003fad20, 0x13, 0xc000344480)
        /usr/local/google/home/kgood/exposure-notifications/internal/verification/verify.go:93 +0x149
github.com/google/exposure-notifications-server/internal/api/publish.(*publishHandler).ServeHTTP(0xc0005ee180, 0xf95360, 0xc000511a40, 0xc0005ec000)
        /usr/local/google/home/kgood/exposure-notifications/internal/api/publish/publish.go:114 +0xb56
github.com/google/exposure-notifications-server/internal/api/handlers.WithMinimumLatency.func1(0xf95360, 0xc000511a40, 0xc0005ec000)
        /usr/local/google/home/kgood/exposure-notifications/internal/api/handlers/delay.go:29 +0xc0
net/http.HandlerFunc.ServeHTTP(0xc000688220, 0xf95360, 0xc000511a40, 0xc0005ec000)
        /usr/lib/google-golang/src/net/http/server.go:2012 +0x44
net/http.(*ServeMux).ServeHTTP(0x15e5180, 0xf95360, 0xc000511a40, 0xc0005ec000)
        /usr/lib/google-golang/src/net/http/server.go:2387 +0x1a5
net/http.serverHandler.ServeHTTP(0xc000416380, 0xf95360, 0xc000511a40, 0xc0005ec000)
        /usr/lib/google-golang/src/net/http/server.go:2812 +0xa3
net/http.(*conn).serve(0xc0005a3ae0, 0xf9a9e0, 0xc000368dc0)
        /usr/lib/google-golang/src/net/http/server.go:1895 +0x86c
created by net/http.(*Server).Serve
        /usr/lib/google-golang/src/net/http/server.go:2938 +0x35c