pigmice2733/peregrine-backend

The future of FRC scouting

peregrine-backend's Issues

Setup blackbox testing with CircleCI

The jest blackbox tests should be in CI.

Remove the comment in the README.md about branch naming

We no longer use initials prefixes in branch names. They should just be descriptive. The README.md should be updated to reflect this.

Add ETag and If-None-Match to tba package

The TBA API supports the ETag and If-None-Match HTTP headers. The TBA package should keep a map of URLs to ETags, send them in an If-None-Match HTTP header in any TBA request, set the ETag sent back, and handle a 304 Not Modified properly. You can take a look at the old code to sorta get an idea of what this would look like. The old code used If-Modified-Since which is a bit different but a similar idea.

@NathanJesudason @j10k11 do either of you want to attempt this now that you have peregrine setup locally? If it would be helpful, I can write the tests for this feature, so you can tell when you've got it working properly, or I can just pair program with one of you at robotics.

Add dependency health check route

It would be nice if instead of a 404 the / route served information about dependency (PostgreSQL, TBA) health.

Use migrations for seeding the DB

We can use SQL migrations for seeding the DB. That would be a lot nicer than our current solution.

Remove leading eventKey from matchKey

For an event like 2018abca, all the match keys will start with 2018abca_ (e.g. 2018abca_qm2, 2018abca_qm6). The match keys should simply be the later part (qm2, qm6).

Add `/events` endpoint

https://github.com/Pigmice2733/peregrine-frontend/blob/master/docs/docs.md#events

Add realm support

Firehose and caching for event and match TBA data

Right now almost every request involving matches requires a roundtrip request to TBA, there should be a time period (several hours probably) where the server will just respond with the data it already has, and during that period the data is updated via the TBA firehose.

Wrap store errors better

There are lots of very complex store functions that do not provide context to the errors they return. They should do that :)

Add uptime to health check

It would be cool if the health check endpoint at / served the service uptime.

Middleware for eventKey, matchKey, etc, etc.

So either:

• We have a middleware that looks in mux.Vars for eventKey, matchKey, bla, bla, bla to update events, matches, bla, bla, bla.
• We have a middleware where we specify dependencies on each route like ihttp.Update(fooHandler, ihttp.Deps{Events: true, Matches: false, Teams: true}) to update events, matches, bla, bla, bla.

Couple `server` and `store` types

IMO it makes more sense to just define json tags on store types than have the types defined in two places. What do you think @brendanburkhart

Make /users route authenticated

And update the docs on the frontend repo.

Deploy to docker hub, use docker for deployment

Respond with empty arrays, not nil arrays

Example:

{
  "data": [
    {
      "id": 1,
      "username": "franklin",
      "firstName": "Franklin",
      "lastName": "Harding",
      "roles": {
        "isAdmin": true,
        "isVerified": true
      },
      "stars": null
    }
  ]
}

Actually write exit codes on migrate tool

Add match info endpoint

See docs: https://github.com/Pigmice2733/peregrine-frontend/blob/master/docs/docs.md#eventeventkeymatchmatchkeyinfo

Add the teams and team info endpoints

Add the /event/{eventKey}/team/{team}/info and the /event/{eventKey}/teams endpoints.

https://github.com/Pigmice2733/peregrine-frontend/blob/dev/docs/docs.md#eventeventkeyteams
https://github.com/Pigmice2733/peregrine-frontend/blob/dev/docs/docs.md#eventeventkeyteamteaminfo

Remove second m.Close from migrate

Add event info endpoint

See the api docs

/event/{eventId}/info

https://github.com/Pigmice2733/peregrine-frontend/blob/master/docs/docs.md#eventeventkeyinfo

Separate test files

There are a lot of tests in api.test.js now, it might make sense to separate them, e.g.: events.test.js, matches.test.js, etc, etc.

Setup continuous deployment with CircleCI

The applications should automatically deploy when a branch is merged (assuming all tests pass).

Add context to ErrNoResults returns

We should be providing context when we return ErrNoResults (that's the whole point of it being a type rather than just an error value), like this:

peregrine-backend/internal/store/events.go

Line 65 in 675f40c

return ErrNoResults(fmt.Errorf("event key %s does not exist", eventKey))

But we aren't doing that everywhere:

• peregrine-backend/internal/store/events.go

  Line 82 in 675f40c

  return event, ErrNoResults(err)

• peregrine-backend/internal/store/matches.go

  Line 75 in 675f40c

  return m, ErrNoResults(err)

• search all files for ErrNoResults( for more.

Add Windows steps to README.md

We should add steps on how to install PostgreSQL on Windows, manage it with pg_ctl, and use psql to create the peregrine database since most of the new Peregrine developers will be using (bleh) Windows. Also some specific steps on how to use install/use dep would be nice since you just download a single binary rather than install it.

• Download/use dep to install dependencies
• Install PostgreSQL on Windows
• Manage PostgresSQL start/stop/restart with pg_ctl
• Create the peregrine database with psql

Refactor New to Init

We have a stupid amount of arguments in New. We should probably refactor it so the user can construct the Server struct themselves and then call server.Init() or something.

Refresh tokens

We should add refresh tokens. This doesn't need to happen until after we get everything else done.

Make the API more RESTful and less gross RPC

Per slack discussion.

No

peregrine-backend/internal/store/store.go

Line 7 in d605468

"time" // Register lib/pq PostreSQL driver

Add ability to delete/update manual events/matches

If events/matches are created, they can't currently be changed. Admins should be able to update or delete events on their realm. Additionally it should not be possible to create a manual event that replaces a TBA event, they need to have different names.

Spelling errors!

https://goreportcard.com/report/github.com/Pigmice2733/peregrine-backend#misspell

Make roles into a struct

Should happen at some point, no rush.

Add SSL

Reports endpoints

Dependent on #32

https://github.com/Pigmice2733/peregrine-frontend/blob/dev/docs/docs.md#eventeventkeymatchmatchkeyreportsteam

Add section to README that describes scouting

Add a section to README for that describes what scouting is, why we scout, what problems peregrine is supposed to solve, etc, etc. Most of this can probably be pulled from our FIRST Faire presentation or maybe from my blog post. You'll need to know markdown.

Authentication and user management

I put these together because I thought they should go together because stars is an easy feature dependent on users.

https://github.com/Pigmice2733/peregrine-frontend/blob/dev/docs/docs.md#authenticate
https://github.com/Pigmice2733/peregrine-frontend/blob/dev/docs/docs.md#usersuserid
https://github.com/Pigmice2733/peregrine-frontend/blob/dev/docs/docs.md#users

Event teams list should include teams that are in matches, but not on the TBA match list

For example, Madtown Throwdown has many teams that bring multiple robots, where the extra/practice robots get their team numbers prefixed with 9 or 8. So the team list route should return a combination of the teams list from TBA + the teams that are in matches

Fix issue with sql connection pool

It would seem we're leaking SQL connections. This is high priority and should be fixed soon.

Oct 24 00:44:52 centos-s-1vcpu-1gb-nyc1-01 peregrine[8771]: {"error":"pq: remaining connection slots are reserved for non-replication superuser connections","level":"error","msg":"retrieving event data","time":"2018-10-24T00:44:52Z"}
Oct 24 00:53:31 centos-s-1vcpu-1gb-nyc1-01 peregrine[8771]: {"admin":false,"body_size":22,"level":"info","method":"GET","msg":"got request","remote_addr":"REDACTED:42086","request_time":0.005135046,"start_time":1540342411,"status_code":500,"time":"2018-10-24T00:53:31Z","url":"/events","u
Oct 24 00:53:31 centos-s-1vcpu-1gb-nyc1-01 peregrine[8771]: {"error":"pq: remaining connection slots are reserved for non-replication superuser connections","level":"error","msg":"retrieving event data","time":"2018-10-24T00:53:31Z"}
Oct 24 00:54:15 centos-s-1vcpu-1gb-nyc1-01 peregrine[8771]: {"admin":false,"body_size":22,"level":"info","method":"GET","msg":"got request","remote_addr":"REDACTED:42450","request_time":0.004834191,"start_time":1540342455,"status_code":500,"time":"2018-10-24T00:54:15Z","url":"/events","u
Oct 24 00:54:15 centos-s-1vcpu-1gb-nyc1-01 peregrine[8771]: {"error":"pq: remaining connection slots are reserved for non-replication superuser connections","level":"error","msg":"retrieving event data","time":"2018-10-24T00:54:15Z"}
Oct 24 00:54:16 centos-s-1vcpu-1gb-nyc1-01 peregrine[8771]: {"admin":false,"body_size":19,"level":"info","method":"GET","msg":"got request","remote_addr":"REDACTED:42450","request_time":0.000034252,"start_time":1540342456,"status_code":404,"time":"2018-10-24T00:54:16Z","url":"/favicon.ic

https://www.reddit.com/r/golang/comments/4j5ysh/reliably_detect_databasesql_connection_leaks/

Log User-Agent

It would be cool to log the User-Agent HTTP header.

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent

Add event matches endpoint

See /event/{eventKey}/matches in the API docs

https://github.com/Pigmice2733/peregrine-frontend/blob/master/docs/docs.md#eventeventkeymatches

Replace config files with .env files

Also, instead of having a bunch of postgres stuff we should just have like PRGN_PG_DSN

Sort matches by scheduled start date

The matches are currently coming back in database-retrieval-order. They should be sent in order of the scheduled start date

Add CONTRIBUTING.md

This should probably be done by somebody that has contributed to this project before.

@simonporter007
@brendanburkhart
@calebeby

Add full district name to event info

See Pigmice2733/peregrine-frontend#93

Proper error handling for `cmd/peregrine`

Use SQL JOINs for getting stars

Currently, we use a separate statement for getting user stars, but we could just use a JOIN, similar to what was just done for matches in this PR, and it would be both faster and more readable.

Removed deleted matches

If they aren't going to happen, tie-breaker matches are deleted from TBA. Deleted matches need to be removed from the database, but custom matches need to be left alone.

Move linting to CircleCI step

The GolangCI bot is pretty unreliable so we should stop using it and instead use circleci to run the golangci-lint cli tool.

Add POST to events and matches so that new matches and events can be added

We need a way to add events and matches via the API.

Add/update tests for ETag stuff in TBA package