terkwood/bugout

AI-driven, Multiplayer Go/Weiqi/Baduk for the web 🐛🤖🦀♟

bugout's Issues

☕️ JVM services: Kafka InvalidPidMappingException

System stops processing moves

This exception was seen in multiple services.

Gateway service showed no errors at all

History provider

📚          ️d026ae80 HISTPROV HistoryProvided(gameId=d026ae80-a7e1-40a3-8e02-d297bf611d5e, replyTo=d8469f2b-9238-45af-94e0-7b23004fbf0d, eventId=425fbe0e-f9c4-4e98-8b30-a233feac61b0, moves=[], epochMillis=1566684125177)
Exception in thread "bugout-history-provider-48398a94-7ebb-4572-98cd-3958b1e152a8-StreamThread-1" org.apache.kafka.streams.errors.StreamsException: task [0_0] Abort sending since an error caught with a previous record (key d026ae80-a7e1-40a3-8e02-d297bf611d5e value {"type":"HistoryProvided","gameId":"d026ae80-a7e1-40a3-8e02-d297bf611d5e","replyTo":"d8469f2b-9238-45af-94e0-7b23004fbf0d","eventId":"425fbe0e-f9c4-4e98-8b30-a233feac61b0","moves":[],"epochMillis":1566684125177} timestamp 1566684124980) to topic bugout-history-provided-ev due to org.apache.kafka.common.KafkaException: org.apache.kafka.common.errors.InvalidPidMappingException: The producer attempted to use a producer id which is not currently assigned to its transactional id.
	at org.apache.kafka.streams.processor.internals.RecordCollectorImpl.recordSendError(RecordCollectorImpl.java:138)
	at org.apache.kafka.streams.processor.internals.RecordCollectorImpl.access$500(RecordCollectorImpl.java:50)
	at org.apache.kafka.streams.processor.internals.RecordCollectorImpl$1.onCompletion(RecordCollectorImpl.java:201)
	at org.apache.kafka.clients.producer.KafkaProducer$InterceptorCallback.onCompletion(KafkaProducer.java:1318)
	at org.apache.kafka.clients.producer.internals.ProducerBatch.completeFutureAndFireCallbacks(ProducerBatch.java:230)
	at org.apache.kafka.clients.producer.internals.ProducerBatch.abort(ProducerBatch.java:158)
	at org.apache.kafka.clients.producer.internals.RecordAccumulator.abortBatches(RecordAccumulator.java:742)
	at org.apache.kafka.clients.producer.internals.Sender.maybeAbortBatches(Sender.java:479)
	at org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:316)
	at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:238)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.kafka.common.KafkaException: org.apache.kafka.common.errors.InvalidPidMappingException: The producer attempted to use a producer id which is not currently assigned to its transactional id.
	at org.apache.kafka.clients.producer.internals.TransactionManager$AddPartitionsToTxnHandler.handleResponse(TransactionManager.java:1204)
	at org.apache.kafka.clients.producer.internals.TransactionManager$TxnRequestHandler.onComplete(TransactionManager.java:1069)
	at org.apache.kafka.clients.ClientResponse.onComplete(ClientResponse.java:109)
	at org.apache.kafka.clients.NetworkClient.completeResponses(NetworkClient.java:561)
	at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:553)
	at org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:307)
	... 2 more
Caused by: org.apache.kafka.common.errors.InvalidPidMappingException: The producer attempted to use a producer id which is not currently assigned to its transactional id.

Judge

⚖️          ️d026ae80 ACCEPT   BLACK @ Coord(x=14, y=13) capturing
Exception in thread "bugout-judge-17432a73-0993-4717-b94d-ba7218e9e859-StreamThread-1" org.apache.kafka.streams.errors.StreamsException: task [0_0] Abort sending since an error caught with a previous record (key d026ae80-a7e1-40a3-8e02-d297bf611d5e value {"type":"MoveMade","gameId":"d026ae80-a7e1-40a3-8e02-d297bf611d5e","replyTo":"bc148b87-04f0-49f0-bfc5-d135d3453bb6","eventId":"f7d635d9-f7c1-4602-bf37-ab375f4dd266","player":"BLACK","coord":{"x":14,"y":13},"captured":[]} timestamp 1566684102451) to topic bugout-move-accepted-ev due to org.apache.kafka.common.KafkaException: org.apache.kafka.common.errors.InvalidPidMappingException: The producer attempted to use a producer id which is not currently assigned to its transactional id.
	at org.apache.kafka.streams.processor.internals.RecordCollectorImpl.recordSendError(RecordCollectorImpl.java:138)
	at org.apache.kafka.streams.processor.internals.RecordCollectorImpl.access$500(RecordCollectorImpl.java:50)
	at org.apache.kafka.streams.processor.internals.RecordCollectorImpl$1.onCompletion(RecordCollectorImpl.java:201)
	at org.apache.kafka.clients.producer.KafkaProducer$InterceptorCallback.onCompletion(KafkaProducer.java:1310)
	at org.apache.kafka.clients.producer.internals.ProducerBatch.completeFutureAndFireCallbacks(ProducerBatch.java:230)
	at org.apache.kafka.clients.producer.internals.ProducerBatch.abort(ProducerBatch.java:158)
	at org.apache.kafka.clients.producer.internals.RecordAccumulator.abortBatches(RecordAccumulator.java:742)
	at org.apache.kafka.clients.producer.internals.Sender.maybeAbortBatches(Sender.java:460)
	at org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:297)
	at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:235)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.kafka.common.KafkaException: org.apache.kafka.common.errors.InvalidPidMappingException: The producer attempted to use a producer id which is not currently assigned to its transactional id.
	at org.apache.kafka.clients.producer.internals.TransactionManager$AddPartitionsToTxnHandler.handleResponse(TransactionManager.java:1041)
	at org.apache.kafka.clients.producer.internals.TransactionManager$TxnRequestHandler.onComplete(TransactionManager.java:909)
	at org.apache.kafka.clients.ClientResponse.onComplete(ClientResponse.java:109)
	at org.apache.kafka.clients.NetworkClient.completeResponses(NetworkClient.java:557)
	at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:549)
	at org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:288)
	... 2 more
Caused by: org.apache.kafka.common.errors.InvalidPidMappingException: The producer attempted to use a producer id which is not currently assigned to its transactional id.

Prevent accidental color switching / Sabaki command spam

This issue appears to be caused by BUGOUT Sabaki spamming a bunch of commands in a row.

Do we need to add a commands array to the GTP Controller shim, and to the WebSocketController? This type of array is present in the node.js GTP module -- see source code

If you trigger a modal dialog in the UI, then the other player triggers a move, the MoveMade event is missed by the UI. This can result in having your player send the wrong color of move. Might be hard to reproduce, but if you see this sort of output in the gateway component...

🤖 92d8dc09 BEEP
🏓 3dc0069d PING   PONG   (273.152ms)
♕ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 4, y: 14 })
♜ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 11, y: 4 })
♖ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 16, y: 15 })
♞ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 16, y: 14 })
♘ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 5, y: 5 })
♝ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 15, y: 15 })
♔ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 10, y: 10 })
♟ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 17, y: 15 })
♘ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 16, y: 16 })
♝ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 16, y: 17 })
♔ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 12, y: 16 })
♚ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 15, y: 16 })
♗ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 16, y: 12 })
♚ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 17, y: 16 })
♙ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 14, y: 6 })
♛ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 13, y: 12 })
♙ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 8, y: 13 })
♝ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 14, y: 7 })
♘ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 16, y: 5 })
♛ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 14, y: 5 })
♖ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 15, y: 6 })
♝ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 6, y: 8 })
♖ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 5, y: 8 })
♚ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 8, y: 8 })
♔ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 8, y: 6 })
♚ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 13, y: 9 })
♘ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 13, y: 6 })
♝ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 17, y: 14 })
♕ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 18, y: 13 })
♝ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 15, y: 14 })
♔ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 17, y: 13 })
♞ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 17, y: 12 })
♖ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 17, y: 11 })
♛ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 18, y: 11 })
♕ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 18, y: 12 })
♚ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 18, y: 10 })
♙ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 17, y: 10 })
♛ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 17, y: 9 })
♔ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 18, y: 9 })
♟ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 4, y: 8 })
♕ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 5, y: 7 })
♟ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 9, y: 3 })
♘ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 9, y: 4 })
♛ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 13, y: 11 })
♔ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 14, y: 11 })
♝ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 14, y: 12 })
♖ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 15, y: 11 })
♟ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 16, y: 11 })
♕ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 16, y: 10 })
♜ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 15, y: 12 })
♙ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 16, y: 13 })
♝ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 15, y: 10 })
♗ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 14, y: 10 })
♛ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 14, y: 9 })
♗ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 15, y: 9 })
♝ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 13, y: 10 })
♔ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 17, y: 8 })
♛ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 9, y: 9 })
♘ 92d8dc09 MOVE   a68c21bb WHITE Some(Coord { x: 8, y: 10 })
♛ 92d8dc09 MOVE   a68c21bb BLACK Some(Coord { x: 11, y: 13 })
🏓 92d8dc09 PING   PONG   (87.259ms)

Judge: Guard against null gamestate in join

If you send a MakeMoveCmd but there is no gamestate available to join:

judge_1                  | MAKE MOVE CMD 00b8d848 BLACK Coord(x=0, y=18)
judge_1                  | Exception in thread "bugout-judge-3474fb5f-ef9f-4a9f-b82d-7234a7120d44-StreamThread-1" java.lang.IllegalArgumentException: Parameter specified as non-null is null: method Judge$process$valueJoiner$1.apply, parameter rightValue
judge_1                  | 	at Judge$process$valueJoiner$1.apply(Judge.kt)
judge_1                  | 	at Judge$process$valueJoiner$1.apply(Judge.kt:18)
judge_1                  | 	at org.apache.kafka.streams.kstream.internals.KStreamKTableJoinProcessor.process(KStreamKTableJoinProcessor.java:73)
judge_1                  | 	at org.apache.kafka.streams.processor.internals.ProcessorNode.process(ProcessorNode.java:117)

Sabaki: auto reconnect websocket

Gateway can help support this.

Instead of "AddClient" struct being processed by the router component, expose "RequestOpenGame" and a "Reconnect To Game" variants

Sabaki: modal dialog for create priv game vs find public game

Related to #42

Sabaki: modal dialog for color choice

The setAppElement example on codepen worked well enough

Related to #84, #42, #83

👂Allow white to hear the first move 👂

With game IDs being automatically provisioned by the gateway, you should be able to put White into a passive state and get moves routed to you correctly.

Minimum POC

• Docker setup (#8)
• hard-coded game ID. Erase topics on every run to test
• create minimum judge (#10)
• create minimum gateway and listen only for events relevant to the game in progress (#4)
• add reverse proxy for gateway websocket (#39)
• create a minimal frontend in React.js (#5)
• no lobby functionality

Map gameId to GameBoard

.... So that data for individual games is tracked correctly . Change existing GameBoard class so that it accounts for this mapping. "AllGameBoards"?

MVP Data Paths

With the POC finished, we need to add four data paths to support a full featured system:

• Back-end: Wait for Public Game (req) / public Game found (reply)... For people who want to queue in the lobby (#78)
• Back-end: private games (#78)
• Front-end & gateway: Private games joining (#53)
• write open games to changelog as soon as they're created (#78)
• Gateway: swap envy for dotenv #97
• color choice: UI and gateway support #84 / #65
• #63 choose color / color chosen backend

enum Visibility { Public, Private} 

struct OpenGame {
  gameId: GameId, visibility: Visibility
} 

We'll need a GameLobby to keep track of all games, public and private, in a global KTable

Whenever a game is joined, it's removed from that list 📃

🤖 Play against KataGo

Goal

Implement a system which allows playing against a KataGo instance running on an NVIDIA Jetson Nano.

Background

We picked up an NVIDIA Jetson Nano board and built KataGo. Documentation is available in the repo now

Tasks

• build KataGo on the nano
• wrap katago stdin/stdout #190
• make a basic docker image #189
• gateway: backend routing (#205)
• gateway: send & recv bot attachment and move-making messages
• use websockets correctly #210
• create botlink service and push game states via websocket (#193, #196, #197)
• connect tinybrain to Botlink service via websocket and process game states (#194)
• document reverse-proxy setup
• clean up micro-changelog init (#187)
• extend Sabaki initial menu, send attach bot command, process makemoves as normal, listenForMove when human plays white
• fix parse error #220
• fix all coords #223

Architecture draft

Use KataGo analysis engine to respond to queries.

Errata

• https://github.com/lightvector/KataGo
• https://github.com/leela-zero/leela-zero
• https://blog.janestreet.com/accelerating-self-play-learning-in-go/
• https://github.com/lightvector/KataGo/blob/master/README.md#how-to-use
• leela-zero/leela-zero#2569 (comment)

An existing service for playing against bots

• https://www.zbaduk.com/tools
• and a related KataGo ticket

Gateway: manage finding public games

Friend #42 and #87

#87 should be completed before this

Docker setup

Some inspiration : https://medium.com/rate-engineering/using-docker-containers-as-development-machines-4de8199fc662

Minimum gateway

Gateway should listen for updates to streams based on gameId, and route them to appropriate clients.

Use a hard-coded gameId for the first draft

🔲 Sabaki: default to not showing move colorization

Move colorization can do a bit of blinking on some devices. Default to off

Make startup delays configurable

The run scripts in changelog, judge] and gateway should have configurable wait values sourced from .env files. This will support operation in various sized environments (small cloud instances vs hefty developer machines)

# alter the sleep value here
./wait-for-it.sh kafka:9092 -s -- sleep 16

Keep sane defaults in place in case the .env files aren't present.

Sabaki: modal dialog for color choice

Related to #84, #42, #83

Minimum judge

• Use an inner join for bugout-make-move-cmd and bugout-game-states
• On MakeMoveCmd, query game state via KTable. If move is allowed, write MoveMadeEv (topic=bugout-move-made-ev). Use Kafka streams branch operator
• validate each move
• compute captured pieces for validated moves
• communicate captured pieces in MoveEvent data type 📢
• account for passing on turns
• Transform bugout-move-made-ev into the queryable KTable on bugout-game-states

Guarantee system synchronization

System isn't smart enough to recover from the state where your browser thinks it's WHITE to move, the server thinks it's WHITE to move, but MY browser thinks it's BLACK to move

Minimum frontend

Use Sabaki

• no auth (hardoded & fake UID)
• resurrect some Sabaki engine logic into a branch
• implement a Sabaki GTP-style controller to communicate with gateway
• implement gateway communication via web sockets
• don't bother to trim anything else out of Sabaki, just take its interface as is
• handle player assignment / engine attachment sanely.

Gateway: crash less aggressively

Gateway tends to fall over in a light breeze. Make sure it can handle unexpected inputs

Explore retention settings

https://kafka.apache.org/documentation/#configuration

https://stackoverflow.com/questions/51461416/aggregration-and-state-store-retention-in-kafka-streams

Use healthcheck in docker-compose

https://stackoverflow.com/questions/42358169/docker-compose-client-connects-to-kafka-too-early

Sabaki: set UI color to white when player selects white

Presently, you have to click the little black/white piece selector in PlayBar to change sides, even if you choose to play as white when prompted by the dialog

Judge: Write to move-rejected-ev on move command failure

Sabaki: Fix corner cases for reconnect

In some cases, a client will disconnect, and then reconnect thinking that it's the wrong player's move. The Reconnected event exposed by gateway already makes it clear which player should be expected to take a turn next. The client should honor that field in order to avoid deadlocking the game for both players.

Related: #48, #49, & Terkwood/Sabaki#11

Prevent white from missing turn one

Detail

When black makes a move before white connects, white will never hear about turn one

TODO

• Finish #76
• Make Sabaki work (deterministically: white should always request history before going into a wait state)

🔢 Store client_id in browser localstorage

We'd like to persist a client_id (not a session_id!) in the browser localstorage and use it across websocket sessions

• See https://developer.mozilla.org/en-US/docs/Web/API/Web_Storage_API/Using_the_Web_Storage_API
• and see https://developer.mozilla.org/en-US/docs/Web/API/Web_Storage_API/Using_the_Web_Storage_API#Testing_for_availability

Replay game on finish

Aggregate MoveMadeEv into a history-focused KTable, and then announce that result to the gateway in a dedicated topic.

This approach lets us separate the game state querying logic into GameBoard and the replay logic into GameHistory

class GameHistory {
  fun add(moveMadeEv: MoveMadeEv): GameHistory { 
    TODO() 
    return this
  }
}

moveMadeEvStream
  .aggregate(GameHistory(), { history -> history.add(move) }, 
    Materialized.as("bugout-game-history-store))
  .toStream()
  .to("bugout-game-replay-ev")

Re-evaluate docker-compose startup timings

There are a bunch of manually-programmed startup timings affecting the kafka-streams services, and gateway. For history-provider and game-lobby, they seem especially suspect. We've noticed both of those services failing to respond on startup, which can hang the system. 😇

One way to work around this is in Sabaki -- for the "provide history" step in the web UI, specifically, we could give it only so much time to respond, before assuming there's no history available.

The game lobby has less leeway -- we can't run the system at all without the lobby functioning. So it would be best if we can ensure sane startup for all JVM services without needing the UI to work around them.

Sabaki: Queue up move when sent prior to gateway connection

If a newly-connected player sends a move prior to Sabaki having established a game ID by talking to gateway, you'll see an "undefined" dialog box and the move will never be communicated. In such a state, we should queue up the move so that it's sent after the connection to gateway is made

Reproducing this issue:

Connect to UI & pick black, and quickly place a piece. You'll see a dialog box with an OK button and "undefined" text

Allow logical color choice

• pick color pref or random
• never have two players with same color
• Kafka streams and tables

Child of #42

Gateway: write move commands to Kafka

Advances #4

🧹 Clean up dead lobby games

In the event that someone creates a game, and then disconnects before an opponent joins, gateway must signal to kafka that the game should be removed from lobby. Use clientIDs to join, here

Child of #42

💸 Automatic startup and shutdown for Kafka host

The Big Picture: Run BUGOUT at Very Low Cost

Let's run BUGOUT at a very low cost, relying mostly on a micro instance to support gateway and the reverse proxy, and starting/stopping the larger JVM services as necessary.

status

Big box starts up cleanly on a t2.medium. Runs Kafka and JVM apps. Reaper is implemented. Need to add startup service (bugle) 📯

todo

• udp channel between bugle and gateway
• gateway listen for message on shutdown channel

initial Grand plan

Provides low AWS cost, uses a single micro instance at all times.

• gateway lives on its own micro instance
• gateway micro instance uses a perm DNS address
• kafka / judge / changelog box is turned off when it isn't needed (see DEAD ZONE MONITOR, below)
• gateway would need to know how to disconnect from kafka gracefully, when kafka goes down
• DEAD ZONE MONITOR: kafka streams service which tracks the set of connected client_ids
• DEAD ZONE MONITOR: when the set reaches size 0 and when a sufficient amount of time has passed, then shut down the big-box container host instance
• DEAD ZONE MONITOR: prior to shutdown, emit a message via kafka. this will let gateway know to disconnect
• REMOTE CONTROL capabilities need to be created
• REMOTE CONTROL should be its own service, isolated from gateway
• REMOTE CONTROL accepts a single command: TURN ON THE BIG BOX
• use rusoto for aws ec2 control
• rusoto can query via instance tags to figure out private DNS of giant container host
• Sabaki ONLY connects to gateway
• Above approach isolates such that Sabaki doesn't have to know anything about REMOTE CONTROL.

CHOICE

Redis pub/sub approach

• Gateway pushes ACTIVITY message here
• ALARM CLOCK is keeping its own state on whether system is alive. If not, wake it up on ACTIVITY
• ALARM CLOCK pushes IT'S ONLINE NOW when system is up
• Gateway reestablishes Kafka connection when system comes up

UDP approach

• this isn't quite right - amend
• gateway fires and forgets enum RemoteControlCommands{ Off, On }
• use tokio udp for one way send from gateway to RC 📺
• Gateway can listen for private DNS of the freshly-initialized big-box over a UDP connection

HTTP approach

• Just use rocket for remote control API layer
• Use request/response. Remote control provides private DNS of giant host instance as the result of a POST /rc/on command

---

Tasks

• finalize machine split
• implement SHUTDOWN 📴 service
• implement ALARM CLOCK 🕒 service
• gateway graceful Kafka disconn
• gateway communication with ALARM CLOCK
• gateway management of existing commands. Don't change the flow drastically , just queue up FindGame /Create Game / Join Private / Choose Color Pref and execute them after system boots
• Sabaki notification that system is booting
• Gateway listens to Redis topic for system Liveness heartbeat, driven by ALARM CLOCK. Gateway state mirrors that value (once per sec heartbeat 💓
• Redis for gateway to track queue of delayed commands

⚠️ Sabaki: use material dialog component for "Invalid game" notice on rejection

Multiplayer Sabaki shows an ugly alert() dialog when you try to join an invalid private game. Use the material dialog component to reskin this.

🐛 Sabaki: send PASS correctly

When attempting to use the Pass control in Sabaki, Gateway received a garbled message with an x coord but no y

💥 1589668e b95307e4 ERROR    message deserialization {"type":"MakeMove","gameId":"b95307e4-a817-4917-8bbe-9291d01c9629","reqId":"2b3d0548-3c7d-4834-b285-aa5de6b4c613","player":"BLACK","coord":{"x":14,"y":null}}

Gateway: clean up router player_up

Entries are added but never removed -- fix this

Represent move success / failure using Kafka streams

Use a topic with data

gameId -> { reqId: abc, success: bool } 

As the key/value pair for the stream

TBD

Need to make sure this works well for the frontend gateway component. Finish end to end POC without checking move correctness before this ticket

Gateway: Use unbounded channels

Advances #4

Add reverse proxy

Add a caddy reverse proxy to serve gateway traffic to the public internet

Advances #3

Mapping users to games

On connection from frontend to gateway, query a Ktable to see if there's an existing mapping from GCP UID to a gameId

Do not introduce an "internal user ID". This would add unnecessary complexity to the app

You either need to use a simple consumer/producer data flow
.... Or implement an additional JVM service behind the gateway to handle such a query using Kafka streams

Drop type tag for KafkaCommands

In Gateway's model, we have a serde "type" tag on KafkaCommands. And in the Kafka Streams services, we have JSON type annotations to deal with that "type" field. We should drop the type tag on KafkaCommands and delete the JSON annotations in the KS services

// model.rs

#[derive(Serialize, Deserialize, Debug)]
#[serde(tag = "type")]
pub enum KafkaCommands {
    MakeMove(MakeMoveCommand),
    ProvideHistory(ProvideHistoryCommand),
}

🔥 Prevent race against gamestates-aggregator 🔥

If two players move in quick succession, the gamestates aggregator(GSA) may fail to catch up.

We should make the gateway aware of the current GSA turn entry, and force it to queue any moves sent before the new turn is announced by GSA

Gateway: don't disconnect the innocent

Gateway is kicking off connections that it shouldn't be

   fn on_timeout(&mut self, event: Token) -> Result<()> {
        match event {
            /*...*/
            EXPIRE => {
                // TODO FIXME HALP
                println!("⌛️ {} EXPIRE CONN", session_code(self));
                self.notify_router_close();
                self.ws_out.close(CloseCode::Away)
            }

Gateway & Sabaki: support color choice

Gateway: provide API for color choice

See #42 (MVP meta-issue) and #83 (initial impl of color chooser streams app)

Sabaki

We also need to change Sabaki to interact with the newly exposed Gateway command flow

Allow joining private games via link

Depends on #42

• use compact uuid, resolve in sabaki or gateway (#62)
• Kafka streams and tables #78
• gateway support
• Sabaki support

Use compact UUIDs for private game link

Only use compact UUID in the frontend and resolve in gateway. Don't pass IDs around to the kafka layer in that format.

See https://medium.com/@huntie/representing-a-uuid-as-a-base-62-hash-id-for-short-pretty-urls-c30e66bf35f9

TODO

• validate compact IDs as base64 (Sabaki)

Game lobby: Fix wait for opponent event

Wait for opponent doesn't come when the initial find public game command is made. It comes at the same time as the Game Ready event

Configure zstandard compression

It's fast and saves lots of space

https://cwiki.apache.org/confluence/display/KAFKA/KIP-110%3A+Add+Codec+for+ZStandard+Compression

See compresstion.type config

https://kafka.apache.org/documentation/#configuration