o1-labs/archive-node-api

This is a GraphQL server that is built with the intention of exposing information from Mina's Archive Node

archive-node-api's Issues

Write technical spec of proposed implementation

The technical spec describing all implementation details is located here:

https://www.notion.so/minaprotocol/PoS-Selection-Algorithm-Testing-OCaml-Archive-Node-API-fc6fbbbeed1c4fe6a433fe67ba4176b7

Rename `sequence_events` to `actions` in SQL query once Berkeley is redeployed

Reflect SnarkyJS changes related to not include fromActionState data in resulting output of fetched actions

• If fromActionState is used, the archive node returns one block too many times (namely, the one that ended in fromActionState).
  • o1-labs/snarkyjs#854

[BUG] Returning incorrect events

Given the following transaction:
https://berkeley.minaexplorer.com/transaction/5JtsLnRiszbYZLHfELTN8RMn841yN81FZbro5sJF5QYtvfkknThC

The following query returns incorrect events:

{
  events(
    input: {address: "B62qk6bVJYmHzQHp8xyKKVsb6MBYEQndeXCRvcRGW2NKqEHjqRb6gyD", from: 281, to: 281}
  ) {
    blockInfo {
      distanceFromMaxBlockHeight
      height
      globalSlotSinceGenesis
      stateHash
      parentHash
      chainStatus
    }
    eventData {
      data
      transactionInfo {
        hash
        memo
        status
      }
    }
  }
}

Not sure if a similar issue exists with actions as well. Need to investigate.

Implement Mina client changes

Update opentelemetry npm plugin to use `@opentelemetry/sdk-trace-base` instead

Add a Dockerfile to build and run the server

[Investigate] - Duplicated actions/events seem to happen randomly

Update README

Related issues:
#19
#17
#16

Bug: Duplicate single event types are filtered out

Outdated GQL example in Readme

Readme example query refers to events -> eventData -> fields where fields was replaced with data.

Switch schema to be .graphql format instead of .ts

Berkeley Redeployment Tracker

Changes that need to be made when Berkeley is redeployed with new schema changes:

#38
#7
#56

Add changes to CI

Add a `distanceFromMaxBlockHeight` field on the `BlockInfo` GraphQL type

Add unique identifier for actions in individual AccountUpdates

Addresses #o1-labs/snarkyjs#798 for Archive Node API related changes

Inconsistent sorting of results

The results of actions/events do not always match the order of the transaction. It is also inconsistent when applying various filters.

query getActions {
  actions(
    input: {address: "B62qq1miZzh8QMumJ2dhJSvPxdeShGQ2G2cH4YXwxNLpPSvKdRVTb3q" }
  ) {
    actionData {
      data
    }
  }
}

This query returns the data out of order:

{
  "data": {
    "actions": [
      {
        "actionData": [
          {
            "data": [
              "22"
            ]
          },
          {
            "data": [
              "20"
            ]
          },
          {
            "data": [
              "19"
            ]
          },
          {
            "data": [
              "18"
            ]
          },
          {
            "data": [
              "17"
            ]
          },
          {
            "data": [
              "16"
            ]
          },
          {
            "data": [
              "15"
            ]
          },
          {
            "data": [
              "14"
            ]
          },
          {
            "data": [
              "13"
            ]
          },
          {
            "data": [
              "12"
            ]
          },
          {
            "data": [
              "11"
            ]
          },
          {
            "data": [
              "10"
            ]
          },
          {
            "data": [
              "9"
            ]
          },
          {
            "data": [
              "8"
            ]
          },
          {
            "data": [
              "21"
            ]
          },
          {
            "data": [
              "7"
            ]
          }
        ]
      },

However, adding additional filtering returns the data as ordered in the actual transaction:

query getActions {
  actions(
    input: {address: "B62qq1miZzh8QMumJ2dhJSvPxdeShGQ2G2cH4YXwxNLpPSvKdRVTb3q", from: 2500, to: 2500 }
  ) {
    actionData {
      data
    }
  }
}

Port over best tip selection algorithm to TypeScript

To have a good way of selecting the best tip among multiple blocks at the tip of the network, we need to implement a selection algorithm. We can port over the current algorithm here in TypeScript to do so: https://github.com/MinaProtocol/mina/blob/bff1e117ae4740fa51d12b32736c6c63d7909bd1/src/lib/consensus/proof_of_stake.ml#L2950

Add Action State to actions resolver

Description

While implementing actions, the actions state was overlooked as part of the integration with SnarkyJS. We should add field to the Actions resolver that returns the action state from an account update after applying a list of actions.

Archive Node API POS Testing

This epic aims to ensure the consistent implementation of the proof of stake selection algorithm for blocks at the highest tip across both our main Mina client and the Archive Node API server. The ongoing work referenced in this pull request represents a small segment of this process.

The concern is keeping both implementations synchronized and compatible, which we plan to validate through a differential test comparing both software pieces. This epic captures the sequence of tasks necessary to reach this objective.

Tasks Involved

• Write a Technical Specification: This involves the preparation of a detailed technical document outlining our proposed approach to the implementation.
• Implement Changes in the OCaml Client: Apply the required modifications to the OCaml client to adapt it to our approach.
• Implement Changes for the Archive Node API Server: We will introduce necessary changes in the Archive Node API server.
• Integration in CI Pipelines: Lastly, we'll incorporate this workflow into the relevant Continuous Integration (CI) pipelines, allowing for automated checks in the future.

By following this plan, we aim to maintain a synchronized and effective implementation of our algorithm across all relevant platforms.

This work is required for #80 to land.

Add sanity checks during the application startup

• For later: it will be good to know from the app if it can be used with the connection string provided right after the startup:
  • Like add checks during/after the app startup:
    • if it is possible to connect to DB configured?
    • if current app is compatible with DB schema connected?
  • And notify users in terminal output if there are errors or all is good.

Archive node silently returns an empty array when given a non-existing fromActionState

When querying getActions with a non-existing fromActionState, the API returns an empty array.

This behavior is unexpected to me. I would have expected an error to be returned.

Change SQL queries to reflect zkapp_fee_payer_body using public key

MinaProtocol/mina#12741

Add RFC to Github from Notion

[README] - Document best tip problem

Rename `zkapp_sequence_states` -> `zkapp_action_states` when Berkeley is redeployed

Extend server usage documentation

Readme should mention that before running the npm run start one actually need to build the app with npm run build.

Allow .env usage

.env (git ignored) and .env.example (under version control) where you can specify defaults/examples instead of For an example of some sensible defaults, see below: README section.

Support different chain-tip selection methods

Suggestions to improve the best tip chain selection was made in a Discord channel, see here: https://discord.com/channels/484437221055922177/1086196859627917343/1087343997812486174

For those without Discord, the contents of the message are:

"
Potential options for the archive node api could be:
the latest finalized block by some num of confirmations n
the latest undisputed block (what I mean by that is that if there are multiple blocks proposed in a slot, the next proposer has selected a block out of those)
the latest unconfirmed block (for example the block in the latest filled slot with the lowest timestamp)
all blocks with the potential of becoming canonical

Doing it this way keeps the possibility of only retrieving confirmed data while also giving the developer the option to modify that behaviour if needed.
"

By supporting these different options, we can let the user define how they want to query the data for the Archive Node.

Add GraphQL Inspector

Description

Install GraphQL Inspector as a Github Action to detect breaking changes in schema changes.

If a breaking change is detected, block the pull request. Then make the pull request unblocked with a breaking-change-approval tag after it's been reviewed.

Add module for performance testing

Description

Add an easy-to-run module that will run some performance testing on the GraphQL server. The module should output a easy to read report of all metrics gathered when running the benchmarks

Incorrect default port in Readme

Server starts on port 8080 by default, not the 4000 as it is mentioned in Readme (it will be also good to mention the PORT envar configuration option).

[Roadmap] - Deploy with 2 community node operators

[Bug] Events/Actions transaction information is incorrectly fetched

Description

Transaction Information related to events/actions are defined on the top level of the actions/events resolvers:

Archive-Node-API/schema.graphql

Line 43 in 90c90cb

type EventOutput {

The GraphQL schema represents an invalid data format for events/actions. The way the GraphQL schema is defined, it implies that events/actions will always have the same transaction information in a block. But this is incorrect since events/actions can be dispatched from different zkApp transactions, which all get included in a single block. Right now, the Archive Node API, will look at the first transaction fetched from the Postgres DB and assume that that transaction emitted all events/actions under a block.

We will need to fix the transaction information to be related to individual emitted actions/events since they can all have different transaction information related to a block.

Add better error checking with missing environment variables on start up

Fix Boolean env vars parsing

By default any non-empty string env var in condition like this if (process.env.VAR) { will be parsed into boolean: true. So we need to check against another string if (process.env.VAR === 'true') {.

Unit tests

Add error checking for multi host Postgres configs

Implement Archive Node API server changes

Better error messages in GraphQL layer and provide a way to hide stack traces in prod

Add a testing plan

Description

We should add a plan on how to test the server accurately. Currently, this is left out of the RFC, but @shimkiv has graciously voluntureed to come up with some ideas. Once a formal plan is in place, it should be added to the RFC.

Related issue: #9

Change SQL queries to ignore non canonical blocks