avohq/flutter-avo-inspector

Avo Inspector SDK for Flutter

flutter-avo-inspector's Issues

Avo Inspector entry point

Here we are going to build the public interface that the library users will use. It will invoke the type extractor built in the previous issue and log the results.

Requirements

• constructor

apiKey: string;
env: "prod" | "dev" | "staging";
version: string;
appName: string;

should save passed data for later access

• shouldLog - a method to enable or disable logs

• trackSchemaFromEvent(eventName: string, eventProperties: { [propName: string]: any }): should print the eventName parameter and the result of the type extractor function applied to the eventProperties parameter if the logs are enabled (see shouldLog method). Should return the extracted schema from the eventProperties.

References

https://github.com/avohq/node-avo-inspector/blob/main/src/AvoInspector.ts

The type parser

The core of this SDK is the type extractor function. This issue is about building one.

Requirements

1. Build a function that accept anything and returns a Json string representing its type.

The resulting types are the following:

• string
• int
• float
• bool
• null
• list
• object
• unknown

The return format is:

{
    propertyType: string;
}

2. When parsing lists, extract the types present in the list in an additional field:

{
    propertyType: string;
    children: string[]; // e.g. [string, null]
}

3. When parsing objects, extract the object fields recursively:

{
    propertyType: string;
    children: {
        propertyName: string; // object field's name
        propertyType: string;
        children?: any
    }[];
}

4. Build a function that accepts a map of string keys to values of any type and returns an array of objects of the following type:

{
    propertyName: string; // map key
    // The other fields are the result of the function build on step 3 applied to the map value 
    propertyType: string;
    children: {
        propertyName: string; // object field's name
        propertyType: string;
        children?: any
    }[];
}

Add tests to verify the correct behavior.

Reference code

We have this logic and tests for it implemented in the Avo Inspector SDKs for other platforms

TypeScript:
The function: https://github.com/avohq/node-avo-inspector/blob/main/src/AvoSchemaParser.ts#L6
Tests: https://github.com/avohq/node-avo-inspector/blob/main/src/__tests__/TestParsing.ts

Java:
The function: https://github.com/avohq/android-avo-inspector/blob/33934530274af6aa7640dfaba54b4d626bbe6e48/avoinspector/src/main/java/app/avo/inspector/AvoSchemaExtractor.java#L172
Tests: https://github.com/avohq/android-avo-inspector/blob/33934530274af6aa7640dfaba54b4d626bbe6e48/avoinspector/src/test/java/app/avo/inspector/MapSimpleTypeSchemaExtractionTests.java

Send session started events to the API

Client should group event schemas into sessions.

Session is a rolling 5 minutes window. If an event schema is send before 5 minutes from the last event schema passed - it's the same session.
If an event is sent after 5 minutes since previous event - it's a new session.

Also, if it is a first event schema sent from current app instance - it's a new session.

Session is identified by a session id - a UUID.

API doc

Endpoint: api.avo.app/inspector/v1/track
Method: POST

Every time new session starts we need to send the following payload to the API:

{
  [
    {
      "appName": ...,
      "appVersion": ...,
      "libVersion": ...,
      "libPlatform": "flutter",
      "apiKey": ...,
      "sessionId": "11332f62-ba2d-11eb-8529-0242ac130003", // UUID
      "messageId": "4cffbdca-ba2e-11eb-8529-0242ac130003", // UUID
      "createdAt": "yyyy-mm-ddThh:mm:ss.SSSZ",
      "type": "sessionStarted",
    },
  ]
}

### HTTP Responses

- Server returns a `200` response for successful API requests.
- Server returns a `400` response for invalid requests.
Invalid requests include:
    - `Content-type header missing`
    - `Invalid API key

Write tests for the session tracking and the payload generated for the API calls.

Reference

https://github.com/avohq/node-avo-inspector/blob/main/src/AvoSessionTracker.ts
https://github.com/avohq/node-avo-inspector/blob/main/src/__tests__/Sessions.ts

Log unsuccessful server responses

Write readme

Set up CI in Github actions

Run the unit tests on every pull request

Send the event schemas to the API

The last piece is to send the data we prepared to the API.

TODO

• Sessions handling: #11
• Batching by size
• Batching by time

API doc

Endpoint: api.avo.app/inspector/v1/track
Method: POST

Every time the trackSchemaFromEvent method is called we need to send the following to the API:

{
  [
    {
      "appName": ...,
      "appVersion": ...,
      "libVersion": ...,
      "libPlatform": "go",
      "apiKey": ...,
      "sessionId": "11332f62-ba2d-11eb-8529-0242ac130003", // UUID
      "messageId": "4cffbdca-ba2e-11eb-8529-0242ac130003", // UUID
      "createdAt": "2021-05-21T09:05:40.735Z",
      "type": "sessionStarted",
    },
    {
      "appName": "...",
      "appVersion": "...",
      "libVersion": "...",
      "libPlatform": "go"
	"sessionId": "11332f62-ba2d-11eb-8529-0242ac130003", //UUID same as in the sessionStarted event
      "messageId": "54cafd77-c6cd-46b0-8459-69b59d44f012", // new UUID
      "createdAt": "2021-05-21T09:05:40.740Z",
      "type": "event",
      // Data we prepared before
      "eventName": "Onboarding Started",
      "eventProperties": [
        {
          "propertyName": "user",
          "propertyType": "object",
          "children": [
            {
              "propertyName": "id",
              "propertyType": "string"
            }
          ]
        },
        {
          "propertyName": "friendNames",
          "propertyType": "list",
           "children": ["string", "null"]
        },
        {
          "propertyName": "isAdmin",
          "propertyType": "bool",
        },
      ]
    },
  ]
}

### HTTP Responses

- Server returns a `200` response for successful API requests.
- Server returns a `400` response for invalid requests.
Invalid requests include:
    - `Content-type header missing`
    - `Invalid API key

Write tests for the trackSchemaFromEvent method and the payload generated for the API calls.

Reference

https://github.com/avohq/node-avo-inspector/blob/main/src/AvoNetworkCallsHandler.ts
https://github.com/avohq/node-avo-inspector/blob/main/src/__tests__/TestNetworkCallsHandler.ts
https://github.com/avohq/node-avo-inspector/blob/main/src/__tests__/TestNetworkCallsHandlerPublic.ts

Respect sampling rate

Example app

Requirements

Create an app that runs locally and can trigger the interface methods with various inputs. Can be console or UI.

Parse object fields in the type parser

When parsing objects, the result should be in a form of

{
  "field_name": "object",
  "children": [
        "object_field_name": "object_field_type",
         ...
    ];
}

Publish to pub.dev

Batch network calls

Collect event schemas and session objects and send them in batches.

There are 2 triggers to send - size and time.
Both should be configurable.
Defaults are 30 items and 30 seconds.
In development default batch size is 1.

The storage should not go over 1000 items.

Items that were not sent yet should be stored on disk.

When the app starts it should check the disk for outstanding items and send them, regardless of batch size.

There should be logs when an item is supplied to batcher and when the item is sent.

You should not be sending more than 1 batch in parallel. If batch sending is in progress next batch should wait.