yngtdd/roadmap

• 👋 Hi, I’m @yngtdd!

roadmap's Issues

Bespoke Versioning - Logs

Log the commit hash of all microservices, grouping them by day.

Versioning Bespoke - Q2

Create new versions that represent breaking changes and feature releases.

Proposed Approach

When a new feature or a breaking change is introduced, we now need a new source tree which does not interfere with previous working versions. The simplest approach would be a single directed graph without branching:

┌──┐      ┌──┐      ┌──┐
│  ├─────►│  ├─────►│  │
└──┘      └──┘      └──┘

For simplicity, we would not allow branching:

1.0        2.0       3.0
┌──┐      ┌──┐      ┌──┐
│  ├─────►│  ├─────►│  │
└──┘      └─┬┘      └──┘
            │
            │       ┌──┐
            └──────►│  │
                    └──┘
                     4.0

This would require that we keep logs for all possible versions, and store the feature and bug set logs for each version for each customer.

Open Questions

1. Would we be able to plan new features across customers?

Cache LVC microservice results

Store the LVC microservice results to be later used by other microservices

Proposed approach:

• Store results in CosmoDB
• Maintain hashes from the LVC results - tracking batches of runs

Minimal LVC - Azure Native

Prove out a first LVC microservice.

Used to test the following two aims:

1. Asynchronous returns of LVC results on a CML by CML basis
2. Reducing overhead waiting on results

Using the following technology:

• Azure Event Grid
• Cloud Event
• Service Bus
• Open Telemetry

Newton LVC Connection

Have Newton UI render the LVC results as they come in asynchronously.

Open Questions

1. How does Newton currently store LVC results?
2. How does Newton currently render results?

Possible Approaches

• tRPC + React to handle communication between backend database and front end

Versioning - Easiest

Use semantic version with release notes at the microservice level. The Newton monorepo's versioning scheme becomes the aggregate of microservice versions.

Generic Microservice

Wireframe a generic, cloud agnostic, event driven Python microservice.

Approach

• Docker container
• Poetry build
• Dapr or Knative
• Subscribes to a topic on event service

Microservice Template

Create a template to create the scaffolding of new microservices.

Versioning Bespoke - Q3

Build a dependency management system. This is on the scope of a package manager that would integrate with both our monorepo, git, and Azure Devops.

Cache Template

Create a way to instantiate generic data storage for microservice caches.

Approach

• CLI to create new data stores that can be plugged into by generic microservices

Versioning - Bespoke Q1

First cut of bespoke versioning

Proposed alpha stage:

• log of all features present in time stamped builds
• log known bugs in the present time stamped builds
• map users to the above logs

Version Dependency Graphs

Create new versions that represent breaking changes and feature releases.

Proposed Approach

When a new feature or a breaking change is introduced, we now need a new source tree which does not interfere with previous working versions. The simplest approach would be a single directed graph without branching:

┌──┐      ┌──┐      ┌──┐
│  ├─────►│  ├─────►│  │
└──┘      └──┘      └──┘

For simplicity, we would not allow branching:

1.0        2.0       3.0
┌──┐      ┌──┐      ┌──┐
│  ├─────►│  ├─────►│  │
└──┘      └─┬┘      └──┘
            │
            │       ┌──┐
            └──────►│  │
                    └──┘
                     4.0

This would require that we keep logs for all possible versions, and store the feature and bug set logs for each version for each customer.

Open Questions

1. Would we be able to plan new features across customers?
2. How do you manage these dependency graphs with the ability to create new generic microservices?

Concrete LVC Microservice

Used to test the following two aims:

1. Asynchronous returns of LVC results on a CML by CML basis
2. Reducing overhead waiting on results

Using the following technology:

Either:

• dapr
• knative

Tool to Revert Versions

Add the ability to switch between Newton versions and deploy them independently.

LVC Newtion UI Connection

Have Newton UI render the LVC results as they come in asynchronously.

Open Questions

1. How does Newton currently store LVC results?
2. How does Newton currently render results?

Possible Approaches

• tRPC + React to handle communication between backend database and front end

LVC Aggregation Microservice

Aggregate the results from the LVC microservice.

Proposed approach:

• Store the CML results in a data store similar to CosmoDB
• Trigger a new event when CosmoDB is updated to call aggregation microservice
• Rewrite the aggregation code

Minimal LVC - Cloud Agnostic

Prove out a first LVC microservice.

Used to test the following two aims:

1. Asynchronous returns of LVC results on a CML by CML basis
2. Reducing overhead waiting on results

Using the following technology:

Either:

• dapr
• knative

Version Dependency Graph - Part 2

Continuing the work from the previous quarter.

Concrete LVC Aggregation

Aggregate the results from the LVC microservice.

Proposed approach:

• Store the CML results in a data store similar to CosmoDB
• Trigger a new event when CosmoDB is updated to call aggregation microservice
• Rewrite the aggregation code

Generic Cache of Results

Create a generic approach to storing results from microservices.

Approach

• Instantiate a generic form of data storage, i.e. blob or datalake