aurae-runtime / ae Goto Github PK

View Code? Open in Web Editor NEW

22.0 22.0 14.0 162 KB

Unix inspired command line client for Aurae

License: Apache License 2.0

Makefile 3.58% Go 96.42%

ae's People

Contributors

Stargazers

Watchers

Forkers

jolheiser rkuris jasonmcintosh j0shgrant daegalus sachinpc rrethy manuelcoppotelli voigt jesse-peters bailey-coding zoidyzoidzoid bradlugo zanshin

ae's Issues

Run and destroy a cell and executable with `ae`

Currently auraed has support for Cells which can be found in /examples written in AuraeScript (TypeScript).

Check out the cells.ts example for the functionality we need to replicate with ae.

This will be required to start:

subcommand: allocate

See the "Allocate" runtime verb in https://github.com/aurae-runtime/aurae/blob/main/api/v0/runtime.proto

Create a subcommand to match the readme. Does not have to "work" just needs the plumbing in place.

Set up automatic builds

Can we please run make test and fail the build if the tests fail? Can someone work with me in Discord to get permissions needed to set this up such that we have an automatic build for the project?

Add PKI and Config

Right now the main repository has a make target that is used to kickstart a local auraed environment.

https://github.com/aurae-runtime/aurae/blob/main/docs/certs.md#generating-client-certificate-material

make pki config

which creates mTLS certs, signs them, and configures a configuration file locally on the filesystem.

Can we please design and introduce the subcommands, flags, and features needed such that we can easily create, sign, manage, and mutate TLS certificates and config files for Aurae directly from the ae tool?

Some examples of what I am thinking:

ae create <domain> # Creates CA
ae create <domain> --user=nova # Create a Nova user
ae sign <domain> --user=nova --root=ca.crt
ae config /path/to/pki/ ~/.aurae/config.toml

Use godocs for packages and functions

Godocs for packages and functions following the style of https://go.dev/blog/godoc.

ae vs aurae

What is ae? Why does it exist?

Responding to a good question that @dmah42 has surfaced. Why does ae exist? Who is it for? Why does it exist?

I am going to take a few minutes and draft some architecture and product thoughts on the topic to help us as a group.

In order to understand ae we should first understand aer.

What is aer?

We intend to auto generate a command line tool named aer built on the Rust client that has an identical scope of a single auraed node.

This tool is for "power-users" exists as a way of quickly developing and debugging the APIs as we change them. For example an auraed developer might make a change to an API and need a quick way to test the API locally against a single daemon.

What is ae?

We intend to maintain a command line tool named ae build on the Go client that has a broader scope than aer.

The scope for ae will be for a cluster of Aurae nodes and will likely have identical functionality to aer in many places - but not all.

The ae command line tool will be for working with a group of Aurae nodes, and will likely include more pragmatic functionality that is meaningful to an enterprise operator.

Who is ae for?

The ae tool should feel familiar to any cloud operator.

Tasks such as "rolling out nginx to prod" should be possible with ae.
Tasks such as "counting the number of threads of a process" should be possible with ae.
Tasks such as "changing the current database connection count for a service" should be possible with ae.
Tasks such as "swinging traffic from one service to another" should be possible with ae.

These are more "practical" or "outcome driven" tasks, and will likely require additional functionality to the lightweight aer tool.

Guiding Principles

Less is more.

We do not want ae to become a junk drawer. In situations where we are considering bringing in new functionality to the project, we prefer to keep it out.

For example imagine we were considering bringing in a --filter flag or a --query flag which would allow for us to filter the output returned by ae. In this situation we could very well identify advanced query patterns and libraries that would make it possible to adopt this feature.

However the feature comes at a maintenance cost, and frankly already exists in other tools. Instead of adopting a filter syntax and the associated burden that comes along with it we instead focus on outputting to formats (such as JSON) which can easily be queried by existing tools (such as jq).

Just because you can, doesn't mean you should.

Also known as "The Jurassic Park" rule.

In situations where functionality is made available or "is possible" because of current technical advancements or capabilities, we prefer to focus on the goals of ae instead of supporting optional features "just because we can".

For example imagine discovering a well written, and well regarded library that takes a Go struct and turns it into many formats which can be printed to the screen such as XML, JSON, YAML, TOML, BSON, and so on....

The project currently has established a need for JSON only, and plans to use jq for filtering and querying the data. In this case bringing the additional output types to the project "just because we can" would be a violation of this rule.

No assumptions

Also known as "no conveniences" or "no magic" policies.

Assumptions and conveniences can delight a user, until the assumption is wrong in which case they can be catastrophically frustrating. We prefer to stay away from assumptions whenever possible. We prefer to stay away from "magic" and "convenience" style features.

A famous example of this is bringing a logger to a project which assumes that the logger will be able to take ownership of the -v flag for verbose. There are many situations (docker -v for volume, or grep -v for reverse) where this assumption is flawed and will conflict with the goals of the project and panic at runtime. We do not want ae to turn into the logger where we assume something incorrectly that ends up causing problems for others.

This will be a delicate principle to consider, as we also will need to hold firm opinions on things. A balance between a strong opinion and false assumptions is often times hard to get right.

For example the ae project will need to read from a configuration file in order to communicate with a remote server. In one case we could assume that the configuration file is always in a familiar location for convenience. In the other case we do not want to force a user to pass a very long flag every time they run the tool. What do we do?

We hold the opinion that the the file should be in a few well documented locations, and create logic to try the locations if no flag is passed. If a user passes a flag we should remove all assumptions from the program and only consider the input. We prefer clarity over magic.

Remote Servers instead of Local Systems

This is an interesting principle to consider. As the functionality of the ae tool grows we will inevitably need to execute logic somewhere.

We prefer to keep the ae tool as "boring" and "unintelligent" as possible. We do not want the command line tool to do a lot of "work" or "processing" locally. In situations where we need to perform logic against a set of Aurae nodes, we need to find a way to take the logic out of ae for compatibility reasons.

We do not want to get into a situation where we have multiple clients attempting to perform slightly different tasks locally against a server. Especially when these clients are subject to take long periods of time for their work to complete.

In the event these types of situations arise, it is a sign that we likely need to deploy a scheduler or server mechanisms somewhere to manage the work on our behalf.

Success: Be able to demonstrate a connection to auraed

subcommand: logs

Introduce a logging subcommand command built on https://github.com/aurae-runtime/aurae/blob/main/api/v0/observe.proto

Fix failing tests

make test fails currently.

We may also want to bring in a testing library (https://github.com/stretchr/testify).

OutputFormat should use the generic test suite

add health check command

ae should take a network block in CIDR notation and attempt to discover healthy nodes within the network
ae should take a list of IP addresses and attempt to discover which have healthy nodes running on them

the input should be from a configuration file or flag

the output should be either a list of healthy nodes or a list of all nodes and their health. any other information from the health endpoint response should also be included.

/* -------------------------------------------------------------------------- *\
 *             Apache 2.0 License Copyright © 2022 The Aurae Authors          *
 *                                                                            *
 *                +--------------------------------------------+              *
 *                |   █████╗ ██╗   ██╗██████╗  █████╗ ███████╗ |              *
 *                |  ██╔══██╗██║   ██║██╔══██╗██╔══██╗██╔════╝ |              *
 *                |  ███████║██║   ██║██████╔╝███████║█████╗   |              *
 *                |  ██╔══██║██║   ██║██╔══██╗██╔══██║██╔══╝   |              *
 *                |  ██║  ██║╚██████╔╝██║  ██║██║  ██║███████╗ |              *
 *                |  ╚═╝  ╚═╝ ╚═════╝ ╚═╝  ╚═╝╚═╝  ╚═╝╚══════╝ |              *
 *                +--------------------------------------------+              *
 *                                                                            *
 *                         Distributed Systems Runtime                        *
 *                                                                            *
 * -------------------------------------------------------------------------- *
 *                                                                            *
 *   Licensed under the Apache License, Version 2.0 (the "License");          *
 *   you may not use this file except in compliance with the License.         *
 *   You may obtain a copy of the License at                                  *
 *                                                                            *
 *       http://www.apache.org/licenses/LICENSE-2.0                           *
 *                                                                            *
 *   Unless required by applicable law or agreed to in writing, software      *
 *   distributed under the License is distributed on an "AS IS" BASIS,        *
 *   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. *
 *   See the License for the specific language governing permissions and      *
 *   limitations under the License.                                           *
 *                                                                            *
\* -------------------------------------------------------------------------- */

Introduce formatter and output mechanism

We need a clean way to output and query the output of the program.

Ideally we will be able to do something like the following:

ae get pods --json # Spits out something we can pipe to jq
ae get pods --yamo # Spits out something we can pipe to yq
ae get pods --go # Spits out syntactically correct Go structs we can copy/paste
ae get pods --table # Spits out a table in stdout that is auto formatted to the terminal

Update README with project vision

Add a section to the README that gives vision to the project.

subcommand: start