clivern/peanut

🐺 Deploy Databases and Services Easily for Development and Testing Pipelines.

Peanut

Deploy Databases and Services Easily for Development and Testing Pipelines.

Dashboard Screenshots

Peanut provides a REST API, Admin Dashboard and a command line tool to deploy and configure the commonly used services like databases, message brokers, graphing, tracing, caching tools ... etc. It perfectly suited for development, manual testing, automated testing pipelines where mocking is not possible and test drives.

Under the hood, it works with the containerization runtime like docker to deploy and configure the service. Destroy the service if it is a temporary one.

Technically you can achieve the same with a bunch of yaml files or using a configuration management tool or a package manager like helm but peanut is pretty small and fun to use & should speed up your workflow!

Supported Services:

• MySQL.
• MariaDB.
• PostgreSQL.
• Redis.
• Etcd.
• Grafana.
• Elasticsearch.
• MongoDB.
• Graphite.
• Prometheus.
• Zipkin.
• Memcached.
• Mailhog.
• Jaeger.
• RabbitMQ.
• Consul.
• Vault.
• Cassandra.
• Minio.
• Docker Registry.
• Ghost.
• Httpbin.
• Nagios.
• Etherpad.

Documentation

Run Peanut on Ubuntu

To run peanut on ubuntu, You can use the following bash script since it may take a while for a cold start. the script will install etcd, docker, docker-compose and peanut.

$ bash < <(curl -s https://raw.githubusercontent.com/Clivern/Peanut/main/deployment/linux/install.sh)

# Get The Public IP
$ curl https://ipinfo.io/ip
x.x.x.x

Peanut will be running on 80 port and UI on this URL http://x.x.x.x. Please open this file /etc/peanut/config.prod.yml and adjust the following line to be your Public IP or hostname

# App configs
app:
    ...
    # Hostname
    hostname: ${PEANUT_API_HOSTNAME:-127.0.0.1}

Then Restart Peanut

$ systemctl restart peanut

To make sure peanut is running, Run the following from your laptop to run some redis instances for 10 minutes.

# To provision redis services for 10 minutes
$ curl -X POST http://$PUBLIC_IP/api/v1/service -d '{"service":"redis","configs": {},"deleteAfter":"10min"}' -H 'x-api-key: ~api~key~here~'

{
  "createdAt": "2021-07-11T09:58:11.076Z",
  "id": "aadd5741-58c5-43c7-94fd-e6c0171fe8be",
  "service": "a8138f52-3ebb-4a34-b403-1be6ad481daf",
  "status": "PENDING",
  "type": "service.deploy"
}


# To list services including the host and port to use for connection
$ curl -X GET http://$PUBLIC_IP/api/v1/service  -H 'x-api-key: ~api~key~here~'

{
  "services": [
    {
      "id": "9d655cbe-caf1-4104-b8e4-b83fd569b509",
      "service": "redis",
      "configs": {
        "address": "127.0.0.1",
        "password": "",
        "port": "49156"
      },
      "deleteAfter": "10min",
      "createdAt": "2021-07-11T09:58:13Z",
      "updatedAt": "2021-07-11T09:58:13Z"
    },
    {
      "id": "a8138f52-3ebb-4a34-b403-1be6ad481daf",
      "service": "redis",
      "configs": {
        "address": "127.0.0.2",
        "password": "",
        "port": "49155"
      },
      "deleteAfter": "",
      "createdAt": "2021-07-11T09:58:12Z",
      "updatedAt": "2021-07-11T09:58:12Z"
    }
  ]
}

There is also a script to upgrade peanut.

$ bash < <(curl -s https://raw.githubusercontent.com/Clivern/Peanut/main/deployment/linux/upgrade.sh)

Linux Deployment Explained

Download the latest peanut binary. Make it executable from everywhere.

$ export PEANUT_LATEST_VERSION=$(curl --silent "https://api.github.com/repos/Clivern/Peanut/releases/latest" | jq '.tag_name' | sed -E 's/.*"([^"]+)".*/\1/' | tr -d v)

$ curl -sL https://github.com/Clivern/Peanut/releases/download/v{$PEANUT_LATEST_VERSION}/peanut_{$PEANUT_LATEST_VERSION}_Linux_x86_64.tar.gz | tar xz

Then install etcd cluster or a single node! please refer to etcd docs or bin directory inside this repository.

Install the virtualization runtime! By default peanut uses docker & docker-compose

$ apt-get update
$ apt-get install docker.io -y
$ systemctl enable docker

$ apt-get install docker-compose -y

Create the configs file config.yml from config.dist.yml. Something like the following:

# App configs
app:
    # Env mode (dev or prod)
    mode: ${PEANUT_APP_MODE:-dev}
    # HTTP port
    port: ${PEANUT_API_PORT:-8000}
    # Hostname
    hostname: ${PEANUT_API_HOSTNAME:-127.0.0.1}
    # TLS configs
    tls:
        status: ${PEANUT_API_TLS_STATUS:-off}
        pemPath: ${PEANUT_API_TLS_PEMPATH:-cert/server.pem}
        keyPath: ${PEANUT_API_TLS_KEYPATH:-cert/server.key}

    # Containerization runtime (supported docker)
    containerization:
        driver: ${PEANUT_CONTAINERIZATION_DRIVER:-docker}

        # Clean up stale images, volumes and networks
        autoClean: ${PEANUT_CONTAINERIZATION_AUTO_CLEAN:-true}

        # Time to cache docker images tags
        cacheTagsTimeInMinutes: ${PEANUT_CONTAINERIZATION_CACHE_TIME:-10080}

    # App Storage
    storage:
        # Type (only local supported)
        type: ${PEANUT_STORAGE_TYPE:-local}
        # Local Path
        path: ${PEANUT_STORAGE_PATH:-/tmp}

    # API Configs
    api:
        key: ${PEANUT_API_KEY:-6c68b836-6f8e-465e-b59f-89c1db53afca}

    # Async Workers
    workers:
        # Queue max capacity
        buffer: ${PEANUT_WORKERS_CHAN_CAPACITY:-5000}
        # Number of concurrent workers
        count: ${PEANUT_WORKERS_COUNT:-4}

    # Runtime, Requests/Response and Peanut Metrics
    metrics:
        prometheus:
            # Route for the metrics endpoint
            endpoint: ${PEANUT_METRICS_PROM_ENDPOINT:-/metrics}

    # Application Database
    database:
        # Database driver
        driver: ${PEANUT_DB_DRIVER:-etcd}

        # Etcd Configs
        etcd:
            # Etcd database name or prefix
            databaseName: ${PEANUT_DB_ETCD_DB:-peanut}
            # Etcd username
            username: ${PEANUT_DB_ETCD_USERNAME:- }
            # Etcd password
            password: ${PEANUT_DB_ETCD_PASSWORD:- }
            # Etcd endpoints
            endpoints: ${PEANUT_DB_ETCD_ENDPOINTS:-http://127.0.0.1:2379}
            # Timeout in seconds
            timeout: 30

    # Log configs
    log:
        # Log level, it can be debug, info, warn, error, panic, fatal
        level: ${PEANUT_LOG_LEVEL:-info}
        # Output can be stdout or abs path to log file /var/logs/peanut.log
        output: ${PEANUT_LOG_OUTPUT:-stdout}
        # Format can be json
        format: ${PEANUT_LOG_FORMAT:-json}

The run the peanut with systemd

$ peanut api -c /path/to/config.yml

Deploy your first redis server!

$ curl -X POST http://127.0.0.1:8000/api/v1/service -d '{"service":"redis"}' -H 'x-api-key: ~api~key~here~'

To run the Admin Dashboard (Development Only):

Clone the project or your own fork:

$ git clone https://github.com/Clivern/Peanut.git

Create the dashboard config file web/.env from web/.env.dist. Something like the following:

VUE_APP_API_URL=http://localhost:8080

Then you can either build or run the dashboard

# Install npm packages
$ cd web
$ npm install
$ npm install -g npx

# Add api server url to frontend
$ echo "VUE_APP_API_URL=http://127.0.0.1:8000" > .env

$ cd ..

# Validate js code format
$ make check_ui_format

# Format UI
$ make format_ui

# Run Vuejs app
$ make serve_ui

# Build Vuejs app
$ make build_ui

# Build the Vuejs app
$ make package

The command line tool

In order to interact with peanut API server, you can either do basic API calls or use the provided command line tool. It is still not finished yet but it will be ready soon.

Supported Services

Here is a list of all supported services so far and the API call to deploy them.

• MySQL.

$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"mysql","configs": {"rootPassword": "root", "database": "peanut", "username": "peanut", "password": "secret"}}' \
    -H 'x-api-key: ~api~key~here~'

• MariaDB.

$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"mariadb","configs": {"rootPassword": "root", "database": "peanut", "username": "peanut", "password": "secret"}}' \
    -H 'x-api-key: ~api~key~here~'

• PostgreSQL.

$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"postgresql","configs": {"database": "peanut", "username": "peanut", "password": "secret"}}' \
    -H 'x-api-key: ~api~key~here~'

• Redis.

$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"redis","configs": {"password": "secret"}}' \
    -H 'x-api-key: ~api~key~here~'

• Etcd.

$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"etcd"}' \
    -H 'x-api-key: ~api~key~here~'

• Grafana.

$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"grafana","configs": {"username": "admin", "password": "admin", "allowSignup": "false", "anonymousAccess": "true"}}' \
    -H 'x-api-key: ~api~key~here~'

• Elasticsearch.

$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"elasticsearch"}' \
    -H 'x-api-key: ~api~key~here~'

• MongoDB.

$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"mongodb","configs": {"database": "peanut", "username": "peanut", "password": "secret"}}' \
    -H 'x-api-key: ~api~key~here~'

• Graphite.

$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"graphite"}' \
    -H 'x-api-key: ~api~key~here~'

• Prometheus.

$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"prometheus"}' \
    -H 'x-api-key: ~api~key~here~'

# Configs can be provided as base64 encoded string (use https://www.base64encode.org/)
$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"prometheus", "configs": {"configsBase64Encoded": "Z2xvYmFsOgogIGV2YWx1YXRpb25faW50ZXJ2YWw6IDE1cwogIHNjcmFwZV9pbnRlcnZhbDogMTVzCnJ1bGVfZmlsZXM6IH4Kc2NyYXBlX2NvbmZpZ3M6CiAgLQogICAgam9iX25hbWU6IHByb21ldGhlCiAgICBzY3JhcGVfaW50ZXJ2YWw6IDVzCiAgICBzdGF0aWNfY29uZmlnczoKICAgICAgLQogICAgICAgIHRhcmdldHM6CiAgICAgICAgICAtICJsb2NhbGhvc3Q6OTA5MCI="}}' \
    -H 'x-api-key: ~api~key~here~'

• Zipkin.

$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"zipkin"}' \
    -H 'x-api-key: ~api~key~here~'

• Memcached.

$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"memcached"}' \
    -H 'x-api-key: ~api~key~here~'

• Mailhog.

$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"mailhog"}' \
    -H 'x-api-key: ~api~key~here~'

• Jaeger.

$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"jaeger"}' \
    -H 'x-api-key: ~api~key~here~'

• RabbitMQ.

$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"rabbitmq"}' \
    -H 'x-api-key: ~api~key~here~'

• Consul.

$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"consul"}' \
    -H 'x-api-key: ~api~key~here~'

• Vault.

$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"vault","configs": {"token": "peanut"}}' \
    -H 'x-api-key: ~api~key~here~'

• Cassandra.

$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"cassandra"}' \
    -H 'x-api-key: ~api~key~here~'

• Minio.

$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"minio","configs": {"username": "admin12345678", "password": "admin12345678"}}' \
    -H 'x-api-key: ~api~key~here~'

• Docker Registry.

$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"registry"}' \
    -H 'x-api-key: ~api~key~here~'

• Ghost.

$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"ghost"}' \
    -H 'x-api-key: ~api~key~here~'

• Httpbin.

$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"httpbin"}' \
    -H 'x-api-key: ~api~key~here~'

• Nagios.

$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"nagios"}' \
    -H 'x-api-key: ~api~key~here~'

• Etherpad.

$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"etherpad"}' \
    -H 'x-api-key: ~api~key~here~'

To create a temporary service, you will need to add extra parameter while creating it.

# It will be deleted after 30 seconds
$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"etcd", "deleteAfter": "30sec"}' \
    -H 'x-api-key: ~api~key~here~'

# It will be deleted after 20 minutes
$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"etcd", "deleteAfter": "20min"}' \
    -H 'x-api-key: ~api~key~here~'

# It will be deleted after 1 hour
$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"etcd", "deleteAfter": "1hours"}' \
    -H 'x-api-key: ~api~key~here~'

# It will be deleted after 3 days
$ curl -X POST http://127.0.0.1:8000/api/v1/service \
    -d '{"service":"etcd", "deleteAfter": "3days"}' \
    -H 'x-api-key: ~api~key~here~'

To list all services running on a single or multi nodes.

$ curl -X GET http://127.0.0.1:8000/api/v1/service \
    -H 'x-api-key: ~api~key~here~'

To delete a service.

$ curl -X DELETE http://127.0.0.1:8000/api/v1/service/:serviceId \
    -H 'x-api-key: ~api~key~here~'

To get async job status like a deployment status.

$ curl -X DELETE http://127.0.0.1:8000/api/v1/job/:serviceId/:jobId \
    -H 'x-api-key: ~api~key~here~'

To get service versions.

$ curl -X GET http://127.0.0.1:8000/api/v1/tag/$serviceType/$fromCacheStatus \
    -H 'x-api-key: ~api~key~here~'

$ curl -X GET http://127.0.0.1:8000/api/v1/tag/mysql/true \
    -H 'x-api-key: ~api~key~here~'

Versioning

For transparency into our release cycle and in striving to maintain backward compatibility, Peanut is maintained under the Semantic Versioning guidelines and release process is predictable and business-friendly.

See the Releases section of our GitHub project for changelogs for each release version of Peanut. It contains summaries of the most noteworthy changes made in each release.

Bug tracker

If you have any suggestions, bug reports, or annoyances please report them to our issue tracker at https://github.com/clivern/peanut/issues

Security Issues

If you discover a security vulnerability within Peanut, please send an email to [email protected]

Contributing

We are an open source, community-driven project so please feel free to join us. see the contributing guidelines for more details.

License

© 2021, Clivern. Released under MIT License.

Peanut is authored and maintained by @clivern.

peanut's Issues

Peanut Command Line Tool

Services to Support

• Cassandra.
• Kafka.
• Minio.
• Clickhouse.
• Docker Registry.
• Infra Workspace.
• VS Code.
• Ghost.
• Httpbin.
• Etherpad.

Service version

Is your feature request related to a problem? Please describe.
Be able to choose the service version in the request payload:

{
   "service":"mysql",
   "version":"8.0.25",
   "configs":{
      "rootPassword":"root",
      "database":"peanut",
      "username":"peanut",
      "password":"secret"
   }
}

Kubernetes driver

Is having a Kubernetes driver something on the roadmap for this project? I'm exploring the possibilities here, not really requesting a feature 🙂

I noticed the code contains something like:

if viper.GetString("app.containerization.driver") == "docker" {
    result.containerization = runtime.NewDockerCompose()
} else {
    panic("Invalid containerization runtime!")
}

which makes me think it's already prepared to be extensible for other drivers than docker+etcd.

Thinking of a possible implementation, there are many ways to do it in Kubernetes. I'd think the most "kubernetes way" would be to add a driver that replaces Docker calls with Kubernetes calls and use Kubernete's CRDs as a storage means instead of etcd.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• Update bitnami/etcd Docker tag to v3.5.13
• Update consul Docker tag to v1.15.4
• Update memcached Docker tag to v1.6.26
• Update module go.etcd.io/etcd/client/v3 to v3.5.13
• Update vault Docker tag to v1.13.3
• Update elasticsearch Docker tag to v8.13.0
• Update module github.com/prometheus/client_golang to v1.19.0
• Update module github.com/spf13/viper to v1.18.2
• Update dependency eslint to v9
• Update dependency vue to v3
• Update dependency vue-router to v4
• Update dependency vuex to v4
• Update goreleaser/goreleaser-action action to v5
• Update grafana/grafana Docker tag to v10
• Update mariadb Docker tag to v11
• Update module gopkg.in/yaml.v2 to v3
• Update mongo Docker tag to v7

Detected dependencies

docker-compose

definition/docker-compose.yml

• elasticsearch 8.12.2
• bitnami/etcd 3.5.11
• grafana/grafana 9.5.18
• graphiteapp/graphite-statsd 1.1.7-11
• jaegertracing/all-in-one 1.57
• mailhog/mailhog v1.0.1
• mariadb 10.11.7
• memcached 1.6.25
• mongo 5.0.0-rc7
• mysql 8.4
• postgres 16.2
• prom/prometheus v2.51.2
• rabbitmq 3.13-management
• bitnami/redis 7.2.4
• openzipkin/zipkin 3.3
• vault 1.13.2
• consul 1.15.2
• cassandra 5.0
• registry 2

github-actions

.github/workflows/api.yml

• actions/checkout v4@0ad4b8fadaa221de15dcec353f45205ec38ea70b
• actions/setup-go v5

.github/workflows/release.yml

• actions/checkout v4@0ad4b8fadaa221de15dcec353f45205ec38ea70b
• actions/setup-go v5
• goreleaser/goreleaser-action v4

.github/workflows/release_package.yml

• actions/checkout v4@0ad4b8fadaa221de15dcec353f45205ec38ea70b
• actions/setup-go v5

.github/workflows/ui.yml

• actions/checkout v4@0ad4b8fadaa221de15dcec353f45205ec38ea70b
• actions/cache v4
• actions/setup-node v4

gomod

go.mod

• go 1.20
• github.com/drone/envsubst v1.0.3
• github.com/franela/goblin v0.0.0-20211003143422-0a4f594942bf@0a4f594942bf
• github.com/gin-gonic/gin v1.9.1
• github.com/manifoldco/promptui v0.9.0
• github.com/prometheus/client_golang v1.18.0
• github.com/satori/go.uuid v1.2.0
• github.com/sirupsen/logrus v1.9.3
• github.com/spf13/cobra v1.8.0
• github.com/spf13/viper v1.16.0
• github.com/stretchr/testify v1.9.0
• go.etcd.io/etcd/client/v3 v3.5.9
• gopkg.in/yaml.v2 v2.4.0

npm

web/package.json

• axios 1.6.8
• buefy 0.9.29
• core-js 3.37.0
• vue 2.7.16
• vue-router 3.6.5
• vuex 3.6.2
• @babel/core 7.24.5
• @babel/eslint-parser 7.24.5
• @vue/cli-plugin-babel 5.0.8
• @vue/cli-plugin-eslint 5.0.8
• @vue/cli-service 5.0.8
• eslint 8.57.0
• eslint-plugin-vue 9.25.0
• vue-template-compiler 2.7.16

---

• Check this box to trigger a request for Renovate to run again on this repository

Peanut Node should delete local services only

Update services ports on peanut startup

Peanut Web UI