Giter Site home page Giter Site logo

tramlinehq / tramline Goto Github PK

View Code? Open in Web Editor NEW
102.0 4.0 6.0 12.85 MB

Release coordination and deployment platform, just for your mobile apps

Home Page: https://tramline.app

License: Apache License 2.0

Ruby 74.65% JavaScript 2.72% CSS 0.60% HTML 21.56% Shell 0.45% Just 0.02%
android fastlane flutter ios mobile mobile-devops react-native release-automation release-engineering release-management

tramline's Introduction

the tramline logo

Tramline

Release apps without drowning in process

Codify your app's release cycle,
deploy builds with increased confidence,
and give visibility to the entire organization.

Website · Latest Updates · Blog

Twitter Follow Discord

GitHub commit activity

Features ✨

Release dashboards

A centralized dashboard to monitor and control all your mobile releases, that gives visibility into the release process to all stakeholders.

Release trains

Using the release train model, you can setup different types of releases as separate trains comprising of different steps. For e.g. your production release train can look completely different from the release train that does frequent internal deploys.

Integrations

Connect with all the essential tools you use during your release cycle: version control, notifications, CI/CD (build) servers, distribution services, and both App Store and Play Store.

Automations

Save time and reduce human error across the board by automating release-specific chores. For e.g.

  • Create a new release branch for every release
  • Create and merge release-specific branches, as determined by your branching strategy
  • Submit build to the Store only after explicit approval
  • Tag the final release build commit
  • Don't allow starting a new release unless previous release-specific commits have landed in the working branch

Analytics

Track and visualize release-specific metadata that you need to make informed decisions: release frequency, build times, review times, etc.

How to set it up yourself ⚙️

These steps assume setting it up on Render only. However, the instructions are standard enough to be adapted for a Heroku deployment or even bare-metal. A Dockerized setup is in the works and will come shortly.

Note: Since Render does not offer background workers under the free plan, you will have to put in your payment details to fully complete this deployment.

Requirements

At minimum, you'll need the following to get Tramline up and running:

  • This repository set up as the primary monolithic backend
  • This repository set up as a background worker
  • Postgres database
  • Redis, preferably persistent

You'll also need to set up integrations for Tramline to be useful:

Setting up integration apps

The guides above should help you setup the OAuth apps as necessary. They may ask you to fill up a redirect URL, this URL should be updated with the final DNS after everything is setup towards the end.

Google Cloud Platform

We need to setup GCP for storing builds in Tramline. After creating your service account as mentioned above, please create a GCS bucket named artifact-builds-prod to host your builds.

Setting up Tramline

The deployment architecture looks like this:

setup architecture

To begin, first clone this repo. This ensures everything that you do is fully under your control.

In case you'd like to run this locally first, please follow the local development instructions.

To host Tramline directly, you'll need to prep your fork:

Set up Rails

bin/setup.mac

Generate production credentials and follow the instructions

bin/setup.creds -e prod

Keep the production.key file safe and don't commit it!

Update production credentials

After adding the encryption credentials, fill in the following details for the integrations in production.yml.enc by running bin/rails credentials:edit --environment production.

Follow the links mentioned earlier to setup the bare-minimum integrations.

For applelink, choose any string as your secret. We will use this later.

Use the following template:

active_record_encryption:
  primary_key:
  deterministic_key:
  key_derivation_salt:

secret_key_base:

dependencies:
  postmark:
    api_token:

  gcp:
    project_id:
    private_key_id:
    private_key: |
    client_email:
    client_id:
    client_x509_cert_url:

integrations:
  slack:
    app_id:
    client_id:
    client_secret:
    signing_secret:
    verification_token:
    scopes: "app_mentions:read,channels:join,channels:manage,channels:read,chat:write,chat:write.public,files:write,groups:read,groups:write,im:read,im:write,usergroups:read,users.profile:read,users:read,users:read.email,commands,usergroups:write"

  applelink:
    iss: "tramline"
    aud: "applelink"
    secret: "any password"

  github:
    app_name:
    app_id:
    private_pem: |

Save the credentials file and commit your changes. Use this button from your fork to kick-off a Render deployment.

Deploy to Render

The blueprint will ask for the RAILS_MASTER_KEY. Use the contents of production.key from the previous step.

Setup applelink

If you would like to use the App Store integration, you'd have to configure the applelink service. You can skip this section otherwise.

  1. To start off, in your fork, uncomment the applelink section from the render.yaml file.
  2. Secondly, to your encryption file, add this to the integrations section, by running bin/rails credentials:edit --environment production. Choose a secret key for authorization.
integrations:
  applelink:
    iss: "tramline.dev"
    aud: "applelink"
    secret: "any password"
  1. Commit your changes and resync the blueprint on Render. This will kick-off the applelink service. This will most likely fail to deploy on the first attempt, because it requires certain environment configuration. To do that, create a Secret File under applelink > Settings > Environment > Secret Files called .env and update with the following details by putting the applelink secret from the previous step under AUTH_SECRET:
RACK_ENV=production
WEB_CONCURRENCY=2
MAX_THREADS=1
PORT=3001
AUTH_ISSUER="tramline"
AUTH_AUD="applelink"
AUTH_SECRET=""
SENTRY_DSN=""

Wrap up

Before we wrap up, we need to fix a couple of ENV variables:

  1. If you have setup applelink in the previous step, you must add an APPLELINK_URL to site-web with the final DNS of the applelink service.
  2. The HOSTNAME in site-web and site-jobs must be updated to point to the DNS of site-web (without the protocol).

Once all services on Render are green, your setup should look like this:

render.com services

That should be it! You can use the default DNS from site-web to launch Tramline. You can configure and tweak more settings later.

Local development 🛠️

Setup

For local development on macOS, clone this repository and run the included setup script:

bin/setup.mac

Note: If you already have a previous dev environment that you're trying to refresh, the easiest thing to do is to drop your database and run setup again.

rails db:drop
bin/setup.mac

Refer to db/seeds.rb for credentials on how to login using the seed users.

Running

  • Place the master.key file in the config directory.
  • Start ngrok for webhooks.
  • Start PostgreSQL and Redis using Homebrew services.
  • Finally, run bin/dev.

Webhooks

Webhooks need access to the application over the Internet and that requires tunneling on the localhost environment. We use ngrok, and you should run it like this:

ngrok http https://localhost:3000

If you'd like to use the custom DNS tunnel, add the following to your ngrok config file,

version: "2"
authtoken: # put your authtoken
region: in
tunnels:
  tramline_dev:
    proto: http
    hostname: # add the tunnel hostname
    addr: https://localhost:3000

You can run this configured tunnel via

ngrok start tramline_dev

or through the Procfile.dev

Adding or updating gems

  • Use bundle add <gem> to add a new gem.
  • To update a gem use bundle update <gem>.

Using the bundle add tool auto-applies the pessimistic operator in the Gemfile. Although Gemfile.lock is the correct source of gem versions, specifying the pessimistic operator makes for a simpler and safer update path through bundler for future users.

Doing this for development/test groups is optional.

SSL

We use SSL locally and certificates are also generated as part of the setup script. It's recommended to use https://tramline.local.gd:3000.

This is the default HOST_NAME that can be changed via .env.development if necessary.

Letter Opener

All e-mails are caught and can be viewed here.

Sidekiq

The dashboard for all background jobs can be viewed here.

Flipper

All feature-flags are managed through flipper. The UI can be viewed here.

Contributing 👩🏻‍💻

We are still in the early stages and would <3 any feedback you have to offer. You can get in touch with us in several ways:

tramline's People

Contributors

kitallis avatar nid90 avatar pratul avatar tachyons avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

tramline's Issues

Enhance the sidebar

  • All apps with configured release train nested under Apps
  • Under the trains, nest the active run

Introduce proper abstraction for the Step runs within a Step

As per our current data modelling, each train can have multiple train runs (AKA releases) and steps. A step_run refers to the execution of train run for the given step for given commit. Ie there is no proper abstraction train run per step. This makes storing approval status complicated. Current workaround is to use sign_required column, but it is a clear smell of missing abstraction

Clean-up release page

Release page is currently a mess as there are too many conditions and SQL queries are being generated from there. It is due to missing abstractions and lack of clean-up.

Persist notifications

Currently we are just generating and sending notifications without any persistence. Having persistence feature makes sending notifications to multiple channels and showing notifications on the UI easy. noticed is a good gem which take care of most of the heavy lifting regarding the notifications

Handle merge conflicts while creating and merging PRs during post-release

We need to make merging PRs sequential but non-blocking, as creating and merging PRS aren't straight forward when there are merge conflicts which need to be manually handled by developer. If there are multiple PRs we need to wait for the first PR to be merged before creating the second PR. We need to persist the state to handle this and can't be done inside a transaction.

This is currently only an issue in Parallel Branch & Back-merge strategy.

Validations and automations for Almost Trunk branching strategy

Approaches

Choosing commits

  1. User marks a commit message
    ... and we automatically cherry pick that commit onto the release branch of the live release.

  2. User picks commits from a list populated from the working branch
    ... and we cherry pick that commit onto the release branch

Direction of copy

  1. Commit lands on release branch and is cherry picked onto the working branch

  2. Commit lands on working branch and is cherry picked onto the release branch

Questions

  • which one of the two approaches is simpler to implement? better for the user?
  • how will failures be handled?

Notes

Handle stopping release with a warning if a diff exists between the release and working branch. Also, do the same for other branching strategies.

Should be able to edit core settings of an app

Constraints

name

no constraint

description

no constraint

bundle identifier

  • no release train should be running
    • because if it is, then the system will attempt to submit a build to the wrong play store account because the build identifier has changed
    • the submission will not go through and the system will start throwing errors

build number

  • only allowed to increase the build number, not decrease it
  • both play store and app store only allow builds with increasing "version numbers" anyway so this constraint makes sense

Better workflow detection mechanism

We are currently creating an extra job on the workflow solely for storing the version_code and refer that once the workflow is finished.

This approach is having following drawbacks

  1. Additional job
  2. Artifact extraction is expensive
  3. We get to link step_runs with workflow only after workflow run is finished
  4. This approach is platform dependent.

Alternate approach is to have

  1. An end point in tramline to receive workflow event
  2. Create a http call from workflow as the first step with workflow details to this endpoint
  3. This can be accomplished by our own cli or simple curl script

Advantages

  1. Early feedback so that we can link step runs with workflow run(CI action) and show in the UI
  2. No expensive processing in callback handler
  3. Similar approach can be implemented in most of CI platforms

Should we have a health check somewhere in settings?

IMO we need an easy way to debug a setup. For example, without running an actual release I cannot figure out if my webhooks and integrations are properly set up or not.

Aboo thinks that such a feature will be useful to validate values with connected integrations.

Parallel branching strategy: update kickoff PR text

Title

[Release kickoff] vX.Y.?

Description

New release train (train name) triggered by user name.
The working branch branch has been merged into release branch branch, as per branching strategy name branching strategy.

Process webhooks using background jobs

We are currently processing webhooks in synchronous manner and return status to Github only after process is completed. This causes issues due to following reasons.

  • Github is not interested in what we do with the payload or result, it only interested whether the payload received successfully or not
  • Some payload handlers take time, sometimes several seconds, this causes webhook to timeout and retry
  • When we process webhooks in synchronous manner, the number of requests we need to handle at the same time increases due to latency and causes bigger request queue.

Unzip archive before saving to object storage

We are currently storing artifacts in zip as provided by GitHub action, this makes uploading to play store difficult and expensive. Also, since different CI providers might provide artificats in different formats, it is better to store in standardized format

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.