Giter Site home page Giter Site logo

st4nson / gitlab-ci-pipelines-exporter Goto Github PK

View Code? Open in Web Editor NEW

This project forked from mvisonneau/gitlab-ci-pipelines-exporter

0.0 1.0 0.0 671 KB

Prometheus / OpenMetrics exporter for GitLab CI pipelines insights

License: Apache License 2.0

Makefile 18.76% Smarty 6.11% Dockerfile 2.93% Go 72.20%

gitlab-ci-pipelines-exporter's Introduction

๐ŸฆŠ gitlab-ci-pipelines-exporter

GoDoc Go Report Card Docker Pulls Build Status Coverage Status

gitlab-ci-pipelines-exporter is a small binary (~10MB) that allows you to monitor your GitLab CI pipelines with Prometheus or any monitoring solution supporting the OpenMetrics format.

TL:DR

Here is a Grafana dashboard I've been able to craft, using those metrics:

grafana_dashboard

Getting started

# Write your config file somewhere on disk
~$ cat <<EOF > $(pwd)/config.yml
# URL and Token with sufficient permissions to access your GitLab's projects
# pipelines informations
gitlab:
  url: https://gitlab.example.com
  token: xrN14n9-ywvAFxxxxxx

# Custom waiting time between polls for projects, their refs and pipelines (in seconds, optional)
projects_polling_interval_seconds: 1800 # only used for wildcards
refs_polling_interval_seconds: 300
pipelines_polling_interval_seconds: 60

# Default regexp for parsing the refs (branches and tags) to monitor (optional, default to master)
# default_refs: "^master$"

# The list of the projects you want to monitor
projects:
  - name: foo/project
  - name: bar/project
    refs: "^master|dev$"

# Dynamically fetch projects to monitor using a wildcard
wildcards:
  - owner:
      name: foo
      kind: group
    refs: "^master|1.0$"
    search: 'bar' # optional
EOF

# If you have docker installed, it is as easy as :
~$ docker run -d \
   --name gitlab-ci-pipelines-exporter \
   -v $(pwd)/config.yml:/etc/config.yml \
   -p 8080:8080 \
   mvisonneau/gitlab-ci-pipelines-exporter:latest \
   -config /etc/config.yml

You should then be able to see the following logs

~$ docker logs -f gitlab-ci-pipelines-exporter
INFO[2019-07-30T18:12:24+01:00] Starting exporter
INFO[2019-07-30T18:12:24+01:00] Configured GitLab endpoint : https://gitlab.example.com
INFO[2019-07-30T18:12:24+01:00] Polling projects every 15s
INFO[2019-07-30T18:12:24+01:00] Polling refs every 10s
INFO[2019-07-30T18:12:24+01:00] Polling pipelines every 60s
INFO[2019-07-30T18:12:24+01:00] 2 project(s) configured
INFO[2019-07-30T18:12:24+01:00] Listing all projects using search pattern : 'bar' with owner 'foo' (group)
INFO[2019-07-30T18:12:24+01:00] Found project : foo/bar
INFO[2019-07-30T18:12:24+01:00] Polling refs for project : foo/project
INFO[2019-07-30T18:12:24+01:00] Polling refs for project : bar/project
INFO[2019-07-30T18:12:24+01:00] Polling refs for project : foo/bar
INFO[2019-07-30T18:12:24+01:00] Found ref 'master' for project 'foo/project'
INFO[2019-07-30T18:12:24+01:00] Found ref 'master' for project 'bar/project'
INFO[2019-07-30T18:12:24+01:00] Found ref 'dev' for project 'bar/project'
INFO[2019-07-30T18:12:24+01:00] Found ref 'master' for project 'foo/bar'
INFO[2019-07-30T18:12:24+01:00] Found ref '1.0' for project 'foo/bar'
INFO[2019-07-30T18:12:24+01:00] Polling foo/project:master (1)
INFO[2019-07-30T18:12:24+01:00] Polling bar/project:master (2)
INFO[2019-07-30T18:12:24+01:00] Polling bar/project:dev (2)
INFO[2019-07-30T18:12:24+01:00] Polling foo/bar:master (1)
INFO[2019-07-30T18:12:24+01:00] Polling foo/bar:1.0 (1)

And this is an example of the metrics you should expect to retrieve

~$ curl -s localhost:8080/metrics | grep gitlab_ci_pipeline
# HELP gitlab_ci_pipeline_last_run_duration_seconds Duration of last pipeline run
# TYPE gitlab_ci_pipeline_last_run_duration_seconds gauge
gitlab_ci_pipeline_last_run_duration_seconds{project="bar/project",ref="master"} 676
gitlab_ci_pipeline_last_run_duration_seconds{project="foo/project",ref="master"} 33
gitlab_ci_pipeline_last_run_duration_seconds{project="bar/project",ref="dev"} 701
gitlab_ci_pipeline_last_run_duration_seconds{project="foo/bar",ref="master"} 570
gitlab_ci_pipeline_last_run_duration_seconds{project="foo/bar",ref="1.0"} 571
# HELP gitlab_ci_pipeline_run_count GitLab CI pipeline run count
# TYPE gitlab_ci_pipeline_run_count counter
gitlab_ci_pipeline_run_count{project="bar/project",ref="master"} 0
gitlab_ci_pipeline_run_count{project="foo/project",ref="master"} 0
gitlab_ci_pipeline_run_count{project="bar/project",ref="dev"} 0
gitlab_ci_pipeline_run_count{project="foo/bar",ref="master"} 0
gitlab_ci_pipeline_run_count{project="foo/bar",ref="1.0"} 0
# HELP gitlab_ci_pipeline_status GitLab CI pipeline current status
# TYPE gitlab_ci_pipeline_status gauge
gitlab_ci_pipeline_status{project="bar/project",ref="master",status="failed"} 0
gitlab_ci_pipeline_status{project="bar/project",ref="master",status="running"} 0
gitlab_ci_pipeline_status{project="bar/project",ref="master",status="success"} 1
gitlab_ci_pipeline_status{project="foo/project",ref="master",status="failed"} 0
gitlab_ci_pipeline_status{project="bar/project",ref="dev",status="running"} 0
gitlab_ci_pipeline_status{project="bar/project",ref="dev",status="success"} 1
gitlab_ci_pipeline_status{project="foo/bar",ref="master",status="running"} 0
gitlab_ci_pipeline_status{project="foo/bar",ref="master",status="success"} 1
gitlab_ci_pipeline_status{project="foo/bar",ref="1.0",status="running"} 0
gitlab_ci_pipeline_status{project="foo/bar",ref="1.0",status="success"} 1
# HELP gitlab_ci_pipeline_time_since_last_run_seconds Elapsed time since most recent GitLab CI pipeline run.
# TYPE gitlab_ci_pipeline_time_since_last_run_seconds gauge
gitlab_ci_pipeline_time_since_last_run_seconds{project="bar/project",ref="master"} 87627
gitlab_ci_pipeline_time_since_last_run_seconds{project="foo/project",ref="master"} 29531
gitlab_ci_pipeline_time_since_last_run_seconds{project="bar/project",ref="dev"} 2950
gitlab_ci_pipeline_time_since_last_run_seconds{project="foo/bar",ref="master"} 2951
gitlab_ci_pipeline_time_since_last_run_seconds{project="foo/bar",ref="1.0"} 2900

Usage

~$ gitlab-ci-pipelines-exporter --help
NAME:
   gitlab-ci-pipelines-exporter - Export metrics about GitLab CI pipeliens statuses

USAGE:
   cli [global options] command [command options] [arguments...]

COMMANDS:
     help, h  Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --log-level level              log level (debug,info,warn,fatal,panic) (default: "info") [$GCPE_LOG_LEVEL]
   --log-format format            log format (json,text) (default: "text") [$GCPE_LOG_FORMAT]
   --listen-address address:port  listen-address address:port (default: ":8080") [$GCPE_LISTEN_ADDRESS]
   --config file                  config file (default: "~/.gitlab-ci-pipelines-exporter.yml") [$GCPE_CONFIG]
   --help, -h                     show help
   --version, -v                  print the version

HELM

If you want to make it run on kubernetes, there is a helm chart for that!

~$ git clone [email protected]:mvisonneau/gitlab-ci-pipelines-exporter.git
~$ cd gitlab-ci-pipelines-exporter/charts
~$ cat <<EOF > values.yml
config:
  gitlab:
    url: https://gitlab.example.com
    token: xrN14n9-ywvAFxxxxxx
  projects:
    - name: foo/project
    - name: bar/project
  wildcards:
    - owner:
        name: foo
        kind: group
EOF
~$ helm package gitlab-ci-pipelines-exporter
~$ helm upgrade -i gitlab-ci-pipelines-exporter ./gitlab-ci-pipelines-exporter-0.0.0.tgz -f values.yml

Develop / Test

If you use docker, you can easily get started using :

~$ make dev-env
# You should then be able to use go commands to work onto the project, eg:
~docker$ make fmt
~docker$ gitlab-ci-pipelines-exporter

Contribute

Contributions are more than welcome! Feel free to submit a PR.

gitlab-ci-pipelines-exporter's People

Contributors

mvisonneau avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.