melezhik/sparky

Sparky is a flexible and minimalist continuous integration server and distribute tasks runner written in Raku.

sparky's Issues

View to see overall sparky stats

An overall view of what sparky have done from "all time".

Something like :

To put somewhere either on main page or on a "statistic" or "status" page

Change shebang to raku ?

FTP for File Triggerring is confusing

While I perfectly understand and love the feature, the acronym collision makes me uncomfortable.

Can we find something more around "file/dir watcher" or something else like "file creation" or "file event".

Tag a new release

It's been a while.

I'm considering self-hosting this, but I'd like to check out/build from a tag.

Animated "running" button/badge

Can we make the "running" button blink ?

Add user management

Some informations and actions should be only accessible to an administrator.
Some informations and actions should be only accessible to its owner(s) and administrator(s).
The administrator should be the one and the only one to be able to manage this user management (create user, delete user, attach/detach to project etc...).

Technically, I think it means adding some tables in DB + handle sessions etc.. in web UI.

bin/sparky-web.raku - rewrite from Bailador to Cro

Reason: Bailador seems is not being in actively developing phase, last commit was more then 1 year ago. It also has some annoying glitches making web application crashed on and off. On other hand Cro seems growing, well maintained and at the end of the day looks more convenient and easier to use ...

Could we enable discussion tab ?

This tab :

To be enabled in general settings from repo :

Outdates build status

Principle

An old build should not have the same value than a recent build.

Imagine a package that have a slow releasing life-cycle.
It has an old green status, but it should not be flashy green but dull gray of with a small banner.

Demo

Here is a classic (github action) badge :

An outdated could be :

Or with banner :

On what criterias a build is outdated ?

At least a docker image version and a different runner (different host or same host but upgraded).
These infos could be found in the sparky.yaml and sparrowfile.

Does it makes sense ?

sparkyd refuses to start if a sparky.yaml is missing

Hello,

When a projects does not contains a sparky.yaml or contains an empty one, sparkyd refuses to boot.

Adding even something dumb like

a: "b"

is fixing the boot.

It is maybe "by design", but the documentation does not clearly mention this (to my understanding it seems to say implicitly the contrary).

Thank you 😁

Thibault

Document a deliver/publish

Is there any plugin to deliver to s3, artifactory, ftp, whatever ?

Could we document it ?

Show last builds on project page

Show last builds (at least) of a project on his project page.

Replay with edit ?

When we do a rebuild, it would be nice to edit "on the fly" the configuration/sparrowfile.

Hightlight code/config snippets

Like scenario and config.

It will improve a lot user experience and beauty 😄

HTML produced by sparky is not valid

The HTML produced by spark web is weird. No html tag, script and css before head, head almost empty etc...
Can we fix it ?

Hamburger menu does not expand on safari mobile

Problem loading configs hash

Hello,

raku bin/sparky-runner.raku --dir=~/.sparky/projects/hello/ is not working because the ~ is not translated to absolute path at this line.

You end up with this empty configuration because we never enter the if block :
merged sparrowdo configuration: { }

Worker workload

Somewhat related to #27 and #28 but I would like a view with a graph to monitor the health of workers. We can play with kubernetes also for this (but it won't always be deployed in k8s...).

What is downstream projects ?

Fixed size for status buttons

Very UI related, but I guess having the "running/failed/success" button with the same width would render better.

SCM triggering support

I'd be good to have sparky builds get triggered by changes in SCM:

sparky.yaml:

  scm:
   url: https://github.com/melezhik/rakudist-teddy-bear.git
   branch: main

CI looks broken, project stuck in queue

Project git-pheix-core-perl6.git stuck in queue: https://sparky.sparrowhub.io/queue. Looks like no runners are available or smth like this.

Also I see last successful build about a week ago: https://sparky.sparrowhub.io/builds

Fixed size for status buttons

Very UI related, but I guess having the "running/failed/success" button with the same width would render better.

Sparky CI Web restarting too frequently

Hello,

I feel like Sparky CI Web is restarting too frequently, like if it was watching logs and restarting on change:

Is it normal?

Build along a "reference build"

For instance when rebuilding a failed build, I would like to rebuild the "reference OK" build : means the last one that was green.
We won't be able to do this completely, because inputs are maybe not available (commit or deps) but we could at least try to emulate some parts : the sparrowfile and sparky.yaml and therefore, as it is inside these files, the docker image and/or host target (but not its state of course).

It requires to store this "last OK config" somehow (versioning ? db ?).

It's not 100% clear in my mind and it's a nice to have, but I really think it has great added value for end user.

On end user side, I would like to have an option to start 2 builds at the same time, one behing the reference and the other the current.

Should play well with #32 😀

Can we assign task to a group of workers ?

More on sparrow side, but can we assign a project to a group of workers, let's say "big nodes"... And some workers defined elsewhere as being "big nodes" can handle the build ?

Add notifications to nav bar

I would like to have a visual feedback to running jobs, something like this :

I do not checked too much the sparky internals but it probably needs to technically implement something like websocket or similar ?

Licence ?

What is the licence of sparky ?

Use columns instead table ?

Spin up a test Sparky instance for dog fooding

I'd be good to have one with public IP. http://rakudist.raku.org/sparky/ should be considered booked for Raku community purposes and is not good candidate if users want just to play with web hooks or other test requests ...

I have free tier aws account which expires pretty soon, any ideas where we could have (free?) resources? We need something 2-4GB RAM and 2-4 CPU core setup.

cc @thibaultduponchelle

Same project config for multiple items

The idea is :

• Define one project dir with a sparrowfile and a sparky.yaml
• Use it for multiple items (repos or modules) through a list (implementation to be defined) (*)

(*) An extra feature would be github/bitbucket/gitlab org/project/whatever scanning

Workers status page ?

The same way we have a list of builds, we could have a list of workers with an info about if the machine is up or down.

Badges ?

It would be more than cool to have badges for easy visual reporting in external services.

Cascading between projects

This is another one that comes from my mind.

Could we chain easily some projects "harmoniously".
At the end of a running build, trigger one or more other projects, potentially passing one or more parameters (sha1 or tags).

It is often needed for docker image creation (one repo contains low level image, one contains technical and is built on top of low level one, one contains functional etc...).
It is also useful for version propagation.

It would be very cool to do not have to implement cascading mechanism by ourself through http form calls or auto git commit.

Print sparky and sparky web logs in web UI

Can be put behind admin auth after #17 is implemented

Filter queue per worker

Sparky needs a logo

This is my favorite issue 😎

Diff between build

I would like to be able to compare builds, especially when it goes from OK to KO.

It means a view where we can see these kind of thing, to be implemented as best effort :D

• the config/sparrowfile diffs if any
• the input diffs (commits if coming from scm, md5sum or similar if tarball) if any
• log diffs
• the diffs between docker images version (only the versions, not deeper)
• the diff between runner

Turn off/refine "flapper protection"

Hello,

in my use case, sparkyd runs forever and ever, as a pm2 task. From what I understand in the code, timeouts are never reset, and once they reach 5 occasions, the project is permanently marked as a "flapper" which defeats the purpose of having the CI running 24/7 continuously.

I can imagine that there are timeouts for whatever reason but I generally don't even want that to make any difference, as long as there is any chance of recovering, whether that's 5 minutes later, 2 hours later or a couple of days later. I think there should be either a way to turn this feature off completely, or at least the timeouts should wear off. If every second attempt times out, that's still okay. If 5 attempts time out in a row, even that might be okay, depending on the timing circumstances. In any case, only massive bursts of timeouts should be considered.

Reboot sparky and sparky web from UI

Ability to ask for a soft restart from UI.

This feature depends on #17

Wall of status

A new page to list all status for a bitbucket project, github org, module namespace etc...

I do this kind of thing, it's a bit hacky and dirty but it is working for a user point of view.
Here is my homemade thing :

buildbot has a page that looks like this, but he mixing workers with status (matrix commit/worker).

Can we connect to a running job ?

I know we can, at least with docker (something like docker -H tcp://thenodewhereitisexecuted exec -ti thelayerhash /bin/bash) but do we have the info somewhere (in the logs or better in the running build ("report") page ?

Extra bonus : It could go with a "pause" feature for short living jobs (I'm greedy sorry)

Can we think about "easy install"

I think the install should be dead simple, which is not the case today.
(Ok there is the docker image but even)

I don't know what to do, but can we imagine at least a "fatpacked" version of sparky (sparky + modules).
It would be even better to have a raku + zef + modules + sparky since raku is not always available on distributions and for sure not with a recent version (and the correct interpreter name).

Maybe a curl .../install.sh | bash would do the trick even if some people does not like this 😋

According to me, it is a very important issue to start with for better adoption.

When using scm or delivering, show link in report page

When building from scm, or delivering things during build, put links in report page.