typesafegithub/github-workflows-kt

Authoring GitHub Actions workflows in Kotlin. You won't go back to YAML!

github-workflows-kt's Issues

Add end-to-end tests

Input: DSL, output: YAML as string.

[Action wrapper request] gradle/wrapper-validation-action

What action do you need?
https://github.com/gradle/wrapper-validation-action

Do you have a wrapper working for you locally?
No.

Run checks in cross-repo PRs

See e. g. #28, the checks weren't ran.

Fail at least in runtime for certain cases

• no triggers defined
• no jobs
• more than one job defined with given name
• "You cannot use branches and branches-ignore to filter the same event in a single workflow." + "You cannot use both the tags and tags-ignore filters for the same event in a workflow.": https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#example-including-and-excluding-branches

[Bug] Consistency check job uses backward slashes when YAML generated on Windows

seems like when you generate the yaml on windows (kotlin installed via scoop)
the cosistency check uses backslashes and will fail in CI

Broken output for multi-line command

Add info to comment in output file that this lib was used

[Action wrapper request] gradle-update/update-gradle-wrapper-action

What action do you need?
https://github.com/gradle-update/update-gradle-wrapper-action

Do you have a wrapper working for you locally?
No.

Add docs

• how to use
• how to contribute, especially add new actions
• deploy automatically on tags publishing? together with the lib

[Bug] Version for 'peterjgrainger/action-create-branch' doesn't exist

This commit 6de3c97 added two action wrappers:

• peterjgrainger/action-create-branch@v2
• repo-sync/pull-request@v2

It turns out that such versions don't exist, I'm getting this from GitHub:

Error: Unable to resolve action `peterjgrainger/action-create-branch@v2`, unable to find version `v2`

FYI @jmfayard

Design more convenient API

https://kotlinlang.slack.com/archives/C02UUATR7RC/p1643817358166909?thread_ts=1643817358.166909&cid=C02UUATR7RC

[Core feature request] Custom runner type from string

What feature do you need?
Runners are already supported by the library: https://github.com/krzema12/github-actions-kotlin-dsl/blob/6e7119ed9b55a9fd096b32046fcfa26fd93ad103/library/src/main/kotlin/it/krzeminski/githubactions/domain/RunnerType.kt#L7-L25

but it's not possible to define a custom one in runtime. Use cases:

• the library lags with adding a new type
• runner is passed from e.g. strategy matrix as a string, then it's not possible to use it

Proposed implementation
Instead of enum, make it a sealed class, with additional

class Custom(val value) : RunnerType

[Core feature request] `env` on workflow, job and step level

What feature do you need?
Please add a link to GitHub docs (https://docs.github.com/en/actions/using-workflows/) which describes the desired feature.
https://docs.github.com/en/actions/learn-github-actions/environment-variables

Do you have an example usage?
Best if a YAML snippet is pasted here, or if your project is open-source, an URL to your workflow.

some tasks need to call scripts / binaries that expect some env variables

- name: Send Webhook Notification
  if: always()
  env:
    JOB_STATUS: ${{ job.status }}
    WEBHOOK_URL: ${{ secrets.WEBHOOK_URL }}
    HOOK_OS_NAME: ${{ runner.os }}
    WORKFLOW_NAME: ${{ github.workflow }}
  run: |
    git clone https://github.com/DiscordHooks/github-actions-discord-webhook.git webhook
    bash webhook/send.sh $JOB_STATUS $WEBHOOK_URL
  shell: bash

Add comment to generated YAML that it's generated

[Core feature request] Allow accessing step outputs

What feature do you need?
Please add a link to GitHub docs (https://docs.github.com/en/actions/using-workflows/) which describes the desired feature.
https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsid

Do you have an example usage?
Best if a YAML snippet is pasted here, or if your project is open-source, an URL to your workflow.

- name: Build and push
  id: docker_build
  uses: docker/build-push-action@v2
  with:
    context: ./
    file: ./Dockerfile
    push: true
    tags: ${{ secrets.DOCKER_HUB_USERNAME }}/discordbot:latest
- name: Image digest
  run: echo ${{ steps.docker_build.outputs.digest }}

even better would be if we could use it somewhat like so:

fun variable(variable: String): String = "\${{ $variable }}"

val dockerBuild = uses(
    name = "Build and push",
    id = "docker_build_push",
    action = DockerBuildPush(
        context = ".",
        file = "./Dockerfile",
        push = true,
        tags = "${variable("secrets.DOCKER_HUB_USERNAME")}/discordbot:latest",
    )
)
run(
    name = "image digest",
    command ="echo ${dockerBuild.outputsVariable("digest")}",
//      command = "echo ${variable("steps.${dockerBuild.id}.outputs.digest")}",
)

but just having access to the id field would let me build the command echo ${{ steps.docker_build.outputs.digest }} myself too

Implement handling environment variables

Consistency check job overwrites any other dependencies

There's a bug here:
https://github.com/krzema12/github-actions-kotlin-dsl/blob/8f12c7f2d4258eb85863305094aaef75901f2ee3/library/src/main/kotlin/it/krzeminski/githubactions/yaml/ToYaml.kt#L19

We should merge existing dependent job with consistencyCheckJob.

[Action wrapper request] actions/setup-java

What action do you need?
https://github.com/actions/setup-java

Do you have a wrapper working for you locally?
No.

Implement using GitHub variables

Consider supporting more than one major version of actions

A user should be able to depend on any chosen major version of a given action, to not force him to migrate to e. g. the newest one.

Generate action wrappers code

There should be a piece of logic that, given basic info about the action, generates Kotlin code.

Make condition the last item in step

Describe workflow from kotlin-python

Allow setting names to steps

[Core feature request] on.pull_request

What feature do you need?
https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#onpull_requestpull_request_targetbranchesbranches-ignore

Do you have an example usage?
https://github.com/krzema12/github-actions-kotlin-dsl/blob/6adac8dd70728090d210f4bb15847c94895657e5/.github/workflows/build.yaml#L5

[Core feature request] Inputs for workflow dispatch

What feature do you need?
https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#onworkflow_dispatchinputs

Do you have an example usage?
See the docs.

Check consistency of generated YAML against source Kotlin script

Imagine we have two files: workflow.main.kts which uses this library, and workflow.yaml. We'd like to ensure both are in sync.

Proposed approach: in the generated YAML add an extra step that would self-check if the generated YAML was generated from the main.kts file. It can be done by calculating a checksum from the main.kts file, using it in the extra initial step and comparing it with current checksum (compare "expected" with "actual").

Thanks to this, if someone edits just one of these files, the workflow will fail fast.

Adjust Action API to enforce specifying version

For some of the actions, e. g.
https://github.com/krzema12/github-actions-kotlin-dsl/blob/b152a2b91c7578103302018f8070ad6a77cf506c/library/src/main/kotlin/it/krzeminski/githubactions/actions/actions/SetupJavaV2.kt#L8

I forgot to write the version. A separate field should help.

Add links to projects using the library

First project: probably kotlin-python

Add missing parameters to all wrappers

When adding the wrappers, I often implemented only these parameters that I need. It was a mistake, I was too lazy. Let's make the wrappers versatile so that others can use them for their specific cases.

Add a "custom action" builder

If there's no wrapping API yet for a given action, it should not block the user from giving its name and version, along with the arguments, until it gets proper wrapper in the library.

Self-use this library

[Core feature request] on.schedule.cron

What feature do you need?
https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#onschedule

Do you have an example usage?
https://github.com/krzema12/github-actions-kotlin-dsl/blob/6adac8dd70728090d210f4bb15847c94895657e5/.github/workflows/update-gradle-wrapper.yml#L5

actions/upload-artifact@v2: more than one path is incorrectly serialized to YAML

Actual:

path: python/box.tests/reports/git-history-plot.svg
python/experiments/generated

Expected:

path: |
  python/box.tests/reports/git-history-plot.svg
  python/experiments/generated

This can be a more generic problem when it comes to actions having parameters that are rendered in separate lines.

Don't duplicate action defaults

See e.g. here:

https://github.com/krzema12/github-actions-kotlin-dsl/blob/50f28297b605964bb87ca2926ac66957d9dee749/library/src/main/kotlin/it/krzeminski/githubactions/actions/Checkout.kt#L13

If the user doesn't specify fetchDepth, we now set 1 here in the library, but it's actually the default value of this action. In case someone doesn't specify fetchDepth, we shouldn't put related key in the resulting YAML, thus allowing the action's default to kick in.

[Action wrapper request] madhead/*

What action do you need?
I am an author of a few published actions:

• madhead/semver-utils
• madhead/read-java-properties
• madhead/check-gradle-version

Do you have a wrapper working for you locally?
Yes, and I'll make a PR.

[Bug] Incorrect YAML when value starts with `*`, `[` or `!`

Action

            uses(
                name = "Publish JUnit report",
                action = object : Action("mikepenz", "action-junit-report", "v2") {
                    override fun toYamlArguments() = linkedMapOf(
                        "report_paths" to "**/build/reports/tests/**",
                    )
                },
                condition = "always()",
            )

Expected

Working action.

Actual

I got an error from GitHub Actions:

Invalid workflow file : .github/workflows/build.yaml#L40
You have an error in your yaml syntax on line 40

It resulted in such YAML:

- name: Publish JUnit report
  uses: mikepenz/action-junit-report@v2
  with:
    report_paths: **/build/reports/tests/**

Workaround

Add single quotes in the value:

"report_paths" to "'**/build/reports/tests/**'",

Library version

v0.6.0

Have unit tests for each action

[Action wrapper request] gradle/gradle-build-action

What action do you need?
https://github.com/gradle/gradle-build-action

Do you have a wrapper working for you locally?
No.

Fix nesting DSL parts (`@DslMarker`)

https://kotlinlang.org/docs/type-safe-builders.html#scope-control-dslmarker

Backfill outputs for existing actions

See #56 - once retrieving output is possible, outputs for existing action wrappers have to be mapped in a type-safe way to be able to consume them.

Compile and run examples from the docs

It's to ensure it works, especially when there are frequent breaking changes.
Just extract the Kotlin part from the README, put it into a main.kts file, run it and compare actual output with expected one. Make it a separate workflow.

Validate action wrapper classes code in build time

• enforce adding major version to the action class name
• ensure that action name contains major version at the end

Configure publishing to Maven Central

Use actions in a type-safe way

actions/upload-artifact@v2
actions/download-artifact@v2

[Core feature request] condition on job

What feature do you need?
Please add a link to GitHub docs (https://docs.github.com/en/actions/using-workflows/) which describes the desired feature.
https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idif

Do you have an example usage?
Best if a YAML snippet is pasted here, or if your project is open-source, an URL to your workflow.

jobs:
  build:
   .....
  notify:
    name: Discord Notification
    runs-on: ubuntu-latest
    needs: # make sure the notification is sent AFTER the jobs you want included have completed
      - build
    if: ${{ always() }} # You always want to be notified: success, failure, or cancelled
    steps:
    - name: Notify
      uses: nobrayner/discord-webhook@v1
      with:
        github-token: ${{ github.token }}
        discord-webhook: ${{ secrets.WEBHOOK_URL }}
        include-details: true

Generate initial code for action wrappers

Create e. g. a Gradle task that would look into action.yml for a given action (https://github.com/gradle/wrapper-validation-action/blob/master/action.yml) and generate as much as possible from it.

Implement missing parameters of existing triggers

E. g. Push is just an object:

https://github.com/krzema12/github-actions-kotlin-dsl/blob/77706e847267debc08f3c5cf895707c8a19de98d/library/src/main/kotlin/it/krzeminski/githubactions/domain/Trigger.kt#L4

while in YAML it accepts some optional arguments. We should first make these triggers as classes instead of objects, and next add necessary parameters.

Provide first-party API to output files directly

Rather than encouraging users to pipe files it'd be easier to to have something like a Workflow.writeToFile(inputPath: Path, outputPath: Path) API. This also allows users like me with a lot of workflows to be able to generate all of the outputs in one go without having to resort to too much shell scripting. Having at least the output path be a property in the workflow would also help simplify this. Ultimately I envision being able to do something like this in a CI workflow to automatically update YAML files without requiring a consistency check job in each individual workflow

#!/usr/bin/env kscript
 
@file:DependsOn("it.krzeminski:github-actions-kotlin-dsl:0.2.0")

//INCLUDE workflow1.kt
//INCLUDE workflow2.kt
//INCLUDE workflow3.kt
//INCLUDE workflow4.kt

arrayOf(workflow1, workflow2, workflow3, workflow4).forEach { workflow ->
  workflow.writeToFile() // implicitly uses an output parameter provided in the workflow itself
}

Make generated YAML more readable and better formatted

Currently the output YAML has some weird empty lines, puts certain keys in weird order, and so on. This is because the library uses kaml library which has its quirks and doesn't give full control over e.g. order of emitted YAML keys.

A proposed solution is to generate the YAML manually. It would also make development easier because on the code level, kaml imposes some limitations as well (e.g. sealed hierarchy has to be used for actions which makes it impossible to divide them into packages).