CI Integration

[froehlicha]

CI Integration

[froehlichA]

My idea is to have the init command generate a ci script that runs when tags are added to commits, builds the mod pack, and uploads it to Curseforge. If you're savvy enough to change it to something that fits your workflow better, you can always go and do just that. But for the "common folk" that just want a working solution, this will suffice.

[MrAwesomeRocks]

Here's a couple of files you could possibly use (assuming the OneFlow branching strategy).

Auto-release

name: Build & Push

on:
  push:
    branches:
      - main
    tags:
      - "v*"

env:
  PROJECT_NAME: "CHANGE_ME"

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  get-tag:
    name: Get latest tag
    runs-on: ubuntu-latest
    outputs:
      tag: ${{ steps.version.outputs.tag }}

    steps:
      - name: Checkout repo
        uses: actions/checkout@v2
        with:
          fetch-depth: 0

      - name: Get project version
        id: version
        uses: "WyriHaximus/github-action-get-previous-tag@v1"
        with:
          fallback: "unknown"

  changelog:
    name: Generate changelog
    runs-on: ubuntu-latest
    outputs:
      changelog: ${{ steps.changelog.outputs.changelog }}

    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Generate changelog
        id: changelog
        uses: metcalfc/[email protected]
        with:
          myToken: ${{ secrets.GITHUB_TOKEN }}

      - name: Write changelog to file
        run: echo '${{ steps.changelog.outputs.changelog }}' > changelog.txt

      - name: Store changelog
        uses: actions/upload-artifact@v2
        with:
          name: changelog
          path: changelog.txt

  build:
    name: Build Modpack
    runs-on: ubuntu-latest
    needs: [get-tag, changelog]

    steps:
      - name: Download PAX
        uses: dsaltares/fetch-gh-release-asset@master
        with:
          repo: froehlichA/pax
          file: "pax"
          token: ${{ secrets.GITHUB_TOKEN }}

      - name: Checkout repo
        uses: actions/checkout@v2
        with:
          path: "modpack"

      - name: Download changelog
        uses: actions/download-artifact@v2
        with:
          name: changelog

      - name: Move changelog
        run:  mv ./changelog.txt ./modpack/CHANGELOG.md

      - name: Create ZIP
        run: |
          sudo chmod +x ./pax
          ./pax export

      - name: Move and rename ZIP
        run: |
          ls -la
          mv .out/${{ env.PROJECT_NAME }}.zip ./${{ env.PROJECT_NAME }}-${{ needs.get-tag.outputs.tag }}.zip

      - name: Upload zip
        uses: actions/upload-artifact@v2
        with:
          name: modpack-zip
          path: ${{ env.PROJECT_NAME }}-${{ needs.get-tag.outputs.tag }}.zip

  release-gh:
    name: Release to Github
    runs-on: ubuntu-latest
    needs: [build, changelog, get-tag]

    steps:
      - name: Download modpack ZIP
        uses: actions/download-artifact@v2
        with:
          name: modpack-zip

      - name: Download changelog
        uses: actions/download-artifact@v2
        with:
          name: changelog

      - name: Create release
        uses: softprops/action-gh-release@v1
        with:
          body_path: changelog.txt
          files: ${{ env.PROJECT_NAME }}-${{ needs.get-tag.outputs.tag }}.zip
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

  release-cf:
    name: Release to CurseForge
    runs-on: ubuntu-latest
    needs: [build, changelog, get-tag]

    steps:
      - name: Download modpack ZIP
        uses: actions/download-artifact@v2
        with:
          name: modpack-zip

      - name: Get release type
        id: release-type
        run: |
          rel_type="release"
          case "${{ needs.get-tag.outputs.tag }}" in
            *alpha*) rel_type="alpha" ;;
            *beta*) rel_type="beta" ;;
            *) rel_type="release" ;;
          esac
          echo "::set-output name=type::$rel_type"

      - name: Upload to CurseForge
        uses: itsmeow/curseforge-upload@v3
        with:
          token: ${{ secrets.CF_API_TOKEN }}
          project_id: 123456
          game_endpoint: minecraft
          file_path: ${{ env.PROJECT_NAME }}-${{ needs.get-tag.outputs.tag }}.zip
          changelog: "${{ needs.changelog.outputs.changelog }}"
          changelog_type: markdown
          game_versions: java:Java 8,Forge,1.16.5
          release_type: ${{ steps.release-type.outputs.type }}
          # relations: mod:type

Auto-upgrade

name: Update modpack

on:
  schedule:
    - cron: '10 0 * * *'
  workflow_dispatch:

jobs:
  update:
    runs-on: ubuntu-latest

    steps:
      - name: Download PAX
        uses: dsaltares/fetch-gh-release-asset@master
        with:
          repo: froehlichA/pax
          file: "pax"
          token: ${{ secrets.GITHUB_TOKEN }}

      - name: Checkout repo
        uses: actions/checkout@v2
        with:
          path: "modpack"

      # This requires a --yes option for upgrade
      - name: Update mods
        run: |
          sudo chmod +x ./pax
          pax upgrade

      - name: Create pull request
        uses: peter-evans/create-pull-request@v3
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          path: ./modpack
          commit-message: "Upgrade mods"
          signoff: true
          branch: workflows/upgrade-mods
          delete-branch: true
          branch-suffix: timestamp
          title: Upgrade Mods
          base: "dev"

[froehlichA]

Nice! I'm definitely gonna take a modified version of the auto-release one, not so sure about the auto-upgrade one cuz modpacks generated that way aren't really guaranteed to work. But thanks for the files!

[MrAwesomeRocks]

That's why I made the auto-upgrade one create a PR. That way, you can test the modpack before releasing, and it should hopefully run often enough to only update one mod at a time.

[DAmNRelentless]

Since PAX is supposed to make the workflow easier when working with multiple people on the same project, I would prefer the GitFlow principle in that case.

[MrAwesomeRocks]

Did you read the article, specifically this section? It's describing the problems with GitFlow for working collaboratively and proposes a solution.

[MrAwesomeRocks]

Oh ok. I see your point. You don't want to over-complicate the workflows. So you'll be primarily using the first script?

[froehlichA]

I haven't extensively looked at it yet, but yes, the final ci script will look similar.

[froehlichA]

Haven't tested it yet, but my version looks something like this:

.github/workflows/main.yml

name: main

on:
  push:
    tags:
      - 'v*'

env:
  PROJECT_ID: "123456"

jobs:
  modpack-info:
    runs-on: ubuntu-latest
    outputs:
      version: ${{ steps.info.outputs.version }}
      loader: ${{ steps.info.outputs.loader }}
      projectname: ${{ steps.info.outputs.projectname }}
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Get modpack info
        id: info
        run: |
            manifestjson=`cat ./modpack/manifest.json`
            version=`echo $(jq -r '.minecraft.version' <<< "$manifestjson")`
            echo "::set-output name=version::$version"
            loader=`echo $(jq -r '.minecraft.modLoaders[0].id' <<< "$manifestjson")`
            loader=`echo ${loader%%-*}`
            loader="$(tr '[:lower:]' '[:upper:]' <<< ${loader:0:1})${loader:1}"
            echo "::set-output name=loader::$loader"
            projectname=`echo $(jq -r '.name' <<< "$manifestjson")`
            echo "::set-output name=projectname::$projectname"

  get-tag:
    runs-on: ubuntu-latest
    outputs:
      tag: ${{ steps.version.outputs.tag }}
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Get tag
        id: version
        uses: "WyriHaximus/github-action-get-previous-tag@v1"

  changelog:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Generate changelog
        id: changelog
        uses: metcalfc/[email protected]
        with:
          myToken: ${{ secrets.GITHUB_TOKEN }}
      - name: Write changelog to file
        run: echo '${{ steps.changelog.outputs.changelog }}' > changelog.txt
      - name: Store changelog
        uses: actions/upload-artifact@v2
        with:
          name: changelog
          path: CHANGELOG.md

  build:
    runs-on: ubuntu-latest
    needs: [changelog, get-tag, modpack-info]
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Download PAX
        uses: dsaltares/fetch-gh-release-asset@master
        with:
          repo: froehlichA/pax
          file: "pax"
          token: ${{ secrets.GITHUB_TOKEN }}
      - name: Download changelog
        uses: actions/download-artifact@v2
        with:
          name: changelog
      - name: Export modpack
        run: |
          sudo chmod +x ./pax
          ./pax export
      - name: Rename zip
        run: |
          mv .out/${{ needs.modpack-info.outputs.projectname }}.zip .out/${{ needs.modpack-info.outputs.projectname }}-${{ needs.get-tag.outputs.tag }}.zip
      - name: Upload zip
        uses: actions/upload-artifact@v2
        with:
          name: modpack
          path: .out/${{ needs.modpack-info.outputs.projectname }}-${{ needs.get-tag.outputs.tag }}.zip
  
  release-github:
    runs-on: ubuntu-latest
    needs: [build, changelog, get-tag, modpack-info]
    steps:
      - name: Download modpack
        uses: actions/download-artifact@v2
        with:
          name: modpack
      - name: Download changelog
        uses: actions/download-artifact@v2
        with:
          name: changelog
      - name: Create release
        uses: softprops/action-gh-release@v1
        with:
          body_path: CHANGELOG.md
          files: ${{ needs.modpack-info.outputs.projectname }}-${{ needs.get-tag.outputs.tag }}.zip
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

  release-curseforge:
    runs-on: ubuntu-latest
    needs: [build, changelog, get-tag, modpack-info]
    steps:
      - name: Download modpack
        uses: actions/download-artifact@v2
        with:
          name: modpack
      - name: Get release type
        id: release-type
        run: |
          rel_type="release"
          case "${{ needs.get-tag.outputs.tag }}" in
            *alpha*) rel_type="alpha" ;;
            *beta*) rel_type="beta" ;;
            *) rel_type="release" ;;
          esac
          echo "::set-output name=type::$rel_type"
      - name: Create release
        uses: itsmeow/curseforge-upload@v3
        with:
          token: ${{ secrets.CF_API_TOKEN }}
          project_id: ${{ env.PROJECT_ID }}
          game_endpoint: minecraft
          file_path: ${{ needs.modpack-info.outputs.projectname }}-${{ needs.get-tag.outputs.tag }}.zip
          changelog: "${{ needs.changelog.outputs.changelog }}"
          changelog_type: markdown
          game_versions: ${{ needs.modpack-info.outputs.loader }},${{ needs.modpack-info.outputs.version }}
          release_type: ${{ steps.release-type.outputs.type }}

• All commits tagged with v* will have the modpack built and uploaded to Github releases.
  • If a commit is tagged with v*-alpha, it's uploaded to Curseforge as a alpha release. Same with v*-beta. (Thanks MrAwesomeRocks!)
• To enable Curseforge releases, the user just has to specify the env.PROJECT_ID and secrets.CF_API_TOKEN variables.

[froehlichA]

Btw, added #29 for the -y parameter specified in MrAwesomeRocks' auto-upgrade script.

[MrAwesomeRocks]

Haven't tested it yet, but my version looks something like this:
.github/workflows/main.yml

• All commits tagged with v* will have the modpack built and uploaded to Github releases.

• If a commit is tagged with v*-alpha, it's uploaded to Curseforge as a alpha release. Same with v*-beta. (Thanks MrAwesomeRocks!)

• To enable Curseforge releases, the user just has to specify the env.PROJECT_ID and secrets.CF_API_TOKEN variables.

Only change I'd recommend is to add java:Java 8 to the `game_versions field.

Btw, added #29 for the -y parameter specified in MrAwesomeRocks' auto-upgrade script.

Thank you!

[MrAwesomeRocks]

Actually, I just noticed 2 things with your script @froehlichA :

1. The modpack-info and get-tag steps can probably be combined.
2. You need to specify an outputs section in the changelog step.

[froehlichA]

The modpack-info and get-tag steps can probably be combined.

Probably, but they only take about 4 seconds each, so it's not a big deal.

You need to specify an outputs section in the changelog step.

Yeah, thanks! :D I also noticed that when testing.

updated ./.github/workflows/main.yml

name: main

on:
  push:
    tags:
      - 'v*'

env:
  PROJECT_ID: "495368"

jobs:
  modpack-info:
    runs-on: ubuntu-latest
    outputs:
      version: ${{ steps.info.outputs.version }}
      loader: ${{ steps.info.outputs.loader }}
      projectname: ${{ steps.info.outputs.projectname }}
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Get modpack info
        id: info
        run: |
            manifestjson=`cat ./modpack/manifest.json`
            version=`echo $(jq -r '.minecraft.version' <<< "$manifestjson")`
            echo "::set-output name=version::$version"
            loader=`echo $(jq -r '.minecraft.modLoaders[0].id' <<< "$manifestjson")`
            loader=`echo ${loader%%-*}`
            loader="$(tr '[:lower:]' '[:upper:]' <<< ${loader:0:1})${loader:1}"
            echo "::set-output name=loader::$loader"
            projectname=`echo $(jq -r '.name' <<< "$manifestjson")`
            echo "::set-output name=projectname::$projectname"

  get-tag:
    runs-on: ubuntu-latest
    outputs:
      tag: ${{ steps.version.outputs.tag }}
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Get tag
        id: version
        uses: "WyriHaximus/github-action-get-previous-tag@v1"

  changelog:
    runs-on: ubuntu-latest
    outputs:
      changelog: ${{ steps.changelog.outputs.changelog }}
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Generate changelog
        id: changelog
        uses: heinrichreimer/[email protected]
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          output: CHANGELOG.md
          stripGeneratorNotice: true
      - name: Store changelog
        uses: actions/upload-artifact@v2
        with:
          name: changelog
          path: CHANGELOG.md

  build:
    runs-on: ubuntu-latest
    needs: [changelog, get-tag, modpack-info]
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Download PAX
        uses: dsaltares/fetch-gh-release-asset@master
        with:
          repo: froehlichA/pax
          file: "pax"
          token: ${{ secrets.GITHUB_TOKEN }}
      - name: Download changelog
        uses: actions/download-artifact@v2
        with:
          name: changelog
      - name: Export modpack
        run: |
          sudo chmod +x ./pax
          ./pax export
      - name: Rename zip
        run: |
          mv .out/${{ needs.modpack-info.outputs.projectname }}.zip .out/${{ needs.modpack-info.outputs.projectname }}-${{ needs.get-tag.outputs.tag }}.zip
      - name: Upload zip
        uses: actions/upload-artifact@v2
        with:
          name: modpack
          path: .out/${{ needs.modpack-info.outputs.projectname }}-${{ needs.get-tag.outputs.tag }}.zip
  
  release-github:
    runs-on: ubuntu-latest
    needs: [build, changelog, get-tag, modpack-info]
    steps:
      - name: Download modpack
        uses: actions/download-artifact@v2
        with:
          name: modpack
      - name: Download changelog
        uses: actions/download-artifact@v2
        with:
          name: changelog
      - name: Create release
        uses: softprops/action-gh-release@v1
        with:
          body_path: CHANGELOG.md
          files: ${{ needs.modpack-info.outputs.projectname }}-${{ needs.get-tag.outputs.tag }}.zip
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

  release-curseforge:
    runs-on: ubuntu-latest
    needs: [build, changelog, get-tag, modpack-info]
    steps:
      - name: Download modpack
        uses: actions/download-artifact@v2
        with:
          name: modpack
      - name: Get release type
        id: release-type
        run: |
          rel_type="release"
          case "${{ needs.get-tag.outputs.tag }}" in
            *alpha*) rel_type="alpha" ;;
            *beta*) rel_type="beta" ;;
            *) rel_type="release" ;;
          esac
          echo "::set-output name=type::$rel_type"
      - name: Create release
        uses: itsmeow/curseforge-upload@v3
        with:
          token: ${{ secrets.CF_API_TOKEN }}
          project_id: ${{ env.PROJECT_ID }}
          game_endpoint: minecraft
          file_path: ${{ needs.modpack-info.outputs.projectname }}-${{ needs.get-tag.outputs.tag }}.zip
          changelog: "${{ needs.changelog.outputs.changelog }}"
          changelog_type: markdown
          game_versions: ${{ needs.modpack-info.outputs.version }}
          release_type: ${{ steps.release-type.outputs.type }}

Apparently uploading files only works with the Minecraft version, curseforge surprisingly complains and returns 400 if you use java: Java 8 or the mod loader (Fabric or Forge).

I've been testing this updated script for some time now, and everything seems to work.

EDIT: Oh yeah, I had to switch out the changelog generator to heinrichreimer/[email protected], because the previous one fails if there is no prior release. To avoid manual labor from users as much as possible, I switched to the one mentioned above - it doesn't generate a changelog from commits though, only from closed issues & PRs.

[MrAwesomeRocks]

Here's a different changelog generator that may work better: https://github.com/marketplace/actions/changelog-ci.

Also, for the game_versions section, try using 4458 instead of java:Java 8, 7499 instead of Fabric, and 7498 instead of Forge. You can do that with an if statement like this:

loader=$(echo "$loader" | tr '[:upper:]' '[:lower:]')
if [[ "$loader" == "forge" ]]
then
    echo "::set-output name=loader::7498"
elif [[ "$loader" == "fabric" ]]
then
    echo "::set-output name=loader::7499"
else
    # something broke
    exit 1
fi

echo "::set-output name=java::4458"

---

Curseforge API version ids:

[froehlichA]

Unfortunately that's not the problem, the loader & java version are correctly converted to gameVersion IDs, But apparently these IDs aren't accepted for modpacks.

Try if the following curl statement works for you:

• curl -X POST -H "X-Api-Token:<token>" https://minecraft.curseforge.com/api/projects/<project-id>/upload-file -F 'file=@/home/alex/Projects/mc/paxdev/.out/barebones.zip' -F 'metadata={"changelog":"# Changelog","gameVersions":[4458,7499,8203],"releaseType":"release","changelogType":"markdown"}'

For me it returns {"errorCode":1009,"errorMessage":"Invalid game version ID: 4458 belongs to an invalid dependency."}

I tried removing java: Java 8 from main.yml, it gave me the same error for the Fabric id, now I'm just using the mc version.

[froehlichA]

Your suggestion for a changelog generator somehow doesn't work either. It can be configured to either generate a changelog using pull requests or commit messages, but gave me a python error with both. I'm sticking with heinrichheimer's for now, but I'd switch to metcalfc's if it would work without an initial release.

[froehlichA]

🎉 CI Integration is now out for v1.4.0, with the default CI config included.

If you would like to try out the config with an existing PAX project, you can use ./pax init -f --skip-manifest, which will only generate the git folder, .gitignore and CI file.

EDIT: Also created a wiki page for automatic releases & the CI system: https://github.com/froehlichA/pax/wiki/Automatic-releases