easimon/maximize-build-space

Github action to maximize the available disk space on Github runners

maximize-build-space's Issues

is this still working?

is this still working?

More possible Cleanup targets

Since the Runner VMs are populated by a script, the knowledge of the contents of its configuration can help with identifying further target to remove.
For example:
https://github.com/actions/runner-images/blob/main/images/linux/toolsets/toolset-2204.json

How about either generating the options of this option directly from the JSON file or create some sort of rollback mechanism, than can take a similar script and just remove the software missing from the modified json file?

swapoff: Not superuser

Thanks for this useful action!

I'm currently getting the following error:

Unmounting and removing swap file.
swapoff: Not superuser.
Error: Process completed with exit code 1.

I find this confusing as I thought all actions were run as root. Any ideas what's happening here? Here's the repository.

Full actions log is below.

Thanks for any help!

Run easimon/maximize-build-space@master
with:
    remove-dotnet: true
    remove-android: true
    remove-haskell: true
    root-reserve-mb: 1024
    temp-reserve-mb: 100
    swap-size-mb: 4096
    overprovision-lvm: false
    pv-loop-path: /pv.img
    tmp-pv-loop-path: /mnt/tmp-pv.img
  env:
    GITHUB_PAT: ***
Run echo "Memory and swap:"
Memory and swap:
              total        used        free      shared  buff/cache   available
Mem:        7116968      570596      347668       32492     6198704     6203892
Swap:       4194300        2316     4191984

NAME          TYPE SIZE USED PRIO
/mnt/swapfile file   4G 2.3M   -2

Available storage:
Filesystem      Size  Used Avail Use% Mounted on
overlay          84G   56G   29G  67% /
tmpfs            64M     0   64M   0% /dev
tmpfs           3.4G     0  3.4G   0% /sys/fs/cgroup
shm              64M     0   64M   0% /dev/shm
/dev/sda1        84G   56G   29G  67% /__w
tmpfs           696M  780K  695M   1% /run/docker.sock
tmpfs           3.4G     0  3.4G   0% /proc/acpi
tmpfs           3.4G     0  3.4G   0% /proc/scsi
tmpfs           3.4G     0  3.4G   0% /sys/firmware

Run set -euo pipefail
Arguments:

  Root reserve:      1024 MiB
  Temp reserve:      100 MiB
  Swap space:        96 MiB
  Overprovision LVM: false
  Mount path:        /__w/emma_model/emma_model
  Root PV loop path: /pv.img
  Temp PV loop path: /mnt/tmp-pv.img
  Removing:     dotnet android haskell 
Removing unwanted software... 
... done
Unmounting and removing swap file.
swapoff: Not superuser.
Error: Process completed with exit code 1.

Git checkout fails after cleanup

1. Ran the cleanup action
2. Did a git checkout, and git lfs checkout
3. Failed with exit code 1

Git LFS initialized.
Fetching the repository
  /usr/bin/git -c protocol.version=2 fetch --no-tags --prune --progress --no-recurse-submodules --filter=blob:none --depth=1 origin +refs/heads/maximize_space*:refs/remotes/origin/maximize_space* +refs/tags/maximize_space*:refs/tags/maximize_space*
  The process '/usr/bin/git' failed with exit code 1
  Waiting 20 seconds before trying again
  /usr/bin/git -c protocol.version=2 fetch --no-tags --prune --progress --no-recurse-submodules --filter=blob:none --depth=1 origin +refs/heads/maximize_space*:refs/remotes/origin/maximize_space* +refs/tags/maximize_space*:refs/tags/maximize_space*
  The process '/usr/bin/git' failed with exit code 1
  Waiting 20 seconds before trying again
  /usr/bin/git -c protocol.version=2 fetch --no-tags --prune --progress --no-recurse-submodules --filter=blob:none --depth=1 origin +refs/heads/maximize_space*:refs/remotes/origin/maximize_space* +refs/tags/maximize_space*:refs/tags/maximize_space*
  Error: The process '/usr/bin/git' failed with exit code 1

Example config:

name: Build sample apk
on:
  workflow_dispatch:
  schedule:
    - cron: '0 7 * * *' # run at 7 AM UTC
jobs:
  build:
    name: Build sample apk from develop branch
    runs-on: ubuntu-latest
    steps:
      - name: Maximize space on the runner
        uses: easimon/maximize-build-space@v7
        with:
          root-reserve-mb: 512
          swap-size-mb: 1024
          remove-haskell: 'true'
          remove-docker-images: 'true'

      # Checkout (with LFS)
      - name: Checkout Repo
        uses: actions/checkout@v3
        with:
          ref: maximize_space
          lfs: true
          sparse-checkout: |
            Assets
            Packages
            ProjectSettings

Mount path permissions

Updates last night seem to have broken this action for me. https://github.com/zenml-io/zenml/actions/runs/7016996228/job/19090533966 you can see the error message. Was just running it with build-mount-path: /var/lib/docker/ set. It was working without issue until last night / this morning.

When I rerun the action without setting that input to var/lib/docker the action itself works again / completes, but then I run out of space on integration tests etc that I'm trying to run on that runner.

So not quite sure about the diagnosis / root cause, but pretty sure it relates to the changes made overnight.

Support only "removal of unnecessary software"

Would you be up for having an input that disables everything except "removal of unnecessary software"?

I'd like to be able to just use (and benefit from) this:
https://github.com/easimon/maximize-build-space/blob/test-report/README.md

(as I have a setup that runs into fatal: write error: No space left on device when using the full setup, which I didn't run into as early before)

Breach of Github T&Cs?

Hi,

Just wondering if any sort of advice has been sought regarding whether or not this breaches the Github T&Cs?

I could do with using this action but I'm concerned it may end up getting my account limited/banned.

Cheers,

James

temp and root partition swapped

For two weeks or so the root partition and the temp partition has been swapped on around 20% of the runners.

Which results in temp getting bigger and root staying small:

/dev/root                     73G   73G  100M 100% /

LVM volume creation fails on ubuntu-20.04

GitHub Actions recently started upgrading Linux virtual environment from ubuntu-18.04 to ubuntu-20.04 (actions/runner-images#1816) that seemingly breaks maximize-build-space.

Run ***/maximize-build-space
  with:
    remove-dotnet: true
    remove-android: true
    root-reserve-mb: 1024
    swap-size-mb: 4096
    pv-loop-path: /pv.img
    remove-haskell: false
Memory and swap:
              total        used        free      shared  buff/cache   available
Mem:        7121276      529448     4858396        6580     1733432     6279844
Swap:       4194300           0     4194300

NAME          TYPE SIZE USED PRIO
/mnt/swapfile file   4G   0B   -2

Available storage:
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        84G   60G   24G  72% /
devtmpfs        3.4G     0  3.4G   0% /dev
tmpfs           3.4G   12K  3.4G   1% /dev/shm
tmpfs           696M  1.1M  695M   1% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           3.4G     0  3.4G   0% /sys/fs/cgroup
/dev/sda15      105M  9.2M   96M   9% /boot/efi
/dev/loop0      136M  136M     0 100% /snap/chromium/1466
/dev/loop2       56M   56M     0 100% /snap/core18/1944
/dev/loop1      138M  138M     0 100% /snap/chromium/1479
/dev/loop3       56M   56M     0 100% /snap/core18/1988
/dev/loop4      163M  163M     0 100% /snap/gnome-3-28-1804/145
/dev/loop5       65M   65M     0 100% /snap/gtk-common-themes/1514
/dev/loop6       70M   70M     0 100% /snap/lxd/19188
/dev/loop7       32M   32M     0 100% /snap/snapd/10707
/dev/loop8       32M   32M     0 100% /snap/snapd/11036
/dev/loop9       70M   70M     0 100% /snap/lxd/19032
/dev/sdb1        14G  4.1G  9.0G  32% /mnt

Arguments:

  Root reserve: 1024 MiB
  Swap space:   4096 MiB
  Mount path:   /home/runner/work/Cropbox.jl/Cropbox.jl
  PV loop path: /pv.img
  Removing:     dotnet android 

Removing unwanted software... 
... done
Unmounting swap and temp disk.
Creating LVM Volume.
  Physical volume "/dev/loop10" successfully created.
  Can't open /dev/sdb1 exclusively.  Mounted filesystem?
  Can't open /dev/sdb1 exclusively.  Mounted filesystem?
Error: Process completed with exit code 5.

Use btrfs with compress to further reduce storage consumption

Before I knew this action I was using btrfs with compress to workaround my own space issue. I see this action is still using ext4, so this could be another way to accommodate large projects.

Mine got minimized hahaha

runs-on: ubuntu-latest

0s
Run df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        84G   53G   31G  63% /
devtmpfs        3.4G     0  3.4G   0% /dev
tmpfs           3.4G  4.0K  3.4G   1% /dev/shm
tmpfs           695M  1.1M  694M   1% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           3.4G     0  3.4G   0% /sys/fs/cgroup
/dev/loop0       64M   64M     0 100% /snap/core20/1623
/dev/loop2       68M   68M     0 100% /snap/lxd/22753
/dev/loop1       48M   48M     0 100% /snap/snapd/16778
/dev/sda15      105M  5.2M  100M   5% /boot/efi
/dev/sdb1        14G  4.1G  9.0G  31% /mnt
tmpfs           695M     0  695M   0% /run/user/1001
8s
Run easimon/maximize-build-space@master
  with:
    root-reserve-mb: 512
    swap-size-mb: 1024
    remove-dotnet: true
    remove-android: true
    remove-haskell: true
    temp-reserve-mb: 100
    overprovision-lvm: false
    pv-loop-path: /pv.img
    tmp-pv-loop-path: /mnt/tmp-pv.img
  env:
    CARGO_TERM_COLOR: always
Run echo "Memory and swap:"
  echo "Memory and swap:"
  free
  echo
  swapon --show
  echo
  
  echo "Available storage:"
  df -h
  echo
  shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
  env:
    CARGO_TERM_COLOR: always
Memory and swap:
              total        used        free      shared  buff/cache   available
Mem:        7110660      550796     5335396       12680     1224468     6247452
Swap:       4194300           0     4194300

NAME          TYPE SIZE USED PRIO
/mnt/swapfile file   4G   0B   -2

Available storage:
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        84G   53G   31G  63% /
devtmpfs        3.4G     0  3.4G   0% /dev
tmpfs           3.4G  4.0K  3.4G   1% /dev/shm
tmpfs           695M  1.1M  694M   1% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           3.4G     0  3.4G   0% /sys/fs/cgroup
/dev/loop0       64M   64M     0 100% /snap/core20/1623
/dev/loop2       68M   68M     0 100% /snap/lxd/22753
/dev/loop1       48M   48M     0 100% /snap/snapd/16778
/dev/sda15      105M  5.2M  100M   5% /boot/efi
/dev/sdb1        14G  4.1G  9.0G  31% /mnt
tmpfs           695M     0  695M   0% /run/user/1001

Run set -euo pipefail
Arguments:

  Root reserve:      512 MiB
  Temp reserve:      100 MiB
  Swap space:        1024 MiB
  Overprovision LVM: false
  Mount path:        /home/runner/work/XXXXX/XXXXX
  Root PV loop path: /pv.img
  Temp PV loop path: /mnt/tmp-pv.img
  Removing:     dotnet android haskell 
Removing unwanted software... 
... done
Unmounting and removing swap file.
Creating LVM Volume.
  Creating LVM PV on root fs.
  Physical volume "/dev/loop3" successfully created.
  Creating LVM PV on temp fs.
  Physical volume "/dev/loop4" successfully created.
  Volume group "buildvg" successfully created
Recreating swap
  Logical volume "swap" created.
Setting up swapspace version 1, size = 1024 MiB (1073737728 bytes)
no label, UUID=3a88015a-c8d7-412d-b453-14790523aa4d
Creating build volume
  Logical volume "buildlv" created.
mke2fs 1.45.5 (07-Jan-2020)
Creating filesystem with 14713856 4k blocks and 3679200 inodes
Filesystem UUID: f54aed80-8c13-4e70-993f-967be6f29c96
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
	4096000, 7962624, 11239424

Allocating group tables:   0/450�������       �������done                            
Writing inode tables:   0/450�������       �������done                            
Creating journal (65536 blocks): done
Writing superblocks and filesystem accounting information:   0/450�������       �������done

Run echo "Memory and swap:"
Memory and swap:
              total        used        free      shared  buff/cache   available
Mem:        7110660      589880     5117596       12688     1403184     6208616
Swap:       1048572           0     1048572

NAME      TYPE       SIZE USED PRIO
/dev/dm-0 partition 1024M   0B   -2

Available storage:
Filesystem                   Size  Used Avail Use% Mounted on
/dev/root                     84G   83G  512M 100% /
devtmpfs                     3.4G     0  3.4G   0% /dev
tmpfs                        3.4G  4.0K  3.4G   1% /dev/shm
tmpfs                        695M  1.1M  694M   1% /run
tmpfs                        5.0M     0  5.0M   0% /run/lock
tmpfs                        3.4G     0  3.4G   0% /sys/fs/cgroup
/dev/loop0                    64M   64M     0 100% /snap/core20/1623
/dev/loop2                    68M   68M     0 100% /snap/lxd/22753
/dev/loop1                    48M   48M     0 100% /snap/snapd/16778
/dev/sda15                   105M  5.2M  100M   5% /boot/efi
/dev/sdb1                     14G   13G  100M 100% /mnt
tmpfs                        695M     0  695M   0% /run/user/1001
/dev/mapper/buildvg-buildlv   55G   24K   55G   1% /home/runner/work/XXXXX/XXXXX

[Suggestion] Use zram as main swap while swap ram as backup

Some compilations need a lot of rams and sometimes, the runner will timeout and it isn't because of lack of ram as you can assign gigs of swap ram, it timeout because the swap ram is taking forever, I believe due to everybody fighting over IO on these drive, it is public runner afterall.

I did a fork and replaced the swap ram code with zramctl. My compilation which usually timeout and if it did not timeout, it usually compile in 45 minutes. With zram, it didn't timeout and compiled in 30 minutes, huge improvement.

With that said, I suggest (an elegant) implementation of zram as primary swap while using swap ram as backup swap. On a side note, the zram device takes a second to create on these public runner.

Collect time passed to free up space based on the selected combination

Summary

It would be useful to track how much time it took to free up the space with each of the options and update it in this table.

My use case

for me, it would help since I want to free up only around 8 GB and I want to pick up the quickest option/combination

Failing to create LVM Volumes (due to Github Runner changes)

A few days ago, it appears that Github Runners had their storage disk size decrease from 84GB to 73GB, since then this action has been failing with the error code fallocate: invalid length value specified. Our actions were configured with the following parameters:

• root-reserve-mb: 40960
• remove-dotnet: true
• remove-android: true
• remove-haskell: true
• remove-codeql: true
• remove-docker-images: true

I'd like to know if others are seeing the same problem.

Logs at the point of failure:

...
untagged: ubuntu:20.04
untagged: ubuntu@sha256:f2034e7195f61334e6caff6ecf2e965f92d11e888309065da85ff50c617732b8
deleted: sha256:f78909c2b360d866b3220655c0b079838258b8891a12ac25fc670f0cbb54229f
deleted: sha256:3a03f09d212915b240e9d216069aba5652ed4765c7e4b098c65e71860d47b8e1

Total reclaimed space: 3.122GB
... done
Unmounting and removing swap file.
Creating LVM Volume.
  Creating LVM PV on root fs.
fallocate: invalid length value specified
Error: Process completed with exit code 1.

Potentially to revert actions

Is it somehow possible to revert / change the space distribution after a specific point. I tried to rerun your action (with root-reserve-mb and temp-reserve-mb set), but unfortunately it doesn't work and I get the error fallocate: invalid length value specified. I guess the action isn't made for something like this yet?

https://github.com/os-builds/kali/runs/7927017365

I'm trying to build a iso, which requires a lot of space. The github actions workflow works until I try to upload the iso. I guess that the upload-artifact action requires space on root oder temp fs?

are LVM operations required

Are the LVM operations required? Would it be possible to just remove the software and skip the LVM step? I only ask because I'm not sure what maximum value I should provide for the root-reserve-mb option. Thanks.

fallocate: invalid length value specified

Started getting this error on my actions. Assumed the repo had grown too big, but even re-running previously successful actions causes this error. Log follows.

Run easimon/maximize-build-space@master
Run echo "Memory and swap:"
Memory and swap:
               total        used        free      shared  buff/cache   available
Mem:         7110608      490944     5421356       22792     1198308     6284588
Swap:        4194300           0     4194300

NAME          TYPE SIZE USED PRIO
/mnt/swapfile file   4G   0B   -2

Available storage:
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        84G   58G   26G  69% /
tmpfs           3.4G  172K  3.4G   1% /dev/shm
tmpfs           1.4G  1.1M  1.4G   1% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
/dev/sda15      105M  6.1M   99M   6% /boot/efi
/dev/sdb1        14G  4.1G  9.0G  31% /mnt
tmpfs           695M   12K  695M   1% /run/user/1001

Run set -euo pipefail
Arguments:

  Root reserve:      30720 MiB
  Temp reserve:      100 MiB
  Swap space:        1024 MiB
  Overprovision LVM: false
  Mount path:        /home/runner/work/map-repo/map-repo
  Root PV loop path: /pv.img
  Temp PV loop path: /mnt/tmp-pv.img
  Removing:     dotnet 
Removing unwanted software... 
... done
Unmounting and removing swap file.
Creating LVM Volume.
  Creating LVM PV on root fs.
fallocate: invalid length value specified
Error: Process completed with exit code 1.

Configuring this action to be less aggressive

Hey,

this is the only action that made our build work, so thank you.
Would it be possible to offer a less aggressive version or configurable version ideally which only deletes some usually never used stuff like the android sdk?

Support for Windows and macOS runners

As composite-based actions now has support for conditionals, we can now run a separate script that runs on Windows and macOS to maximize available disk space.

support docker build workflows

The first three steps will work for docker build, can you integrate it into the action?

jobs:
  build_docker:
    runs-on: ubuntu-latest
    steps:
      - name: Backup docker files
        run: |
          echo "backup moby/buildkit image"
          sudo docker image save -o ${GITHUB_WORKSPACE}/images.tar moby/buildkit
          echo "Back up /var/lib/docker folder structure and other files"
          sudo rsync -aPq /var/lib/docker/ ${GITHUB_WORKSPACE}/docker 


      - name: Maximize build space
        uses: easimon/maximize-build-space@master
        with:
          overprovision-lvm: 'true'          
          remove-dotnet: 'true'
          # instead of using default value to mount to build path, /var/lib/docker/ is really the place we need more spaces.
          build-mount-path: '/var/lib/docker/'

      - name: Restore docker files
        run: |
          sudo rsync -aPq ${GITHUB_WORKSPACE}/docker/ /var/lib/docker
          sudo rm -rf ${GITHUB_WORKSPACE}/docker
          sudo ls ${GITHUB_WORKSPACE} -l
          sudo docker image load -i ${GITHUB_WORKSPACE}/images.tar
          sudo rm ${GITHUB_WORKSPACE}/images.tar

      - name: Checkout
        uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3      
              
      - name: list images
        run: docker images

      - name: check storage
        run: |
          sudo df -h

free: command not found

I was trying out this action today on ubuntu-latest and kept getting the following error:

Run easimon/maximize-build-space@master
Run echo "Memory and swap:"
Memory and swap:
/__w/_temp/c9089276-45ab-4176-ac40-c7afff117bd0.sh: line 2: free: command not found
Error: Process completed with exit code 127.

I wonder if we need to install packages separately? Or is it assumed that the commands required by the action comes pre-installed in the github runners?

fallocate command fails with 'invalid length value specified' using default swap size

I noticed our github actions started failing around 2 days ago and it keeps failing for the same reason. Looking into it, I noticed it's because the fallocate command in this github action is failing claiming invalid length value even though, we use the default swap size and haven't made any changes to our github actions. See the log below:

My guess is Github changing its ubuntu-latest version from 20.04 to 22.04 may have something to do with it.

Here is the link to our failed github action job.

Run easimon/maximize-build-space@v8
Run echo "Memory and swap:"
Memory and swap:
               total        used        free      shared  buff/cache   available
Mem:        16365024      607300    14552084       22820     1205640    15380000
Swap:        4194300           0     4194300

NAME          TYPE SIZE USED PRIO
/mnt/swapfile file   4G   0B   -2

Available storage:
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        73G   54G   20G  74% /
tmpfs           7.9G  172K  7.9G   1% /dev/shm
tmpfs           3.2G  1.1M  3.2G   1% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
/dev/sdb15      105M  6.1M   99M   6% /boot/efi
/dev/sda1        74G  4.1G   66G   6% /mnt
tmpfs           1.6G   12K  1.6G   1% /run/user/1001

Run set -euo pipefail
Arguments:

  Root reserve:      40000 MiB
  Temp reserve:      10000 MiB
  Swap space:        4096 MiB
  Overprovision LVM: false
  Mount path:        /home/runner/work/devbox/devbox
  Root PV loop path: /pv.img
  Temp PV loop path: /mnt/tmp-pv.img
  Removing:     dotnet android haskell codeql 
Removing unwanted software... 
... done
Unmounting and removing swap file.
Creating LVM Volume.
  Creating LVM PV on root fs.
fallocate: invalid length value specified
Error: Process completed with exit code 1.

Missing License

We would like to use this action for our own build pipelines. But the project is missing a license that allows us to do so.

Can you please add a license to your project? I would suggest this license: https://choosealicense.com/licenses/mit/

Thank You!
Nico Dreher

GitHub runners changes causes the root reserve argument to be non-functional

After a recent GitHub runner change, which changed the file structure of the runners to remove /mnt, the root-reserve-mb flag no longer changes how much storage remains in the root directory.

Old filesystem:

Cleaner results on the old filesystem:

New filesystem:

Cleaner results on the new filesystem:

Arguments:

Note that the arguments we used with the old file system is identical to this except root-reserve-mb was set to 30000.

Mount point of LVM Volume is to specific

The action is a great tool, bit it has a minor drawback.
The freed space is mounted the following way (output of the action itself, name of the repository changed)

/dev/mapper/buildvg-buildlv   39G   24K   39G   1% /home/runner/work/repo/repo

But I sometimes get the following message

System.IO.IOException: No space left on device : /home/runner/runners/2.308.0/_diag/Worker_20230912-055007-utc.log

[truncated]

I know, that I can change the free space of the root partition, but this doesn't really scale in terms in reusing the configuration in different projects.

Why not just changing the mount point of /dev/mapper/buildvg-buildlv to /home/runner/?