Manage GitHub Actions Artifact Storage

If you are reading this post, I hope you have already faced the error: "Create Artifact Container failed: Artifact storage quota has been hit. Unable to upload any new artifacts". If not and if you are about to set up GitHub actions for your repository, read this article before you start to avoid facing the above error down the line.

Image Credits: Pixabay@Pexels

GitHub: the famous code hosting platform stretches beyond being just a hosting platform. GitHub Actions is one such addition that automates workflows in a repository. Though GitHub is generous enough to provide free storage and computing power, there is a limit on your free meal. For more details about the quota, refer to the official document.

Product	Storage	Minutes (per month)
GitHub Free	500 MB	2,000
GitHub Pro	1 GB	3,000
GitHub Free for organizations	500 MB	2,000
GitHub Team	2 GB	3,000
GitHub Enterprise Cloud	50 GB	50,000

GitHub Quota

This article focuses on the storage limitation though similar precaution has to be taken on computation too. Let's say you have a pretty active development team or a community that keeps pushing changes every hour. Suppose your workflow builds an uber jar and packs them into docker containers using two workflow operations; you may choose to upload the uber jar to the artifact storage at the end of the build operation and download it in the deploy operation. However, if your account is a GitHub Free account (with a 500 MB storage limitation) and your uber jar size is around 100 MB, you can only have five artifacts in the artifact storage. If you didn't clear the previous builds, your sixth build will fail to upload the artifact with the following error: Create Artifact Container failed: Artifact storage quota has been hit. Unable to upload any new artifacts.

There are two different options to fix this problem automatically.

1. Setting up the maximum retention period for artifacts and logs
2. Delete old artifacts in your workflow before publishing new artifacts

Artifact and Log Retention

GitHub provides an option to set the retention period for artifacts and logs for each project. By default, GitHub keeps the artifacts and build logs for 90 days, but you can adjust this value based on your GitHub subscription. Based on the frequency of workflow execution, you may find it useful or not. For example, if your artifacts are small enough and your workflows are executed only once a day, deleting them after some days may be a good option. On the other hand, if your artifacts are too big and built on an hourly basis (or frequent enough to eat all your storage within a day), even one day retention period wouldn't work.

Step 1:

To adjust the retention period, go to the Settings of your repository.

 

Step 2:

Expand the Actions section, and navigate to the sub-section: General.

There you can change the retention period. If your repository is under an organization, you can set the retention period for the whole organization. In that case, the maximum retention period of any repository in that organization cannot exceed the retention period of the organization.

Delete Old Artifacts

Suppose this is your workflow definition. Note that the artifact is named "package" and uploaded to the artifact storage in the last step.

name: CI

on:
  push:
    branches: [master]
  pull_request:
    branches: [master]

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Set up JDK 8
        uses: actions/setup-java@v3
        with:
          java-version: "8"
          distribution: "adopt"
          cache: maven

      - name: Build with Maven
        run: mvn clean package

      - name: Copy Artifact
        run: mkdir staging && cp target/*.jar staging

      - name: Upload Artifact
        uses: actions/upload-artifact@v3
        with:
          name: package
          path: staging

 

Step 1:
To delete the old artifacts, add the following step just above the "Upload Artifact" step. This step lists all the artifacts from previous builds and deletes only the artifacts named "package". Feel free to remove the filtering logic if you want to delete all the artifacts. Refer to the GitHub Actions API to learn more about the properties.

      - name: Delete Old Artifacts
        uses: actions/github-script@v6
        id: artifact
        with:
          script: |
            const res = await github.rest.actions.listArtifactsForRepo({
              owner: context.repo.owner,
              repo: context.repo.repo,
            })

            res.data.artifacts
              .filter(({ name }) => name === 'package')
              .forEach(({ id }) => {
                github.rest.actions.deleteArtifact({
                  owner: context.repo.owner,
                  repo: context.repo.repo,
                  artifact_id: id,
                })
              })

After the changes, your workflow must look like this:

name: CI

on:
  push:
    branches: [master]
  pull_request:
    branches: [master]

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Set up JDK 8
        uses: actions/setup-java@v3
        with:
          java-version: "8"
          distribution: "adopt"
          cache: maven

      - name: Build with Maven
        run: mvn clean package

      - name: Delete Old Artifacts
        uses: actions/github-script@v6
        id: artifact
        with:
          script: |
            const res = await github.rest.actions.listArtifactsForRepo({
              owner: context.repo.owner,
              repo: context.repo.repo,
            })

            res.data.artifacts
              .filter(({ name }) => name === 'package')
              .forEach(({ id }) => {
                github.rest.actions.deleteArtifact({
                  owner: context.repo.owner,
                  repo: context.repo.repo,
                  artifact_id: id,
                })
              })

      - name: Copy Artifact
        run: mkdir staging && cp target/*.jar staging

      - name: Upload Artifact
        uses: actions/upload-artifact@v3
        with:
          name: package
          path: staging

Step 2:
Save the changes and push the changes to the repository.

 

Any workflow triggered after these changes will delete the previous artifacts but leave the build logs. Combining this with the retention period for build logs will keep your repository stays within the storage limit.

• Twitter
• Facebook
• LinkedIn
• Tumblr
• Pinterest