Automation for software development

Continuous Integration (CI) refers to the build and unit testing stages of the software release process. Every committed revision can trigger an automated build and test. With Continuous Delivery (CD), code changes are automatically built, tested, and prepared for a release to production.

Source: Solidstudio (solidstudio.io). Asset path.

GitHub Actions

GitHub Actions is a Continuous Integration and Continuous Delivery (CI/CD) platform that automates building, testing, and deploying code. You can set up workflows that build and test every pull request in your repository, and optionally deploy changes after those pull requests are merged, triggered automatically based on your chosen rules.

Exploring GitHub Actions

Setting up a demo workflow in GitHub Actions.

Tips

• GitHub has templates available. Go to the Actions tab in your repository and select new workflow for an overview.
• You can find workflow examples shared in the community.
• Add workflow_dispatch as a trigger for your GitHub Actions workflow. With this trigger, you can manually run an action, instead of having to rely on external triggers. This is quite useful when testing your workflow.
• Test your workflow in a separate branch to avoid committing many small changes during debugging of the workflow.

Automating testing

A common usecase of automation is to trigger automatic testing when pushing changes and creating pull requests.

Python

The example below is taken from the CodeRefinery lesson on Continuous Integration.

Python testing

name: Python package testing

on:
  push:
    branches: [ "main", "develop" ]
  pull_request:
    branches: [ "main", "develop" ]
  workflow_dispatch:
  
jobs:
  test:
    permissions:
      contents: read
      pull-requests: write

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v4
    - name: Set up Python 3.10
      uses: actions/setup-python@v5
      with:
        python-version: "3.10"
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        python -m pip install flake8 pytest pytest-cov
        if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
    - name: Lint with flake8
      run: |
        # stop the job if there are Python syntax errors or undefined names
        flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
        # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
        flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
    - name: Test with pytest and calculate coverage
      run: |
        pytest --cov-report "xml:coverage.xml"  --cov=.
    - name: Create Coverage 
      if: ${{ github.event_name == 'pull_request' }}
      uses: orgoro/coverage@v3.1
      with:
          coverageFile: coverage.xml
          token: ${{ secrets.GITHUB_TOKEN }}

MATLAB

MATLAB has multiple pre-defined GitHub Actions available to use in your workflows.

MATLAB testing

name: Generate Test and Coverage Artifacts
on:
  push:
    branches: [ "main", "develop" ]
  pull_request:
    branches: [ "main", "develop" ]
  workflow_dispatch:
 
jobs:
  test:
    name: Run MATLAB Tests
    runs-on: ubuntu-latest
    steps:
      - name: Check out repository
        uses: actions/checkout@v4
      - name: Set up MATLAB
        uses: matlab-actions/setup-matlab@v2
        with:
          release: R2024a
      - name: Run tests
        uses: matlab-actions/run-tests@v2
        with:
          source-folder: src
          test-results-junit: test-results/results.xml
          code-coverage-cobertura: code-coverage/coverage.xml
      - name: Upload coverage reports to Codecov
        uses: codecov/codecov-action@v4
        with:
          file: code-coverage/coverage.xml
          token: ${{ secrets.CODECOV_TOKEN }}

This workflow uses Codecov to analyse the coverage report (free service for public repositories).

⮕ Learn more about Codecov

Automating documentation generation

GitHub Pages is a static site hosting service that takes HTML, CSS, and JavaScript files straight from a repository on GitHub, optionally runs the files through a build process, and publishes a website.

In order to deploy the documentation generated by the workflow below, you need to navigate to Settings -> Pages in your repository and set:

1. Source: “Deploy from a branch”
2. Branch: gh-pages from root

It is a best practice to only deploy new documentation to the gh-pages branch upon a Pull Request to the main branch. This avoids mismatches between the available source code and the documentation.

Sphinx building with gh-pages from branch

name: Sphinx documentation

on: [push, pull_request, workflow_dispatch]

permissions:
  contents: write

jobs:
  docs:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - name: Install dependencies
        run: |
          pip install -r docs/requirements.txt
      - name: Sphinx build
        run: |
          sphinx-build docs/ docs/_build/
      - name: Deploy to GitHub Pages
        uses: peaceiris/actions-gh-pages@v4
        if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
        with:
          publish_branch: gh-pages
          github_token: ${{ secrets.GITHUB_TOKEN }}
          publish_dir: docs/_build/
          force_orphan: true

Customizing the workflow

This example assumes you have a separate requirements.txt in the /docs folder. Update the location of the requirements.txt if you store the Sphinx dependencies in the root.

Warning

• With the GitHub Free plan, Pages cannot be deployed from a private repository.
• GitHub Pages sites are publicly available on the internet, even if the repository for the site is private.

Workflows for building Python packages

You can automate publishing a new version of your Python package with a GitHub Action. Notice that in the workflow below, the trigger is the creation of a new release on GitHub.

Building a Python package

name: Upload Python Package

on:
  release:
    types: [published]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install build
      - name: Build package
        run: python -m build
      - name: Publish package
        uses: pypa/gh-action-pypi-publish@release/v1
        with:
          password: ${{ secrets.PYPI_API_TOKEN }}

This action should only run after all other workflows (such as testing and building) have passed.

Additional concepts

Variables and secrets

Secrets are variables that you create in an organization, repository, or repository environment. The secrets that you create are available to use in GitHub Actions workflows. GitHub Actions can only read a secret if you explicitly include the secret in a workflow.

⮕ More information on using secrets in GitHub Actions.

Matrix strategy

A matrix strategy lets you use variables in a single job definition to automatically create multiple job runs that are based on the combinations of the variables. For example, you can use a matrix strategy to test your code in multiple versions of a language or on multiple operating systems.

A job will run for each possible combination of the variables. In the example below, the workflow will run 12 jobs, one for each combination of the os and python-version variables.

jobs:
  example_matrix:
    strategy:
      matrix:
        os: [ubuntu-latest, windows-latest, macos-latest]
        python-version: ['3.10', '3.11', '3.12', '3.13']       
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}

⮕ More information on using a matrix for your jobs.

Note

If you want deterministic runner environments, specify exact versions (e.g., macos-15) instead of latest, since the latest aliases will migrate over time.

Artifacts

Artifacts are files or sets of files that are produced during the execution of a workflow and need to be stored or shared between jobs in a workflow.

To upload an artifact, you typically add a step in your workflow:

steps:
  - name: Upload build output
    uses: actions/upload-artifact@v4
    with:
      name: build-output
      path: <path/to/build/output>

Artifacts can be downloaded in subsequent jobs of the same workflow run using the actions/download-artifact action. This is done with a step like:

steps:
  - name: Download build output
    uses: actions/download-artifact@v4
    with:
      name: build-output
      path: <path/to/download>

By default, artifacts are retained for 90 days, but this can be configured per workflow or at the repository/organization level.

GitLab pipelines

As previously mentioned in the Project management section, TU Delft has its own GitLab instance. However, it does not (yet) have preconfigured servers available to run GitLab pipelines, the equivalent of GitHub Actions.

In order to set up a pipeline, you will need to request a TU Delft Virtual Private Server and configure a GitLab runner there. The DCC has developed a step-by-step guide (see below), along with guidance on setting up a CI pipeline for a MATLAB environment.

Continuous Integration with GitLab

Set up CI/CD for your project in TU Delft GitLab.

Learn more »

Setting up a GitLab runner for MATLAB

Create a CI pipeline for a MATLAB environment in TU Delft GitLab.

Learn more »

Sonar

To automate checking your code quality, you can also make use of a third-party service. SonarQube Cloud (previously known as SonarCloud) is a cloud-based code analysis service designed to detect coding issues in many different programming languages. The free plan allows you to analyze an unlimited number of public repositories. Private projects will not be importable on this plan.

1. Make your repository public.
2. Link your GitHub repository via their login page.
3. Follow the instructions to set up the code analysis.

You can also integrate SonarQube Cloud code analysis in GitHub Actions. Typically, you would create a new workflow file, for example .github/workflows/sonar.yml, and configure triggers to your needs. You will need to set up a token in GitHub Secrets and configure what needs to be analyzed in sonar-project.properties.

Example of SonarQube Cloud GitHub Action

name: SonarQube Cloud Workflow
on:
  push:
    branches:
      - main
  pull_request:
      types: [opened, synchronize, reopened]

jobs:
  sonarqube:
    name: SonarQube
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0  
      - name: SonarQubeScan
        uses: SonarSource/sonarqube-scan-action@v4
        env: 
          SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}

The basic usage step-by-step is described in their GitHub repository.

Learn more

• Possible analysis parameters