Skip to main content

Test your custom registry ports using kmpkg with GitHub Actions

Once you have set up a custom registry of kmpkg ports, you may want to add Continuos Integration (CI) to validate that all your dependencies can be built successfully.

The main kmpkg registry at kumose/kmpkg is tested by the kmpkg team using Azure DevOps Pipelines. This ensures that adding new packages or updating existing ones does not break consumers.

In this article, we show you how to set up a CI environment to test the kmpkg ports in your own registry.

In this article, you'll learn to set up a CI workflow for your registry running in GitHub Actions:

  • Set up a binary cache and asset cache for your GitHub Actions workflows
  • Set up a workflow to test your registry's ports

Prerequisites​

Set up a binary cache and asset cache for your GitHub Actions workflows​

Testing a large collection of ports is an expensive task both in terms of time and computing power. We strongly recommend that before engaging CI for your ports, you invest in setting up a binary cache and an asset cache for your GitHub Action workflows.

A binary cache provides the most benefit for CI scenarios by ensuring that unmodified packages aren't rebuilt on every CI run. An asset cache mirrors artifacts downloaded for your packages during a run and uses the cached artifacts in subsequent runs. It can also help mitigate issues where the upstream repository is unreliable: for example, a broken download URL.

For detailed instructions on how to set up these caches read our binary caching and asset caching articles.

Example: Enable asset and binary caching in a GitHub Actions workflow​

env:
  FEED_URL: https://nuget.pkg.github.com/<OWNER>/index.json

steps:
  - name: Install kmpkg ports
    run: "${{ github.workspace }}/kmpkg/kmpkg install"
    env: 
      X_KMPKG_ASSET_SOURCES: "clear;x-azurl,https://my.domain.com/container,{{ secrets.SAS }},readwrite"
      KMPKG_BINARY_SOURCES: "clear;nuget,https://nuget.pkg.github.com/<OWNER>/index.json,readwrite"

This example shows how to set up a binary cache and asset cache in a GitHub Actions workflow. You should adapt this snippet to use on your own workflow's YAML file.

Breaking down this snippet:

X_KMPKG_ASSET_SOURCES is the environment variable used to configure asset caches in kmpkg. In this example, it is set to x-azurl,https://my.domain.com/container,{{secrets.SAS}},readwrite. The x-azurl backend instructs kmpkg to use an Azure Storage container as the storage provider. The x-azurl is followed by three parameters separated by commas (,).

  • https://my.domain.com/container is a storage container URL.
  • {{secrets.SAS}} is a GitHub Actions secret variable that contains a SAS token to authenticate to the storage container.
  • readwrite sets read and write permissions for the asset cache. This means that this asset cache is used to store artifacts as well as to restore artifacts from it.

KMPKG_BINARY_SOURCES is the environment variable used to configure binary caches in kmpkg. In this example, it is set to clear;x-gha,readwrite. This enables the GitHub Actions Cache backend for the binary cache. An extra step is required in your workflow to enable this backend successfully.

The following step should be included as-is in your GitHub Action workflow steps. This step exports two environment variables required by the x-gha backend to work and should be run before any task that involves kmpkg.

- name: Enable GitHub Actions Cache backend
  uses: actions/github-script@v7
  with:
  script: |
    core.exportVariable('ACTIONS_CACHE_URL', process.env.ACTIONS_CACHE_URL || '');
    core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || '');

Learn more about how all of these work by reading the documentation on the asset cache and binary cache features.

Set up a workflow to test your registry's ports​

After you have set up a binary cache and asset cache for your CI environment, the next step is to set up a workflow to test all your registry's ports. You can decide whether this workflow runs on a schedule or if it is triggered by new commits or pull requests.

The main kmpkg registry uses the kmpkg ci command, which has been tailored to the kmpkg project's needs and is not intended to remain stable or be used by consumers of kmpkg. As such, it is not suited to use for testing your own kmpkg registries. We recommend to follow the steps outlined in this article instead.

Use a manifest file to include all your ports​

Instead of using the kmpkg ci command, we suggest using a manifest file to create a build that depends on all the packages in your registry.

The following example, creates a manifest file to test all the ports in a hypothetical kmpkg registry. Replace the list of dependencies to include all the ports in your registry and place it in the root of your repository.

kmpkg.json

{
  "dependencies": [
    "beicode",
    "beison"
  ]
}

Your own ports may have dependencies on the main kmpkg registry or other third-party registries, in which case, you need to add those registries in a kmpkg-configuration.json file. While kmpkg can resolve packages from the main registry without additional configuration, we strongly recommend that you explicitly add it to the registries list for version control purposes. This ensures that you have control of the set of underlying port versions. Check out the kmpkg x-update-baseline command to help managing the baseline of your registries.

kmpkg-configuration.json

{
  "default-registry": null,
  "registries": [
    {
      "kind": "git",
      "repository": "https://github.com/kumose/kmpkg",
      "baseline": "42bb0d9e8d4cf33485afb9ee2229150f79f61a1f",
      "packages": ["*"]
    }
  ]
}

Read the kmpkg.json and kmpkg-configuration.json reference articles to learn more. And the manifest mode documentation to learn how these work together.

Acquire kmpkg in your GitHub Actions workflow​

Next, you need to acquire kmpkg to use it in your workflow. Add the following steps to install kmpkg.

steps:
- uses: actions/checkout@v4
  with:
    repository: "https://github.com/kumose/kmpkg"
    path: "kmpkg"

- name: Bootstrap kmpkg
  run: "${{ github.workspace }}/kmpkg/bootstrap-kmpkg.sh"
  shell: bash

Once these steps are completed you should have a kmpkg executable to work with.

Run kmpkg install to build your ports​

The last step is to tell kmpkg to build all your ports. You may have noticed that your own registry is absent from the kmpkg-configuration.json created a couple of steps above. The reason is that you want to test the version of the ports currently in the working directory as opposed to the versions published in your repository.

To that goal, you need to add your registry's ports as overlay ports by setting the KMPKG_OVERLAY_PORTS environment variable to your registry's ports directory.

The snippet below shows how to set up your registry's ports as overlay ports and runs kmpkg install in manifest mode to install all of your custom ports.

  - name: some kmpkg task
    run: "${{ github.workspace }}/kmpkg/kmpkg install"
    env: 
      X_KMPKG_ASSET_SOURCES: "clear;x-azurl,https://my.domain.com/container,{{ secrets.SAS }},readwrite"
      KMPKG_BINARY_SOURCES: "clear;x-gha,readwrite"
      KMPKG_OVERLAY_PORTS: "${{ github.workspace }}/ports"

In this example we assume that the kmpkg.json file is created in the root of your registry's repository.

Putting it all together your workflow's YAML file should look similar to this:

.github/workflows/test-ports.yml

name: Test kmpkg ports

on:
  push:
    branches: ["main"]
  pull_request:
    branches: ["main"]

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v4
    
    - name: Acquire kmpkg
      uses: actions/checkout@v4
      with:
        repository: "kumose/kmpkg"
        path: kmpkg

    - name: Bootstrap kmpkg
      run: "${{ github.workspace }}/kmpkg/bootstrap-kmpkg.sh"
      shell: bash

    - name: Enable GitHub Actions Cache backend
      uses: actions/github-script@v7
      with:
        script: |
          core.exportVariable('ACTIONS_CACHE_URL', process.env.ACTIONS_CACHE_URL || '');
          core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || '');

    - name: Build ports
      run: ${{ github.workspace }}/kmpkg/kmpkg install
      env:
        X_KMPKG_ASSET_SOURCES: "clear;x-azurl,https://your.domain.com/container,${{ secrets.SAS }},readwrite"
        KMPKG_BINARY_SOURCES: "clear;x-gha,readwrite"
        KMPKG_OVERLAY_PORTS: "${{ github.workspace }}/ports"
      shell: bash

This is the basic structure for a CI workflow to test your registry's ports. You may require some extra work to authenticate to private repositories or to your NuGet feed.

You may also want to add steps to automate the generation of the kmpkg.json file or a step that verifies that ports newly added to your registry are not left out of the tests.

Next steps​

The following articles may be useful to you when setting up a CI environment.