Add K3s End-to-End Workflow in GitHub (Experimental)

[mamercad]

Add an experimental workflow in GitHub using K3s for testing; fixes #243. Here's an example in my fork.

[cognifloyd]

Exciting stuff

[arm4b]

BTW, speaking of CircleCI Issue, - #243 it's a bit outdated (sorry for that!). CircleCI is free for us now and is using 'large' 4 vCPUs & 15GB RAM executor. Added more context here: #243 (comment)

We're fine running minikube w CircleCI. Consistent runs for some good time and that environment probably provides the best testing compatibility experience with the real K8s cluster.

But perhaps adding K3s as an experimental run would also be nice for the community?
The flip side is that it may add some support burden if something fails in k3s?

[mamercad]

BTW, speaking of CircleCI Issue, - #243 it's a bit outdated (sorry for that!). CircleCI is free for us now and is using 'large' 4 vCPUs & 15GB RAM executor. Added more context here: #243 (comment)

We're fine running minikube w CircleCI. Consistent runs for some good time and that environment probably provides the best testing compatibility experience with the real K8s cluster.

But perhaps adding K3s as an experimental run would also be nice for the community? The flip side is that it may add some support burden if something fails in k3s?

Sweet, glad CircleCI is $free. As far as the support burden goes, we can label the workflow as "experimental" and not require it to pass, right?

[mamercad]

Let me update the changelog and readme ...

[mamercad]

Let me update the changelog and readme ...

Should be all set, sorry for not catching this @cognifloyd.

[mamercad]

We should probably bug this (unless it's already covered somewhere and I missed it):

# TODO: Document which versions we support and cover them.

[cognifloyd]

We should probably bug this (unless it's already covered somewhere and I missed it):

# TODO: Document which versions we support and cover them.

Yes, please create an issue for it. I'm not sure what range of k8s versions we support.

[arm4b]

Looking at the result

k3s is what really unique about this pipeline and distinguishes it from the existing CircleCI e2e tests 👍
We probably don't need to run the linting in both CircleCI and Github.

Thinking that it would be nice to have a consistent naming across the checks and show more specifics about the environment, leaving only e2e minikube on CircleCI:

• ci / helm-unittest
• ci / helm-lint
• ci / k8s-lint
• ci / helm-e2e (v1.25.4+k3s1)
• ci/circleci: helm-e2e (vX.Y.Z+minikube)
• license/cla

[mamercad]

Looking at the result

k3s is what really unique about this pipeline and distinguishes it from the existing CircleCI e2e tests 👍 We probably don't need to run the linting in both CircleCI and Github.

Thinking that it would be nice to have a consistent naming across the checks and show more specifics about the environment, leaving only e2e minikube on CircleCI:

• ci / helm-unittest
• ci / helm-lint
• ci / k8s-lint
• ci / helm-e2e (v1.25.4+k3s1)
• ci/circleci: helm-e2e (vX.Y.Z+minikube)
• license/cla

I'm on board, these are good suggestions for the next iteration!

[cognifloyd]

For GHA, the word before the first / is the name of the workflow.

For external reporters, the service reports its name before the :. CircleCI chose to report their name with the ci/ prefix.

I think listing "CI" is redundant. GHA is CI. All the listed checks are CI. Ideally I would like to see something meaningful for the names of workflows and jobs. We're stuck with the pointless ci/ prefix on ci/circleci. But everything else can be more helpful.

Currently we have lint combined with e2e tests to avoid running the e2e tests unless the lint checks pass. If we are willing to skip that optimization, then I propose we split things into these workflows and jobs:

• GHA
  • Lint workflow
    • helm-lint job
    • k8s-lint job
  • Unit tests workflow
    • helm-unittest job
  • E2E Tests workflow
    • k3s job
• CircleCI
  • E2E Tests minikube

That would lead to something that looks like this:

• Lint / helm-lint (pull_request)
• Lint / k8s-lint (pull_request)
• Unit Tests / helm-unittest (pull_request)
• E2E Tests / k3s (experimental) (v1.25.4+k3s1) (pull_request)
• ci/circleci: E2E Tests / minikube

For CircleCI, we can add the slash in our workflow name, assuming they don't disallow that character, to make it read more like the pattern GHA supplies.

If, on the other hand, you want the CI / prefix, then we have to either:

1. prefix the workflow names with CI (making it CI / Lint and CI / E2E Tests), or
2. combine all our GHA workflows into a single workflow named "CI".

I do not want to combine the workflows (option 2) because that makes restarting them more painful, and it makes the workflow file much longer and harder to maintain.