We are using Jenkins as CI runner and keep its configuration as code in the repo. The address of the instance we use is https://devops-ci.elastic.co/view/cloud-on-k8s/.
There are few layers in most of our jobs:
- Job definition - description of the job.
- Jenkinsfile - loads vault credentials, sets up configuration.
- CI makefile - creates container to run CI in, consolidates dev and CI setups.
- Main makefile - contains logic, delegates to specific tools as needed.
- tools - e.g. for e2e test running and cluster provisioning.
For debugging and development purposes it's possible to run CI jobs from dev box. It requires minimal setup and it mirrors CI closely, starting at CI makefile layer.
Once, run:
# fill out:
export VAULT_ADDR=YOUR_VAULT_INSTANCE_ADDRESS
export GITHUB_TOKEN=YOUR_PERSONAL_ACCESS_TOKEN
Per repro, depending on the job, set up .env and deployer-config.yml files by using pipeline-gen andset-deployer-config.sh. Example:
> .buildkite/e2e/pipeline-gen/pipeline-gen -f p=gke,s=8.6.2 -e | tee .env
E2E_JSON=true
GO_TAGS=release
export LICENSE_PUBKEY=/Users/krkr/dev/src/github.com/elastic/cloud-on-k8s/.ci/license.key
E2E_IMG=docker.elastic.co/eck-dev/eck-e2e-tests:2.8.0-SNAPSHOT-f01854af
OPERATOR_IMAGE=docker.elastic.co/eck-dev/eck-operator-krkr:2.8.0-SNAPSHOT-f01854af
E2E_PROVIDER=gke
TEST_OPTS=-race
TEST_LICENSE=/Users/krkr/dev/src/github.com/elastic/cloud-on-k8s/.ci/test-license.json
MONITORING_SECRETS=
PIPELINE=e2e/gke
CLUSTER_NAME=eck-e2e-gke-dzau-0
BUILD_NUMBER=0
E2E_STACK_VERSION=8.6.2
> export $(cat .env | xargs)
> .buildkite/scripts/test/set-deployer-config.sh
> make -C .ci TARGET="run-deployer e2e-run" ciThe CI Makefile will take care of setting up correct credentials in the deployer-config.yml file. For more details about settings in this file, see deployer.
This will run e2e tests using the same:
- container
- credentials
- settings
- call path
as the CI job.
You can build and run CI container interactively with:
make -C .ci ci-interactive