This doc will track the use-case and how-to’s of the Habitat core-plans maintainership’s CI pipeline. To get started make sure your local machine has all of the following requirements. If you do not have access to the core team’s shared 1password group, you will likely need to ask a core maintainer to refresh the pipelines for you.
The Plans-CI system was built as a post-merge CI pipeline for executing automated tests against package change groups created by Builder. The CI pipeline is currently running as an “acceptance” service and consists of several different pieces all being operated by a concourse ci platform. There are two repositories that house the various scripts and images that concourse uses in it’s tasks:
The intention is to provide automation around functional testing for core-plans that ship services alongside binaries in order to guarantee a high level of quality in core packages for Habitat users.
Most of the operations in the pipeline are automated and have environmental triggers in order for tasks to get executed. There are two instances (to my knowledge) in which a user must interact with the pipeline directly and that is when a new plan or plans are added to the repo or when creating a new pipeline on a new concourse instance. Both of these operations are performed in the same way.
First you need to validate that you have all of the items listed under
Requirements fulfilled. Once you have both binaries, a core-plans checkout and a copy of the shared
settings.yml file we can get started.
Change directories into the
ci dir in your core plans checkout and make sure you see a
and a dummysettings.yml` :
$ cd core-plans/ci $ ls base.yml repipe scripts settings.yml
Now (as you could probably guess) replace the dummy
settings.yml with the one downloaded from 1password. Cat the file to make sure the values no longer read
REDACTED. Take note of the values for
$ rm settings.yml $ cp ~/Downloads/settings.yml . $ cat settings.yml
Before we can update the pipeline we need to sign into the concourse instance with the
fly binary. Enter the values for concourse.user and concourse.password as required
$ fly -t plans-ci login -c https://concourse.acceptance.habitat.sh logging in to team 'main' username: <concourse.username> password: <concourse.password> target saved
You should now be able to refresh (or do an initial configuration) of the pipeline by executing the
$ ./repipe <a whole bunch of output will drop right here> apply configuration? [yN]: y configuration updated unpaused 'habitat-ci'
This operation could take a while as there are a significant number of core packages.
The plans-ci pipeline triggers off of inbound changes to unstable packages. The intention being that as new
bldr- foobarbaz groups are created, we want to validate and test each package in the build group to ensure an inbound change hasn’t broken runtime behaviors.
The first phase of the pipeline is a build group validation. This task is the equivalent to running
hab bldr job status -s and verifying that all of the jobs in the pipeline are complete and no single job has a “Failure” status
The test phase is executed on successful validation of a build group. The phase uses the delmo binary and the test harness shipped with the package. It spins up the containers defined in the docker-compose.yml from the test harness against a docker-api in AWS.
The last phase of the pipeline collects all of the changes from a the build group and creates a “release” channel once the tests have all completed successfully. This provides an easy indicator of whether a package has been fully tested successfully and enables us to promote those release channels (as necessary) into “stable”.