Running your README in CI

The continuous integration (CI) environment has become an essential component in modern software development workflows. I've seen CI most often used to run basic builds -- ensuring code compiles, passes lint/security/code scanning rules, ensure unit tests pass. Adventurous teams even trigger end to end builds or visual regression tests. From my experience teams usually have custom build scripts specifically setup for CI, such as GitHub actions YAML scripts or separate make files. There's certainly nothing wrong with that, however in this blog post I'll endeavor to convince you to either replace that build system, or perhaps augment it, with a full developer-like build based on your README.

Motivation

If you've not yet read The Unicorn Project by Gene Kim I'll give you the quick summary: high performing teams run builds constantly and can onboard a new developer from zero to testing code in a production like environment in minutes or hours, not days or weeks.

We use CI to verify our most important goals, that code is defect free and can be deployed with confidence to production. Why not also use CI to ensure we can deploy locally with confidence?

The Hot Take

Run a build in CI using the same steps as your developer README. If you're README says

yarn
yarn start

then make sure you do that in CI. If your developer build says

docker compose up

then install docker in your CI environment and do that. If your README has multiple pages of convoluted steps required to check out multiple repositories and run multiple commands, then you've got some developer experience debt to simplify and automate that process so it can be automated simply.

But what about secrets and configuration?

A common objection/concern to running a build in CI has to do with all the dependencies required, with secrets and configuration being the most painful. Many teams have a "secrets" file that is passed down from developer to developer or some other bureaucratic process. A better path is to commit secret names or locations, not values, to your repository. You can then use runtime tools (e.g. Berglas or whisper) to authenticate with a secret store to pull the values. All that's left then is to setup your CI environment with authentication (perhaps via Workload identity federation) so it can pull down a non-production version of the configuration it needs.

Benefits

Running a developer-like build in a CI environment is a valuable opportunity to validate the build process and ensure that all dependencies, configurations, and components are working correctly. This step guarantees that a developer can build and deploy the application without any hiccups, reducing their onboarding time and improving their initial productivity.

A developer-build executed in a CI environment helps the technical team identify any integration issues or broken code early on. Early detection means we can prevent breakages from being merged, and we're much more likely to fix the breakage efficiently if it's caught quickly, reducing build maintenance cost as well.