Diamond shaped dependency builds with Jenkins

A typical diamond dependency pipeline

Software has become an essential part of our daily lives, and we can no longer imagine living in a world in which it wouldn’t be possible to accomplish our day to day tasks from the comfort of our home. On the other hand, the companies that are providing the services, have to constantly keep up with the customer expectations and ensure their software works reliably.

The demand to improve the quality of our software gave rise to the continuous integration servers that are running complicated build pipelines after every little code change. With time, these build pipelines usually become large and complicated to maintain, due to all the moving pieces that building a large product brings. But, it doesn’t have to be that way.

One way to manage the complexity of the software build pipeline is with a well known automation server Jenkins and its dependency builder plugin - DepBuilder. A common pattern that appears in the build pipeline of a large software project is a diamond dependency build problem. The dependencies of the project are shaped in a diamond like way with one dependency starting the build, multiple dependencies in the middle of the pipeline, and one dependency wrapping up the build.

Case study

Say we are building a large application that is supposed to work on multiple platforms. Since building the entire project manually is time consuming and error prone, we would like to automate the deployment process. The application consists of a:

  • Server
  • Mobile application
  • Desktop application
  • A shared library

Our desired build pipeline looks similar to the one on the image below:

Diamond dependency build pipeline

Setting up the build pipeline with the DepBuilder plugin is rather simple:

  1. Create a new DepBuilder project
  2. Define the pipeline script
  3. Run the build and watch the progress

1. Create a new DepBuilder project

Click on the New Item in the left sidebar on the main Jenkins dashboard, select the DepBuilder project and pick an appropriate pipeline name:

Create a new DepBuilder project

2. Define the pipeline script

The diamond shaped build pipeline should be defined in the DepBuilder’s domain specific language (DSL), which allows you to connect your existing Jenkins jobs together with a set of a very simple rules (see documentation).

The DepBuilder DSL is very strict and will warn you about typos, missing build agents, cyclic dependencies and other problems before attempting to run a long build process. This allows you to update your pipeline script without worrying about typos breaking your entire build. If your DSL script is valid, your build pipeline works.

The image below displays the user interface of the previously created DepBuilder project with our desired diamond shaped build pipeline definition:

Diamond dependency build setup

// the build script from the image above
_BUILD {
    maxDuration: 2:30
}

TimeLib -> Server
TimeLib -> MobileApp
TimeLib -> DesktopApp

Server -> "Build Installer"
MobileApp -> "Build Installer"
DesktopApp -> "Build Installer"

In case of any error in your script, a helpful error message will appear under the script input box. The graph visualization below the script input will also update based on the contents of your build script.

But, wait! What happens if one of the jobs in the build pipeline fails?

This behavior is configurable on a dependency per dependency basis. By default if one of the dependencies fails and therefore the build cannot continue, the build will be terminated and the dependency and pipeline status light will both turn red (see also ui documentation).

If you would like to change the default build failure behavior, you can easily do that via dependency settings. Say, we would like to build the installer even if one of its dependencies did not build successfully. You can simply add the following setting the previous script and the installer will be built regardless of the status of one of its immediate parents.

"Build Installer" {
    onParentFailure: BUILD
}

3. Run the build and watch the progress

Once you are satisfied with your pipeline configuration, you should be able to run the build and observe the live progress on the pipeline’s dashboard.

The image below is showing a build progress of our diamond pipeline introduced at the beginning of this article. Each node in the visualization graph contains a job name, build number and its build duration. The color of the graph node represents the build status of that specific dependency:

  • Blue - success
  • Yellow - aborted
  • Red - build error
  • Fading blue - in progress
  • White - not build

Diamond dependency build in progress

Is your build pipeline slow?

Upgrade to DepBuilder Pro in order to build dependencies faster with no additional changes to your existing build scripts.

The build visualization graph is packed with information:

Annotated Jenkins build pipeline visualization

You are all done

We have successfully set up a diamond dependency build pipeline on Jenkins. To learn more about the DepBuilder features, make sure to check out the: