How to share bamboo Java specs between several projectsThu 21 May 2020
Atlassian Bamboo CI/CD is a wonderful tool both for building your projects and deploying them. To help the development and the maintenance of the various plans and jobs and pipelines, Bamboo features the "CI/CD-as-code" approach through the so-called Bamboo-Specs, and with two possibilities: YAML and Java.
YAML is quite common for CI/CD in cloud services such as Travis or Appveyor. It is a nice descriptive language that outstands by its simplicity, but can quickly become hard to get done right as your projects scale up. On the other hand, Java, much less common in the CI/CD world, is a powerful programming language that comes with a tremendous amount of support for programming:
- Full fledged object oriented programming language with conditions, inheritance, loops and so on
- Reflexivity which allows excellent tooling such as refactoring
- IDE great support (on-the-fly compilation and error checking, type inference, documentation, highlighting etc)
- unit testing
All in all, Java turns out to be much more powerful as a language than YAML. In this short article, the focus will be on showing how to structure your project such that reusable components on one side, and plan specific businesses on the other, are separated. This will let you reuse this library in other projects, compose and maintain a lot of things at once, and amortize the developments efforts across several projects.
Reusable project organization
Let's dig now into an possible organization for reusable CI/CD code. But first ...
Disclaimer I am not a Java nor a Maven expert: there are certainly better ways of doing what we need to achieve. Any hints that I can understand are most welcome!
The main steps are as follow:
- first you need a Bamboo Java-spec Maven project. Atlassian's documentation gives you a way to get started and this part will be skipped.
- we then need to change a bit the folder structure
- finally we embed the initial Maven project in a bigger/umbrella Maven project that has connection with other existing projects. Those other projects are obviously the common libraries that are bringing us relevant and reusable services for developing further our current CI/CD project.
To make things clearer, let's call the main CI/CD project yayi (that contains the build definition for my project yayi) while the reusable CI/CD library will be called simply common.
The folder structure is as follow. I deliberately left another project,
code_doc, to highlight the organization together with the same
├── common │ ├── pom.xml │ └── src ├── code_doc │ ├── code_doc-specs │ │ ├── pom.xml │ │ └── src │ └── pom.xml └── yayi ├── pom.xml └── yayi-specs ├── pom.xml └── src
The explanations are:
code_doc-specsand not to be confused with the
code_docfolders) contain the project as created by running the skeleton (see Atlassian's documentation).
- the folder
yayicontains the parent/umbrella project that we will discuss in the next section, and the
yayi-specmaven project is a subfolder of it,
- the folder
commonwill contain our CI/CD toolbox code that we will share among several of our projects.
The umbrella project
Maven works with
pom.xml files that describes the project structure in an XML form. Among the various sections this XML can contain, there is a section called
module in there can refer to another Maven folders, which means folders that contain another
pom.xml. This Maven feature seems to be a good start for what we need.
Just as a reminder, this is where this file is:
├── common │ ├── pom.xml │ └── src ├── code_doc │ ├── code_doc-specs │ │ ├── pom.xml │ │ └── src │ └── pom.xml └── yayi ├── pom.xml <-- here └── yayi-specs ├── pom.xml └── src
and this is the content of the parent Maven project:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> <modelVersion>4.0.0</modelVersion> <parent> <groupId>com.atlassian.bamboo</groupId> <artifactId>bamboo-specs-parent</artifactId> <version>6.10.3</version> <!-- this is the version of your Bamboo instance, please change accordingly --> <relativePath/> </parent> <groupId>org.yayimorphology.nan</groupId> <artifactId>yayi-specs-parent</artifactId> <!-- name of the maven project as it appears in my IDE --> <version>1.0.0</version> <packaging>pom</packaging> <modules> <module>../common</module> <module>yayi-specs</module> </modules> </project>
The important parts are:
The type of the packaging that should be
pomas it links to other
Indeed, our umbrella/parent project does not produce any artifact by itself, it is there to make a single Maven project out of several ones: this is what this
pompackaging is about.
The links to the other projects that are relative to the current (parent)
pom.xmlfile, and within a
<modules> <module>../common</module> <module>yayi-specs</module> </modules>
Of course this layout can be adapted to your own.
parentsection as a copy-paste of the skeleton project. Omitting this makes the execution of the deployment difficult.
version are also needed. I use a dummy version
1.0.0 everywhere as I am not distributing compiled artifacts, a
groupId that is similar to the website containing my Bamboo instance and an
artifactId that is derived from the name of the project. There is no particular constraint I know of for those values as no distribution is involved.
Adapting the project skeleton
We now need to advertise a direct dependency of our
yayi project to the
common one. This is done by changing the
pom.xml. As a friendly reminder, we are now here:
├── common │ ├── pom.xml │ └── src ├── code_doc │ ├── code_doc-specs │ │ ├── pom.xml │ │ └── src │ └── pom.xml └── yayi ├── pom.xml <- we were here └── yayi-specs ├── pom.xml <-- now we are here └── src
The dependency injection to the
common project is simply done by adding a
dependency section, as follow:
<dependencies> <!-- those dependencies come from the skeleton --> <dependency> <groupId>com.atlassian.bamboo</groupId> <artifactId>bamboo-specs-api</artifactId> </dependency> <dependency> <groupId>com.atlassian.bamboo</groupId> <artifactId>bamboo-specs</artifactId> </dependency> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <scope>test</scope> </dependency> <!-- the new section --> <dependency> <groupId>org.yayimorphology.nan</groupId> <artifactId>common-specs</artifactId> <version>1.0.0</version> </dependency> </dependencies>
Again there is no particular constraints, but the
version should match the one provided by the
common project. We took also the same
groupId as for the umbrella project or any of our projects.
Note that without this explicit dependency, the functions that are needed from
common might not be found. This omission may appear clearly as a compilation failure only when you actually have code in
yayi-specs referencing code in
common and you may miss this configuration at the beginning of your developments.
The common project
The common project is also a Maven project and contains a
pom.xml. In fact, you can virtually make it a copy paste of the
yayi-specs Maven project:
- it generates a
- it inherits from the same
bamboo-specs-parentsuch that we have nice automated commands,
- only the
artifactIdand (maybe) the
versionshould be specific, and match the ones of the dependency we injected into
A relatively good start would be to just copy/paste (yeah, you heard me) the
pom.xml of the
yayi-specs without the (self) dependency to
├── common │ ├── pom.xml <-- now we are here │ └── src ├── code_doc │ ├── code_doc-specs │ │ ├── pom.xml │ │ └── src │ └── pom.xml └── yayi ├── pom.xml └── yayi-specs ├── pom.xml <- copy pasted from here └── src
The source code cannot be exactly copy/pasted from the
common should not contain any class annotated with
@BambooSpec, as this is a specific instruction for the Bamboo commands.
How it looks like
We like pictures, even when it is to show Eclipse. This is now how it looks like when importing the main
yayi project into this IDE:
Here we see clearly the 3 projects:
yayi-specs-parent, which is at the end just a kind of "super project"
Running the commands
Since you know everything about OOP, I leave your first hello world as an exercise. However, the structure above has one drawback: we need to run a specific
install command every time we make a change in the
cd yayi mvn test mvn -Ppublish-specs
cd yayi mvn test mvn install # this is the new command mvn -Ppublish-specs
I spent quite some time to avoid that, but I gave up :) . The
install command places all the artifacts of the projects into the MAven local cache (usually in
~/.m2), and the
-Ppublish-specs profile takes
jar file from there.
There is one gotcha and I was caught by it several times already, so be warned: you may publish your projects to Bamboo with an outdated version of
common. Maybe it would be a good thing to create a script that does all this mechanically.
There you are with your first, reusable, Bamboo Java spec project. As you will develop further in this path and see the number of projects in Bamboo grow, you will quickly see the benefits of creating shareable components. As examples:
- AWS setup for deploying "things" (docker images, CloudFront, etc)
- composition of short pipelines into longer ones, such as
pythonvirtual environment (creation and sourcing for each task),
yarninstallation and configuration , etc
- dynamic docker image building and direct injection in the subsequent build tasks
- local cache management
- and many more!
In addition to all this, you know already a lot about library life cycles and management from your OOP experience, unit testing and so on, and this library creation will not be too much effort.
Cherry on top: Bamboo is free for open source projects and can run on a Raspberry Pi 4 with low effort!
This project example can be found on Bitbucket (it would have been odd to put a link about Atlassian on a GitHub repository).
As you see, I did not invest too much time in having the perfect integration, this is absolutely not my focus: I want my project to build on Bamboo and to be able to make evolution to those project and my builds easily. That comes at the cost of running sometimes extra commands, so be it.