Jenkins is one of the best continuous integration (CI) servers you can find.
Jenkins has almost one and a half thousand plugins available for downloading, which allow you to solve any kind of problem you might ever encounter during deployment, development or testing
Performance Metrics Analysis Plugins
As the main audience of our blog is performance test engineers, we will start with the handiest plugins for them. One of the main performance testing trends is to make it continuous, meaning that performance verification should be a step in your continuous delivery pipeline. That’s why it is very important to track performance metrics when running the tests in your CI environment. However, it is important to keep in mind that these plugins will not fully solve the analysis part of your performance testing and they are mostly used for giving better visibility but not for detailed metrics analyses and investigation.
The Performance plugin adds a separate build step for running your performance tests. In addition to that, this plugin helps generate graphic charts for your build, based on result test files from popular testing tools like JMeter, Taurus or JUnit. The plugin can be used both in Jenkins pipelines and in GUI jobs configuration.
Plugin link: https://plugins.jenkins.io/performance
If you prefer using Gatling as your performance testing tool, then you will definitely find this plugin very useful. The ‘Gatling plugin’ provides performance trends overview based on Gatling test result files. After tests execution, you will see detailed reports per each execution build.
Plugin link: https://plugins.jenkins.io/gatling
Performance Publisher Plugin
The main advantage of the Performance Published plugin is that it is designed to work with any testing tools. However, it is up to you to create specifically formatted test result files based on XML format. The idea of the ‘Performance publisher’ plugin is that it works with a specific XML format and doesn’t have out of the box integration with different test tools. Instead, it gives you clear instructions as to which XML file you need to generate with your own tool to create very descriptive and detailed reports.
Plugin link: https://plugins.jenkins.io/perfpublisher
The BlazeMeter plugin allows you to execute load tests using BlazeMeter’s platform. This way of performance tests execution is very handy because it allows you to not care about tests infrastructure and additional tools for storing the metrics as the BlazeMeter integration will do it for you, providing real-time execution monitoring as well as metrics storage.
Plugin link: https://plugins.jenkins.io/BlazeMeterJenkinsPlugin
BlazeMeter provides you with:
- Simple Scalability – It’s easy to create large-scale JMeter tests. You can run far larger loads far more easily with BlazeMeter than you could with an in-house lab.
- Rapid-Start Deployment – BlazeMeter’s recorder helps you get started with JMeter right away, and BlazeMeter also provides complete tutorials and tips.
- Web-Based Interactive Reports – You can easily share results across distributed teams and overcome the limitations of JMeter’s standalone UI.
- Built-In Intelligence – The BlazeMeter Cloud provides on-demand geographic distribution of load generation, including built-in CDN-aware testing.
Tests Analysis Plugins
Jenkins is the perfect friend when you need to run your tests as part of continuous integration. By using Jenkins and its plugins you can run any type of test. However, Jenkins doesn’t have any out of the box functionality to analyze tests results after execution, while this is one of the most important aspects of testing.
There is a list of commonly used plugins for each specific tests runner like JUnit, TestNG or NUnit. We will not mention these plugins as they are de facto based on the tests runner you use. At the same time there are a few other plugins that might be useful to have in addition to the main one.
Test Results Analyzer
The Test Results plugin is one of my favorites because it is very easy to install and at the same time it gives great visibility on tests results and execution trend patterns. The ‘Tests Result Analyzer’ has different types of graphs and a very detailed matrix table that will show you the results of each test for all the historical builds you had before. This can be very useful for identifying unstable tests.
Plugin link: https://plugins.jenkins.io/test-results-analyzer
The ‘bootstraped-multi-test-result-report plugin’ looks really nice and allows you to create HTML reports based on tests’ results. The main advantage of this plugin is that it makes interactive reports where you can see an overall picture of all results as well as detailed results regarding steps statuses.
Reports examples: https://web-innovate.github.io/cucumber-reports/featuresOverview.html
Scaling Jenkins Setup
Jenkins setup is very easy when you think about Jenkins as a CI machine running the tests all on its own. But when you are getting more and more jobs inside your server, it becomes obvious that one instance of the Jenkins server is not enough to run all your jobs, assuming they might be running in parallel. For these needs, Jenkins has a support for the “master/slaves” mode. In this case, Jenkins will have several instances (slaves) that will be responsible for running the builds, while there will be one main instance (master) responsible for managing these slaves.
There are plenty of different approaches for configuring the Jenkins master and its slaves, but the best way is when you can dynamically allocate new slaves’ instances only when they are needed and remove them when having these instances is useless. Both of the to-be-mentioned plugins are at the top of the best practices as to how you can manage the scalability of your Jenkins server and it’s definitely worth knowing both of them.
The ‘Kubernetes plugin’ is based on an integration with an open source framework for automating deployment, called Kubernetes. If you already have or are considering having Kubernetes as a backbone for your infrastructure, this plugin is definitely for you. While a transition to the Kubernetes infrastructure is a long and heavy road, you will be paid back at the end.
Plugin link: https://plugins.jenkins.io/kubernetes
Self-Organizing Swarm Modules Plugin
The ‘Self-Organizing Swarm Modules plugin’ is another way to manage your scalable Jenkins continuous integration server, by providing a way to spin up and tear down additional Jenkins slaves. The decision between the ‘Kubernetes plugin’ and the ‘Self-Organizing Swarm Modules plugin’ is only based on which infrastructure you prefer to run for the Jenkins: is it Kubernetes or Docker Swarm? If you have such choice, I would recommend checking this article that can give you nice visibility on the pros and cons of each of them.
Plugin link: https://plugins.jenkins.io/swarm
- Easy CD/CI at Scale with Jenkins, Docker Swarm and Docker Secrets
- Jenkins Slave Nodes – using the Swarm Plugin
As your number of jobs grows, you might notice how difficult it is to manage all your jobs with the same productivity you had when everything just started. Even a few dozen Jenkins jobs on one page starts to become overcrowded. Fortunately, there are a number of plugins that are designed specifically to solve this issue. Let’s check some of them.
The number of downloads for this plugin, more than 116,000, speaks for itself. The ‘Folders plugin’ is one of the best ways for grouping your Jenkins jobs. Moreover, this plugin allows nestable folders, which can perfectly organize your continuous integration server.
Plugin link: https://plugins.jenkins.io/cloudbees-folder
View Job Filters Plugin
The ‘View Job Filters plugin’ is a smart handy way to create different views for your Jenkins jobs. Do you want to group your jobs based on the build status? Or maybe based on their trends or triggers? You will find all of these filtering options and many others in the ‘View Job Filters plugin’, which definitely makes your life easier.
Plugin link: https://plugins.jenkins.io/view-job-filters
The ‘categorized-view plugin’ is another way to handle Jenkins jobs’ organization. This plugin allows you to categorize your jobs based on regular expressions by using labels that you can create. This can save a lot of time if you have lots of jobs with the same prefix/suffix that you would like to group together.
Plugin link: https://plugins.jenkins.io/categorized-view
Getting Better Visibility and Jobs Status Control
In the previous section, we mentioned a couple of plugins that can make your life easier by providing a better way to organize how to store jobs inside Jenkins. However, jobs organization is just one problem you might face during the continuous growth of the number of tests. The next issue that you will definitely notice sooner or later is that it becomes difficult to track statuses across all the jobs you have: which of them failed, which of them passed and which of them are very unstable and require some maintenance. The next couple of plugins are designed to solve this issue as well.
Dashboard View Plugin
The ‘Dashboard view plugin’ adds an additional Jenkins dashboard, where you can track the overall jobs status including time tracking of how long each job takes and the execution time they took in total. All these statistics are very important to keep your Jenkins under control.
Plugin link: https://plugins.jenkins.io/dashboard-view
Build Monitor View Plugin
The ‘Build Monitor View plugin’ is one of my favorite plugins. I prefer running it on the TV screens inside the office space, as it gives a very straightforward view on build’s statuses. The main advantage of this plugin is that it perfectly fits any computer screen size.
Plugin link: https://plugins.jenkins.io/build-monitor-plugin
Improving Jenkins UI and UX
Last but not least, we all like nice-looking tools. The fact that Jenkins is one of the aged continuous integration tools impacted its design, which didn’t have any significant renovations for a long time. That is, until last year when Jenkins introduced their Jenkins Blue Ocean. In the list below we will give you just a few plugins that are directed to improve Jenkins’ UI and UX.
I cannot imagine using Jenkins without the ‘Greenballs plugin’. I remember that I was very confused some years ago when I installed my first Jenkins and realized that all the successful builds were marked blue instead of green, which is the way most of us are used to. The reason behind this is the fact that the creator of Jenkins, Kawaguchi Kohsuke, is from Japan, and Japanese people consider green as a shade of blue. This interesting fact became the reason of confusion, which was treated as a bug by many of engineers across the world. But as soon as you install this plugin, all successful builds will be marked as green instead of blue. Hooray!
Plugin link: https://plugins.jenkins.io/greenballs
Blue Ocean Plugin
Jenkins didn’t have any major UI/UX upgrades until last year, when they introduced the Blue Ocean plugin, which significantly improves Jenkins visualization and clarity. Therefore, if you use an older version of Jenkins, you can install the ‘Blue Ocean plugin’ as a separate plugin, while in the latest versions you will get it out of the box. I believe that the Jenkins team made a great step in the right direction by implementing this new user experience, as it is perfectly designed for Jenkins pipelines usages, which is one of the most critical features of advanced Jenkins users.
Plugin link: https://plugins.jenkins.io/blueocean
Simple Theme Plugin
You might be not ready for Jenkins Blue Ocean design for a couple of reasons. The main one is that Blue Ocean is not fully completed at this moment and you can not use all the Jenkins features when using the new design, so you have to jump between new and old design views, which makes user experience not so good. But what if the new Blue Ocean redesign is not 100% completed yet but you still want to bring new life into your old-fashioned Jenkins design? The ‘Simple Theme plugin’ will become your friend in this case. This plugin allows adding a custom CSS to your Jenkins server. By having this ability in place, you can find a couple of custom Jenkins themes as well as implement your own (for example, with a company logo).
Plugin link: https://plugins.jenkins.io/simple-theme-plugin