Sunday, 21 December 2014

Continuous Integration with Jenkins



Table of Contents


1. Continuous Integration with Jenkins 

1.1. Continuous Integration Overview
Continuous integration is a process in which all development work is integrated at a predefined time or event and the resulting work is automatically tested and built. The idea is that development errors are identified very early in the process. 

1.2. Jenkins 

Jenkins is one open source tool to perform continuous integration. The basic functionality of Jenkins is to monitor a version control system and to start and monitor a build system (for example, Apache Ant or Maven) if changes occur. Jenkins monitors the whole build process and provides reports and notifications to alert maintainers on success or errors.
Jenkins can be extended by additional plug-ins, e.g., for building and testing Android applications. 

1.3. Requirements for using Jenkins 

To use Jenkins you need:
  • An accessible source code repository, e.g., a Git repository, with your code checked in.
  • A working build script, e.g., a Maven script, checked into the repository
Jenkins can be started via the command line or can run in a web application server. Under Linux you can also install Jenkins as system service. 

2. Installation 

2.1. Installing Jenkins on Ubuntu
Jenkins provides Debian/Ubuntu packages which install Jenkins and register Jenkins as start service. See the following URL for details.
http://pkg.jenkins-ci.org/debian/
This installs a /etc/init.d/jenkins start script which starts Jenkins automatically at boot time.
If you start it locally, you find it running under the following URL: http://localhost:8080/
2.2. Using native packages on other platforms
For most platforms you have native packages, see the Jenkins Homepage.
2.3. Using the .WAR file of Jenkins
Download the jenkins.war file from Jenkins Homepage.
You can also start Jenkins directly via the command line with java -jar jenkins*.war. If you start it locally, you find it running under the following URL: http://localhost:8080/
To run it in your Tomcat server, put the .WAR file into the webapps directory. If you start Tomcat, your Jenkins installation will be available under http://localhost:8080/jenkins
3. Apache Ant, Tomcat 

If you want to install Jenkins in a web container, you can use, for example, Tomcat or Jetty. See the Apache Tomcat Tutorial.
In this example we use Apache Ant to build a simple Java project. Please see the Apache Ant tutorial for details.
We use Git in this example. Please see the Git tutorial for details.
4. Configure Jenkins 

4.1. Entering the JDK and your build system
Before using Jenkins you need to tell it where your JDK and ant installation is. Open Jenkins in your browser and click Manage Jenkins and then Configure System. 



Enter the correct path to your JDK, Apache Ant and Maven and press the Save button below. Jenkins can also install these for your automatically.


4.2. Secure Jenkins
It is recommended to secure Jenkins. Manage Jenkins and then Configure Global Security. Select the Enable security flag. The easiest way is to use Jenkins own user database. Create at least the user "Anonymous" with read access. Also create entries for the users you want to add in the next step. 


On the login page, select Create an account to create the users you just gave access.
Go to Manage Jenkins, Manage and Assign Roles and then Assign Roles to grant the newly created user additional access rights.
Navigate to Manage Roles to define access restrictions in detail. Pattern is a regex value of the job name. The following grants unregistered users read-only access to your build jobs that start with the C-MASTER or M-MASTER prefix and only those.
5. Support for the Git version control systems
Jenkins supports the Git version control system via a plugin. Select the Manage Jenkins → Manager Plugins link. Here you have to install the Git Plugin.
To clone a Git repostory via Jenkins you need to enter the email and user name for your Jenkins system. For this switch into your job directory and run the git config command.
# Need to configure the Git email and user for the Jenkins job

# switch to the job directory
cd /var/lib/jenkins/jobs/Android/workspace

# setup name and email
sudo git config user.name "jenkins"
sudo git config user.email "test@gmail.com"
6. Setting up a Jenkins job
The build of a project is handled via jobs in Jenkins. Select New Item from the menu
Afterwards enter a name for the job and and select Freestyle Job. Press OK to create a new Job in Jenkins.
The next page allows you to configure your job. If you for example using Git, enter the URL to the Git repository. If the repository is not public, you may also need to configure the credentials.
Specify when and how your build should be triggered. The following example polls the Git repository every 15 min and triggers a build, if something has changed in the repo.
I typically delete the workspace before a build to avoid any side-effect. In the Build section you can add a build step, e.g., a Maven build.
Press Save to finish the job definition. Press Build Now on the job page to validate the job works as expected.
After a while the job should go to green or blue (depending on your configuration), if successful. Click on the job and afterwards on Console Output to see the log file in case of an error or to validate that log looks as expected.
7. Android Jenkins build job prerequisites
To create a build job on Jenkins you need to have a working build setup. See Android build tutorial.
8. Android builds with Jenkins
Jenkins supports the automatic building and testing of Android applications. To start your tests on an emulator you can use the Android Emulator Plugin. The Android Emulator Plugin supporting starting and unlocking and blocks the build until the emulator has started.
A detailed description of this plugin can be found under the following URL: Android Emulator Plugin Jenkins page.
To install this plug-in use the Manage Jenkins → Manager Plugins link and search for "Android". Select it from the list and select to install it and restart Jenkins.
Tip
For automated tests it is good practice to have another job which executes the tests. This job can be connected to the build job of the application so that it runs automatically after this compile step.
To create an Android build job on Jenkins, select New Job, enter a job name and select the Build a free-style software project option.
You configure from where the source should be cloned.
You configure the emulator which should be started. Ensure that you do not select the Show emulator window option, as your build server should not depend on the availability of a display server.
Warning
You may see the following error message: "Error: Invalid --abi ... for the selected target" in the log of your job. In this case you have to install the desired Andoid images manually with the following command.
// run this as Jenkins user
/var/lib/jenkins/tools/android-sdk/tools/android update sdk --no-ui
Use /var/lib/jenkins/tools/android-sdk/tools/android list targets to see what ABIs are installed. If no are listed manual install one.
Configure the Apache Ant build file, use the Advanced option to specify the location of the build file.
Tip
The Android Emulator Plugin supports a new job with the Build multi-configuration project option. This option allows you to test multiple emulator configurations at the same time. You can, for example, test different languages, densities, screen resolutions, etc.
Typically you have two Jobs, one for a simple build and test run and a multi-configuration project to test the build on different device configurations.
Tip
You can combine Android Emulator Plugin with the Amazon-EC2-Plug-in to run the build and the tests on several machines simultaneously.
9.1. Jenkins backup and copying files
Jenkins stores all the settings, logs and build artifacts in its home directory, for example, in /var/lib/jenkins under the default install location of Ubuntu.
To create a backup of your Jenkins setup, just copy this directory.
The jobs directory contains the individual jobs configured in the Jenkins install. You can move a job from one Jenkins installation to another by copying the corresponding job directory. You can also copy a job directory to clone a job or rename the directory.
Click reload config button in the Jenkins web user interface to force Jenkins to reload configuration from the disk.
See Adminstration of Jenkins for details.
9.2. Managing Jenkins with Git
Jenkins supports the https://wiki.jenkins-ci.org/display/JENKINS/SCM+Sync+configuration+plugin plug-in which allows you to store every change in a Git repo.
It is also possible to manually maintain the Jenkins configuration in a Git repo.

About Jenkins

Executive summary
·           Continuous integration systems are a vital part of any Agile team because they help enforce the ideals of Agile development
·           Jenkins, a continuous build tool, enables teams to focus on their work by automating the build, artifact management, and deployment processes
·           Jenkins’ core functionality and flexibility allow it to fit in a variety of environments and can help streamline the development process for all stakeholders involved

Agenda
  • ·           Continuous Integration (CI)
  • ·           What is it?
  • ·           What are the benefits?
  • ·           Continuous Build Systems
  • ·           Jenkins
  • ·           What is it?
  • ·           Where does it fit in?
  • ·           Why should I use it?
  • ·           What can it do?
  • ·           How does it work?
  • ·           Where is it used?
  • ·           How can I get started?
  • ·           Putting it all together
  • ·           Conclusion
  • ·           References

CI - Defined

  “Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible” – Martin Fowler

CI – What does it really mean?
  At a regular frequency (ideally at every commit), the system is:
  Integrated
  All changes up until that point are combined into the project
  Built
  The code is compiled into an executable or package
  Tested
  Automated test suites are run
  Archived
  Versioned and stored so it can be distributed as is, if desired
  Deployed
  Loaded onto a system where the developers can interact with it
 
CI – Benefits
  •   Immediate bug detection
  •   No integration step in the lifecycle
  •   A deployable system at any given point
  •  Record of evolution of the project.
­CI – The tools

  • ·           Code Repositories
  • ·           SVN, Mercurial, Git
  • ·           Continuous Build Systems
  • ·           Jenkins, Bamboo, Cruise Control
  • ·           Test Frameworks
  • ·           JUnit,Cucumber, CppUnit
  • ·           Artifact Repositories
  • ·           Nexus, Artifactory, Archiva

Jenkins

·           Branched from Hudson
·           Java based Continuous Build System
·           Runs in servlet container
·           Glassfish,Tomcat
·           Supported by over 400 plugins
·           SCM,Testing, Notifications, Reporting, Artifact Saving,Triggers, External Integration
·           Under development since 2005




Jenkins - History
  2005 - Hudson was first release by Kohsuke Kawaguchi of Sun Microsystems
  2010 – Oracle bought Sun Microsystems
  Due to a naming dispute, Hudson was renamed to Jenkins
  Oracle continued development of Hudson (as a branch of the original)
  

Jenkins Fitting in

 

Why Jenkins? Flexibility!

  Jenkins is a highly configurable system by itself
  The additional community developed plugins provide even more flexibility
  By combining Jenkins with Ant, Gradle, or other Build Automation tools, the possibilities are limitless

Why Jenkins? Award winning!

  InfoWorld Bossies Award, 2011

  O'Reilly Open-Source Award, 2011

ALM&SCM, SDTimes 100, 2010, 2011

  GlassFish Community Innovation Award 2008

  Duke's Choice Award 2008

Why Jenkins? Free/OSS
  Jenkins is released under the MIT License
  There is a large support community and thorough documentation
  Its easy to write plugins
  Think something is wrong with it? You can fix it!

What can Jenkins do?

  Generate test reports
  Integrate with many different Version Control Systems
  Push to various artifact repositories
  Deploys directly to production or test environments
  Notify stakeholders of build status
  …and much more

How Jenkins works - Setup

  When setting up a project in Jenkins, out of the box you have the following general options:
  Associating with a version control server
  Triggering builds
  Polling, Periodic, Building based on other projects
  Execution of shell scripts, bash scripts, Ant targets, and Maven targets
  Artifact archival
  Publish JUnit test results and Javadocs
  Email notifications
  As stated earlier, plugins expand the functionality even further

How Jenkins works - Building

  Once a project is successfully created in Jenkins, all future builds are automatic
  Building
  Jenkins executes the build in an executer
  By default, Jenkins gives one executer per core on the build server
  Jenkins also has the concept of slave build servers
  Useful for building on different architectures
  Distribution of load


How Jenkins works - Reporting

  Jenkins comes with basic reporting features
  Keeping track of build status
  Last success and failure
  Weather” Build trend
  These can be greatly enhanced with the use of pre-build plugins
  Unit test coverage
  Test result trending
  Findbugs, Checkstyle, PMD


Jenkins by example Main Page

Jenkins by example Project Status

  •   Project status pages provide more details about a given project
  •   The status of the last several builds
  •   Charting (depending on plugins)
  •   Dependencies
  •  
  •  

  • Jenkins by example Project Status


    Enhancing Jenkins
    *—  Jenkins plugin system can enable a wide range of features including (but certainly not limited to)

    • SCM
       Mercurial, Git, Subversion

    •   Testing
       Selenium,Windmill,TestLink

    •   Notifications
       IRC,Twitter, Jabber

    •   Reporting
       Doxygen, PMD, Findbugs

    •   Artifact Saving
       Artifactory, Amazon S3, SCP

    •   Triggers
       Jabber, Directory Watchers

    •   External Integration
       GitHub, Bugzilla, JIRA

    •   And most importantly The CI Game
       A points based game where developers compete against each other to develop the most stable, well- tested code




    Running Jenkins yourself

    •   Jenkins is packaged as a WAR, so you can drop it into whichever servlet container you prefer to use
      •   Jenkins comes pre-packaged with a servlet if you just want a light- weight implementation
        1.   Native/Supported packages exist for
        2.   Windows 
            Ubuntu/Debian
            Redhat/Fedora/CentOS
            Mac OSX
            openSUSE
            FreeBSD
            OpenBSD
            Solaris/OpenIndiana
            Gentoo

    Running Jenkins yourself Updates

      Jenkins has two release lines
      Standard releases
      Weekly bug fixes and features
      Long-Term Support releases
      Updates about every 3 months
      Uses a “Stable but older” version from the standard release line
      Changes are limited to backported, well-tested modifications

    Letting someone else run Jenkins

      There are also cloud-based solutions that can provide a Jenkins instance
      Cloudbees - http://www.cloudbees.com/
      ShiningPanda - https://www.shiningpanda.com/

    Tying it into Agile

      For an Agile team, Jenkins provides everything needed for a robust continuous build system
      Jenkins supports Agile principles by constantly providing access to working copies of software
      Jenkins’ extensibility allows the system to adapt to many different pre-existing environments

    Putting it all together

      While an integral part of a CI system, Jenkins is by no means the only component
      In order for a CI system to function, a common repository for the codebase needs to exist
      A database of artifacts needs to exist, so deliveries can be made at past iterations
      The last step in a CI process is the deployment of the components built
      …and none of this matters if the developers don’t use the system; procedures need to ensure the system is used as intended

    Conclusion

      Continuous integration is a necessity on complex projects due to the benefits it provides regarding early detection of problems
      A good continuous build system should be flexible enough to fit into pre-existing development environments and provide all the features a team expects from such a system
      Jenkins, a continuous build system, can be an integral part of any continuous integration system due to its core feature set and extensibility through a plugin system

    References

      Continuous Integration Martin Fowler
      Hudson
      Hudson Continuous Integration Server
      The Hudson Book
      Jenkins
      https://wiki.jenkins-ci.org
      Monkey Image
      What is Continuous Integration
    +Integration