Introduction to Chef

What is Chef

Chef is an automation platform that “turns infrastructure into code,” . Its a powerful configuration management system that can be used to programmatically control your infrastructure environment. Chef is a popular configuration management system that allows you to deploy predictable, complex environments to machines within your organization.

Chef Server

The Chef Server acts as a central repository for cookbooks as well as for information about every node it manages.

There are three types of chef server available.

• Chef Solo: Actually chef solo is not a chef server. In fact it removes the need of having a central chef server to test configurations on nodes.
• Open Source Chef: This is completely free and open source chef, which you can install anywhere.
• Hosted Chef: This is a paid, where opscode will manage your central chef server, which you can access/configure using the web interface. This makes you free from the responsibility of managing a central chef server yourself.

Chef Workstation

This machine holds all the configuration data that can later be pushed to the central chef server. Several chef command line utilities will be available in the system, which can be used to interact with nodes, update configurations etc. This is a place from which most of the work happens on a day to day basis. Chef DK is installed on the workstation

Workstation will have two main components

1. Knife Utility: Its a Command Line Tool used to communicate with the central chef server from workstation. Adding, removing, changing configurations, of nodes in central chef server will be carried out by using this knife utility
2. A Local Chef Repository: This is a place where every configuration components of chef server is stored. This chef repository can be synchronized with the chef central server (using knife).

Chef Client & Node

A node is any physical or virtual machine in your network that is managed by the Chef server. The Chef client is a piece of software that runs on each node and securely communicates with the Chef server to get the latest configuration instructions. The Chef client uses the instructions to bring the node to its desired state. It uses Ruby as its reference language for creating cookbooks and defining recipes, with an extended DSL for specific resources. 

Chef Development Kit

This is installed on your workstation. The ChefDK gives you the tools you need to develop and test your infrastructure automation code locally from your workstation, before deploying changes into production.Chef DK also includes InSpec, a powerful language for writing infrastructure tests.

Cookbook

A cookbook is the fundamental unit of configuration and policy distribution in Chef. A cookbook defines a scenario and contains everything that is required to support that scenario:

• Recipes: A recipe is the most fundamental configuration element within the organization. Its a collection of resources. It is stored in a cookbook. 
• Attribute values: An attribute can be defined in a cookbook (or a recipe) and then used to override the default settings on a node.
• File distributions: A file distribution is a specific type of resource that tells a cookbook how to distribute files, including by node, by platform, or by file version.
• Templates: A template is a file written in markup language that uses Ruby statements to solve complex configuration scenarios.
• Definitions: A definition is used to create new resources by stringing together one (or more) existing resources.
• Libraries: A library allows the use of arbitrary Ruby code in a cookbook, either as a way to extend the chef-client language or to implement a new class.
• Custom Resources: A custom resource is an abstract approach for defining a set of actions and (for each action) a set of properties and validation parameter
• Metadata: A metadata file is used to ensure that each cookbook is correctly deployed to each node.
• Resources:A resource instructs the chef-client to complete various tasks like installing packages, running Ruby code, or accessing directories and file systems. The chef-client includes built-in resources that cover many common scenarios. For the full list of resources that are built-in to the chef-client, see /resources.html.

Jenkins Pipeline - An overview

Jenkins pipeline is the script file, typically written using Groovy language, where you can mention all the tasks that you want your Jenkins to run for your job. This replaces the manual way of configuring your job from Jenkins UI.

For example, when you create a simple free style job from Jenkins UI, you mention GIT repository, and probably add one task of compiling and packaging the program. As a result of this, when you run a build Jenkins will checkout the source code from the repository, and compile and package it to give you a war/jar file, depending upon your program. Now, if any further change needs to be done, say adding a Unit testing step, you would typically go to Jenkins UI and add a build step with Unit testing. This manual way of editing UI is at times not scalable in enterprise environment when you have 10s and 100s of jobs.

Jenkins pipeline provides an easier way to manage your build configuration. Instead of configuring your steps on Jenkins UI, you create a text file named 'Jenkinsfile', and add your steps like source code checkout, compile, package, test etc... in the file. And now once you run the build, Jenkins will read this file and execute the Job. Any further change can easily be done in the Jenkinsfile.

Below are the steps to configure Jenkins Pipeline

• In your root of your project, create a file called 'Jenkinsfile'
• Open it and add few steps, as follows

                     node (‘label’){

                     git “https://github.com/jenkinsci/jenkins.git”

             //triggers the tool install as usual

                     def java = tool ‘jdk-1.8.0’

            def maven = tool ‘maven-3’

           withEnv([“JAVA_HOME=$java”,          “PATH+MAVEN=$maven/bin:${env.jAVA_HOME}/bin”])

               sh “mvn clean deploy ..”

                   }

                   }

Setup Master-Slave node in Jenkins

Setup Master-Slave node in Jenkins

The machine on which your Jenkins runs your build is called a build server. In an enterprise environment,  you might have number of builds running parallel. These builds might be of different projects and different releases, but happen to trigger at the same time, and hence these run in parallel. As any process, the build process also will consume memory while it is running. While it all depends the JVM memory setting and the size of the server, however running multiple builds simultaneously will take a toll on the performance of the server. Also in an enterprise environment, it is not advisable to run all builds on one server. To solve this problem, Jenkins has the ability to run Jenkins in a distributed environment. This means, instead of running all jobs on just one machine, jobs can be distributed to run on various other machines as configured on the Jenkins master. This is called Master-Slave Configuration. The one machine where Jenkins is installed becomes the Master and all other machines which are configured to run the builds, are the Slave Machines.

In order to configure Slave machines on Jenkins, follow below steps

• Go to Manage Jenkins - Manage Nodes and click on 'Add New Node'
• Give a name to your name say 'Slave1' and check radio button 'Permanent Agent' and click on OK.
• A screen to configure Slave details will come up.. provide a description as required. Enter # of executors as per the requirement. This number will enable you to run parallel builds on this slave.
• Mention the directory path of the slave machine. This will be the workspace for Jenkins and the source code will be checked out on this path when build is run on this machine
• Give  a 'Label' as needed
• Select Usage: 'Use this build as much as possible'. If this is selected, builds will run on this, whenever all agents of master are occupied
• Select Usage: 'Only build jobs with label expression matching this node' If this is selected, the build will run on this slave, only if the job label matches with the name of this slave.. e;g: slave1_job1
• Select Launch Method: Jenkins provides few options to launch and bring up the slave agent.
• Launch Agent via Web start: 
• This is helpful when the slave machine is  a Windows machine
• Configure your node to use the "Launch slave agents via Java Web Start" launch method. Click Save
• Note the command required to launch the slave
• On the new slave node's Jenkins page, note the slave command line shown. It will be like: java -jar slave.jar -jnlpUrl http://<JenkinsHostName>:8080/computer/<nodeName>/slave-agent.jnlp -secret <some_long_hex_string>
• Obtain the slave.jar file and copy it to your new Windows slave node. In the command line noted in the last step, the "slave.jar" is a hyperlink. Click it to download the slave.jar file. Copy the slave.jar file to a permanent location on your slave machine
• Ensure that you have a java version available on your slave machine. If not, obtain and install a copy of Java
• Run the command manually from a CMD window on your slave to confirm that it works
• Open the CMD window
• Run the command (the one like java -jar slave.jar -jnlpUrl http://<JenkinsHostName>:8080/computer/<nodeName>/slave-agent.jnlp -secret <some_long_hex_string>)
• Go back to the node's web page in Jenkins.  If everything works then page should say "Connected via JNLP agent"

• Launch Agent via SSH
• Jenkins has a built-in SSH client implementation that it can use to talk to remote sshd and start a slave agent. This is the most convenient and preferred method for Unix slaves, which normally has sshd out-of-the-box.
• Select the Launch method as 'Launch Agent via SSH'
• In this set up, you'll supply the connection information (the slave host name, user name, and ssh credential). Note that the slave will need the master's public ssh key copied to ~/.ssh/authorized_keys.
• Jenkins will do the rest of the work by itself, including copying the binary needed for a slave agent, and starting/stopping slaves. 
• If your project has external dependencies (like a special ~/.m2/settings.xml, or a special version of java), you'll need to set that up yourself, though.