How to build a basic 2 nodes Jenkins architecture on AWS4th June 2019
…or how to start dealing with distributed builds mode…
A single instance of Jenkins may not be sufficient when dealing with several projects where different builds are performed by different teams during the day.
Luckily it is possible to remedy this drawback by adding one or more
slave nodes within our architecture.
A slave node is an instance configured to automate execution on behalf of the master node and will handle heavy work by letting the master node perform basic operations (including the GUI part).
In this tutorial I will explain to you in a very simple and fast way how to configure a basic two nodes architecture on AWS cloud.
For this PoC i launched two Centos 7- t2small instances with 8gb EBS volume on AWS (remember that you’ll need a public ip or an elastic ip in order to download the software we need).
So, if the subnet where you’re launching your instances is set up by default to not release an ip public, just keep in mind that you’ll have to change the subnet settings or — better — associate an elastic ip to the instances.
IMPORTANT: you need to configure a security group to allow ssh connection (tcp, port 22) in order to allow the communication between the two nodes.
Once you have 2 instances up and running we can proceed configuring the master node:
STEP 1 — MASTER NODE
- Connect via ssh to the master instance
- Perform the following steps in order to install the components we need (Java and Jenkins):
- Install Java 8 (current version of Jenkins doesn’t support Java 10 and 11. So if you have multiple installations of Java on your server(s) make sure Java 8 is the default one:
sudo yum install java-1.8.0-openjdk-devel
- Enable Jenkins repo by importing the GPG key:
curl --silent --location http://pkg.jenkins-ci.org/redhat-stable/jenkins.repo | sudo tee /etc/yum.repos.d/jenkins.repo
- Add the repo to your ec2 instance:
sudo rpm --import https://jenkins-ci.org/redhat/jenkins-ci.org.key
- Now we can install Jenkins on our master node:
sudo yum install jenkins -y
- Then we can start the service:
sudo systemctl start jenkins
- … check if the service is running properly:
systemctl status jenkins
- Last but not least… enable Jenkins to start on every system boot:
sudo systemctl enable jenkins
STEP 2 — SETTING UP JENKINS
In order to configure Jenkins, we need to open a a browser and type the IP of our ec2 instance with the Jenkins port, so:
- If the installation went fine, you’ll see a screen like this one:
- You will need to use this command to check the initial admin password:
sudo cat /var/lib/jenkins/secrets/initialAdminPassword
- At this point you should be able to see a 32 character long alphanumeric string, copy it and paste it into the password field.
Once done, you’ll be asked if you want to install the suggested plugins or if you want to hand pick only some of them. Proceed with the “Installed suggested plugins”.
Jenkins will start installing plugins for us, nice !
- Once the plugin installation is done, you can proceed creating an administrator user:
- Going ahead, you’ll be asked for a URL of your Jenkins instance. You can leave the auto-generated one:
- At this point, your configuration is done:
STEP 3— SETTING UP THE SLAVE NODE (THE FUNNY PART)
- Now that we managed to complete our Jenkins master instance, we can proceed with the slave (worker) instance.
To do so, connect via ssh to the other instance we previously created.
- Install Java (as we did for the master node):
sudo yum install java-1.8.0-openjdk-devel
- Now we need to create a new jenkins user. The master node will log into the slave node as this user (… and all the builds will be executed as this user as well):
sudo useradd -d /var/lib/jenkins jenkins
- We need to generate an ssh keypair. Jenkins will use this key to authenticate on the the slave node to perform the association (please note that you can perform this step on the agent node itself or on any other Linux instance):
- Assumed that you are familiar with ssh keys, the default location where the files will be written is “~/.ssh/”.
Copy the content of id_rsa.pub, we’ll need it in a moment.
- Now it’s time to add the public key we’ve just copied into the authorized_keys file of our jenkins user (the one we have already created).
First of all, we need to create both the directory and the authorized_keys file:
sudo mkdir /var/lib/jenkins/.ssh
sudo vi /var/lib/jenkins/.ssh/authorized_keys
- Now you can copy the content of the id_rsa.pub into the authorized_keys file.
STEP 4— ADD THE SLAVE NODE VIA JENKINS GUI
Time to come back on our Jenkins GUI, then click on:
- Manage Jenkins
- Manage Nodes
- Then click on “New Node” and you will see this screen:
- Give a name to your Jenkins slave node and click on the “Permanent Agent” radio button.
- At this point you need to configure the slave node details via this screen:
In order to successfully complete the configuration, select the following options:
- Select Launch Slave Agents via SSH for Launch Method.
- Enter the hostname or IP address of your agent node in the Host field.
- Click the Add button next to Credentials and select the Jenkins scope.
- For the credential, set Kind to SSH username with private key.
- Enter jenkins for the username.
- For the private key, select Enter directly. Then, copy the contents of your private key file (~/.ssh/id_rsa by default) and paste it into the Key box.
- If you used a passphrase when generating the key, enter that for Passphrase, otherwise, leave it blank.
- Enter a descriptive id and description, then click Add.
- Click Save to save the new node.
Now, if everything wen fine, you should be able to see the node that has correctly joined our master:
Now you can start playing around with distributed builds, have fun!
How to build a basic 2 nodes Jenkins architecture on AWS was originally published in Hacker Noon on Medium, where people are continuing the conversation by highlighting and responding to this story.