AWS Jenkins Pipeline Project
CI/CD Pipeline using Jenkins, SonarQube, Nexus and Graphana
Outline
The goal of the project was to practise creating a CI/CD Pipeline using Jenkins, SonarQube, Nexus and Graphana. It aims to replicate more complicated corporate level CI/CD Pipelines which are used to build, test, deploy and monitor applications.
The pipeline includes several steps within Jenkins such as testing with maven, storing artifacts, as well as configuration of linux VMs, configuration of SonarQube and configuration of Nexus. The architecture of this project is as follows:
EC2
A security group is created to allow traffic from Jenkins and the other services to our EC2 instances. The EC2 instances are created in a VPC and a key pair is created to allow SSH traffic to the instance. Our security group has the following inbound rules:
We will have 7 instances: Master, Slave1, Slave2, SonarQube, Nexus, Jenkins, Monitor. Master and worker nodes will have the following commands run for setup:
1. Update System Packages [On Master & Worker Node]
sudo apt-get update
2. Install Docker[On Master & Worker Node]
sudo apt install docker.io -y
sudo chmod 666 /var/run/docker.sock
3. Install Required Dependencies for Kubernetes[On Master & Worker Node]
sudo apt-get install -y apt-transport-https ca-certificates curl gnupg
sudo mkdir -p -m 755 /etc/apt/keyrings
4. Add Kubernetes Repository and GPG Key[On Master & Worker Node]
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.28/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.28/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list
5. Update Package List[On Master & Worker Node]
sudo apt update
6. Install Kubernetes Components[On Master & Worker Node]
sudo apt install -y kubeadm=1.28.1-1.1 kubelet=1.28.1-1.1 kubectl=1.28.1-1.1
7. Initialize Kubernetes Master Node [On MasterNode]
sudo kubeadm init --pod-network-cidr=10.244.0.0/16
8. Configure Kubernetes Cluster [On MasterNode]
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
9. Deploy Networking Solution (Calico) [On MasterNode]
kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml
10. Deploy Ingress Controller (NGINX) [On MasterNode]
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.49.0/deploy/static/provider/baremetal/deploy.yaml
We ensure the Master and Slave nodes are updated and installed with the required dependencies for Kubernetes, and that nodes are connected to each other before continuing.
Setup Jenkins on Ubuntu
This involves installing Java JDK, Jenkins and Docker onto the Jenkins server. There are various credentials from Git, SonarQube and Nexus which are used to configure the Jenkins server. Thus, we are able to code the Jenkinsfile which contains the build, test and deploy steps for the application.
There is also email notification setup through SMTPS port 465 to send and email regarding build status, and reports of the build process.
Running Pipeline
Upon running the pipeline, we get our artifacts generated as .jar and .pom files, as well as our trivy reports. Inside the SonarQube server we can also see code examination and issues found.
Monitoring
In our monitoring EC2 instance, we install prometheus + blackbox exporter and graphana. We edit the prometheus.yml file to add the blackbox job. We also add blackbox as a data source inside graphana.
We will import a dashboard template into graphana to visualine our data without too much setup.
We can also add monitoring to our system metrics of Jenkins by installing promtetheus node_exporter into our Jenkins server. This also requires a new job to be added to our prometheus.yml file in our monitoring server:
We will import a dashboard for node_exporter as well similair to before. It provides many metrics on our Jenkins VM such as CPU and memory usage.
Technologies Used
AWS EC2
Jenkins
SonarQube
Nexus
Graphana
Trivy
Website
n/a