Jenkins Architecture Overview

The Jenkins ecosystem consists of two primary components: the Jenkins Controller (Master) and Jenkins Agents.

Jenkins Controller (Master)

• Acts as the central hub, managing workflow orchestration, job scheduling, and configuration.
• Serves the Jenkins UI, providing a unified interface for users.
• Delegates job execution to agents, ensuring efficient workload distribution.

Jenkins Agents

• Separate machines or containers that execute tasks defined in Jenkins jobs.
• Identified by unique labels, allowing the controller to assign jobs to the right agent.
• Enable distributed job execution, making it possible to scale Jenkins infrastructure as needed.

Benefits of Distributed Architecture

• Scalability: Handle increased workload by adding more agents.
• Resource Management: Isolate builds and tests to prevent resource conflicts.
• Environment Isolation: Run jobs in specific environments, ensuring consistency and reliability

Day 28 is one of the most important steps in Jenkins scaling: setting up agents (nodes).
Below is a step by step, to follow in order to complete Task 01 (Agent Setup) and Task 02 (Running jobs on agents).

🔹 Task 01: Create an Agent

We’ll create a new EC2 instance and connect it as a Jenkins agent.

Step 1: Create an AWS EC2 Instance for the Agent

1. Go to AWS console → EC2 → Launch Instance.
2. Choose Ubuntu 22.04 as AMI.
3. Choose instance type (t2.micro is fine for testing).
4. Configure:

• Security group: allow SSH (22), Jenkins port (8080) if needed, and Docker ports (e.g., 5000) if your jobs run containers.
  1. Download your .pem key (you’ll need it to SSH).

Step 2: Install Required Software on Agent

SSH into the agent machine:

ssh -i mykey.pem ubuntu@<agent-public-ip>

Update and install Java + Docker (must match versions on master):

sudo apt update && sudo apt upgrade -y

# Install Java (example: OpenJDK 17, check your master’s version!)
sudo apt install -y openjdk-17-jdk

# Install Docker
sudo apt install -y docker.io
sudo systemctl enable docker
sudo systemctl start docker

# Allow jenkins user to run docker (create later)
sudo usermod -aG docker ubuntu

Step 3: Configure Jenkins Master to Add New Node

On your Jenkins Master UI:

1. Go to Manage Jenkins → Nodes and Clouds → New Node.
2. Enter a name (e.g., agent-1).
3. Choose Permanent Agent.
4. Configure:

• Remote root directory: /home/ubuntu/jenkins-agent (or your preferred path).
• Labels: e.g., docker-agent, build-agent.
• Usage: "Use this node as much as possible".
• Launch method: Launch agents via SSH.

Step 4: Setup SSH Authentication

On master, generate an SSH key for Jenkins (if not already):

sudo su - jenkins
ssh-keygen -t rsa -b 4096 -C "jenkins-agent"

Copy the public key to the agent:

ssh-copy-id -i ~/.ssh/id_rsa.pub ubuntu@<agent-public-ip>

Or manually add it to agent:

nano ~/.ssh/authorized_keys

Paste master’s public key.

Now test:

ssh ubuntu@<agent-public-ip>

Step 5: Verify Connection in Jenkins

• Go back to Nodes → agent-1.
• Click Launch Agent via SSH.
• Status should change to Connected.

Now you have a working Jenkins agent.

🔹 Task 02: Run Previous Jobs on New Agent

Step 1: Add Label to Agent

On Jenkins Master:

• Go to agent-1 config → Labels.
• Example: docker-agent.

Step 2: Configure Pipeline to Use Agent

Update your Day 26 or Day 27 pipeline to run on the agent:

pipeline {
    agent { label 'docker-agent' }

    stages {
        stage('Checkout') {
            steps {
                git branch: 'main', url: 'https://github.com/<your-repo>.git'
            }
        }

        stage('Build Docker Image') {
            steps {
                sh 'docker build -t myapp:latest .'
            }
        }

        stage('Run Container') {
            steps {
                sh 'docker run -d -p 5000:5000 myapp:latest'
            }
        }
    }
}

Step 3: Trigger Build

• Run your pipeline.
• Jenkins master will delegate execution to the agent.
• Check in Nodes → agent-1 → Build Executor Status → you should see your job running there.

Step 4: Verify

• Run docker ps on your agent EC2 → you should see your container running.
• Open browser → http://<agent-public-ip>:5000 → app should be live.

At this point:

• Master only controls jobs.
• Agent(s) executes heavy lifting (building, testing, running containers).
• You can scale horizontally by adding more agents for different workloads.