Lately I have been building a lot of systems that need secrets passed to them at build time. Initially, we were using the Jenkins credentials manager to hold the secrets. As the number of secret we had to manage for builds grew, along with all the other secrets we have, we decided that we needed a more robust secrets management system. We tried to use AWS’s secret management system, but we ran into some issues, specifically with formatted credentials such as SSL and SSH certificates. In the end we decided to deploy HashiCorp’s Vault.

There are a number of good tutorials on how to deploy Vault, so I won’t go into the details here. Instead, I’ll just cover how I configured Vault to support Jenkins and the Jenkins configuration.

Setting up the app role

The first step is to decide how to have Jenkins authenticate with Vault. Since we have multiple teams that need access to only their secrets backend, we decided not to use IAM for authentication. Instead, we decided to go with the approle backend. This allows us to configure a set of credentials for each team, and limit those credentials to only the secrets paths that the team should have access to.

The first thing is to configure the access policy that will be assigned to the role when it is created. Create a file called Jenkins-example.hcl

vi Jenkins-example.hcl

and add the following content (updating the paths for your secrets).

path "/auth/token/create" {
  capabilities = ["update"]
}

path "global/*" {
  capabilities = [ "read", "list" ]
}

path "development/*" {
  capabilities = [ "read", "list" ]
}

Once the policy file is created, you need to push it to the server.

vault policy write Jenkins-example Jenkins-example.hcl

Once the policy is in place, the next step is to enable the approle engine and create the Jenkins-example user.

# Enable the approle backend
vault auth-enable approle 

# Create the approle for Jenkins
vault write auth/approle/role/jenkins-example 
secret_id_ttl=60m \
token_ttl=60m \
token_max_tll=120m \
policies=“jenkins-example”

# Get the role-id and secret-id for storing in Jenkins
vault read auth/approle/role/jenkins-example/role-id
vault write -f auth/approle/role/jenkins-example/secret-id

Keep the output from the role-id and secret-id for later addition to Jenkins.

Configuring Jenkins

On the Jenkins server, log in to the console, navigate to configure->plugins and install the HashiCorp Vault plugin. Once it is installed, you can add the credentials to the Jenkins credentials store, storing it as jenkins-vault-approle.

With the configuration complete, you can now use Vault in your pipeline jobs.

Configuring a Pipeline Job

There are two ways to use the Vault plugin on Jenkins. If you need to need to pull out a specific secret for your build, you can use withVault to pull the secrets and set them to variables.

node {
    // define the secrets and the env variables
    // engine version can be defined on secret, job, folder or global.
    // the default is engine version 2 unless otherwise specified globally.
    def secrets = [
        [path: 'secret/testing', engineVersion: 1, secretValues: [
            [envVar: 'testing', vaultKey: 'value_one'],
            [envVar: 'testing_again', vaultKey: 'value_two']]],
        [path: 'secret/another_test', engineVersion: 2, secretValues: [
            [vaultKey: 'another_test']]]
    ]

    // optional configuration, if you do not provide this the next higher configuration
    // (e.g. folder or global) will be used
    def configuration = [vaultUrl: 'http://vault.example.com',
                         vaultCredentialId: 'jenkins-vault-approle',
                         engineVersion: 1]
    // inside this block your credentials will be available as env variables
    withVault([configuration: configuration, vaultSecrets: secrets]) {
        sh 'echo $testing'
        sh 'echo $testing_again'
        sh 'echo $another_test'
    }
}

Or, you can use the with credentials plugin, which won’t ull any secrets but will set the VAULT_ADDR variable and the VAULT_TOKEN variable. This is extremely useful for allowing Terraform to interact with Vault.

withCredentials([
        [
            $class: 'VaultTokenCredentialBinding',
            credentialsId: 'my-vault',
            vaultAddr: 'https://vault.example.com'
        ]
    ])
        {
        
        }

That should be it. Now we can pull our secrets credentials from a much more robust system.