For the last few years, we have been hosting our Jira and Confluence instances with a third party (there are some things we do that are not supported in the SaaS versions). We recently decided to upgrade them to their respective data center versions and move them in-house, managing them ourselves. Atlassian provides AWS quickstarts for both Jira and Confluence, but they are pure CloudFormation templates and we prefer to use Terraform for deploying all of our infrastructure.
Initially, I felt that I could just take what they had in their CloudFormation templates and re-write them in Terraform. It was tedious work, but not too difficult to do. Some of the components created, such as RDS instances, load balancers and security groups are things that we have modules for, so I could just re-use those. The problem came building out the auto-scaling group for the server instances.
While CloudFormation has come a long way in the last few years, I have generally stayed with Terraform because I have been multi-cloud, and it makes it a little easier. That said, CloudFormation can offer two benefits you can’t get from Terraform. One is the ability to do rolling upgrades when the template is deployed. The other is (and this is something I recently just learned) the cfn-init helper script and its ability to include metadata on instances. This can be important when you need to pass passwords and other sensative information to an instance. You could pass it in user_data, but then it would be visible to anyone who has access to the console.
My solution was pair down the CloudFormation script to only deploy the auto-scaling and launch configuration resources and then use Terraform to deploy the template and everything else. The Jira & Confluence code examples are a little large, but were the primary reason that I needed to change my deployment method since they include the database passwords for the instances that go into a configuration file. To make this post a little more readable, I’ll show you what I added to the deployment to include a Datadog agent with the deploy.
First step is to include a parameter in the CloudFormation template for the Datadog API key. Make sure that you add the NoEcho: true so that they API key doesn’t end up in the parameters section of the CloudFormation console.
Parameters:
DatadogApiKey:
Description: Datadog API Key
Type: String
Default: ""
NoEcho: true
Next, I setup the launch configuration with the metadata. Since I use Ansible for my configuration management, I create a playbook to install the Datadog agent from their Ansible Galaxy role. Then I create the directory, install Ansible, install the Datadog.datadog role from Ansible Galaxy, and then run ansible. Finally, in the user_data section, I run the cfn-init command to read the metadata and act accordingly.
ClusterNodeLaunchConfig:
Type: AWS::AutoScaling::LaunchConfiguration
Metadata:
AWS::CloudFormation::Init:
config:
files:
/opt/dd/playbook.yaml:
content: !Sub |
---
- name: Install Datadog
hosts: localhost
connection: local
become: yes
vars:
datadog_api_key: ${DatadogApiKey}
datadog_agent_version: "6.13.0-1"
datadog_additional_groups: confluence
datadog_config:
tags:
- "env:dev"
- "datacenter:aws-1"
log_level: INFO
apm_config:
enabled: false
logs_enabled: false
datadog_checks:
system_probe_config:
enabled: true
roles:
- Datadog.datadog
commands:
070_create_dd_dir:
test: "test ! -d /opt/dd/"
command: mkdir -p /opt/dd
ignoreErrors: false
081_install_ansible:
command: pip install ansible && pip install --upgrade jinja2
082_run_ansible_galaxy:
command: ansible-galaxy install Datadog.datadog
ignoreErrors: true
083_run_dd_ansible:
command: ansible-playbook /opt/dd/playbook.yaml
ignoreErrors: true
Properties:
......
UserData:
Fn::Base64:
!Join
- ""
-
- "#!/bin/bash -xe\n"
- "yum update -y aws-cfn-bootstrap\n"
- !Sub ["/opt/aws/bin/cfn-init -v --stack ${StackName}", {StackName: !Ref "AWS::StackName"}]
- !Sub [" --resource ClusterNodeLaunchConfig --region ${Region}\n", {Region: !Ref "AWS::Region"}]
- !Sub ["/opt/aws/bin/cfn-signal -e $? --stack ${StackName}", {StackName: !Ref "AWS::StackName"}]
- !Sub [" --resource ClusterNodeGroup --region ${Region}", {Region: !Ref "AWS::Region"}]
Finally, I add the terraform code to run the cloudformation template.
resource "aws_cloudformation_stack" "confluence" {
name = "confluence"
parameters = {
DatadogApiKey = var.datadog_api_key
}
capabilities = ["CAPABILITY_IAM"]
template_body = file("templates/confluence.template.yaml")
}
Now I can deploy an auto-scaling group that not only does rolling updates, but can also have passwords that won’t be visible through the AWS console.
Comments