Commands
This is used to download and configure providers in your terraform code:
terraform init
Resource: https://learn.hashicorp.com/tutorials/terraform/eks
Run the terraform code
terraform apply
Destroy all terraform resources
terraform destroy
List all resources
terraform state list
Resource: https://github.com/hashicorp/terraform/issues/12917
Remove something from state
This will remove packet_device called worker from your existing state:
terraform state rm 'packet_device.worker'
Resource: https://www.terraform.io/docs/cli/commands/state/rm.html
Cause rebuild
terraform taint $RESOURCE_NAME
# example:
terraform taint aws_security_group.allow_all
Resource: https://www.terraform.io/docs/cli/commands/taint.html
Makefile Template
all: init plan apply
init:
@echo Initializing the terraform project, please wait...
terraform init
plan:
@echo Generating an execution plan, please wait...
terraform plan
apply:
@echo Deploying terraform code, please wait...
terraform apply
destroy:
@echo Tearing down the entire terraform deployment, incoming 'are you absolutely sure you want to do this?'
terraform destroy
Import existing resources
This particular example will import the OPTIONS method from an API gateway.
Put the following in main.tf:
resource "aws_api_gateway_method" "options_method" {
}
Then run this command to import it:
/usr/local/bin/terraform import aws_api_gateway_method.options_method <api_gateway_id>/<api_resource_id>/OPTIONS
You can find the output by running this command:
terraform show
Another example (import the POST gateway method):
put the following in main.tf:
# POST
resource "aws_api_gateway_method" "post_method" {
}
command to import:
/usr/local/bin/terraform import aws_api_gateway_method.post_method <api_gateway_id>/<api_resource_id>/POST
One last example (import stage):
put the following in main.tf:
resource "aws_api_gateway_stage" "<stage_name>" {
}
command to import:
/usr/local/bin/terraform import aws_api_gateway_stage.<stage_name> <api_gateway_id>/<stage_name>
Example with security group
Terraform code:
resource "aws_security_group" "my_sg" {
}
Command to import:
terraform import aws_security_group.my_sg sg-xxxxxxxxx
To see the changes:
terraform show
AWS
Secrets Manager
Create blank secret:
resource "aws_secretsmanager_secret" "IRCSecrets" {
name = "irc/client/credentials"
description = "My IRC client credentials"
}
Resource: https://gist.github.com/anttu/6995f20e641d4f30a6003520f70608b3
Create IAM role to run on an instance and attach it
iam.tf:
# Policy for role that uses STS to get credentials to access ec2 instances
resource "aws_iam_role" "ec2_iam_role" {
name = "ec2_iam_role"
assume_role_policy = file("iam_role_policy.json")
tags = {
Name = "ec2_iam_role"
}
}
# Group together roles that apply to an instance
resource "aws_iam_instance_profile" "ec2_iam_instance_profile" {
name = "ec2_iam_instance_profile"
role = aws_iam_role.ec2_iam_role.name
}
# Policy for ansible control nodes
resource "aws_iam_role_policy" "ec2_iam_role_policy" {
name = "ec2_iam_role_policy"
role = ec2_iam_role.id
policy = file("ec2_iam_role_policy.json")
}
iam_role_policy.json:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
ec2_iam_role_policy.json - this is going to be variable based on what you want your ec2 instance to do. Here's an eaxmple that allows it to do a bunch of logging stuff:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"logs:CreateLogStream",
"logs:DescribeLogGroups",
"logs:DescribeLogStreams",
"logs:PutLogEvents"
],
"Resource": [
"arn:aws:logs:ap-southeast-1:0000:log-group:*",
"arn:aws:logs:ap-southeast-1:0000:log-group:production:*"
]
}
]
}
ec2.tf:
resource "aws_instance" "ec2_node" {
ami = "ami-07dd19a7900a1f049"
instance_type = "t3.medium"
key_name = "ec2-key"
# Enable termination protection
disable_api_termination = true
vpc_security_group_ids = [aws_security_group.name1.id, aws_security_group.name2.id]
subnet_id = "your_subnet_id"
associate_public_ip_address = true
root_block_device {
volume_size = 100
delete_on_termination = true
}
tags = {
Name = "ec2_node"
}
iam_instance_profile = "aws_iam_instance_profile.ec2_iam_instance_profile.name"
}
Resources:
https://adrianhesketh.com/2016/06/27/creating-aws-instance-roles-with-terraform/
https://devopslearning.medium.com/aws-iam-ec2-instance-role-using-terraform-fa2b21488536
https://stackoverflow.com/questions/62953164/create-and-attach-iam-role-to-ec2-using-terraform
Import existing IAM role
- Create a directory and run
terraform init - Create a placeholder like so
resource "aws_iam_role" "yourrolename" {
name = "yourrolename"
assume_role_policy = "{}"
}
- Run this command to import the existing role:
terraform import aws_iam_role.yourrolename <the name of the existing role>
- Run
terraform showto get the block of terraform code that you'll want to implement
Resource: https://mklein.io/2019/09/30/terraform-import-role-policy/
GCP
GCS Backend
If you want to manage your terraform state with a remote backend (you do if you have multiple people managing the infrastructure), you will need to run a couple of command before your first terraform init.
Create the bucket you'll be storing the state in:
REGION=us-west1
gsutil mb -p $(gcloud projects list --format="value(project_id)" --filter="yourprojectname") -l $REGION gs://name-of-bucket-to-store-state
Next, enable object versioning to avoid any corruption with your state file:
gsutil versioning set on gs://name-of-bucket-to-store-state
Finally, create a backend.tfvars with the following commands:
echo -e "bucket = \"$(gsutil ls | grep yourprojectname | awk -F '[/:]' '{print $4}')"\" | tee backend.tfvars
echo -e "prefix = \"terraform/state\"" | tee -a backend.tfvars
Add this block to your terraform code:
terraform {
backend "gcs" {}
}
At this point, you can run the following to init your terraform:
terraform init -backend-config backend.tfvar
This will take the variables we defined in the backend.tfvar we created previously and apply them to the gcs backend in the above terraform code.
From here, feel free to run plan and then apply.
Resources:
https://betterprogramming.pub/effective-ways-of-managing-your-terraform-state-44bc53043d5 - great introduction to the concept of terraform state
https://medium.com/swlh/terraform-securing-your-state-file-f6c4e13f02a9 - walkthrough of how to set things up with gsutil
Create ansible hosts file
aws_instance.managed*
ansible_template_builder.tf:
resource "local_file" "ansible_hosts" {
content = templatefile("templates/hosts.tmpl",
{
private-ip = aws_instance.managed_system.private_ip,
public-id = aws_instance.managed_system.id
}
)
filename = "${path.module}/hosts"
}
templates/hosts.tmpl:
[some_group]
%{ for index, ip in private-ip ~}
${ip} ansible_user=ansible ansible_ssh_private_key_file=/home/ubuntu/.ssh/key_file ansible_python_interpreter=/usr/bin/python3 # ${public-id[index]}
%{ endfor ~}
Resource:
https://www.linkbynet.com/produce-an-ansible-inventory-with-terraform
Packer
Create packer file
packer_builder.tf:
resource "local_file" "ami_name_to_use" {
content = templatefile("templates/ami_name_to_use.json.tpl", {
subnet_id = var.subnet_id,
sg_id = var.sg_id,
vpc_id = var.vpc_id,
size = var.size,
region = var.region,
ami_id = var.base_ami_to_use,
ssh_user = var.ssh_user,
prefix = var.prefix
})
filename = "${var.packer_code_path}/ami_name_to_use.json"
file_permission = "0644"
provisioner "local-exec" {
when = destroy
command = "rm ${self.filename}"
}
}
templates/ami_name_to_use.json.tpl:
{
"description": "Description of the AMI image purpose.",
"builders": [{
"type": "amazon-ebs",
"iam_instance_profile": "${iam_instance_profile}",
"security_group_ids": "${security_group_ids}",
"ami_name": "${ami_name}",
"ami_description": "Some description for the AMI",
"instance_type": "${instance_type}",
"force_deregister": true,
"force_delete_snapshot": true,
"region": "${region}",
"vpc_id": "${vpc_id}",
"subnet_id": "${subnet_id}",
"source_ami": "${source_ami}",
"ssh_username": "${ssh_user}",
"associate_public_ip_address": true,
"tags": {
"Name" : "Some name for the AMI",
"OS":"Ubuntu",
"OSVER": "20.04"
}
}],
"provisioners": [{
"type": "file",
"source": "scripts",
"destination": "/tmp"
},
{
"type": "shell",
"inline": [
"while [ ! -f /var/lib/cloud/instance/boot-finished ]; do echo 'Waiting for cloud-init...'; sleep 1; done"
]
},
{
"type": "shell",
"script": "./scripts/provision_ami_name_to_use.sh"
}]
}
Create security group with instance's public ip
If you need to specify a security group that relies on an instance's public IP address and you don't want to use an EIP, you can do the following:
resource "aws_instance" "my_system" {
ami = var.my_ami
instance_type = var.instance_type
key_name = "my-key"
subnet_id = module.vpc.public_subnets[0]
associate_public_ip_address = true
root_block_device {
volume_size = var.disk_size
delete_on_termination = true
}
tags = {
Name = "My System"
}
vpc_security_group_ids = [ aws_security_group.service_sg.id ]
}
resource "aws_security_group" "service_sg" {
name = "my_service"
description = "Some great description"
egress {
from_port = 0
to_port = 0
protocol = "-1"
ipv6_cidr_blocks = ["::/0"]
cidr_blocks = ["0.0.0.0/0"]
description = "Allow egress everywhere"
}
vpc_id = module.vpc.vpc_id
tags = {
Name = "service_sg"
}
}
resource "aws_security_group_rule" "instance_to_itself" {
type = "ingress"
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = ["${aws_instance.my_system.public_ip}/32"]
security_group_id = aws_security_group.service_sg.id
}
Resource: https://stackoverflow.com/questions/38246326/cycle-error-when-trying-to-create-aws-vpc-security-groups-using-terraform - discovered aws_security_group_rule from here