Difference between revisions of "Terraform"

From Christoph's Personal Wiki
Jump to: navigation, search
(→‎External links)
(→‎Concepts)
Line 211: Line 211:
 
}
 
}
 
</pre>
 
</pre>
 +
 +
===Outputs===
 +
 +
[https://www.terraform.io/docs/configuration/outputs.html Outputs] define values that will be highlighted to the user when Terraform applies, and can be queried easily using the [https://www.terraform.io/docs/commands/output.html output command].
 +
 +
<pre>
 +
resource "aws_instance" "example" {
 +
  ami          = "${lookup(var.AMIS, var.AWS_REGION)}"
 +
  instance_type = "t2.micro"
 +
}
 +
 +
output "ip" {
 +
  value = "${aws_instance.example.public_ip}"
 +
}
 +
</pre>
 +
 +
You can refer to any attribute by specifying the following elements in your variable:
 +
* The resource type (e.g., <code>aws_instance</code>)
 +
* The resource name (e.g., <code>example</code>)
 +
* The attribute name (e.g., <code>public_ip</code>)
 +
 +
See [https://www.terraform.io/docs/providers/aws/d/instance.html here] for a complete list of attributes for AWS EC2 instances.
 +
 +
* You can also use the attributes found in a script:
 +
<pre>
 +
resource "aws_instance" "example" {
 +
  ami          = "${lookup(var.AMIS, var.AWS_REGION)}"
 +
  instance_type = "t2.micro"
 +
 +
  provisioner "local-exec" {
 +
    command = "echo ${aws_instance.example.private_ip} >> private_ips.txt"
 +
  }
 +
}
 +
</pre>
 +
 +
===Terraform state===
 +
 +
* Terraform keeps the '''remote state''' of the infrastructure
 +
* It stores it in a file called <code>terraform.tfstate</code>
 +
* There is also a backup of the ''previous'' state in <code>terraform.tfstate.backup</code>
 +
* When you execute <code>`terraform apply`</code>, a new <code>terraform.tfstate</code> and backup is created
 +
* This is how Terraform keeps track of the remote state
 +
** If the remote state changes and you run <code>`terraform apply`</code> again, Terraform will make changes to meet the correct ''remote state'' again.
 +
** E.g., you manually '''terminate''' an instance that is managed by Terraform, after you run <code>`terraform apply`</code>, it will be '''started''' again.
 +
* You can keep the <code>terraform.tfstate</code> in version control (e.g., [[git]]).
 +
** This will give you a history of your <code>terraform.tfstate</code> file (which is just a big JSON file)
 +
** This allow you to collaborate with other team members (however, you can get conflicts when two or more people make changes at the same time)
 +
* Local state works well with simple setups. However, if your project involves multiple team members working on a larger setup, it is better to store your state remotely
 +
** The Terraform '''state''' can be saved remotely, using the [https://www.terraform.io/docs/backends/index.html backend functionality] in Terraform.
 +
* The default state is a '''local backend''' (the local Terraform state file)
 +
* Other backends include:
 +
** AWS S3 (with a locking mechanism using DynamoDB)
 +
** Consul (with locking)
 +
** Terraform Enterprise (the commercial solution)
 +
* Using the backend functionality has definite benefits:
 +
** Working in a team, it allows for collaboration (the remote state will always be available for the whole team)
 +
** The state file is not stored locally and possible sensitive information is only stored in the remote state
 +
** Some backends will enable '''remote operations'''. The <code>`terraform apply`</code> will then run completely remotely. These are called [https://www.terraform.io/docs/backends/types/index.html enhanced backends].
  
 
==Bash completion==
 
==Bash completion==

Revision as of 19:48, 26 April 2018

Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers as well as custom in-house solutions. It is a popular tool in DevOps.

Introduction

  • Infrastructure as Code
  • Used for the automation of your infrastructure
  • It keeps your infrastructure in a certain state (compliant)
    • E.g., 2 web instances and 2 volumes and 1 load balancer
  • It makes your infrastructure auditable
    • That is, you can keep your infrastructure change history in a version control system (e.g., git)

A high-level difference and/or reason to use Terraform over CAPS (Chef, Ansible, Puppet, Salt) is that these others have a focus on automating the installation and configuration of software (i.e., keeping the machines in compliance and in a certain state). Terraform, however, can automate provisioning of the infrastructure itself (e.g., in AWS or Google). One can, of course, do the same with, say, Ansible. However, Terraform really shines in infrastructure management and automation.

Examples

Basic example #1

The following is a super simple example of how to use Terraform to spin up a single AWS EC2 instance.

  • Create a working directory for your Terraform project:
$ mkdir ~/dev/terraform
  • Create a Terraform file describing the AWS EC2 instance to create:
$ cat << EOF >> instance.tf
provider "aws" {
  access_key = "<REDACTED>"
  secret_key = "<REDACTED>"
  region     = "us-west-2"
}

resource "aws_instance" "xtof-terraform" {
  ami           = "ami-a042f4d8"  # CentOS 7.4
  instance_type = "t2.micro"
}
EOF
  • Initialize your Terraform working directory:
$ terraform init
  • Create your EC2 instance:
$ terraform plan
$ terraform apply

Note: A better method to use is: 

$ terraform plan -out myinstance.terraform
$ terraform apply myinstance.terraform

By using the two separate above commands, Terraform will first show you what changes it will make without doing the actual changes. The second command will ensure that only the changes you saw on screen are applied. If you would just use terraform apply, more changes could have been added, because the remote infrastructure can change or files could have been edited (e.g., by someone else on your team). In short, always use the plan/apply file method.

  • Destroy the above instance:
$ terraform destroy

Basic example #2

The following expounds upon what we did in "Basic example #1", except we are building a more "Best Practices" approach. We will continue to build these examples.

  • Create a working directory (aws.create_ec2_instance) with the following files:
aws.create_ec2_instance/
├── .gitignore
├── instance.tf
├── provider.tf
├── terraform.tfvars
└── vars.tf

$ cat << EOF > .gitignore
*/terraform.tfvars
*/terraform.tfstate
*/terraform.tfstate.backup
*/.terraform

The contents of each of the above files should look like the following: 

$ cat << EOF >> instance.tf 
resource "aws_instance" "example" {
  ami           = "${lookup(var.AMIS, var.AWS_REGION)}"
  instance_type = "t2.micro"
}
EOF

$ cat << EOF >> provider.tf 
provider "aws" {
  access_key = "${var.AWS_ACCESS_KEY}"
  secret_key = "${var.AWS_SECRET_KEY}"
  region = "${var.AWS_REGION}"
}
EOF

$ cat << EOF >> terraform.tfvars 
AWS_ACCESS_KEY = "<REDACTED>"
AWS_SECRET_KEY = "<REDACTED>"
EOF

$ cat << EOF >> vars.tf 
variable "AWS_ACCESS_KEY" {}
variable "AWS_SECRET_KEY" {}
variable "AWS_REGION" {
  default = "us-west-2"
}
variable "AMIS" {
  type = "map"
  default = {
    us-west-2 = "ami-b2d463d2"
    us-east-1 = "ami-13be557e"
    eu-west-1 = "ami-0d729a60"
  }
}
EOF
  • Initialize the Terraform working directory:
$ terraform init
  • Now, "plan" your execution with:
$ terraform plan -out myinstance.terraform
...
+ aws_instance.example
    ami:                         "ami-b2d463d2"
    associate_public_ip_address: "<computed>"
    availability_zone:           "<computed>"
    ebs_block_device.#:          "<computed>"
    ephemeral_block_device.#:    "<computed>"
    instance_state:              "<computed>"
    instance_type:               "t2.micro"
    key_name:                    "<computed>"
    network_interface_id:        "<computed>"
    placement_group:             "<computed>"
    private_dns:                 "<computed>"
    private_ip:                  "<computed>"
    public_dns:                  "<computed>"
    public_ip:                   "<computed>"
    root_block_device.#:         "<computed>"
    security_groups.#:           "<computed>"
    source_dest_check:           "true"
    subnet_id:                   "<computed>"
    tenancy:                     "<computed>"
    vpc_security_group_ids.#:    "<computed>"

Plan: 1 to add, 0 to change, 0 to destroy.
  • Now, "apply" (or actually create the EC2 instance):
$ terraform apply myinstance.terraform

Concepts

Provisioners

File uploads
resource "aws_instance" "example" {
  ami           = "${lookup(var.AMIS, var.AWS_REGION)}"
  instance_type = "t2.micro"

  provisioner "file" {
    source      = "app.conf"
    destination = "/etc/myapp.conf"
  }
}
Connection
# Copies the file as the instance_username user using SSH
provisioner "file" {
  source      = "conf/myapp.conf"
  destination = "/etc/myapp.conf"

  connection {
    type     = "ssh"
    user     = "${var.instance_username}"
    password = "${var.instance_password}"
  }
}
  • Copy a script to the instance and execute it:
resource "aws_key_pair" "mykey" {
  key_name   = "christoph-aws-key"
  #public_key = "ssh-rsa my-public-key"
  public_key = "${file("${var.PATH_TO_PUBLIC_KEY}")}"
}

resource "aws_instance" "example" {
  ami           = "${lookup(var.AMIS, var.AWS_REGION)}"
  instance_type = "t2.micro"
  key_name      = "${aws_key_pair.mykey.key_name}"

  provisioner "file" {
    source      = "src/script.sh"
    destination = "/tmp/script.sh"
  }
  provisioner "remote-exec" {
    inline = [
      "chmod +x /tmp/script.sh",
      "sudo /tmp/script.sh"
    ]
  }

  connection {
    type        = "ssh"
    user        = "${var.instance_username}"
    private_key = "${file("${var.PATH_TO_PRIVATE_KEY}")}"
  }
}

Outputs

Outputs define values that will be highlighted to the user when Terraform applies, and can be queried easily using the output command. 

resource "aws_instance" "example" {
  ami           = "${lookup(var.AMIS, var.AWS_REGION)}"
  instance_type = "t2.micro"
}

output "ip" {
  value = "${aws_instance.example.public_ip}"
}

You can refer to any attribute by specifying the following elements in your variable:

  • The resource type (e.g., aws_instance)
  • The resource name (e.g., example)
  • The attribute name (e.g., public_ip)

See here for a complete list of attributes for AWS EC2 instances.

  • You can also use the attributes found in a script:
resource "aws_instance" "example" {
  ami           = "${lookup(var.AMIS, var.AWS_REGION)}"
  instance_type = "t2.micro"

  provisioner "local-exec" {
    command = "echo ${aws_instance.example.private_ip} >> private_ips.txt"
  }
}

Terraform state

  • Terraform keeps the remote state of the infrastructure
  • It stores it in a file called terraform.tfstate
  • There is also a backup of the previous state in terraform.tfstate.backup
  • When you execute `terraform apply`, a new terraform.tfstate and backup is created
  • This is how Terraform keeps track of the remote state
    • If the remote state changes and you run `terraform apply` again, Terraform will make changes to meet the correct remote state again.
    • E.g., you manually terminate an instance that is managed by Terraform, after you run `terraform apply`, it will be started again.
  • You can keep the terraform.tfstate in version control (e.g., git).
    • This will give you a history of your terraform.tfstate file (which is just a big JSON file)
    • This allow you to collaborate with other team members (however, you can get conflicts when two or more people make changes at the same time)
  • Local state works well with simple setups. However, if your project involves multiple team members working on a larger setup, it is better to store your state remotely
  • The default state is a local backend (the local Terraform state file)
  • Other backends include:
    • AWS S3 (with a locking mechanism using DynamoDB)
    • Consul (with locking)
    • Terraform Enterprise (the commercial solution)
  • Using the backend functionality has definite benefits:
    • Working in a team, it allows for collaboration (the remote state will always be available for the whole team)
    • The state file is not stored locally and possible sensitive information is only stored in the remote state
    • Some backends will enable remote operations. The `terraform apply` will then run completely remotely. These are called enhanced backends.

Bash completion

 $ cat << EOF >> /etc/bash_completion.d/terraform
_terraform()
{
   local cmds cur colonprefixes
   cmds="apply destroy fmt get graph import init \
      output plan push refresh remote show taint \
      untaint validate version state"

   COMPREPLY=()
   cur=${COMP_WORDS[COMP_CWORD]}
   # Work-around bash_completion issue where bash interprets a colon
   # as a separator.
   # Work-around borrowed from the darcs work-around for the same
   # issue.
   colonprefixes=${cur%"${cur##*:}"}
   COMPREPLY=( $(compgen -W '$cmds'  -- $cur))
   local i=${#COMPREPLY[*]}
   while [ $((--i)) -ge 0 ]; do
      COMPREPLY[$i]=${COMPREPLY[$i]#"$colonprefixes"}
   done

        return 0
} &&
complete -F _terraform terraform
EOF

External links

Retrieved from "http://wiki.christophchamp.com/index.php?title=Terraform&oldid=7227"