Amazon CloudFormation Part 1: What is it?
This is post 1 of 2 in the series “Amazon CloudFormation”
- Amazon CloudFormation Part 1: What is it?
- AWS CloudFormation Tutorial Part 2: Creating a CloudFormation Template
In this post, we are going to talk about Amazon CloudFormation. According to Amazon’s official site, CloudFormation “provides a common language for you to describe and provision all the infrastructure resources in your cloud environment.” In other words, it is a kind of language that allows engineers to describe, using just code, what Amazon hardware and software resources meet their infrastructure needs.
This new way of instantiating resources on the Cloud is part of a new approach called Infrastructure as Code (IAC), this technique allows data centers to be created by defining the infrastructure through files with code that can be read by machines instead of using physical hardware or tools. This IT management process can handle either physical or virtual resources because its main goal is to promote declarative approaches and the use of version control systems to improve the maintainability of the IT environment.
Now we can talk about the CloudFormation workflow, which services are available on it, the structure of a template, and the advantages of using this kind of service instead of the conventional approach.
What is CloudFormation?
In the image below, we can see the big picture of how CloudFormation works:
How Cloudformation works (Amazon)
First, you need to code from scratch the resources you need for your development, QA or production environment. This can be done using two formats: YAML or JSON. In this blog series, we are just going to use JSON, but YAML is pretty similar. As you can imagine, Amazon already has thousands of template snippets for its most common resources (VPN, EC2, RDS, S3) and some predefined infrastructure templates you can modify or use according to your needs.
After that, you need to log in to your Amazon account (we are going to see the whole creation process in the next post: Amazon Cloudformation Part 2: Creating a CloudFormation Template) and upload the template using one of several ways: uploading your own template with a S3 bucket, uploading it from your local machine or using a predefined template provided by Amazon.
The next step is to execute the template inside CloudFormation. You can do that in three ways: the CloudFormation console, the command line tools provided by Amazon, or any other API that allows the execution of CloudFormation commands. This step verifies if the template has the correct format and syntax, and that nothing will break the creation process. When the creation process starts, you can check the entire process and verify if the template was successfully created.
When CloudFormation finishes the process, you can see in all the service consoles that everything you wanted to create is already there. If you want to understand how this works, the Amazon official video is a great start.
Services Available on CloudFormation
As you probably already know, Amazon offers a wide set of Cloud and global solutions worldwide, from computing and storage to the Internet of things and game development. You can see a complete list of Amazon services here.
Some services Amazon provides to its customers
Not all services are currently supported by CloudFormation, but Amazon is working on adding more and more resources to the common language. Currently, the supported resources are pretty good, so very different projects and requirements can be created using this service. On the official CloudFormation site, you can see the list of supported resources. The image below is a helpful way to imagine the current available resources:
General Template Structure
Now we are getting closer to writing code; that is our real goal. In order to create our first CloudFormation template, we need to understand the template’s syntax and properties. The following code snippet was taken from the Template Anatomy section on the Amazon CloudFormation site:
JSON Template Structure
{ "AWSTemplateFormatVersion" : "version date", "Description" : "JSON string", "Metadata" : { template metadata }, "Parameters" : { set of parameters }, "Mappings" : { set of mappings }, "Conditions" : { set of conditions }, "Transform" : { set of transforms }, "Resources" : { set of resources }, "Outputs" : { set of outputs } }
The AWSTemplateFormatVersion field (optional) identifies the capabilities of the template, and currently the only valid value is “2010-09-09.” This property receives only a string value. The Description also receives a string between 0 to 1024 bytes in length, and it is a section for commenting on the main features implemented in the template.
The property Metadata is also optional and allows you to define extra information about resources. This is the only section you cannot update during a stack update (we are going to see this in detail in Amazon CloudFormation Part 3: Advanced CloudFormation). This property receives a JSON object. An example of this property is here:
Metadata example
"Metadata" : { "Instances" : {"Description" : "Information about the instances"}, "Databases" : {"Description" : "Information about the databases"} }
The Parameters section (optional) allows you to customize or make your template more generic by accepting multiple options during the creation process; for example, a list of available EC2 instances. This property also receives a JSON object:
Parameters example
"Parameters" : { "InstanceTypeParameter" : { "Type" : "String", "Default" : "t2.micro", "AllowedValues" : ["t2.micro", "m1.small", "m1.large"], "Description" : "Enter t2.micro, m1.small, or m1.large. Default is t2.micro." } }
The Mappings property is also optional and allows you to define a group of key-value pairs to use with the resources; for example, a list of available regions. This also receives a JSON object:
Mappings example
"Mappings" : { "RegionMap" : { "us-east-1" : { "HVM64" : "ami-0ff8a91507f77f867"}, "us-west-1" : { "HVM64" : "ami-0bdb828fd58c52235"}, "eu-west-1" : { "HVM64" : "ami-047bb4163c506cd98"}, "ap-southeast-1" : { "HVM64" : "ami-08569b978cc4dfa10"}, "ap-northeast-1" : { "HVM64" : "ami-06cd52961ce9f0d85"} } }
Transform allows you to declare macros to be executed during the creation process. This section is optional and not very used in common cases. Resources is the only required section on the template and the most important. Here, you declare all the resources you need for your infrastructure as a JSON object:
Resources example
"Resources" : { "MyEC2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "ImageId" : "ami-0ff8a91507f77f867" } } }
The Outputs section is also a JSON object in which you can declare values you want to import to other stacks (we are going to see this in detail in the future post Amazon Cloudformation Part 3: Advanced CloudFormation), print after a successful creation (to check if everything was done as expected), or just see on the CloudFormation console:
Outputs example
"Outputs" : { "InstanceID" : { "Description": "The Instance ID", "Value" : { "Ref" : "EC2Instance" } } }
Finally, Conditions provides a way to create or modify resources according to a set of conditions you define. In the following example, we can see how this works with all the other template sections:
Conditions example
{ "AWSTemplateFormatVersion" : "2010-09-09", "Mappings" : { "RegionMap" : { "us-east-1" : { "AMI" : "ami-0ff8a91507f77f867", "TestAz" : "us-east-1a" }, "us-west-1" : { "AMI" : "ami-0bdb828fd58c52235", "TestAz" : "us-west-1a" }, "us-west-2" : { "AMI" : "ami-a0cfeed8", "TestAz" : "us-west-2a" } } }, "Parameters" : { "EnvType" : { "Description" : "Environment type.", "Default" : "test", "Type" : "String", "AllowedValues" : ["prod", "test"], "ConstraintDescription" : "must specify prod or test." } }, "Conditions" : { "CreateProdResources" : {"Fn::Equals" : [{"Ref" : "EnvType"}, "prod"]} }, "Resources" : { "EC2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]} } }, "MountPoint" : { "Type" : "AWS::EC2::VolumeAttachment", "Condition" : "CreateProdResources", "Properties" : { "InstanceId" : { "Ref" : "EC2Instance" }, "VolumeId" : { "Ref" : "NewVolume" }, "Device" : "/dev/sdh" } }, "NewVolume" : { "Type" : "AWS::EC2::Volume", "Condition" : "CreateProdResources", "Properties" : { "Size" : "100", "AvailabilityZone" : { "Fn::GetAtt" : [ "EC2Instance", "AvailabilityZone" ]} } } }, "Outputs" : { "VolumeId" : { "Value" : { "Ref" : "NewVolume" }, "Condition" : "CreateProdResources" } } }
Example taken from Amazon’s official site
In the example, we declared a mapping with some regions available on Amazon; also, we defined a parameter to choose between a “prod” or test environment. In the Conditions section, we defined the conditional EnvType == prod, so just when the environment is “production,” you will create the resources that have this condition in their properties. Inside the “Resources” section, we created an “EC2,” “VolumeAttachment” and “Volume,” but just these last two resources will be created when the environment is “production” because they have “Condition” : “CreateProdResources”.
Finally, in the “Outputs” section, we printed the “newVolume” reference as part of the “VolumeId” key, together with the result of the condition “CreateProdResources.”
Conclusions and Findings
CloudFormation has some benefits over the conventional creation of resources on Amazon. Some of them are general to Amazon, and others were found as process results in our company:
• Modeling all your infrastructure in a single text file.
• Safe building and rebuilding of resources.
• Version control.
• Flexibility of software to design your infrastructure; you just need a code editor.
• Billing control; the engineer in charge of resources can have more control over billable resources. They just need to approve templates with the required resources.
• Reuse of generic templates defined inside the company (internal and external).
• Enhancement of architectural and operational best practices.
In the next blog post, Amazon CloudFormation Part 2: Creating a Cloudformation Template, we are going to make our first real template and look at the entire creation process step by step.
