Automating Your AWS Workloads

Lesson purpose

Automation is a core best practice for cloud architecture. Tasks as small as patching one Linux instance or as large as building a complete multitier environment can be written as code and repeated quickly and consistently. This lesson explains the two styles of automation, infrastructure as code with CloudFormation, the AWS developer tools that automate building, testing, and deploying software (CI/CD), EC2 Auto Scaling, and configuration management with Systems Manager and OpsWorks. It supports CLF-C02 objectives 1.2, 3.1, 3.3, 3.7, and 3.8.

Learning objectives

Explain the benefits of automation and why it still depends on people.
Distinguish the imperative and declarative approaches, with examples of each.
Define infrastructure as code (IaC) and its benefits.
Describe CloudFormation templates, parameters, stacks, stack updates, change sets, stack policies, and drift detection.
Compare CloudFormation with scripted AWS CLI commands.
Describe CodeCommit, CodeBuild, CodeDeploy, CodePipeline, CodeStar, and CodeArtifact.
Define continuous integration and continuous delivery.
Explain EC2 Auto Scaling: launch templates, groups, desired capacity, self-healing, and dynamic, scheduled, and predictive scaling.
Describe Systems Manager features and the OpsWorks options for configuration management.

1. Why automate?

Benefits of automation:

Rapid testing and experimentation — build a fresh environment in minutes, try something, and tear it down.
Reduced expenses — less manual labor, and resources run only when needed.
Minimized human error — code does the same thing the same way every time.
Safe, complete rollbacks — undo changes by returning to a known-good version of the code.

A common misconception is that automation replaces people. It actually takes considerable human skill to design and maintain; the payoff is that people spend more time on productive work and less time fixing avoidable problems.

Automation is not all-or-nothing. Start where it helps most and expand.

2. Imperative vs. declarative automation

	Imperative	Declarative
You write	The exact steps (how)	The desired end result (what)
Examples	Bash or PowerShell scripts, AWS CLI commands, EC2 user data, Systems Manager Command documents	CloudFormation templates, Puppet modules, Chef recipes
Who figures out the steps	You	The service

Imperative example — create a security group step by step with the CLI:

aws ec2 create-security-group --group-name web-sg --description "Allow HTTP"
aws ec2 authorize-security-group-ingress --group-name web-sg --protocol tcp --port 80 --cidr 0.0.0.0/0

The first command creates an empty security group in the default VPC.
The second adds one inbound rule allowing HTTP (TCP port 80) from anywhere.
If you later want a change, you must work out and run the right commands yourself.

Declarative example — describe the finished security group and let CloudFormation build it:

AWSTemplateFormatVersion: "2010-09-09"
Description: Web server security group
Parameters:
  EnvironmentTag:
    Type: String
    Default: Test
Resources:
  WebSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Allow HTTP
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: 0.0.0.0/0
      Tags:
        - Key: Environment
          Value: !Ref EnvironmentTag

Parameters defines an input (EnvironmentTag) with a default value, so the template can be customized without editing it.
Resources lists what should exist. WebSecurityGroup is a logical name; Type says what kind of AWS resource it is.
Properties describes the desired configuration: one inbound HTTP rule and an Environment tag.
!Ref EnvironmentTag inserts the parameter’s value.

The two approaches are not opposites: a declarative service ultimately runs imperative steps for you. Declarative code hides those steps behind a results-focused description.

Infrastructure as code (IaC)

Infrastructure as code means defining infrastructure and configuration in code files. Because a machine runs the code, human error drops. The code also serves as documentation and can be kept in version control to track and roll back changes.

3. AWS CloudFormation

CloudFormation creates and configures AWS resources from code that describes what you want.

Templates

Text files written in the CloudFormation language, in JSON or YAML.
Describe the resources to create, so they double as infrastructure documentation; store them in S3 or Git for version history.
Reusable: one template can build identical dev, test, and production environments that stay separate (CloudFormation adds unique identifiers to resource names to avoid conflicts).
Parameters accept inputs, such as a tag value. A parameter with a default is optional; without one, the user must provide a value.

Design principle to highlight: test environments should mirror production as closely as possible, and templates make that easy.

Stacks

Launching a template creates a stack (its name must be unique in the account and Region) — a container for all the template’s resources, managed as one unit.
Deleting a stack deletes all of its resources, which is ideal for temporary test environments and prevents forgotten resources from running up charges.

Changing stacks safely

Stack update — edit the template and update the stack; CloudFormation works out what to add, change, or remove, including dependent resources.
Change set — a preview of exactly what an update would change, so you can review before executing (or discard).
Stack policy — a JSON document that prevents updates to some or all resources, guarding critical resources from accidental changes. It can be temporarily overridden when an update is truly needed.
Events — each stack keeps a timestamped record of creates, updates, and deletes.
Drift detection — CloudFormation does not stop someone from changing a stack’s resources manually in the console. Drift detection finds and reports those out-of-band changes.

CloudFormation vs. scripting with the AWS CLI

A CLI script can build resources quickly, but the resources are not grouped in a stack and updates are hard: you must know how to change each resource without breaking others. With CloudFormation you change the template or parameters, and CloudFormation figures out the steps.

Exercise 11.1 (updated)

The book uses CloudFormation Designer to view the sample LAMP stack template. AWS has replaced Designer with AWS Infrastructure Composer, which shows the same idea: a visual diagram of the resources (here, an EC2 instance and a security group) linked to the template code. Open a sample template, select the instance in the diagram, and find the matching section of code.

4. AWS developer tools and CI/CD

The developer tools automate getting new code from a developer’s laptop to production. They can also deploy infrastructure code, not only applications.

Source control: CodeCommit

Hosts private Git repositories (“repos”) that integrate with AWS. Git is a version control system for source code, templates, documents, or any file.
Every commit is kept, so older versions can be restored.
Developers clone code locally, change it, and commit it back.
Differencing shows exactly what changed between versions — the key difference between version control and simple file storage such as S3.

Modern note: AWS’s availability of CodeCommit to new customers has changed in recent years, and many teams use GitHub, GitLab, or Bitbucket as the source for AWS pipelines instead.

Build and test: CodeBuild

A fully managed build service. A build prepares code for deployment: running tests, checking syntax, compiling into executables or installers, or creating Docker images and AMIs.
Instructions live in a build specification (buildspec) file stored with the source code.
Sources: CodeCommit, GitHub, Bitbucket, or S3. Output artifacts go to S3.
Each build runs in a fresh, isolated build environment (an OS plus a Docker image with language runtimes and tools) that is discarded afterward, so leftovers never affect the next build. Preconfigured environments exist for Java, Ruby, Python, Go, Node.js, Android, .NET, PHP, and Docker, or you can supply your own.
Several compute sizes are available (the book lists small, medium, and large); all support Linux, and larger sizes support Windows.

Deploy: CodeDeploy

Deploys applications to EC2 instances, on-premises servers, ECS, and Lambda, pulling files from S3, GitHub, or Bitbucket, guided by an application specification (appspec) file.

EC2/on-premises: requires the CodeDeploy agent (tested on Amazon Linux, Ubuntu, Windows Server, and RHEL). Target instances by tag, Auto Scaling group, or manual selection, and deploy one at a time, half at a time, all at once, or in between.
Deployment types:
- In-place — stop the app on existing instances, deploy the new version, restart.
- Blue/green — deploy to a new set of instances (for example, a copy of the Auto Scaling group), shift load balancer traffic to them, and optionally retire the old ones. Rollback is easy because the old (“blue”) environment still exists.
ECS: deploys new container images and shifts load balancer traffic from old containers to new ones (EC2 and Fargate launch types).
Lambda: publishes a new function version and shifts traffic gradually or all at once.

Orchestrate: CodePipeline

CodePipeline automates the whole path from source to production as a series of stages (at least a source stage and a deploy stage). Each stage has actions that can run in sequence or in parallel:

Source — S3, CodeCommit, GitHub, or Amazon ECR (container images).
Build and Test — CodeBuild or third-party tools such as Jenkins, CloudBees, and TeamCity.
Approval — a manual sign-off, allowed anywhere after the source stage.
Deploy — CloudFormation (infrastructure), CodeDeploy, ECS, or S3 (for example, a static website).
Invoke — runs a Lambda function.

CodePipeline fills gaps between services. Example from the book: CodeDeploy can’t deploy directly from CodeCommit, but a pipeline can take code from CodeCommit, test it with CodeBuild, package it to S3, wait for approval, and then call CodeDeploy.

CI and CD

Continuous integration (CI) — developers commit code to a shared repository many times a day, and every commit is automatically built and tested, so bugs are caught early.
Continuous delivery (CD) — extends CI so every tested change is packaged and ready to deploy; production deployment happens with a push of a button after manual approval. (Continuous deployment removes the manual approval.)

CodeStar and CodeArtifact

CodeStar combined the developer tools into project templates (Ruby on Rails, Node.js, Express.js, Django) with team roles (owner, contributor, viewer), a dashboard, and browser-based editing in Cloud9. AWS discontinued CodeStar in 2024, and Cloud9 is closed to new customers; know the concept for older material.
CodeArtifact — a managed repository for software packages (artifacts) such as libraries. It works with pip, npm, yarn, NuGet, and Maven, can share packages privately or publicly, and tracks downloads. It also reduces supply-chain risk: keep vetted copies of public libraries instead of pulling the latest (possibly compromised) version from the internet.

5. EC2 Auto Scaling

Auto Scaling launches and terminates EC2 instances automatically so you have just enough capacity — supporting the design principle “stop guessing capacity.”

Launch templates and launch configurations

Both define what an instance looks like: AMI, instance type, storage, and user data scripts that install software at launch.

Launch template	Launch configuration
Newer and recommended	Legacy
Can be versioned and modified	Cannot be changed after creation
Can launch instances without Auto Scaling	Auto Scaling only

Modern note: AWS has deprecated launch configurations; use launch templates.

Auto Scaling groups

Instances belong to an Auto Scaling group, which can register them automatically with an Application Load Balancer target group so traffic is spread evenly.
Minimum and maximum sizes set the limits; desired capacity (between them) is how many instances the group maintains.
Self-healing: if an instance fails an EC2 health check (instance running and reachable) or an ELB health check (the application responds), Auto Scaling terminates it and launches a replacement.

Scaling actions

Dynamic scaling — scale out (add instances) when demand rises and scale in (remove instances) when it falls, based on a metric such as average CPU or number of users.
Scheduled scaling — change capacity at set times for predictable peaks (for example, class registration week).
Predictive scaling — analyzes past traffic (at least one day’s data) to forecast peaks and schedules capacity ahead of time.

Example: min 2, desired 2, max 6 across two AZs. CPU climbs above 70 percent, so the group scales out to 4; at night it scales back to 2. If one instance fails its health check, it is replaced automatically.

6. Configuration management

Configuration management keeps systems in a known, consistent state — correct operating system settings, installed software, and patches — and reports when they drift.

AWS Systems Manager (imperative)

Requires the SSM Agent on managed EC2 or on-premises instances.

Command documents — scripts (Bash or PowerShell) that run once, on a schedule, or on a trigger such as a new instance launch: install software, apply patches, or collect an inventory.
Configuration Compliance — reports whether instances match the desired state (for example, a required application version or current patches).
Automation documents — automate AWS administrative tasks otherwise done in the console or CLI: snapshot an EBS volume, launch or terminate instances, create a CloudFormation stack, or build an AMI.
Distributor — packages software as a ZIP archive in S3 and installs it on instances.
AppConfig — manages, validates, and deploys application configuration data, such as feature flags that turn a new feature on for some servers or users first. Configurations can be stored in S3, Git, or Systems Manager; validation rules catch bad JSON before it’s deployed; applications poll AppConfig for updates.

AWS OpsWorks (declarative)

OpsWorks brought the Chef and Puppet configuration management platforms to AWS:

OpsWorks for Puppet Enterprise and OpsWorks for Chef Automate — managed Puppet master or Chef servers that configure agent-equipped EC2 and on-premises nodes using declarative Puppet modules or Chef recipes. AWS ran the servers; you still needed Chef or Puppet expertise.
OpsWorks Stacks — application-focused: a stack of layers (for example, an application layer, a database layer, and a load balancer). OpsWorks layers define sets of instances with auto-healing and load- or time-based scaling and use Chef Solo on each instance; service layers connect RDS, ELB, or ECS clusters. An OpsWorks stack is not a CloudFormation stack.

Modern status: AWS ended support for all three OpsWorks services in 2024. Teams now use Systems Manager, or run Chef, Puppet, or Ansible themselves. The CLF-C02 exam and older material may still mention OpsWorks.

Which approach does each service use?

Imperative	Declarative
Bash/PowerShell scripts, AWS CLI, EC2 user data, Systems Manager, CodeBuild, CodeDeploy	CloudFormation, OpsWorks (Puppet modules, Chef recipes)

Classroom application

Imperative vs. declarative: Show the CLI commands and the YAML template side by side. Ask students what happens if each is run twice, and what each requires to change port 80 to 443.
Stack lifecycle demo: Create a stack from a small template, change a parameter with a change set, view the events tab, make a manual change in the console and run drift detection, then delete the stack and confirm every resource is gone.
Pipeline drawing: Have students diagram a pipeline for a static website: GitHub source → CodeBuild tests → manual approval → deploy to S3. Label CI and CD.
Auto Scaling scenario: Set min, desired, and max for a campus website that is quiet at night, busy during the day, and very busy during registration week. Choose dynamic, scheduled, or predictive scaling for each pattern.
Configuration drift: Discuss how Systems Manager Patch Manager and Configuration Compliance would find servers missing a security patch.

Common misconceptions

“Automation replaces people.” It requires skilled people to build and maintain; it frees them for more valuable work.
“Declarative and imperative are opposites.” Declarative tools run imperative steps behind the scenes.
“CloudFormation prevents manual changes.” It doesn’t; drift detection reports them, and stack policies only block stack updates.
“Deleting a stack leaves the resources behind.” Deleting a stack deletes its resources (unless a resource is set to be retained).
“Version control is just file storage.” Version control tracks every change and shows differences; S3 stores objects.
“Continuous delivery means code goes straight to production.” Continuous delivery includes a manual approval; continuous deployment does not.
“Auto Scaling only adds servers.” It also removes them and replaces unhealthy ones.
“An OpsWorks stack is a CloudFormation stack.” They are unrelated despite the shared word.

Lesson summary

Automation turns tasks into code, bringing speed, lower cost, fewer errors, and safe rollbacks. Imperative automation spells out each step (scripts, the CLI, user data, Systems Manager); declarative automation describes the end state and lets the service work out the steps (CloudFormation, Puppet, Chef). Infrastructure as code makes environments repeatable, documented, and version-controlled. CloudFormation builds stacks from JSON or YAML templates with parameters, and supports stack updates, change sets, stack policies, event history, and drift detection; deleting a stack removes all its resources. The developer tools automate software delivery: CodeCommit for Git repositories, CodeBuild for builds and tests, CodeDeploy for in-place and blue/green deployments to EC2, on-premises servers, ECS, and Lambda, and CodePipeline to orchestrate stages into CI/CD pipelines, with CodeArtifact storing vetted packages. EC2 Auto Scaling uses launch templates and Auto Scaling groups to maintain desired capacity between a minimum and maximum, replaces unhealthy instances, and scales dynamically, on a schedule, or predictively. Configuration management keeps systems consistent: Systems Manager uses Command and Automation documents, Configuration Compliance, Distributor, and AppConfig, while the now-retired OpsWorks services brought declarative Chef and Puppet management to AWS.