Difference between revisions of "AWS/DevOps"

From Christoph's Personal Wiki
Jump to: navigation, search
(→‎Domain 1: SDLC Automation)
(→‎SDLC Automation)
Line 52: Line 52:
 
==SDLC Automation==
 
==SDLC Automation==
  
−
* Introduction
+
===Introduction===
−
* What is CI/CD?
+
 
−
* AWS CodeCommit
+
===What is CI/CD?===
−
* AWS CodeBuild
+
 
−
* AWS CodeDeploy
+
; The CI/CD Pipeline
 +
 
 
* AWS CodePipeline
 
* AWS CodePipeline
−
* Testing
+
** Source Stage
−
* Artifacts
+
*** AWS CodeCommit (think "git")
−
* Deployment Strategies
+
** Deploy Stage - Development
 +
*** AWS CodeDeploy -> EC2 instance
 +
** Deploy Stage - Production
 +
*** AWS CodeDeploy -> EC2 instance
 +
 
 +
===AWS CodeCommit===
 +
 
 +
===AWS CodeBuild===
 +
 
 +
* A fully managed build service
 +
* Compiles your code
 +
* Runs unit tests
 +
* Produces artifacts that are ready to deploy
 +
* Eliminates the need to provision/manage/scale your own build servers
 +
* Provides pre-packaged build environments
 +
* Allows you to build your own customized build environment
 +
* Scales automatically to meet your build requirements
 +
 
 +
; Benefits of CodeBuild
 +
* It is fully managed
 +
** You do not have to set up any build servers, nor patch, update, or maintain them.
 +
* It is on-demand
 +
** It automatically scales to meet your requirements. No more migrating to larger EC2 servers because your builds are taking too long. You only pay for the minutes (seconds?) you consume.
 +
* It is preconfigured
 +
** It comes with many pre-configured build environments for the most popular programming languages. You just need to configure it to use your build script.
 +
 
 +
: SEE: [https://docs.aws.amazon.com/codebuild/latest/userguide/troubleshooting.html AWS - Troubleshooting CodeBuild]
 +
 
 +
===AWS CodeDeploy===
 +
 
 +
; What is CodeDeploy?
 +
 
 +
* A fully managed deployment service that automates deployments to:
 +
** Amazon EC2 instances
 +
** On-premise instances
 +
** AWS Lambda functions
 +
* Makes it easier to:
 +
** Rapidly deploy new features
 +
** Update Lambda function versions
 +
** Avoid downtime during deployment
 +
** Handle the full complex deployment process without human intervention
 +
 
 +
<!--
 +
https://github.com/ACloudGuru/devops/blob/master/01/04/userdata.txt
 +
https://github.com/ACloudGuru/devops/raw/master/01/02/apetguru.zip
 +
-->
 +
 
 +
===AWS CodePipeline===
 +
 
 +
CodePipeline is the "CD" of CI/CD.
 +
 
 +
; Benefits
 +
 
 +
* Automatic
 +
** From the check-in of your code to deployment on to your service, CodePipeline takes care of it all.
 +
* Easy to set up
 +
** CodePipeline has no servers to provision, it is dead simple to configure and get working. There are pre-built plugins or you can roll your own.
 +
* Configurable
 +
** You can create, configure, and modify all stages of your software release process with ease. You can implement automated testing and customize the deployment process.
 +
 
 +
===Testing===
 +
 
 +
; Why do we test?
 +
 
 +
* Meet the requirements defined
 +
* Ensure the code performs in an accepatble period of time
 +
* Ensure the code is usable
 +
* Ensure the code responds correctly to all kinds of inputs
 +
* Achieves the result the programmer desired
 +
 
 +
; Types of testings (see Wikipedia)
 +
 
 +
; Automated testing
 +
 
 +
* Automatic execution oif test
 +
* Comparision of actual outcomes to predicted outcomes
 +
* Fast, continuos feedback
 +
* Immediate notifcation
 +
* Save resources
 +
 
 +
; Unit test example
 +
<pre>
 +
assert()
 +
</pre>
 +
 
 +
===Artifacts===
 +
 
 +
; What are artifacts?
 +
 
 +
An artifact is a product or by-product produced during the software development process.
 +
 
 +
For example:
 +
* Compiled binaries
 +
* Source code
 +
* Documentation
 +
* Use cases
 +
* Class diagrams
 +
 
 +
Artifacts are stored in S3 (note: this has ''nothing'' to do with AWS Artifact!)
 +
 
 +
===Deployment Strategies===
 +
 
 +
; Single Target Deployment (build -> target)
 +
* Use for small development projects, especially when legacy or non-highly-available infrastructure is involved.
 +
* When it is initiated, a new application version is installed on the target server.
 +
* A brief outage occurs during installation. There are no secondary servers, so testing is limited. Rollback involves removing the new version and install the previous.
 +
 
 +
; All-at-Once Deployment (build -> x2 targets)
 +
* Deployment happens in one step, just like single target deployment.
 +
* With this method, the destination is multiple targets.
 +
* More complicated than single target; often requiring orchestration tooling.
 +
* Shares negatives of single target. No ability to test, still has deployment outages, and less than ideal rollback.
 +
 
 +
; Minimum in-service Deployment (initial build stage -> t1 t2 t3 ...)
 +
* Deployment happens in multiple stages
 +
* Deployment happens to as many targets as possible while maintaining the minimum in-service targets.
 +
* A few moving parts, orchestration and health checks are required.
 +
* Allows automated testing, deployment targets are assessed and testsd prior to continuing.
 +
* Generally, no downtime.
 +
* Often quicker and less stages than a rolling deployment.
 +
 
 +
; Rolling Deployment
 +
* Deployment happens in multiple stages. Number of targets per stage is user-defined.
 +
* Moving parts; orchestration and health-checks are required.
 +
* Overall applicable health is not necessariliy maintained.
 +
* Can be the leasat efficient deployment time based on time-taken.
 +
* Allows automated testing; deployment targets are assessed and tested prior to continuing.
 +
* Generally, no downtime, assuming number of targets per run is not large neough to impact the application.
 +
* Can be paused, allowing limited multi-version testing (combined with small targets per stage).
 +
 
 +
; Blue/Green Deployment (aka Red/Black)
 +
* Requires advanced orchestration tooling
 +
* Carries significant cost - maintiang 2 environments for the duration of deployments.
 +
* Deployment process is rapid - entire environemnt (blue or green) is deployed all at once.
 +
* Cutover and migration is clean and controlled (e.g., DNS change)
 +
* Rollback is equally clean (e.g., DNS regression)
 +
* Health and performance of entire "green" environment can be tested prior to cutover.
 +
* using advanced template systems, such as CloudFormation, the entire process can be fully automated.
 +
 
 +
; Canary Deployment
 +
* Like Blue/Green, but keep blue active and route percentage of traffic to green
 +
* In AWS, use Route53 w/weighted round-robin
  
 
==Configuration Management and Infrastructure as Code==
 
==Configuration Management and Infrastructure as Code==

Revision as of 23:30, 18 February 2021

This article will cover topics related to the AWS Certified DevOps Engineer - Professional exam and certification.

Domains

The 6 domains outlined in the AWS blueprint for the certification include:

  1. Software Development LifeCycle (SDLC) Automation [22%]
  2. Configuration Management and Infrastructure as Code [19%]
  3. Monitoring and Logging [15%]
  4. Policies and Standards Automation [10%]
  5. Incident and Event Response [18%]
  6. High Availability, Fault Tolerance, and Disaster Recovery [16%]

Domain 1: SDLC Automation

  • 1.1 Apply concepts required to automate a CI/CD pipeline
  • 1.2 Determine source control strategies and how to implement them
  • 1.3 Apply concepts required to automate and integrate testing
  • 1.4 Apply concepts required to build and manage artifacts securely
  • 1.5 Determine deployment/delivery strategies (e.g., A/B, Blue/Green, Canary, Red/Black) and how to implement them using AWS Services

Domain 2: Configuration Management and Infrastructure as Code

  • 2.1 Determine deployment services based on deployment needs
  • 2.2 Determine application and infrastructure deployment models based on business needs
  • 2.3 Apply security concepts in the automation of resource provisioning
  • 2.4 Determine how to implement lifecycle hooks on a deployment
  • 2.5 Apply concepts required to manage systems using AWS configuration management tools and services

Domain 3: Monitoring and Logging

  • 3.1 Determine how to set up the aggregation, storage, and analysis of logs and metrics
  • 3.2 Apply concepts required to automate monitoring and event management of an environment
  • 3.3 Apply concepts required to audit, log, and monitor operating systems, infrastructures, and applications
  • 3.4 Determine how to implement tagging and other metadata strategies

Domain 4: Policies and Standards Automation

  • 4.1 Apply concepts required to enforce standards for logging, metrics, monitoring, testing, and security
  • 4.2 Determine how to optimize cost through automation
  • 4.3 Apply concepts required to implement governance strategies

Domain 5: Incident and Event Response

  • 5.1 Troubleshoot issues and determine how to restore operations
  • 5.2 Determine how to automate event management and alerting
  • 5.3 Apply concepts required to implement automated healing
  • 5.4 Apply concepts required to set up event-driven automated actions

Domain 6: High Availability, Fault Tolerance, and Disaster Recovery

  • 6.1 Determine appropriate use of multi-AZ versus multi-region architectures
  • 6.2 Determine how to implement high availability, scalability, and fault tolerance
  • 6.3 Determine the right services based on business needs (e.g., RTO/RPO, cost)
  • 6.4 Determine how to design and automate disaster recovery strategies
  • 6.5 Evaluate a deployment for points of failure

SDLC Automation

Introduction

What is CI/CD?

The CI/CD Pipeline
  • AWS CodePipeline
    • Source Stage
      • AWS CodeCommit (think "git")
    • Deploy Stage - Development
      • AWS CodeDeploy -> EC2 instance
    • Deploy Stage - Production
      • AWS CodeDeploy -> EC2 instance

AWS CodeCommit

AWS CodeBuild

  • A fully managed build service
  • Compiles your code
  • Runs unit tests
  • Produces artifacts that are ready to deploy
  • Eliminates the need to provision/manage/scale your own build servers
  • Provides pre-packaged build environments
  • Allows you to build your own customized build environment
  • Scales automatically to meet your build requirements
Benefits of CodeBuild
  • It is fully managed
    • You do not have to set up any build servers, nor patch, update, or maintain them.
  • It is on-demand
    • It automatically scales to meet your requirements. No more migrating to larger EC2 servers because your builds are taking too long. You only pay for the minutes (seconds?) you consume.
  • It is preconfigured
    • It comes with many pre-configured build environments for the most popular programming languages. You just need to configure it to use your build script.
SEE: AWS - Troubleshooting CodeBuild

AWS CodeDeploy

What is CodeDeploy?
  • A fully managed deployment service that automates deployments to:
    • Amazon EC2 instances
    • On-premise instances
    • AWS Lambda functions
  • Makes it easier to:
    • Rapidly deploy new features
    • Update Lambda function versions
    • Avoid downtime during deployment
    • Handle the full complex deployment process without human intervention


AWS CodePipeline

CodePipeline is the "CD" of CI/CD.

Benefits
  • Automatic
    • From the check-in of your code to deployment on to your service, CodePipeline takes care of it all.
  • Easy to set up
    • CodePipeline has no servers to provision, it is dead simple to configure and get working. There are pre-built plugins or you can roll your own.
  • Configurable
    • You can create, configure, and modify all stages of your software release process with ease. You can implement automated testing and customize the deployment process.

Testing

Why do we test?
  • Meet the requirements defined
  • Ensure the code performs in an accepatble period of time
  • Ensure the code is usable
  • Ensure the code responds correctly to all kinds of inputs
  • Achieves the result the programmer desired
Types of testings (see Wikipedia)
Automated testing
  • Automatic execution oif test
  • Comparision of actual outcomes to predicted outcomes
  • Fast, continuos feedback
  • Immediate notifcation
  • Save resources
Unit test example
assert()

Artifacts

What are artifacts?

An artifact is a product or by-product produced during the software development process.

For example:

  • Compiled binaries
  • Source code
  • Documentation
  • Use cases
  • Class diagrams

Artifacts are stored in S3 (note: this has nothing to do with AWS Artifact!)

Deployment Strategies

Single Target Deployment (build -> target)
  • Use for small development projects, especially when legacy or non-highly-available infrastructure is involved.
  • When it is initiated, a new application version is installed on the target server.
  • A brief outage occurs during installation. There are no secondary servers, so testing is limited. Rollback involves removing the new version and install the previous.
All-at-Once Deployment (build -> x2 targets)
  • Deployment happens in one step, just like single target deployment.
  • With this method, the destination is multiple targets.
  • More complicated than single target; often requiring orchestration tooling.
  • Shares negatives of single target. No ability to test, still has deployment outages, and less than ideal rollback.
Minimum in-service Deployment (initial build stage -> t1 t2 t3 ...)
  • Deployment happens in multiple stages
  • Deployment happens to as many targets as possible while maintaining the minimum in-service targets.
  • A few moving parts, orchestration and health checks are required.
  • Allows automated testing, deployment targets are assessed and testsd prior to continuing.
  • Generally, no downtime.
  • Often quicker and less stages than a rolling deployment.
Rolling Deployment
  • Deployment happens in multiple stages. Number of targets per stage is user-defined.
  • Moving parts; orchestration and health-checks are required.
  • Overall applicable health is not necessariliy maintained.
  • Can be the leasat efficient deployment time based on time-taken.
  • Allows automated testing; deployment targets are assessed and tested prior to continuing.
  • Generally, no downtime, assuming number of targets per run is not large neough to impact the application.
  • Can be paused, allowing limited multi-version testing (combined with small targets per stage).
Blue/Green Deployment (aka Red/Black)
  • Requires advanced orchestration tooling
  • Carries significant cost - maintiang 2 environments for the duration of deployments.
  • Deployment process is rapid - entire environemnt (blue or green) is deployed all at once.
  • Cutover and migration is clean and controlled (e.g., DNS change)
  • Rollback is equally clean (e.g., DNS regression)
  • Health and performance of entire "green" environment can be tested prior to cutover.
  • using advanced template systems, such as CloudFormation, the entire process can be fully automated.
Canary Deployment
  • Like Blue/Green, but keep blue active and route percentage of traffic to green
  • In AWS, use Route53 w/weighted round-robin

Configuration Management and Infrastructure as Code

  • Introduction
  • AWS CloudFormation
  • AWS CloudFormation Intrinsic Functions
  • AWS CloudFormation Wait Conditions
  • AWS CloudFormation Nested Stacks
  • AWS CloudFormation Deletion Policies
  • AWS CloudFormation Stack Updates
  • AWS CloudFormation Change Sets
  • AWS CloudFormation Custom Resources
  • AWS Elastic Beanstalk
  • AWS Elastic Beanstalk extensions
  • AWS Config
  • Amazon ECS
  • AWS Managed Services
  • AWS Lambda
  • AWS Lambda Step Functions
  • AWS OpsWorks

Monitoring and Logging

  • Introduction
  • CloudWatch
  • CloudWatch Custom Metrics
  • CloudWatch Events
  • CloudWatch Logs
  • AWS X-Ray

Policies and Standards Automation

  • Introduction
  • AWS Service Catalog
  • AWS Trusted Advisor
  • AWS Systems Manager
  • AWS Organizations
  • AWS Secrets Manager
  • Amazon Macie
  • AWS Certificate Manager

Incident and Event Response

  • Introduction
  • Amazon GuardDuty
  • Amazon Inspector
  • Amazon Kinesis

High Availability, Fault Tolerance, and Disaster Recovery

  • Introduction
  • AWS Single Sign-On
  • Amazon CloudFront
  • AutoScaling and Lifecycle hooks
  • Amazon Route53
  • Amazon RDS
  • Amazon Aurora
  • Amazon DynamoDB
  • Amazon DynamoDB Keys and Streams

Other Services You Need to Know About

  • Introduction
  • Tagging
  • Amazon Elastic File System
  • Amazon ElastiCache
  • Amazon S3 Glacier
  • AWS Direct Connect
  • AWS Lambda Function Dead Letter Queues
  • Amazon CloudSearch
  • Amazon Elasticsearch Service
  • Amazon DynamoDB Accelerator
  • AWS Server Migration Service

External links

Retrieved from "http://wiki.christophchamp.com/index.php?title=AWS/DevOps&oldid=7948"