Difference between revisions of "AWS/DevOps"
From Christoph's Personal Wiki
(→Monitoring and Logging) |
(→Policies and Standards Automation) |
||
| Line 256: | Line 256: | ||
==Policies and Standards Automation== | ==Policies and Standards Automation== | ||
| − | + | ===Introduction=== | |
| − | + | ||
| − | + | ===AWS Service Catalog=== | |
| − | + | ||
| − | * AWS | + | ===AWS Trusted Advisor=== |
| − | * AWS Secrets Manager | + | |
| − | + | ===AWS Systems Manager=== | |
| − | * AWS Certificate Manager | + | |
| + | A management service that assists with: | ||
| + | * Collecting software inventory | ||
| + | * Applying OS patches | ||
| + | * Creating system images | ||
| + | * Configuring operating systems | ||
| + | * Manage Hybrid Cloud systems from a single interface (AWS and on-prem) | ||
| + | * Reducing costs | ||
| + | |||
| + | ; Run Command | ||
| + | |||
| + | Lets you run a given command(s) across all of your EC2 instances (or a group of them). | ||
| + | |||
| + | ===AWS Organizations=== | ||
| + | |||
| + | ===AWS Secrets Manager=== | ||
| + | |||
| + | ===Amazon Macie=== | ||
| + | |||
| + | Macie is a security server that uses machine learning to automatically discover, classify, and protect sensitive data in AWS. | ||
| + | |||
| + | * Macie: | ||
| + | ** Can recognize any Personally Identifiable Information (PII) | ||
| + | ** Provides a dashboard | ||
| + | ** Monitors data access activity for anomalies | ||
| + | ** Generates detailed alerts when it detects risk of unauthorized access or accidental data leaks | ||
| + | ** As of February 2021 <CHECK>, it only protects data in S3, with more AWS data stores planned for the future. | ||
| + | ** It gives you superior visibility of data | ||
| + | ** Simple to set up and easy to manage | ||
| + | |||
| + | ===AWS Certificate Manager=== | ||
==Incident and Event Response== | ==Incident and Event Response== | ||
Revision as of 23:35, 18 February 2021
This article will cover topics related to the AWS Certified DevOps Engineer - Professional exam and certification.
Contents
Domains
The 6 domains outlined in the AWS blueprint for the certification include:
- Software Development LifeCycle (SDLC) Automation [22%]
- Configuration Management and Infrastructure as Code [19%]
- Monitoring and Logging [15%]
- Policies and Standards Automation [10%]
- Incident and Event Response [18%]
- High Availability, Fault Tolerance, and Disaster Recovery [16%]
Domain 1: SDLC Automation
- 1.1 Apply concepts required to automate a CI/CD pipeline
- 1.2 Determine source control strategies and how to implement them
- 1.3 Apply concepts required to automate and integrate testing
- 1.4 Apply concepts required to build and manage artifacts securely
- 1.5 Determine deployment/delivery strategies (e.g., A/B, Blue/Green, Canary, Red/Black) and how to implement them using AWS Services
Domain 2: Configuration Management and Infrastructure as Code
- 2.1 Determine deployment services based on deployment needs
- 2.2 Determine application and infrastructure deployment models based on business needs
- 2.3 Apply security concepts in the automation of resource provisioning
- 2.4 Determine how to implement lifecycle hooks on a deployment
- 2.5 Apply concepts required to manage systems using AWS configuration management tools and services
Domain 3: Monitoring and Logging
- 3.1 Determine how to set up the aggregation, storage, and analysis of logs and metrics
- 3.2 Apply concepts required to automate monitoring and event management of an environment
- 3.3 Apply concepts required to audit, log, and monitor operating systems, infrastructures, and applications
- 3.4 Determine how to implement tagging and other metadata strategies
Domain 4: Policies and Standards Automation
- 4.1 Apply concepts required to enforce standards for logging, metrics, monitoring, testing, and security
- 4.2 Determine how to optimize cost through automation
- 4.3 Apply concepts required to implement governance strategies
Domain 5: Incident and Event Response
- 5.1 Troubleshoot issues and determine how to restore operations
- 5.2 Determine how to automate event management and alerting
- 5.3 Apply concepts required to implement automated healing
- 5.4 Apply concepts required to set up event-driven automated actions
Domain 6: High Availability, Fault Tolerance, and Disaster Recovery
- 6.1 Determine appropriate use of multi-AZ versus multi-region architectures
- 6.2 Determine how to implement high availability, scalability, and fault tolerance
- 6.3 Determine the right services based on business needs (e.g., RTO/RPO, cost)
- 6.4 Determine how to design and automate disaster recovery strategies
- 6.5 Evaluate a deployment for points of failure
SDLC Automation
Introduction
What is CI/CD?
- The CI/CD Pipeline
- AWS CodePipeline
- Source Stage
- AWS CodeCommit (think "git")
- Deploy Stage - Development
- AWS CodeDeploy -> EC2 instance
- Deploy Stage - Production
- AWS CodeDeploy -> EC2 instance
- Source Stage
AWS CodeCommit
AWS CodeBuild
- A fully managed build service
- Compiles your code
- Runs unit tests
- Produces artifacts that are ready to deploy
- Eliminates the need to provision/manage/scale your own build servers
- Provides pre-packaged build environments
- Allows you to build your own customized build environment
- Scales automatically to meet your build requirements
- Benefits of CodeBuild
- It is fully managed
- You do not have to set up any build servers, nor patch, update, or maintain them.
- It is on-demand
- It automatically scales to meet your requirements. No more migrating to larger EC2 servers because your builds are taking too long. You only pay for the minutes (seconds?) you consume.
- It is preconfigured
- It comes with many pre-configured build environments for the most popular programming languages. You just need to configure it to use your build script.
AWS CodeDeploy
- What is CodeDeploy?
- A fully managed deployment service that automates deployments to:
- Amazon EC2 instances
- On-premise instances
- AWS Lambda functions
- Makes it easier to:
- Rapidly deploy new features
- Update Lambda function versions
- Avoid downtime during deployment
- Handle the full complex deployment process without human intervention
AWS CodePipeline
CodePipeline is the "CD" of CI/CD.
- Benefits
- Automatic
- From the check-in of your code to deployment on to your service, CodePipeline takes care of it all.
- Easy to set up
- CodePipeline has no servers to provision, it is dead simple to configure and get working. There are pre-built plugins or you can roll your own.
- Configurable
- You can create, configure, and modify all stages of your software release process with ease. You can implement automated testing and customize the deployment process.
Testing
- Why do we test?
- Meet the requirements defined
- Ensure the code performs in an accepatble period of time
- Ensure the code is usable
- Ensure the code responds correctly to all kinds of inputs
- Achieves the result the programmer desired
- Types of testings (see Wikipedia)
- Automated testing
- Automatic execution oif test
- Comparision of actual outcomes to predicted outcomes
- Fast, continuos feedback
- Immediate notifcation
- Save resources
- Unit test example
assert()
Artifacts
- What are artifacts?
An artifact is a product or by-product produced during the software development process.
For example:
- Compiled binaries
- Source code
- Documentation
- Use cases
- Class diagrams
Artifacts are stored in S3 (note: this has nothing to do with AWS Artifact!)
Deployment Strategies
- Single Target Deployment (build -> target)
- Use for small development projects, especially when legacy or non-highly-available infrastructure is involved.
- When it is initiated, a new application version is installed on the target server.
- A brief outage occurs during installation. There are no secondary servers, so testing is limited. Rollback involves removing the new version and install the previous.
- All-at-Once Deployment (build -> x2 targets)
- Deployment happens in one step, just like single target deployment.
- With this method, the destination is multiple targets.
- More complicated than single target; often requiring orchestration tooling.
- Shares negatives of single target. No ability to test, still has deployment outages, and less than ideal rollback.
- Minimum in-service Deployment (initial build stage -> t1 t2 t3 ...)
- Deployment happens in multiple stages
- Deployment happens to as many targets as possible while maintaining the minimum in-service targets.
- A few moving parts, orchestration and health checks are required.
- Allows automated testing, deployment targets are assessed and testsd prior to continuing.
- Generally, no downtime.
- Often quicker and less stages than a rolling deployment.
- Rolling Deployment
- Deployment happens in multiple stages. Number of targets per stage is user-defined.
- Moving parts; orchestration and health-checks are required.
- Overall applicable health is not necessariliy maintained.
- Can be the leasat efficient deployment time based on time-taken.
- Allows automated testing; deployment targets are assessed and tested prior to continuing.
- Generally, no downtime, assuming number of targets per run is not large neough to impact the application.
- Can be paused, allowing limited multi-version testing (combined with small targets per stage).
- Blue/Green Deployment (aka Red/Black)
- Requires advanced orchestration tooling
- Carries significant cost - maintiang 2 environments for the duration of deployments.
- Deployment process is rapid - entire environemnt (blue or green) is deployed all at once.
- Cutover and migration is clean and controlled (e.g., DNS change)
- Rollback is equally clean (e.g., DNS regression)
- Health and performance of entire "green" environment can be tested prior to cutover.
- using advanced template systems, such as CloudFormation, the entire process can be fully automated.
- Canary Deployment
- Like Blue/Green, but keep blue active and route percentage of traffic to green
- In AWS, use Route53 w/weighted round-robin
Configuration Management and Infrastructure as Code
- Introduction
- AWS CloudFormation
- AWS CloudFormation Intrinsic Functions
- AWS CloudFormation Wait Conditions
- AWS CloudFormation Nested Stacks
- AWS CloudFormation Deletion Policies
- AWS CloudFormation Stack Updates
- AWS CloudFormation Change Sets
- AWS CloudFormation Custom Resources
- AWS Elastic Beanstalk
- AWS Elastic Beanstalk extensions
- AWS Config
- Amazon ECS
- AWS Managed Services
- AWS Lambda
- AWS Lambda Step Functions
- AWS OpsWorks
Monitoring and Logging
- Introduction
CloudWatch
$ aws cloudwatch put-metric-data \
--metric-name randomNumber \
--namespace Random \
--value $(shuf -i 1-1000 -n1) \
--region=us-west-2
- CloudWatch Custom Metrics
- CloudWatch Events
CloudWatch Logs
$ sudo dpkg -i amazon-cloudwatch-agent.deb $ sudo vi /opt/aws/amazon-cloudwatch-agent/etc/common-config.toml # create and edit config $ sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:cloudwatchconfig.cfg -s
AWS X-Ray
Think managed "Kiali + Jaeger"
Policies and Standards Automation
Introduction
AWS Service Catalog
AWS Trusted Advisor
AWS Systems Manager
A management service that assists with:
- Collecting software inventory
- Applying OS patches
- Creating system images
- Configuring operating systems
- Manage Hybrid Cloud systems from a single interface (AWS and on-prem)
- Reducing costs
- Run Command
Lets you run a given command(s) across all of your EC2 instances (or a group of them).
AWS Organizations
AWS Secrets Manager
Amazon Macie
Macie is a security server that uses machine learning to automatically discover, classify, and protect sensitive data in AWS.
- Macie:
- Can recognize any Personally Identifiable Information (PII)
- Provides a dashboard
- Monitors data access activity for anomalies
- Generates detailed alerts when it detects risk of unauthorized access or accidental data leaks
- As of February 2021 <CHECK>, it only protects data in S3, with more AWS data stores planned for the future.
- It gives you superior visibility of data
- Simple to set up and easy to manage
AWS Certificate Manager
Incident and Event Response
- Introduction
- Amazon GuardDuty
- Amazon Inspector
- Amazon Kinesis
High Availability, Fault Tolerance, and Disaster Recovery
- Introduction
- AWS Single Sign-On
- Amazon CloudFront
- AutoScaling and Lifecycle hooks
- Amazon Route53
- Amazon RDS
- Amazon Aurora
- Amazon DynamoDB
- Amazon DynamoDB Keys and Streams
Other Services You Need to Know About
- Introduction
- Tagging
- Amazon Elastic File System
- Amazon ElastiCache
- Amazon S3 Glacier
- AWS Direct Connect
- AWS Lambda Function Dead Letter Queues
- Amazon CloudSearch
- Amazon Elasticsearch Service
- Amazon DynamoDB Accelerator
- AWS Server Migration Service
