Category:AWS
This category/article is (currently) just a random collection of my notes on Amazon Web Services (AWS). I will organize each service-type into a separate article, when I find the time.
Contents
- 1 Identity and Access Management (IAM)
- 2 Elastic Compute Cloud (EC2)
- 3 Elastic Load Balancer (ELB)
- 4 CloudWatch
- 5 CloudTrail
- 6 AWS Command Line Interface (CLI)
- 7 SDKs
- 8 Simple Storage Service (S3)
- 9 Lambda
- 10 Databases
- 11 DynamoDB
- 12 Route 53
- 13 Simple Queue Service (SQS)
- 14 Simple Notification Service (SNS)
- 15 Simple Workflow Service (SWF)
- 16 CloudFormation
- 17 Elastic Beanstalk
- 18 Virtual Private Cloud (VPC)
- 19 AWS Shared Responsibility
- 20 Exams overview
- 21 AWS Certifications
- 22 The AWS Partner Program
- 23 Glossary
- 23.1 Storage
- 23.2 Databases
- 23.3 Migration
- 23.4 Analytics
- 23.5 Security & Identity
- 23.6 Management Tools
- 23.7 Application Services
- 23.8 Developer Tools
- 23.9 Mobile Services
- 23.10 Business Productivity
- 23.11 Internet of Things (IoT)
- 23.12 Desktop & App Streaming
- 23.13 Artificial Intelligence
- 23.14 Messaging
- 24 Links
Identity and Access Management (IAM)
- SEE: AWS IAM FAQs
AWS Identity and Access Management (IAM) enables you to securely control access to AWS services and resources for your users. Using IAM, you can create and manage AWS users and groups, and use permissions to allow and deny their access to AWS resources.
IAM is a feature of your AWS account offered at no additional charge. You will be charged only for use of other AWS services by your users.
- IAM Roles are more secure than storing your access key and secret access key on individual EC2 instances. This is consider a "best practice".
- Roles are easier to manage
- Roles can only be assigned when that EC2 instance is being provisioned (i.e., after provisioning an EC2 instance and adding an IAM Role to that instance, you are not able to delete the Role or add another Role. You can, however, add to or modify the existing Policies attached to the Role attached to the instance.)
- Roles are universal, you can use them in any AWS region.
Elastic Compute Cloud (EC2)
SEE: AWS/EC2
Elastic Load Balancer (ELB)
- SEE: Amazon ELB FAQs
- Load Balancer types:
- Application Load Balancer (ALB)
- Layer 7 Load Balancer
- Makes routing decisions at the application layer (HTTP/HTTPS), supports path-based routing, and can route requests to one or more ports on each EC2 instance or container instance in your VPC
- Classic Load Balancer (ELB)
- Layer 4 Load Balancer
- Makes routing decisions at either the transport layer (TCP/SSL) or the application layer (HTTP/HTTPS), and support either EC2-Classic or a VPC.
- Application Load Balancer (ALB)
- ELBs are not free; one is charged by the hour and on a per GB basis of usage
- ELB supported ports:
- ec2-vpc: 1-65535
- ec2-classic: 25, 80, 443, 465, 587, 1024-65535
- ELB supported protocols:
- HTTP, HTTPS, TCP, SSL
Instances monitored by ELBs are reported as either:
- InService
- OutofService
Health Checks check the instance's health by simply "talking" to it over HTTP/HTTPS (looking for specific files on the instance)
NOTE: One can have multiple SSL certificates (for multiple domain names) on a single Elastic Load Balancer.
CloudWatch
SEE: CloudWatch FAQs
Amazon CloudWatch is a monitoring service for AWS cloud resources and the applications you run on AWS. You can use Amazon CloudWatch to collect and track metrics, collect and monitor log files, set alarms, and automatically react to changes in your AWS resources. Amazon CloudWatch can monitor AWS resources such as Amazon EC2 instances, Amazon DynamoDB tables, and Amazon RDS DB instances, as well as custom metrics generated by your applications and services, and any log files your applications generate. You can use Amazon CloudWatch to gain system-wide visibility into resource utilization, application performance, and operational health. You can use these insights to react and keep your application running smoothly.
- Services CloudWatch can monitor include: EC2, Classic ELB, ALB, EBS, S3, SNS, Lambda, DynamoDB, IoT, etc.
- Standard (free) monitoring = every 5 minutes
- Detailed (not free) monitoring = every 1 mintue
- Default CloudWatch EC2 monitoring metrics
- CPU (e.g., CPU utilization, credit usage, credit balance)
- Disk (e.g., read/write bytes/ops)
- Network (e.g., traffic in/out, packets in/out)
- Status Checks (instance-level and host/hypervisor-level)
- Able to create custom metrics
CloudWatch Dashboards allow you to create customizable dashboards to see what is happening within your AWS account.
- Bashboard widgets
- Line (plot): compare metrics over time
- Stacked area (plot): compare the total over time
- Number: instantly see the latest value for a metric
- Text: free text with markdown formatting. Example:
# Heading ## Sub-heading Paragraphs are separated by a blank line. Text attributes *italic*, **bold**, ~~strikethrough~~ . A [link](http://amazon.com). A link to this dashboard: [MyWebServer](#dashboards:name=MyWebServer). [button:Button link](http://amazon.com) [button:primary:Primary button link](http://amazon.com) Table | Header ----|----- CloudWatch | Dashboards ``` Text block ssh my-host ``` List syntax: * CloudWatch * Dashboards 1. Graphs 1. Text widget
CloudWatch Alarms allow you to set alarms that notify you (e.g., via email) when particular thresholds (you set) are hit.
CloudWatch Events helps you to respond to state changes in your AWS resources. When your resources change state they automatically send events into an event stream. You can create rules that match selected events in the stream and route them to targets to take action. You can also use rules to take action on a pre-determined schedule. For example, you can configure rules to:
- Automatically invoke an AWS Lambda function to update DNS entries when an event notifies you that Amazon EC2 instance enters the Running state
- Direct specific API records from CloudTrail to a Kinesis stream for detailed analysis of potential security or availability risks
- Periodically invoke a built-in target to create a snapshot of an Amazon EBS volume
CloudWatch Logs helps you to aggregate, monitor, and store logs. Note: You must install an agent on the EC2 instance to use this service. For example, you can:
- Monitor HTTP response codes in Apache logs
- Receive alarms for errors in kernel logs
- Count exceptions in application logs
Note the difference between CloudWatch and CloudTrail.
CloudTrail
AWS CloudTrail is a web service that records AWS API calls for your account and delivers log files to you. The recorded information includes the identity of the API caller, the time of the API call, the source IP address of the API caller, the request parameters, and the response elements returned by the AWS service.
With CloudTrail, you can get a history of AWS API calls for your account, including API calls made via the AWS Management Console, AWS SDKs, command line tools, and higher-level AWS services (such as AWS CloudFormation). The AWS API call history produced by CloudTrail enables security analysis, resource change tracking, and compliance auditing.
AWS Command Line Interface (CLI)
The AWS Command Line Interface (CLI) is a unified tool to manage your AWS services. With just one tool to download and configure, you can control multiple AWS services from the command line and automate them through scripts.
The AWS CLI introduces a new set of simple file commands for efficient file transfers to and from Amazon S3.
SDKs
- HTTP codes:
- 200 - The request has succeeded
- 3xx - Redirection
- 4xx - Client error (think 404 not found)
- 5xx - Server error (think Apache service not running, etc.)
- Available SDKs:
- Android, iOS, JavaScript (browser)
- Java
- .Net
- Node.js
- PHP
- Python
- Ruby
- Go
- C++ (preview)
- SDK default regions:
- default region (for most SDKs): us-east-1
- Some SDKs have default regions set (Java)
- Some SDKs do not (Node.js)
Services that are free include: CloudFormation, Elastic Beanstalk, Autoscaling, Opsworks, etc. (however, the resources they create are not free; e.g., EC2 instances, ELBs)
Simple Storage Service (S3)
SEE: AWS/S3
Lambda
- SEE: AWS Lambda
AWS Lambda lets you run code without provisioning or managing servers. You pay only for the compute time you consume - there is no charge when your code is not running. With Lambda, you can run code for virtually any type of application or backend service - all with zero administration. Just upload your code and Lambda takes care of everything required to run and scale your code with high availability. You can set up your code to automatically trigger from other AWS services or call it directly from any web or mobile app.
AWS Lambda is a compute service where you can upload your code and create a Lambda function. Lambda takes care of provisioning and managing the servers that you use to run the code. You do not have to worry about operating systems, patching, scaling, etc. You can use Lambda in the following ways:
- As an event-driven compute service where Lambda runs your code in response to events. These events could be changes to data in an S3 bucket or a DynamoDB table.
- As a compute service to run your code in response to HTTP requests using Amazon API Gateway or API calls made using AWS SDKs.
Data Centre -> IaaS -> PaaS -> Containers -> Serverless (e.g., Lambda)
What is Lambda? It encapsulates all of the following for you:
- Data Centres
- Hardware
- Assembly code/protocols
- High-level languages
- Operating systems
- Application layer / AWS APIs
- Languages Lambda supports (as of February 2017):
- Python
- Node.js
- Java
- C#
- Pricing
- Number of requests
- First 1 million requests are free. $0.20 per 1 million requests thereafter
- Duration
- Duration is calculated from the time your code begins executing until it returns or otherwise terminates, rounded up to the nearest 100ms. The price depends on the amount of memory you allocate to your function. You are charged $0.00001667 for every GB-second used.
- Number of requests
Databases
- SEE: Amazon RDS FAQs
- RDS
- Relational Database Server
- RDS (OLTP) Relational Database Types:
- Aurora
- MySQL Server
- MariaDB
- PostgreSQL
- MS SQL Server
- Oracle
- Non-relational Databases:
- DynamoDB (document-oriented)
- Collection (=Table)
- Document (=Row)
- Key-value pairs (=Fields)
- DynamoDB (document-oriented)
E.g., JSON/NoSQL document
{ "_id": "345sdf45asdf", "firstname": "John", "surname": "Smith", "age": "23", "address": [ { "street": "123 First Street", "suburb": "Wallingford" } ] }
- Redshift (OLAP)
- Elasticache
- Memcached
- Redis
DMS: (Online) Database Migration Service (convert Oracle to MySQL, etc.)
- Data Warehousing (business intelligence; e.g., Cognos, Jaspersoft, etc.)
- Online Transaction Processing (OLTP); vs.
- Online Analytics Processing (OLAP)
DynamoDB
- SEE: Amazon DynamoDB FAQs (import to read for the exams!)
Amazon DynamoDB is a fast and flexible NoSQL database service for all applications that need consistent single-digit millisecond latency at any scale. It is a fully managed database and supports both document and key-value data models. Its flexible data model and reliable performance make it a great fit for mobile, web, gaming, ad-tech, IoT, and many other applications.
- Stored on SSD storage
- Spread across 3 geographically distinct data centres (note: not distinct AZs)
- Eventual Consistent READs (default)
- Consistency across all copies of data is usually reached within a second. Repeating a read after a short time should return the update data. (best read performance.)
- Strongly Consistent READs
- A strongly consistent read returns a result that reflects all writes that received a successful response prior to the read.
- The DynamoDB basics:
- Tables
- Items (think a row of data in the table)
- Attributes (think of a column of data in a table)
Example DynamoDB table:
Item 101 (has the following attributes): { Title = "Book 101 Title" ISBN = "111-1111111111" Authors = "Author 1" Price = "-2" Dimensions = "8.5 x 11.0 x 0.5" PageCount = "500" InPublication = true ProductCategory = "Book" } Item 201 (has the following attributes): { Title = "18-Bicycle 201" Description = "201 description" BicycleType = "Road" Brand = "Brand-Company A" Price = "100" Color = [ "Red", "Black" ] ProductCategory = "Bike" }
An item can have any number of attributes, although there is a limit of 400 KB on the item size. An item size is the sum of lengths of its attribute names and values (binary and UTF-8 lengths); it helps if you keep the attribute names short.
- Supports up to 35 levels of nesting (JSON {foo,{bar,{baz,...)
- For any AWS account, there is an initial limit of 256 tables per region (one can, however, request an increase from Amazon)
- You can decrease the
ReadCapacityUnits
orWriteCapacityUnits
settings for a table, but no more than four times per table in a single UTC calendar day. In a single operation, you can decrease the provisioned throughput for a table, for any global secondary indexes on that table, or for any combination of these.
- Pricing
- Provisioned Throughput Capacity:
- Write throughput $0.0065 per hour for every 10 units
- Read throughput $0.0065 per hour for every 50 units
- First 25GB stored per month is free
- Storage costs of $0.25 per GB per month thereafter
- Provisioned Throughput Capacity:
Pricing example:
Assume that one's application needs to perform 1 million writes and 1 million reads per day, while storing 28 GB of data.
- First, one needs to calculate how many writes and reads per second one needs.
- 1 million evenly spread writes per data is equivalent to:
1,000,000 (writes) / 24 (hours) / 60 (minutes) / 60 (seconds) = 11.6 writes per second
A DynamoDB Write Capacity Unit (WCU) can handle 1 write per second, so you need 12 WCUs (round up 11.6 to 12).
For write throughput, one is charged on $0.0065 per hour for every 10 units, thus:
($0.0065/10) * 12 WCUs * 24 hours = $0.1872 per day
Similarly, to handle 1 million strongly consistent reads per day, one needs 12 Read Capacity Units (RCUs).
For read throughput, one is charged $0.0065 per hour for every 50 units, thus:
So, ($0.0065/50) * 12 RCUs * 24 hours = $0.0374 per day
Storage cost is $0.25 per GB per month. Assume the database is 28 GB. One gets the first 25 GB for free, so one only pays for 3 GB of storage, which is $0.75 per month.
Total Cost = $0.1872 per day + $0.0374 per day + storage of $0.75 per month, thus:
(30 * ($0.1872 + $0.0374)) + $0.75 = $7.488
Answer: $7.488/month
With the Free Tier you get:
- 25 Read Capacity Units
- 25 Write Capacity Units
- DynamoDB Indexes and Streams
- Primary Keys (two types of primary keys available):
- Single Attribute (think uniqueID)
- Partition Key (Hash Key) composed of one attribute
- Composite (think uniqueID and a date range)
- Partition Key and Sort Key (Hash & Range) composed of two attributes
- Single Attribute (think uniqueID)
- Partition Key
- DynamoDB uses the partition key's value as input to an internal hash function. The output from the hash function determines the partition (this is simply the physical location in which the data is stored).
- No two items in a table can have the same partition key value!
- Composite Key (Partition Key and Sort Key)
- DynamoDB uses the partition key's value as input to an internal hash function. The output from the hash function determines the partition (this is simply the physical location in which the data is stored).
- The two items can have the same partition key, but the must have a different sort key.
- All items with the same partition key are stored together, in sorted order by the sort key value.
- Indexes
- Local Secondary Index
- an index that has the same hash key as the table, but a different range key. A local secondary index is "local" in the sense that every partition of a local secondary index is scoped to a table partition that has the same hash key.
- has the same partition key and a different sort key.
- can only be created at table creation. They cannot be removed or modified later.
- maximum of 5 local secondary indexes per table
- each table can have up to 20 projected non-key attributes, in total across all local secondary indexes within the table. Each index may also specify that all non-key attributes from the primary index are projected. (note: "projections" are the set of attributes that is copied into a local secondary index.)
- Global Secondary Index
- an index with a hash or a hash-and-range key that can be different from those on the table. A global secondary index is considered "global" because queries on the index can span all items in a table, across all partitions.
- has a different partition key and a different sort key.
- can be created at table creation or added later.
- maximum of 5 global secondary indexes per table
- Local Secondary Index
- DynamoDB Streams
- used to capture any kind of modification of the DynamoDB tables.
- If a new item is added to the table, the stream captures an image of the entire item including all of its attributes.
- If an item in updated, the stream captures the "before" and "after" image of any attributes that were modified in the item.
- If an item was deleted from the table, the stream captures an image of the entire item before it was deleted.
- DynamoDB Streams are stored for a maximum of 24 hours.
- Can trigger a Lambda function (e.g., replicate table in another region and/or create an SES to send an email to the user for, say, when they first register to the website as a "welcome" email)
- One is able to export DynamoDB tables to a CSV file
- DyanmoDB allows for push-button scalability (with zero downtime)
- DynamoDB Query
- find items in a table using only primary key attribute values. You must provide a partition attribute name and a distinct value to search for.
- One can (optionally) provide a sort key attribute name and value, and use a comparison operator to refine the search results.
- Be default, a Query returns all of the data attributes for items with the specified primary key(s), however, you can use the
ProjectExpression
parameter so that the query only returns some of the attributes, rather than all of them. - Query results are always sorted by the sort key. If the data type of the sort key is a number, the results are returned in numeric order; otherwise, the results are returned in order of ASCII character code values. By default, the sort order is ascending. To reverse the order, set the
ScanIndexForward
parameter to false. - By default, Queries are eventually consistent, but can be changed to be strongly consistent.
- DynamoDB Scan
- A Scan operation examines every item in the table. By default, a Scan returns all of the data attributes for every item; however, you can use the
ProjectExpression
parameter so that the Scan only returns some of the attributes, rather than all of them.
What should one typically use: A Query or a Scan? Generally, a Query operation is more efficient than a Scan operation.
A Scan operation always scans the entire table, then filters out values to provide the desired result, essentially adding the extra step of removing data from the result set. Avoid using a Scan operation on a large table with a filter that removes many results, if possible. Also, as a table grows, the Scan operation slows. The Scan operation examines every item for the requested values, and can use up the provisioned throughput for a large table in a single operation.
For quicker response times, design your tables in a way that can use the Query
, GetItem
, or BatchGetItem
APIs, instead. Alternatively, design your application to use Scan operations in a way that minimizes the impact on your table's request rate.
What happens if you exceed your throughput? 400 HTTP Status Code - ProvisionedThroughputExceededException
You exceeded your maximum allowed provisioned throughput for a table or for one or more global secondary indexes.
- DynamoDB Conditional Writes
- If item = $10, then update to $12 (conditional writes are idempotent)
- E.g., two users try to update the same item at the same time
- DynamoDB Atomic Counters
- DynamoDB supports Atomic Counters, where you use the
UpdateItem
operation to increment or decrement the value of an existing attribute (or "field" in a table) without interfering with other write requests. (All write requests are applied in the order in which the were received.) For example, a web application might with to maintain a counter per visitor to their site, the application would need to increment this counter regardless of its current value. - Atomic Counters are not idempotent. This mean that the counter will increment each time you call
UpdateItem
. If you suspect that a previous request was unsuccessful, your application could retry theUpdateItem
operation; however, this would risk updating the counter twice. This might be acceptable for a web site counter, because you can tolerate slightly over- or under-counting visitors. However, in a banking application, it would be safer to use conditional updates rather than atomic counters.
If you application needs to read multiple items, you can use the BatchGetItem
API. A single BatchGetItem
request can retrieve up to 1 MB of data, which can contain as many as 100 items. In addition, a single BatchGetItem
request can retrieve items from multiple tables.
DynamoDB API
Note: This is an incomplete list. The following are the main API calls one can expect to see on an exam.
CreateTable
- Creates a table and specifies the primary index used for data access.
UpdateTable
- Updates the provisioned throughput values for the given table.
DeleteTable
- Deletes a table.
DescribeTable
- Returns table size, status, and index information.
ListTables
- Returns a list of all tables associated with the current account and endpoint.
PutItem
- Creates a new item, or replaces an old item with a new item (including all the attributes). If an item already exists in the specified table with the same primary key, the new item completely replaces the existing item. You can also use conditional operators to replace an item only if its attribute values match certain conditions, or to insert a new item only if that item does not already exist.
BatchWriteItem
- Inserts, replaces, and deletes multiple items across multiple tables in a single request, but not as a single transaction. Supports batches of up to 25 items to Put or Delete, with a maximum total request size of 16 MB.
UpdateItem
- Edits an existing item's attributes. You can also use conditional operators to perform an update only if the item's attribute values match certain conditions.
DeleteItem
- Deletes a single item in a table by primary key. You can also use conditional operators to perform a delete an item only if the item's attribute values match certain conditions.
GetItem
- The
GetItem
operation returns a set of Attributes for an item that matches the primary key. TheGetItem
operation provides an eventually consistent read by default. If eventually consistent reads are not acceptable for your application, useConsistentRead
. BatchGetItem
- The
BatchGetItem
operation returns the attributes for multiple items from multiple tables using their primary keys. A single response has a size limit of 16 MB and returns a maximum of 100 items. Supports both strong and eventual consistency. Query
- Gets one or more items using the table primary key, or from a secondary index using the index key. You can narrow the scope of the query on a table by using comparison operators or expressions. You can also filter the query results using filters on non-key attributes. Supports both strong and eventual consistency. A single response has a size limit of 1 MB.
Scan
- Gets all items and attributes by performing a full scan across the table or a secondary index. You can limit the return set by specifying filters against one or more attributes.
- A
Scan
operation on a table or secondary index has a limit of 1MB of data per operation. After the 1MB limit, it stops the operation and returns the matching values up to that point, and aLastEvaluatedKey
to apply in a subsequent operation, so that you can pick up where you left off.
Using Web Identity Providers with DynamoDB
One can authenticate users using Web Identity providers (e.g., Facebook, Google, Amazon, or any other Open-ID Connect-compatible identity provider). This is done using AssumeRoleWithWebIdentity
API.
You will need to create a role first.
- Authenticate with Identity Provider (e.g., Facebook): Log into Facebook with your username + password
- Facebook returns a Web Identity Token
- AssumeRoleWithWebIdentity request (containing Web Identity Token, App ID of provider, and ARN of role) is sent to AWS Security Token Service
- Amazon then issues Temporary Security Credentials (limit from 15 minutes to 1 hour; default 1 hour)
- Credentials contain:
- AccessKeyID, SecretAccessKey, SessionToken
- Expiration (time limit)
- AssumeRoleID
- SubjectFromWebIdentity Token (the unique ID that appears in an IAM policy variable for this particular identity provider)
- Using the above credentials, the user is allowed to access DynamoDB
Example exam questions
- DynamoDB provisioned throughput calculations
- Unit of Read provisioned throughput
- All reads are rounded up to increments of 4 KB in size.
- Eventually consistent Reads (default) consist of 2 reads per second.
- Strongly consistent Reads consist of 1 read per second.
- Unit of Write provisioned throughput
- All writes are 1 KB in size.
- All writes consist of 1 write per second.
- Unit of Read provisioned throughput
- The Magic Formula
(size of Read rounded to nearest 4 KB chunk / 4 KB) x (number of items) = read throughput # divide by 2 if eventually consistent.
Question: You have a motion sensor which writes 600 items of data every minute. Each item consists of 5kb. Your application uses eventually consistent reads. What should you set the read throughput to?
- First calculate how many READ units per item we need
- 5 KB rounded up to nearest 4 KB increment = 8 KB
- 8 KB / 4 KB = 2 read units per item
- 600 / 60 = 10 items per second
- 2 x 10 read items = 20
- Using eventually consistent reads, we get 20 / 2 reads per second = 10
Answer: 10 units of read throughput
Question: You have an application that requires to read 10 items ("rows" in a table) of 1 KB per second using eventual consistency. What should you set the read throughput to?
- First calculate how many Read units per item we need
- 1 KB rounded up to nearest 4 KB increment = 4 KB
- 4 KB / 4 KB = 1 read units per item
- 1 x 10 read items = 10
- Using eventual consistency, we get 10 / 2 reads per second = 5
Answer: 5 units of read throughput
Question: You have an application that requires to read 10 items of 6 KB per second using eventual consistency. What should you set the read throughput to?
- First calculate how many read units per item we need
- 6 KB round up to nearest increment of 4 KB = 8 KB
- 8 KB / 4 KB = 2 read units per item
- 2 x 10 read items = 20
- Using eventual consistency, we get 20 / 2 reads per second = 10
Answer: 10 units of read throughput
Question: You have an application that requires to read 5 items of 10 KB per second using eventual consistency. What should you set the read throughput to?
- First calculate how many read units per item we need
- 10 KB rounded up to the nearest increment of 4 KB = 12 KB
- 12 KB / 4 KB = 3 read units per item
- 3 x 5 read items = 15
- Using eventual consistency, we get 15 / 2 reads per second = 7.5 => 8
Answer: 8 units of READ throughput
Question: You have an application that needs to read 25 items of 13kb in size per second. Your application uses eventually consistent reads. What should you set the READ throughput to?
- First calculate how many read units per item we need
- 13 KB rounded up to the nearest increment of 4 KB = 16 KB
- 16 KB / 4 KB = 4 read units per item
- 4 x 25 read items = 100
- Using eventual consistency, we get 100 / 2 reads per second = 50
Answer: 50 units of READ throughput
Question: You have an application that requires to read 5 items of 10 KB per second using strong consistency. What should you set the READ throughput to?
- First calculate how many read units per item we need
- 10 KB rounded up to the nearest increment of 4 KB = 12 KB
- 12 KB / 4 KB = 3 read units per item
- 3 x 5 read items = 15
- Using strong consistency, we do not divide by 2
Answer: 15 units of READ throughput
Question: You have a motion sensor which writes 600 items of data every minute. Each item consists of 5kb. Your application uses strongly consistent reads. What should you set the READ throughput to?
- First calculate how many read units per item we need
- 5 KB rounded up to the nearest increment of 4 KB = 8 KB
- 8 KB / 4 KB = 2 read units per item
- 600 / 60 = 10 reads per second
- 2 x 10 read items = 20
- Using strong consistency, we do not divide by 2
Answer: 20 units of READ throughput
Question: You have an application that needs to read 25 items of 13kb in size per second. Your application uses strongly consistent reads. What should you set the READ throughput to?
- First calculate how many read units per item we need
- 13 KB rounded up to the nearest increment of 4 KB = 16 KB
- 16 KB / 4 KB = 4 read units per second
- 4 x 25 read items = 100
- Using strong consistency, we do not divide by 2
Answer: 100 units of READ throughput
Question: You have a motion sensor which writes 300 items of data every 30 seconds. Each item consists of 5kb. Your application uses eventually consistent reads. What should you set the READ throughput to?
- First calculate how many read units per item we need
- 5 KB rounded up to the nearest increment of 4 KB = 8 KB
- 8 KB / 4 KB = 2 read units per second
- 300 items of data every 30 seconds = 10 read items per second
- 2 x 10 read items = 20
- Using eventual consistency, we get 20 / 2 reads per second = 10
Answer: 10 units of READ throughput
Question: You have an application that requires to WRITE 5 items, with each item being 10 KB in size per second. What should you set the WRITE throughput to?
- Each write unit consists of 1 KB of data. You need to write 5 items per second with each item using 10 KB of data
- 5 x 10 KB = 50 write units
Answer: 50 units of WRITE throughput
Question: You have an application that requires to write 12 items of 100 KB per item each second. What should you set the WRITE throughput to?
- Each write unit consists of 1 KB of data. You need to write 12 items per second with each item having 100 KB of data
- 12 x 100 KB = 1200 write units
Answer: 1200 units of WRITE throughput
Question: You have a motion sensor which writes 600 items of data every minute. Each item consists of 5kb. What should you set the WRITE throughput to?
- Each write units consists of 1 KB of data. You need to write 10 items per second with each having 5 KB of data
- 10 x 5 KB = 50 write units
Answer: 50 units of WRITE throughput
Route 53
- SEE: Route 53 FAQs
- DNS 101
- Root Zone Database — by the Internet Assigned Numbers Authority (IANA)
- SEE: List of DNS record types for a complete list of record types.
Route 53 is a global service (i.e., not on a per AWS region).
Note: ELBs do not have pre-defined IPv4 addresses. You resolve to them using a DNS name.
- The Start of Authority (SOA) record stores information about:
- The name of the server that supplied the data for the zone;
- The administrator of the zone;
- The current version of the data file;
- The number of seconds a secondary name server should wait before checking for updates;
- The number of seconds a secondary name server should wait before retrying a failed zone transfer;
- The maximum number of seconds that a secondary name server can use data before it must either be refreshed or expire; and
- The default number of seconds for the time-to-live (TTL) file on resource records.
- Name Server (NS) records:
- Used by Top Level Domain servers to direct traffic to the Content DNS server, which contains the authoritative DNS records.
- A Records:
- An A record is the fundamental type of DNS record and the "A" in A record stands for "Address".
- The A record is used by a computer to translate the name of the domain to the IP address (e.g., http://www.example.com => http://1.2.3.4).
- TTL
- The length that a DNS record is cached on either the Resolving Server or the user's own local PC is equal to the value of the "Time To Live" (TTL) in seconds. The lower the TTL, the faster changes to DNS records take to propagate throughout the Internet.
- CNAMES
- A Canonical Name (CName) can be used to resolve one domain name to another. For example, you may have a mobile website with the domain name http://m.example.com that is used for when users browse to your domain name on their mobile devices. You may also want the name http://mobile.example.com to resolve to this same address.
- CNAME lookups on AWS incur charges.
- Alias Records
- Used to map resource record sets in your hosted zone to ELBs, CloudFront distributions, or S3 buckets that are configured as websites.
- Alias records work like a CNAME record, in that you can map one DNS name (www.example.com) to another "target" DNS name (elb1234.elb.amazonaws.com).
- The key difference: A CNAME cannot be used for naked domain names (zone apex; e.g., example.com, not www.example.com). You cannot have a CNAME for http://example.com, it must be either an A record or an Alias.
- Alias resource record sets can save you time because Route 53 automatically recognizes changes in the record sets that the alias resource record set refers to.
- For example, suppose an alias resource record set for example.com points to an ELB at lb1-1234.us-west-2.elb.amazonaws.com. If the IP address of the load balancer changes, Route 53 will automatically reflect those changes in DNS answers for example.com without any changes to the hosted zone that contains resource record sets for example.com.
- Alias Record lookups on AWS are free. Given the choice (on an exam), always choose an Alias Record over a CNAME (if possible).
Note: ELBs do not have a pre-defined IPv4 addresses, you always resolve them using a DNS name.
- Route 53 Routing Policies
- Simple
- The default routing policy when you create a new record set.
- This is most commonly used when you have a single resource that performs a given function for your domain (e.g., one web server that serves content for http://example.com).
- Weighted
- Allows you to split your traffic based on different weights assigned (e.g., send 20% of your traffic to us-east-1 and 80% to us-west-2).
- Latency
- Allows you to route your traffic based on the lowest network latency for your end user (i.e., which region will give them the fastest response time).
- In order to use latency-based routing, you create a latency resource record set for the EC2 (or ELB) resource in each region that hosts your website. When Route 53 receives a query for your site, it selects the latency resource set for the region that gives the user the lowest latency. Route 53 then responds with the value associated with that resource record set.
- Failover
- Used when you want to create an active/passive setup. For example, you may want you primary site to be in us-west-2 and your secondary DR site in us-east-1.
- Route 53 will monitor the health of your primary site using a health check, which monitors the health of your end points.
- Geolocation
- Lets you choose where your traffic will be sent based on the geographic location of your users (i.e., the location from which DNS queries originate). For example, you might want all queries from Europe to be routed to a fleet of EC2 instances that are specifically configured for your European customers. These servers may have the local language(s) of your European customers and all prices are displayed in Euros.
Simple Queue Service (SQS)
- SEE: Amazon SQS FAQs
Very first service offered by AWS
- SQS vs. RabbitMQ:
- SQS is a managed service. So one does not have to worry about operational aspects of running a messaging system including administration, security, monitoring, etc. Amazon will do this for you and will provide support if something were to go wrong.
- SQS is Elastic and can scale to very large rate/volumes (unlimited according to AWS)
- Availability of SQS has a lot of 9's in it and is backed by Amazon, which is one less thing to worry about in your application.
Amazon SQS is a web service that gives you access to a message queue that can be used to store messages while waiting for a computer to process them.
SQS is a distributed queue system that enables web service applications to quickly and reliably queue messages that one component in the application generates to be consumed by another component. A queue is a temporary repository for messages that a awaiting processing.
Using SQS, you can decouple the components of an application so they run independently, with SQS easing message management between components. Any component of a distributed application can store messages in a fail-safe queue. Messages can contain up to 256 KB of text in any format. Any component can later retrieve the messages programmatically using the SQS API.
The queue acts as a buffer between the component producing and saving data, and the component receiving the data for processing. This mean that queue resolves issues that arise if the producer is producing work faster than the consumer can process it, or if the producer or consumer are only intermittently connected to the network.
Amazon SQS ensures delivery of each message at least once, and support multiple readers and writers interacting with the same queue. A single queue can be used simultaneously by many distributed application components, with no need for those components to coordinate with each other to share the queue.
SQS is engineered to always be available and deliver messages. One of the resulting trade-offs is that SQS does not guarantee first in, first out delivery of messages. For many distributed applications, each message can stand on its own, and as long as all messages are delivered, the order is not important. If your system requires that order be preserved, you can place sequencing information in each message, so that you can reorder the messages when the queue returns them.
To illustrate, suppose you have a number of images files to encode. In a SQS worker queue, you create a SQS message for each file specifying the command (jpeg-encode) and the location of the file in S3. A pool of EC2 instances running the needed image processing software does the following:
- Asynchronously pulls the task messages from the queue;
- Retrievers the named file;
- Processes the conversion (e.g., create a thumbnail, add a watermark, etc.)
- Write the image back to Amazon S3;
- Writes a "task complete" message to another queue;
- Deletes the original task message; and then
- Checks for more messages in the worker queue
Visibility Timeout Clock only starts when the component server (i.e., EC2 instance) pulls the message from the queue
- SQS with Auto-scaling
- SQS does not offer FIFO (first in, first out)
- 12 hour visibility time out
- SQS is engineered to provide "at least once" delivery of all messages in its queues. Although most of the time each message will be delivered to your application exactly once, you should design your system so that processing a message more than once does not create any errors or inconsistencies.
- 256kb message size (as of May 2016)
- Billed at 64kb "chunks"
- A 256kb message will be 4 x 64kb "chunks"
- SQS Pricing
- First 1 million SQS requests per month are free
- $0.50 per 1 million SQS requests per month thereafter ($0.0000005 per SQS request)
- A single request can have from 1 to 10 messages, up to a maximum total payload of 256kb
- Each 64kb "chunk" of payload is billed as 1 request. For example, a single API call with a 256kb payload will be billed as four requests.
- If you see "decouple", think SQS
- SQS Delivery:
- SQS messages can be delivered multiple times and in any order (no first in, first out; last in, last out)
- SQS - Default visibility timeout:
- Default visibility timeout is 30 seconds
- Maximum timeout is 12 hours
- When you receive a message from a queue and begin processing it, you may find the visibility timeout for the queue is insufficient to fully process and delete that message. To give yourself more time to process the message, you can extend its visibility timeout by using the
ChangeMessageVisibility
action to specify a new timeout value. SQS restarts the timeout period using the new value.
- SQS Long Polling:
- SQS long polling is a way to retrieve messages from your SQS queues. While the traditional SQ short polling returns immediately, even if the queue being polled is empty, SQS long polling does not return a response until a message arrives in the queue, or the long poll times out. SQS long polling makes it easy and inexpensive to retrieve messages from your SQS queue as soon as they are available.
- Maximum Long Poll timeout is 20 seconds
- Example exam question: Polling in tight loops is burning CPU cycles and costing the company money. How would you fix this? (Answer: Enable SQS Long Polling.)
- SQS - Fanning Out:
- Create an SNS topic first using SNS. Then create a and subscribe multiple SQS queues to the SNS topic. Now whenever a message is sent to the SNS topic, the message will be fanned out to the SQS queues (i.e., SNS will deliver the message to all the SQS queues that are subscribed to the topic).
Question: You are designing a new application which involves processing payments and delivering promotional emails to customers. You plan to use SQS to help facilitate this. You need to ensure that the payment process takes priority over the creation and delivery of emails. What is the best way to achieve this?
Answer: Use 2 SQS queues for the platform. Have the EC2 fleet poll the payment SQS queue first. If this queue is empty, then poll the promotional emails queue.
Question: Your EC2 instances download jobs from the SQS queue, however they are taking too long to process them. What API call can you use to extend the length of time to process the jobs?
Answer: ChangeMessageVisibility
Question: You have a fleet of EC2 instances that are constantly polling empty SQS queues which is burning CPU compute cycles and costing your company money. What should you do?
Answer: Enable SQS Long Polling
Simple Notification Service (SNS)
- SEE: Amazon SNS FAQs
SNS is a web service that makes it easy to set up, operate, and send notifications from the cloud. It provides developers with a highly scalable, flexible, and cost-effective capability to publish messages from an application and immediately deliver them to subscribers or other applications.
SQS follows the "publish-subscribe" (pub-sub) messaging paradigm, with notifications being delivered to clients, using a "push" mechanism that eliminates the need to periodically check or "poll" for new information and updates. With simple APIs requiring minimal up-front development effort, no maintenance or management overhead and pay-as-you-go pricing, SNS gives developers an easy mechanism to incorporate a powerful notification system with their applications.
Push notifications to Apple, Google, Fire OS, and Windows devices, as well as Android devices in China with Baidu Cloud Push.
Besides pushing cloud notifications directly to mobile devices, SNS can also deliver notifications by SMS text message or email, to Amazon Simple Queue Service (SQS) queues or to any HTTP endpoint.
To prevent messages from being lost, all messages published to SNS are stored redundantly across multiple availability zones.
- SNS - Topics:
- SNS allows you to group multiple recipients using "topics". A topic is an "access point" for allowing recipients to dynamically subscribe to identical copies of the same notification. One topic can support deliveries to to multiple endpoint types. For example, one can group together iOS, Android, and SMS recipients. When you publish once to a topic, SNS delivers appropriately formatted copies of your message to each subscriber.
- Subscriptions via email require the receiving email owner to confirm the subscriptions in order to receive notifications from the given topic (prevents spam). Subscription requests expire after 3 days, if the receiving email owner does not confirm the subscription.
Example SNS email notification:
{ "Type" : "Notification", "MessageId" : "436d9234-f427-5be8-aa54-dd98ae4e286dba0", "TopicArn" : "arn:aws:sns:us-west-2:01234:MyTestSNSTopic", "Subject" : "This is a test", "Message" : "Hello, world!", "Timestamp" : "2016-05-10T21:53:37.981Z", "SignatureVersion" : "1", "Signature" : "PInj15UDcwMI==", "SigningCertURL" : "https://sns.us-west-2.amazonaws.com/SimpleNotificationService-9390147a5624348ee.pem", "UnsubscribeURL" : "https://sns.us-west-2.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-west-2:01234:MyTestSNSTopic:1a4eead5-110c-4acb-894a-91cdf358aabc", "MessageAttributes" : { "AWS.SNS.MOBILE.MPNS.Type" : {"Type":"String","Value":"token"}, "AWS.SNS.MOBILE.MPNS.NotificationClass" : {"Type":"String","Value":"realtime"}, "AWS.SNS.MOBILE.WNS.Type" : {"Type":"String","Value":"wns/badge"} } }
- SNS benefits:
- Instantaneous, push-based delivery (no polling)
- Simple APIs and easy integration with applications
- Flexible message delivery over multiple transport protocols
- Inexpensive, pay-as-you-go model with no up-front costs
- Web-based AWS Management Console offers the simplicity of a point-and-click interface
- SNS vs. SQS:
- Both messaging services in AWS
- SNS => Push
- SQS => Pulls (Polls)
- SNS Pricing:
- Users pay $0.50 per 1 million SNS requests
- $0.06 per 100,000 notification deliveries over HTTP
- $0.75 per 100 notification deliveries over SMS
- $2.00 per 100,000 notification deliveries over email
SNS data format = JSON
- SNS protocols include:
- HTTP
- HTTPS
- Email-JSON
- Amazon SQS
- Application
NOTE: Messages can be customized for each protocol
Simple Workflow Service (SWF)
- SEE: Amazon SWF FAQs
Amazon Simple Workflow Server (Amazon SWF) is a web service that makes it easy to coordinate work across distributed application components. SWF enables applications for a range of use cases, including media processing, web application back-ends, business process work-flows, and analytics pipelines, to be designed as a coordination of tasks. Task represent invocations of various processing steps in an application, which can be performed by executable code, web service calls, human actions, and scripts.
- SWF Workers
- programs that interact with SWF to get tasks, process received tasks, and return the results.
- SWF Decider
- a program that controls the coordination of tasks, i.e., their ordering, concurrency, and scheduling according to the application logic.
The workers and the decider can run on cloud infrastructure, such as Amazon EC2, or on machines behind firewalls. SWF brokers the interactions between workers and the decider. It allows the decider to get consistent views into the progress of tasks and to initiate new tasks in an ongoing manner. At the same time, SWF stores tasks, assigns them to workers when they are ready, and monitors their progress. It ensures that a task is assigned only once and is never duplicated. Since SWF maintains the application's state durably, workers and deciders do not have to keep track of execution state. They can run independently and scale quickly.
- SWF Domains
- Your work-flow, activity types, and the workflow execution itself are all scoped to a domain. Domains isolate a set of types, executions, and task lists from others within the same account. One can register a domain by using the AWS Management Console or by using the
RegisterDomain
action in the SWF API. - The parameters of an SWF Domain are specified in JSON format. E.g.:
https://swf.us-west-2.amazonaws.com RegisterDomain { "name": "123456789", "description": "images", "workflowExecutionRetentionPeriodInDays": "60" }
Maximum Workflow can be 1 year and the value is always measured in seconds.
- SWF vs. SQS:
- SWF presents a task-oriented API, whereas SQS offers a message-oriented API.
- SWF ensures that a task is assigned only once and is never duplicated. What SQS, one needs to handle duplicate messages and may also need to ensure that a message is processed only once.
- SWF keeps track of all the tasks and events in an application. With SQS, one needs to implement one's own application-level tracking, especially if one's application uses multiple queues.
- Does the service require human interaction? If so, one should use SWF.
- Does the service need to run for (much) more than 12 hours? If so, one should use SWF. If less than 12 hours, SQS might be the correct service to use.
- Maintaining your application's execution state (e.g. which steps have completed, which ones are running, etc.) is a perfect use case for SWF.
- Amazon SWF is useful for automating work-flows that include long-running human tasks (e.g. approvals, reviews, investigations, etc.) Amazon SWF reliably tracks the status of processing steps that run up to several days or months.
CloudFormation
CloudFormation => Scripted infrastructure (IaaS)
Using the CloudFormation service is free. However, any resources it creates/consumes/provisions (e.g., EC2 instances, Load Balancers, etc.) are not free.
CloudFormation templates are written in JSON format. E.g., CloudFormation LAMP stack template (see here for full template):
{ "AWSTemplateFormatVersion" : "2010-09-09", "Description" : "AWS CloudFormation Sample Template LAMP_Single_Instance: Create a LAMP stack using a single EC2 instance and a local MySQL database for storage. This template demonstrates using the AWS CloudFormation bootstrap scripts to install the packages and files necessary to deploy the Apache web server, PHP and MySQL at instance launch time. **WARNING** This template creates an Amazon EC2 instance. You will be billed for the AWS resources used if you create a stack from this template.", "Parameters" : { "KeyName": { "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance", "Type": "AWS::EC2::KeyPair::KeyName", "ConstraintDescription" : "must be the name of an existing EC2 KeyPair." }, ...
- CloudFormation "output":
- SEE: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/outputs-section-structure.html
- Remember the "
Fn::GetAtt
" function
If a CloudFormation stack creation fails, the default is to terminate and roll-back all resources created on failure (i.e., delete all of the resources it was trying to create). One can disable roll back to leave all resources in their current state (failed or not). This is useful for troubleshooting your own templates.
Question : You are creating a virtual data centre using cloud formation and you need to output the DNS name of your load balancer. What command/function would you use to achieve this?
Answer: Fn::GetAtt
function
Elastic Beanstalk
AWS Elastic Beanstalk is an easy-to-use service for deploying and scaling web applications and services developed with Java, .NET, PHP, Node.js, Python, Ruby, Go, and Docker on familiar servers such as Apache, Nginx, Passenger, and IIS.
You can simply upload your code and Elastic Beanstalk automatically handles the deployment, from capacity provisioning, load balancing, auto-scaling, to application health monitoring. At the same time, you retain full control over the AWS resources powering your application and can access the underlying resources at any time.
Using the AWS Elastic Beanstalk service is free. However, any AWS resources it creates/consumes/provisions to store and run your applications are not free.
- Environment tier:
- Web Server Environment - Provides resources for an AWS Elastic Beanstalk web server in either a single instance or load-balancing, auto scaling environment.
- Worker Environment - Provides resources for an AWS Elastic Beanstalk worker application in either a single instance or load-balancing, auto scaling environment.
- Environment type:
- Single instance
- Load balancing, auto-scaling
- Preconfigured platforms:
- PHP, Node.js, Python, Ruby, Tomcat, IIS, Java, Go, Docker
Virtual Private Cloud (VPC)
- SEE: Amazon VPC FAQs
- Think of a VPC as a virtual data centre in the cloud.
- AWS definition of a VPC:
- Amazon Virtual Private Cloud (VPC) lets you provision a logically isolated section of the AWS Cloud where you can launch AWS resources in a virtual network that you define. You have complete control over your virtual networking environment, including select of you own IP address range, creation of subnets, and configuration of route tables and network gateways.
- You can easily customize the network configuration for your AWS virtual Private Cloud. For example, you can create a public-facing subnet for your webservers that has access to the Internet, and place backend systems, such as databases or application service in a private-facing subnet, with no Internet access. You can leverage multiple layers of security, including security groups and network access control lists, to help control access to EC2 instance in each subnet.
- Additionally, one can create a Hardware Virtual Private Network (VPN) connection between your corporate datacentre and your VPC and leverage the AWS cloud as an extension of your corporate datacentre. "Hybrid Cloud".
- What can one do with a VPC?
- Launch instances into a subnet of one's choosing
- Assign custom IP address ranges in each subnet
- Configure route tables between subnets
- Create internet gateways and attach them to subnets (or not)
- Much better security control over your AWS resources
- Instance security groups
- Create subnet network access control lists (ACLs)
- Number of allowed VPCs in each AWS Region (by default): 5
- Default VPC vs. Custom VPC
- Default VPC is user friendly (automatically created when one creates an AWS account). It allows one to immediately deploy instances.
- All subnets in a default VPC have an internet gateway attached (i.e., all subnets are public)
- Each EC2 instance has both a public and private IP address
- If one were to delete the default VPC, the only way to get it back is to contact AWS
- VPC Peering
- Allows one to connect one VPC with another via a direct network route using private IP addresses.
- Instances behave as if they were on the same private network.
- One can peer VPCs with other AWS accounts as well as with other VPCs in the same account.
- One cannot create a VPC larger than /16
- Peering is done in a "star configuration", i.e., 1 central VPC peers with 4 others. No transitive peering!
- A "star configuration" peering:
+-------+ | VPC C | +-------+ ^ | v +-------+ +-------+ +-------+ | VPC B | <----> | VPC A | <----> | VPC E | +-------+ +-------+ +-------+ ^ | v +-------+ | VPC D | +-------+
In the above example, instances on VPC-B can not send/receive traffic on VPC-C via VPC-A. One would need to create a VPC peer directly from VPC-B and VPC-C. That is, with a star configuration (as shown above), there is no transitive peering.
- VPC tenancy:
- Default - EC2 instances are created on shared hardware
- Dedicated - EC2 instances are created on dedicated hardware (regardless of the tenancy attribute specified at launch). This the more expensive option.
By default, when one creates a VPC, a route tables is automatically created for the VPC.
NOTE: If one deletes one's account's default VPC (and/or the associated default subnets), the only way to get them back is to raise a ticket with Amazon.
- VPC subnets:
- Use the CIDR format to specify your subnet's IP address block (e.g., 10.0.0.0/24). Note that block sizes must be between a /16 netmask and /28 netmask. Also, note that a subnet can be the same size as your VPC.
- Subnets are always mapped to one availability zone (AZ). Subnets can not be mapped across multiple AZs. 1 subnet = 1 AZ.
- VPC Internet gateways:
- By default, when one creates an Internet Gateway, it is detached. One must attach it to a VPC in order to use it.
- One can only have 1 Internet Gateway per VPC.
Security Groups can span multiple VPCs and VPC subnets.
- Network Address Translation (NAT) Server:
- Allow instances with only private IPs to reach the Internet via the NAT server (via, say, HTTP/HTTPS and all other protocols/ports closed, including SSH)
- One must disable Source/Destination Check on NAT instances for them to work properly.
Network Access Control Lists (ACLs)
- A network ACL is an optional layer of security that acts as a firewall for controlling traffic in and out of a subnet.
- ACLs ~ "firewall"-like rules
- ACLs are a numbered list of rules, which are followed in order, starting with the lowest number first. They control network ingress/egress for all AWS resources within a given subnet.
- The highest ACL number allowed is 32,766
- ACLs have a default (editable) number list that allows all inbound/outbound traffic
- One can create a custom ACL, which starts out with no inbound/outbound traffic allowed, until one adds a rule
- ACLs are applied to an entire subnet (and override the security groups associated with a given instance on that subnet). For an example, if a security group applied to a given instance has port 80 allowed, but the ACL for the subnet the instance is on has port 80 denied, the ACL rule takes precedence (i.e., port 80 will be denied on all instances within that subnet, regardless of what the security group allows).
- Unless one creates a custom ACL and associates it with a given subnet, that ACL will use the default role and rules.
- One can not have multiple ACLs associated with the same subnet. However, a given ACL can be associated with multiple subnets.
- If one dis-associates a custom ACL from a given subnet(s), the subnet reverts back to the default ACL
Example labs
- VPC Lab:
- Create a custom VPC
- Define an IP Address Range (e.g., 10.0.0.0/16)
- By default, this created a Network ACL and Routing Table
- Create a custom Route Table
- Create 3 subnets (e.g., 10.0.1.0/24, 10.0.2.0/24, and 10.0.3.0/24)
- public subnet: 10.0.1.0/24; private subnets: 10.0.2.0/24 and 10.0.3.0/24
- Create an Internet Gateway
- Attach Internet Gateway to the custom VPC
- Associate the public subnet with the custom Route Table
- Provision an EC2 instance with an Elastic IP address (in the public subnet)
- Provision an EC2 instance with only a private IP address (on the private subnet)
- Create a custom VPC
- NAT Lab:
- Create a custom security group
- Allow inbound traffic to 10.0.1.0/24 and 10.0.2.0/24 on HTTP and HTTPS
- Allow outbound traffic on HTTP and HTTPS to anywhere
- Provision a NAT instance inside the public subnet
- Import! Make sure to select "Disabled Source/Destination Check" on this NAT instance!
- Set up a route on the private subnet to route traffic through the NAT instance
- Shared Responsibility Model for AWS Infrastructure Services
- Infrastructure services include: EC2, EBS, Auto-scaling, VPC, etc
- Amazon responsibility (i.e., managed by AWS):
- AWS global infrastructure (regions, availability zones, edge locations)
- Foundation services (compute, storage, databases, networking)
- AWS endpoints
- AWS IAM
- Customer responsibility (i.e., managed by AWD customers):
- Server-side encryption, network traffic protection
- Operating system, network, and firewall configuration(s)
- Platform and application management
- Customer data
- Customer IAM
- Shared Responsibility Model for AWS Container Services
- Container services include: Relational Database Services (RDS), Elastic Map Reduce (EMR), and Elastic Beanstalk
- AWS takes on more of the responsibility (e.g., operating system and network configuration; platform and application management)
- Customer still has the responsibility for firewall configuration
- Shared Responsibility Model for AWS abstracted services
- Abstracted services include: S3, Glacier, DynamoDB, SQS, and Simple Email Service (SES), Lambda
- AWS takes on even more of the responsibility (e.g., network traffic protection provided by the platform; server-side encryption provided by the platform)
- Customer still has the responsibility for client-side data encryption, data integrity authentication, and customer data
Exams overview
AWS Certified Developer - Associate
- Time allotted: 80 minutes
- 55 questions on the exam
- $150 example registration fee
- Conducted Online at an approved centre
AWS Certified Solutions Architect - Associate
- Time allotted: 80 minutes
- 60 questions on the exam
- $150 example registration fee
- Conducted Online at an approved centre
- AWS platforms covered:
- Security & Identity
- Compute
- Storage
- Databases
- Networking & Content Delivery
- Messaging
- Desktop & App Streaming (only at a very high-level)
- Management Tools (only at a very high-level)
- AWS Global Infrastructure (what all of the above platforms/services reside in)
- As of December 2016: 14 Regions and 38 Availability Zones (AZs)
- In 2017: 4 more Regions and 11 more AZs
- Edge Locations are CDN Endpoints for CloudFront (as of December 2016, there are ~66 Edge Locations)
AWS Certifications
NOTE: All AWS certification exams are taken on-site and proctored.
- Associate Level ($150 each):
- AWS Certified Developer - Associate
- AWS Certified Solutions Architect - Associate
- AWS Certified SysOps Administrator - Associate
- Professional Level ($300 each):
- AWS Certified DevOps Engineer - Professional
- AWS Certified Solutions Architect - Professional
- Specialty (Beta, as of January 2017)
- AWS Certified Security - Specialty
- AWS Certified Big Data - Specialty
- AWS Certified Advanced Networking - Specialty
The AWS Partner Program
The AWS Partner Program | |||
---|---|---|---|
Partner | Associate Certs | Professional Certs | |
Standard | 2 | 0 | |
Advanced | 4 | 2 | |
Premier | 20 | 8 |
Glossary
see: Official AWS Glossary
- AMI
- Amazon Machine Image
- ARN
- Amazon Resource Name
- EBS
- Elastic Block Storage (victual disks for EC2 instances)
- EC2
- Elastic Compute Cloud
- EC2 Container Service (ECS)
- Elastic Beanstalk
- EFS
- Elastic File System
- ELB
- Elastic Load Balancer
- Lambda
- Serverless code
- Lightsail
- Out-of-the-box Cloud
- STS
- Security Token Service
- VPC
- Virtual Private Cloud
- Route53
- DNS + ability or register domain names
- CloudFront
- Content Delivery Network (CDN) / Edge Locations
- DirectConnect
Storage
- S3
- Simple Storage Service (object-based storage)
- Glacier
- Data archival (for objects in S3). Low cost.
- EFS
- Elastic File Service (block-based storage; shareable)
- Storage Gateway
- Connect S3 to on-premise DC
Databases
- RDS
- Relational Database Service (e.g., MySQL, MariaDB, Aurora, Postgres, etc.)
- DynamoDB
- NoSQL (non-relational database)
- Redshift
- Data warehousing
- Elasticache
Migration
- Snowball
- Move large amounts of data into the Cloud (e.g., contents of a HDD)
- DMS
- Database Migration Service (e.g. in-house Oracle DB into AWS RDS:Aurora)
- Server Migration Service
- Virtual machine migration (e.g., on-premise VMware VMs into AWS)
Analytics
- Athena
- Run SQL queries on S3 (e.g., CSV/JSON files)
- EMR
- Elastic MapReduce (process large amounts of data {Big Data}; e.g., log files)
- CloudSearch
- Elastic Search
- Kinesis
- Stream and analyse live/real-time data (e.g., financial data, social media feeds, etc.)
- Data Pipeline
- Allows moving data from one location to another (e.g., from S3 to DynamoDB or vice versa, etc.)
- Quick Sight
- Business analytics tool
Security & Identity
- IAM
- AWS Identity and Access Management
- Inspector
- Agent-based service to inspect EC2 instances, etc.
- Certificate Manager
- Free SSL certs
- Directory Service
- Active Directory in the Cloud
- WAF
- Web Application Firewall (e.g., protect against SQL injections, etc.)
- Artifacts
- Compliance Reports (e.g., ISO 27001 certification, etc.)
Management Tools
- Cloud Watch
- Monitor performance of AWS (e.g. EC2 => CPU/RAM util)
- Cloud Formation
- Infrastructure as Code (document-based; JSON/YAML)
- Cloud Trail
- Audit AWS resource usage
- OpsWorks
- Chef for AWS
- Config
- Monitor AWS environment (e.g., send alert if someone creates an IAM role that breaks company policy)
- Trusted Advisor
- Automate performance, security, fault-tolerance, cost, etc.
Application Services
- Step Functions
- Visualize what is going on inside an application (and/or microservice)
- SWF
- Simple Workflow Service (coordinate automated vs. human tasks)
- API Gateway
- Create, publish, maintain APIs in the Cloud
- AppStream
- Stream desktop applications to users
- Elastic Transcoder
- Change video format (e.g., for viewing on different devices)
Developer Tools
- CodeCommit
- GitHub in AWS
- CodeBuild
- Compile code in the Cloud
- CodeDeploy
- Deploy code to EC2 instances
- CodePipeline
- Keep track of versions of code (e.g., dev, test, prod, UAT)
Mobile Services
- Mobile Hub
- Cognito
- Device Farm
- Mobile Analytics
- Pinpoint
- Google Analytics for mobile applications
Business Productivity
- WorkDocs
- WorkMail
- Exchange for AWS
Internet of Things (IoT)
- IoT
Desktop & App Streaming
- WorkSpaces
- Virtual Desktops in the Cloud / Virtual Desktop Infrastructure (VDI) solutions
- AppStream 2.0
- Stream dekstop applications to users
Artificial Intelligence
- Lex
- Think "Alexa in the Cloud" or Alexa on a RaspberryPi
- Polly
- Text-to-Speech (text => mp3 in S3)
- Machine Learning
- Rekognition
- Analyse pictures with tagging and facial recognition
Messaging
- SNS
- Simple Notification Service
- SQS
- Message Queue Service
- SES
- Simple Email Service
Links
- AWS Whitepapers
- Training and certification
- https://acloud.guru/courses
- https://aws.amazon.com/certification/certified-developer-associate/
- What you need to get AWS certified — by A Cloud Guru
- Register for AWS certification exams
- Miscellaneous
Pages in category "AWS"
The following 8 pages are in this category, out of 8 total.