AWS/S3 - Revision history

Christoph: /* Features */

2017-04-07T18:15:07Z

‎Features

Christoph: /* Features */

2017-01-30T22:44:52Z

‎Features

Christoph: /* Features */

2017-01-30T22:41:26Z

‎Features

Christoph: /* AWS Snowball */

2017-01-30T22:30:13Z

‎AWS Snowball

Christoph: /* AWS Snowball */

2017-01-28T00:55:28Z

‎AWS Snowball

Christoph at 00:21, 28 January 2017

2017-01-28T00:21:11Z

Christoph: /* AWS Storage Gateway */

2017-01-28T00:20:46Z

‎AWS Storage Gateway

Christoph at 00:03, 28 January 2017

2017-01-28T00:03:46Z

Christoph: /* Features */

2017-01-27T23:34:29Z

‎Features

Christoph: /* Features */

2017-01-26T00:30:36Z

‎Features

@@ Line 122: / Line 122: @@
 ** One can clear the cache of an object stored on an Edge Location before the TTL expires, but one will be charged for that service. Create an "invalidation" request to clear the cache of a given object.
 ** Restrict viewer access by using either Signed URLs or Signed Cookies (e.g., only allow paying users to view your content)
-** Allows for reo-restricting access (i.e., whitelist or blacklist countries)
+** Allows for geo-restricting access (i.e., whitelist or blacklist countries)
 * [http://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html S3 Transfer Acceleration]:

@@ Line 17: / Line 17: @@
 ** Note that the largest size file you can transfer to S3 using a PUT operation is 5 GB
 * Unlimited storage
 * Files/objects are stored in "buckets"
 * S3 is a universal namespace (i.e., bucket names must be unique globally; think domain names)

@@ Line 54: / Line 54: @@
 * S3 bucket names must contain only lower case characters
 * S3 stores data in alphabetical order (lexigraphical order)
 * Largest file size one can transfer to S3 using a PUT operation: 5 GB ([http://docs.aws.amazon.com/AmazonS3/latest/dev/UploadingObjects.html Use multi-part upload for files larger than 5GB])
 * S3 - Securing your buckets:

@@ Line 194: / Line 194: @@
 ** 80 TB Snowball in all regions. Snowball uses multiple layers of security designed to protect your data, including tamper-resistant enclosures, 256-bit encryption, and an industry-standard Trusted Platform Module (TPM) designed to ensure both security and full chain-of-custody of your data. Once the data transfer job has been processed and verified, AWS performs a software erasure of the Snowball appliance.
 * Snowball Edge (re:Invent 2016)
-** 100TB data transfer device with on-board storage and compute capabilities. You can use Snowball Edge to move large amounts of data into and out of AWS, as a temporary storage tier for large local datasets, or to support local workloads in remote of offline locations.
+** 100TB data transfer device with on-board storage and compute capabilities (e.g., Lambda functions). You can use Snowball Edge to move large amounts of data into and out of AWS, as a temporary storage tier for large local datasets, or to support local workloads in remote of offline locations.
 ** Snowball Edge connects to your existing appliances and infrastructure using standard storage interfaces, streamlining the data transfer process and minimizing setup and integration. Snowball Edge can cluster together to form a local storage tier and process your data on-premise, helping to ensure your applications continue to run even when they are not able to access the Cloud.
 * Snowmobile (re:Invent 2016)

← Older revision		Revision as of 00:55, 28 January 2017
Line 184:		Line 184:

	==AWS Snowball==		==AWS Snowball==
		+
		+	Prior to AWS Snowball, there was AWS Import/Export Disk, which accelerated the moving of large amounts of data into and out of the AWS Cloud using portable storage devices for transport. AWS Import/Export Disk transfers your data directly onto and off of storage devices using Amazon's high-speed internal network and bypassing the Internet.
		+
		+	AWS Snowball was introduce at re:Invent 2015.
		+
		+	Types of Snowballs:
		+	* Snowball (re:Invent 2015)
		+	** petabyte-scale data transport solution that uses secure appliances to transfer large amounts of data into and out of AWS S3. Using Snowball addresses common challenges with large-scale data transfers, including high network costs, long transfer times, and security concerns. Transferring data with Snowball is simple, fast, secure, and can be as little as one-fifth the cost of high-speed Internet.
		+	** 80 TB Snowball in all regions. Snowball uses multiple layers of security designed to protect your data, including tamper-resistant enclosures, 256-bit encryption, and an industry-standard Trusted Platform Module (TPM) designed to ensure both security and full chain-of-custody of your data. Once the data transfer job has been processed and verified, AWS performs a software erasure of the Snowball appliance.
		+	* Snowball Edge (re:Invent 2016)
		+	** 100TB data transfer device with on-board storage and compute capabilities. You can use Snowball Edge to move large amounts of data into and out of AWS, as a temporary storage tier for large local datasets, or to support local workloads in remote of offline locations.
		+	** Snowball Edge connects to your existing appliances and infrastructure using standard storage interfaces, streamlining the data transfer process and minimizing setup and integration. Snowball Edge can cluster together to form a local storage tier and process your data on-premise, helping to ensure your applications continue to run even when they are not able to access the Cloud.
		+	* Snowmobile (re:Invent 2016)
		+	** Exabyte-scale data transfer service using to move extremely large amounts of data to AWS. You can transfer up to 100PB per Snowmobile, a 45-foot long ruggedized shipping container, pulled by a semi-trailer truck. Snowmobile makes it easy to move massive amounts of data to the Cloud, including video libraries, image repositories, or even a complete data centre migration. Transferring data with Snowmobiles is secure, fast, and cost effective.

	==External links==		==External links==

@@ Line 182: / Line 182: @@
 # Tape Gateway (aka Gateway Virtual Tape Library). Backup/archiving solution.
 #: Offers a durable, cost-effective solution to archive your data in the AWS Cloud. The Virtual Tape Library (VTL) interface it provides lets you leverage your existing tape-based backup application infrastructure to store data on virtual tape cartridges that you create on your tape gateway. Each tape gateway is pre-configured with a media changer and tape drives, which are available to your existing client backup applications as iSCSI devices. You add tape cartridges as you need to archive your data. Supported by NetBackup, Backup Exec, Veam, etc. Virtual Tape backed by Amazon Glacier.
 ==External links==

@@ Line 172: / Line 172: @@
 Types of Storage Gateways:
-# File Gateway (NFS). Store flatfiles in S3
+# File Gateway (NFS). Store flatfiles in S3.
 #: Files are stored as objects in your S3 buckets, accessed through a Network File System (NFS) mount point. Ownership, permissions, and timestamps are durably stored in S3 in the user-metadata of the object associated with the file. Once objects are transferred to S3, they can be managed as native S3 objects, and bucket policies such as versioning, lifecycle management, and cross-region replication apply directly to objects stored in your bucket.
 # Volumes Gateway (iSCSI). A Virtual HDD.
@@ Line 178: / Line 178: @@
 #: Data written to these volumes can be asynchronously backed up as point-in-time snapshots of your volumes, and stored in the cloud as AWS EBS snapshots.
 #: Snapshots are incremental backups that capture only changed blocks. All snapshot storage is also compressed to minimize your storage charges.
-#:* Stored Volumes
+#:* Stored Volumes: These let you stored your primary data locally, while asynchronously backing up that data to AWS. Stored Volumes provide your on-premise applications with low-latency access to their entire datasets, while providing durable, off-site backups. You can create storage volumes and mount them as iSCSI devices from your on-premise application servers. Data written to your stored volumes is stored on your on-premise storage hardware. This data is asynchronously backup up to S3 in the form of Amazon Elastic Block Store (EBS) snapshots. 1 GB - 16 TB in size for Stored Volumes.
-#: Cached Volumes
+#:* Cached Volumes: These let you use S3 as your primary data storage while retaining frequently accessed data locally in your storage gateway. Cached volumes minimize the need to scale your on-premise storage infrastructure, while still providing your applications with low-latency access to their frequently accessed data. You can create storage volumes up to 32 TiB in size and attach to them as iSCSI devices from your on-premise application servers. Your gateway stores data that you write to these volumes in S3 and retains recently read data in your on-premise storage gateway's cache and upload buffer storage. 1 GB - 32 TB in size for Cached Volumes.
-# Tape Gateway (VTL). Backup/archiving solution.
+# Tape Gateway (aka Gateway Virtual Tape Library). Backup/archiving solution.
 ==External links==

@@ Line 165: / Line 165: @@
 </div>
 <br clear="all"/>
 ==External links==

@@ Line 55: / Line 55: @@
 * S3 stores data in alphabetical order (lexigraphical order)
 * Largest file size one can transfer to S3 using a PUT operation: 5 GB ([http://docs.aws.amazon.com/AmazonS3/latest/dev/UploadingObjects.html Use multi-part upload for files larger than 5GB])
 * S3 Static Website:
@@ Line 81: / Line 99: @@
 * S3 - Lifecyle Management
-** Transition objects to Infrequent Access Storage Class (must wait a minimum of 30 days from initial upload for the object to transition to the new storage class; minimum object size 128KB) or Glacier Storage Class after x amount of days.
+** Transition objects to Infrequent Access Storage Class (must wait a minimum of 30 days from initial upload for the object to transition to the new storage class; minimum object size 128KB) or Glacier Storage Class after ''x'' amount of days.
 ** Infrequent retrieval: ~milliseconds
 ** Glacier retrieval: 3 - 5 hours
 ** With Versioning: Transition object versions as well (including deleting old/current versions)
 ** Can not use Reduced Durability Storage Class with Lifecycle Management
 * CloudFront (a content delivery network {CDN})
@@ Line 94: / Line 114: @@
 ** Distribution: This is the name given to the CDN, which consists of a collection of Edge Locations.
 *** Web Distribution: Typically used for websites.
-*** RTMP: Used for media streaming (e.g., Adobe Flash)
+*** [[:wikipedia:Real-Time Messaging Protocol|Real-Time Messaging Protocol]] (RTMP): Used for media streaming (e.g., Adobe Flash)
 ** Edge location are not just READ only, one can write to them as well (i.e., PUT and object to them)
 ** Objects are cached for the life of the TTL (Time To Live)
-** One can clear the cache of an object stored on an Edge Location before the TTL expires, but one will be charged for that service.
+** One can clear the cache of an object stored on an Edge Location before the TTL expires, but one will be charged for that service. Create an "invalidation" request to clear the cache of a given object.
 * [http://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html S3 Transfer Acceleration]:

@@ Line 64: / Line 64: @@
 ** One needs to enable CORS for one S3 bucket to reference objects in another S3 bucket
-* S3 Versioning
+* S3 - Versioning
 ** Stores all versions of an object (including all writes, including deleting the object)
 ** Great backup tool
@@ Line 72: / Line 72: @@
 ** Cross Region Replication, requires versioning enabled on the source bucket.
-* S3 Lifecyle Management
+* S3 - Cross Region Replication
 ** Transition objects to Infrequent Access Storage Class (must wait a minimum of 30 days from initial upload for the object to transition to the new storage class; minimum object size 128KB) or Glacier Storage Class after x amount of days.
 ** Infrequent retrieval: ~milliseconds