Servicios de almacenamiento en AWS

Henry AlvaradoSolutions Architect, AWS Colombiagomhenry@amazon.com

Experience Day, Cali

Block vs File vs ObjectBlock StorageRaw StorageData organized as an array of unrelated blocksHost File System places data on diske.g.: Microsoft NTFS, Unix ZFS

File StorageUnrelated data blocks managed by a file (serving) systemNative file system places data on disk

Object StorageStores Virtual containers that encapsulate the data, data attributes, metadata and Object IDsAPI Access to dataMetadata Driven, Policy-based, etc

Storage - Characteristics

Durability Availability Security Cost Scalability Performance Integration

Measure of expected data loss

Measure of expected downtime

Security measures in place

Amount per storage unit, e.g. $ / GB

Upward flexibility

Performance metrics

Ability to interact with

Some of the ways we look at storage

AWS has a variety of storage optionsAmazon EBS (Elastic Block Storage)

Amazon Elastic File System (EFS)

Amazon S3 (Simple Storage Service)

Amazon Glacier

AWS Storage Gateway

Amazon Snowball & Snowball Edge

AWS Snowmobile New!New!

Amazon EBS

• Persistent block level storage for EC2• Pay only for what you provision• Native redundancy and write cache• Consistent and low-latency performance• Optimized for random I/O• Native support for encryption at rest (data volumes)

AWS EBS FeaturesDurable Secure

Low-latency SSD Consistent I/O PerformanceStripe multiple volumes for higher I/O performance

Identity and Access PoliciesEncryption

ScalableUnlimited capacity when you need itEasily scale up and down

Performance Backup

Designed for five9’s reliabilityRedundant storage across multiple devices within an AZ

Point-in-time SnapshotsCopy snapshots across AZ and Regions

EBS Volume TypesSolid-State Drives (SSD) Hard disk Drives (HDD)

Volume Type General Purpose SSD (gp2)* Provisioned IOPS SSD (io1)

Throughput Optimized HDD (st1)

Cold HDD (sc1)

Description General purpose SSD volume that balances price and performance for a wide variety of transactional workloads

Highest-performance SSD volume designed for mission-critical applications

Low cost HDD volume designed for frequently accessed, throughput-intensive workloads

Lowest cost HDD volume designed for less frequently accessed workloads

Use Cases • Recommended for most workloads

• System boot volumes• Virtual desktops• Low-latency

interactive apps• Dev and test

environments

• Critical business applications that require sustained IOPS performance, or more than 10,000 IOPS or 160 MiB/s of throughput per volume

• Large database workloads

• Streaming workloads requiring consistent, fast throughput at a low price

• Big data• Data warehouses• Log processing• Cannot be a boot

volume

• Throughput-oriented storage for large volumes of data that is infrequently accessed

• Scenarios where the lowest storage cost is important

• Cannot be a boot volume

Volume Size 1 GiB - 16 TiB 4 GiB - 16 TiB 500 GiB - 16 TiB 500 GiB - 16 TiBMax. IOPS**/Volume 10,000 20,000 500 250Max. Throughput/Volume† 160 MiB/s 320 MiB/s 500 MiB/s 250 MiB/sMax. IOPS/Instance 65,000 65,000 65,000 65,000Max. Throughput/Instance 1,250 MiB/s 1,250 MiB/s 1,250 MiB/s 1,250 MiB/sDominant Performance Attribute

IOPS IOPS MiB/s MiB/s

*Default volume type**gp2/io1 based on 16KiB I/O size, st1/sc1 based on 1 MiB I/O size† To achieve this throughput, you must have an instance that supports it, such as r3.8xlarge or x1.32xlarge.

Elastic File System (EFS)

Amazon EFS is Simple

• Fully managed- No hardware, network, file layer- Create a scalable file system in seconds!

• Seamless integration with existing tools and apps- NFS v4.1—widespread, open- Standard file system access semantics- Works with standard OS file system APIs

• Simple pricing = simple forecasting

1

Amazon EFS is Elastic

• File systems grow and shrink automatically as you add and remove files

• No need to provision storage capacity or performance

• You pay only for the storage space you use, with no minimum fee

2

• File systems can grow to petabyte scale

• Throughput and IOPS scale automatically as file systems grow

• Consistent low latencies regardless of file system size

• Support for thousands of concurrent NFS connections

Amazon EFS is Scalable3

Amazon S3 (Simple Storage Service)

• Web accessible object store• Pay for exactly what you use• Highly durable (99.999999999% design)• Limitlessly scalable• Natively online• Two flavors:

– Standard Storage - $0.023 * per GB / mo– Standard – Infrequent Access Storage (min size 128KB) – $0.0125* per GB / mo + Data

retrieval cost* (US East (N Virginia) pricing)

Amazon S3 (Simple Storage Service)• Parallel I/O for max speed (Multipart Upload, Ranged GETs)

• Resource-level IAM permissions• Bucket Policies & ACLs• Direct access through APIs• Server Side Encryption• Static Website Hosting• Data Lifecycle Rules• Amazon Athena – New

– Interactive Query Service that makes it easy to analyze data in Amazon S3 using standard SQL

Object Storage Tiering

S3 Standard

• Primary data• Big Data

Analytics• Small objects• Temporary

scratch space

S3 - IA

• File sync and share• Active Archive• Enterprise backup• Media transcoding• Geo-redundancy/

DR

Glacier

• Deep/offline archives

• Tape vaulting replacement

• WORM-compliant data

Data tiering using S3 Life Cycle Policies

Amazon Glacier• Low-Cost Archival Storage• Secure

• SSL & AES-256

• Durable• Designed for 99.999999999% durability

• Optimized for data archiving and backup• Suitable for RTO measured in hours• Includes storage costs and retrieval costs

• Three retrieval options: Expedited, Standard, Bulk • As little as $0.004 per GB/Month (US East pricing)• Integrated with S3

Lifecycle

AvailableS3: 99.99%

S3-IA: 99.9%

PerformantLow Latency

High Throughput≥ 30 Days≥ 128K

≥ 90 Days

Durable99.999999999%

ScalableElastic capacity No preset limits

> 0K$0.004 / GB per month

$0.0125 / GB per month

“Hot” DataActive and/or

Temporary Data

“Warm” DataInfrequently

Accessed Data

“Cold” DataArchive and

Compliance Data

≥ 0 Days> 0KStarts at $0.023 / GB per month

1-5 mins

$0.01/GB retrieval

Storage Tiered To Your Requirements

S3-IA

Glacier

S3

3 new retrieval options

3–5 hrs 5–12 hrs

Expedited Standard Bulk$0.03 / GB $0.01 / GB $0.0025 / GB

Amazon CloudFront

• Content delivery network (CDN)• Distribute content to end users with low latency and high

data transfer rates• Supports cookie and query string forwarding• Accelerate data uploaded from end users• Multi-format live streaming

AWS Import/Export

• Accelerates moving large amounts of data into and out of the AWS cloud by shipping a portable storage devices such as eSATA/SATA based hard drives or USB flash drives

• Faster than Internet transfer and more cost effective than upgrading your connectivity

• Supports data transfer into Amazon S3 buckets, Amazon EBS snapshots, and Amazon Glacier

• Common use cases are database migrations, offsite backups, and disaster recovery

AWS Import/Export Snowball

• Petabyte-scale data transport solution that uses secure appliances to transfer large amounts of data into and out of AWS

• Snowball client encrypts and compresses data before transfering the data to the Snowball appliances

• Supports 1-Gigabit Ethernet, 10-Gigabit Ethernet, and 10 Gigabit SFP+

• If it takes more that a week to upload your data to AWS, then consider using Snowball

Introducing AWS Snowmobile• 45-foot long ruggedized shipping

container

• Up to 100PB of capacity

• Load data S3 or Glacier

• Dedicated security personnel, GPS tracking, alarm monitoring, 24/7 video surveillance, and optional escort security while in transit

• Data encrypted with 256-bit encryption keys, managed through KMS

AWS Storage Gateway

• Connect an on-premises software appliance VM with cloud-based storage like Amazon S3 or Amazon Glacier

• VM runs on VMWare ESXi or Microsoft Hyper-V• Mount as iSCSI device, and expose volumes as

Common Internet File System (CIFS), or Network File System (NFS) mount points to client machines

• Securely upload data to the AWS cloud for cost effective backup and rapid disaster recovery

Gracias