Databases on AWS

Uriel Ramírez, Solutions Architect

Armando Barrales, Solutions Architect
April 2020
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Agenda
• AWS Database Services
• Traditional vs AWS Data services model
• Amazon RDS
• Amazon Redshift
• Amazon DynamoDB
• Amazon ElastiCache
• Amazon DocumentDB
• Amazon Aurora

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS database services
Purpose-built databases, the right tool for the right job

Aurora RDS DynamoDB ElastiCache Neptune

Key value In-memory Graph

DocumentDB Timestream QLDB

Document Time series Ledger

Database Migration Service

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Traditional Database Architecture

Client Tier

one database
for all App/Web Tier
workloads
RDBMS

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Traditional Database Architecture

Client Tier
Key-value access
Complex queries
App/Web Tier
OLAP transactions
Analytics
RDBMS
All forced into the
relational database

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Data Tier Architecture
Client Tier
On AWS choose best
database service for
each workload App/Web Tier

Data Tier

Cache Data Warehouse Time Series Blob Store

NoSQL RDBMS Quantum Ledger Search

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Workload Driven Data Store Selection
hot reads analytics
logging
NoSQL complex queries Periodic rich search
simple query & transactions data
Graph / Key Value / Document Untampered
data

Data Tier

Cache Data Warehouse Time Series Blob Store

NoSQL RDBMS Quantum Ledger Search

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 AWS Database Services for the Data Tier
hot reads analytics
logging
NoSQL complex queries Periodic rich search
simple query & transactions data
Graph / Key Value / Document Untampered
data

Data Tier
Amazon Amazon Amazon
Amazon S3
ElastiCache Redshift Timestream

Amazon DynamoDB Amazon Amazon Amazon

Neptune / DocumentDB RDS QLDB ElasticSearch

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 purpose
noun | pur·pose | \ˈpər-pəs\

The reason for which something is done or created

or for which something exists

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Things with
purpose

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon RDS
Managed relational database service with a choice of popular database engines

Easy to administer Performant & scalable Available & durable Secure and compliant

Easily deploy and maintain Scale compute Automatic Multi-AZ data Data encryption at rest and in
hardware, OS and DB and storage with a few clicks; replication; automated transit; industry compliance
software; built-in monitoring minimal downtime for your backup, snapshots, and and assurance programs
application failover

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
If you host your databases on-premises…
App optimization
Scaling
High availability
Database backups
DB s/w patches
DB s/w installs
OS patches
OS installation
Server maintenance
Rack & stack
Power, HVAC, net

you

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
If you host your databases in Amazon EC2…
App optimization
Scaling
High availability
Database backups
DB s/w patches
DB s/w installs
OS patches
OS installation OS installation
Server maintenance Server maintenance
Rack & stack Rack & stack
Power, HVAC, net Power, HVAC, net

you

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
If you choose Amazon RDS…
App optimization
Scaling Scaling
High availability High availability
Database backups Database backups
DB s/w patches DB s/w patches
DB s/w installs DB s/w installs
OS patches OS patches
OS installation OS installation
Server maintenance Server maintenance
Rack & stack Rack & stack
Power, HVAC, net Power, HVAC, net

you

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Key Amazon RDS Features Multi-AZ
Amazon RDS Improve Increase Reduce
Configuration Availability Throughput Latency
Push-Button Scaling
Multi AZ
availabilit availabilit
Read Replicas y zone y zone
Provisioned IOPS Region

Push-Button Scaling Read Replicas

Provisioned IOPS

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Aurora
MySQL and PostgreSQL compatible relational database built for the cloud
Performance and availability of commercial-grade databases at 1/10th the cost

Performance Availability Highly Fully

& scalability & durability secure managed
5x throughput of standard Fault-tolerant, self-healing Network isolation, Managed by RDS: no
MySQL and 3x of standard storage; six copies of encryption at hardware provisioning,
PostgreSQL; scale-out up data across three AZs; rest/transit software patching, setup,
to15 read replicas continuous backup to S3 configuration, or backups

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Everything you get from Amazon RDS…
App optimization App optimization App optimization
Scaling Scaling Scaling
High availability High availability High availability
Database backups Database backups Database backups
DB software patches DB software patches DB software patches
Managed DB software installs DB software installs DB software installs Managed
by you OS patches OS patches OS patches by AWS
OS installation OS installation OS installation
Server maintenance Server maintenance Server maintenance
Rack and stack Rack and stack Rack and stack
Power, HVAC, net Power, HVAC, net Power, HVAC, net

Database on-premises Database on EC2 Amazon RDS

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 for as low as
$934/TB per year

Petabyte scale
Massively parallel
Amazon Columnar Store
Redshift

Relational data warehouse

Fully managed = no admin

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Redshift – Data Warehousing
Fast, powerful, and simple data warehousing at 1/10 the cost
Massively parallel, petabyte scale

Fast Inexpensive Scalable Secure

$
Columnar storage As low as $1000 per Resize your cluster up Data encrypted at rest
technology to improve I/O terabyte per year, and down as your and transit. Isolate
efficiency and parallelize 1/10th the cost of performance and clusters with VPC.
queries. Data load scales traditional data capacity needs Manage your own keys
linearly. warehouse solutions change with KMS

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Redshift cluster architecture
Massively parallel, shared nothing architecture
Streaming Backup/Restore from S3 JDBC/ODBC

Redshift Cluster
Leader node
Leader Node
• SQL endpoint
• Stores metadata
• Coordinates parallel SQL processing
Compute Nodes
Compute nodes
• Local, columnar storage
• Executes queries in parallel Efficient Data Loads
• Load, backup, restore Streaming Backup/Restore

• 2, 16, or 32 slices

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Redshift Spectrum
Run SQL queries directly against data in S3 using thousands of nodes

High concurrency: Multiple No ETL: Query data in-place Full Amazon Redshift SQL
clusters access same data using open file formats support

S3 SQL

Fast at exabyte scale Elastic and highly available On-demand, pay-per-query

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 NoSQL database
Seamless scalability
Zero admin
Amazon
DynamoDB
Single-digit millisecond latency
Multi-Master
Multi-Region
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon DynamoDB

Highly available
Fully managed Consistently fast at any scale and durable

Integrates with AWS Lambda,

Secure Cost-effective
Amazon Redshift, and more
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Highly available and durable

Designed to support
Built for high durability 99.99%
of availability

WRITES READS

3-way replication Strongly or eventually consistent

Persisted to disk No latency trade-off
(Custom SSD) Data is always replicated to
three Availability Zones

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
NoSQL vs. SQL for a new app: how to choose?
Want simplest possible DB Need joins, transactions, frequent
management? table scans?
Want app to manage DB integrity? Want DB engine to manage DB
integrity?
Team has SQL skills?

Amazon Amazon
DynamoDB RDS

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Introducing Amazon ElastiCache
Fully-managed, Redis or Memcached compatible, low-latency, in-memory data
store

Extreme Fully Easily

Performance Managed Scalable
In-memory data store and AWS manages all Read scaling with
cache for sub-millisecond hardware and software replicas. Write and memory
response times setup, configuration, scaling with sharding.
monitoring Non disruptive scaling

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 µs is the new ms

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon ElastiCache
• In-memory cache in the cloud
• Improve latency and throughput for read-heavy
workloads
• Supports open-source caching engines
• Memcached
• Redis
• Fully managed
• Multi-AZ

Examples
• Caching of MySQL database query results
• Caching of post-processing results
• Caching of user session and frequently accessed data
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 (Now GA)

Fully managed document database

MongoDB-compatible
Amazon Multi-AZ
DocumentDB

Performance at scale

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Why use a document database?

The JSON document model maps naturally to

application data

Each document can have a different data

structure and is independent of other documents

Index on any key in a document, and run ad hoc and

aggregation queries across your data set

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Use cases for document databases

Content Mobile Personalization

Management

Catalog Retail and User profiles

Marketing
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Use case: Profile Management
users table tankfight_users table
id usernam first_nam last_name id hi_scor global_ra
e e e nk
18127 sue1942 Susan Benoit 18127 3185400 5139
6 6

{
id: 181276,
{ username: 'sue1942',
id: 181276, name: {first: 'Susan',
username: 'sue1942', last: 'Benoit'},
name: {first: 'Susan', tankfight: {
last: 'Benoit'}, hi_score: 3185400,
} global_rank: 5139
}
}

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
MongoDB – #1 NoSQL database engine

Over 60 million downloads

Easy for developers to get started
MongoDB API is rich and powerful
Easy to programmatically query

Source: http://db-engines.com
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
MongoDB Architecture
Sharded cluster scaling dramatically increases operational complexity

Availability Zone 1 Availability Zone 2 Availability Zone 3

Shard Shard Shard

Router Router Router

reads / writes

Shard
Shard Shard
Primary Primary Primary
replication

Secondaries Secondaries Secondaries

Shard Balancing

Config Config Config

Server Server Server

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Running MongoDB is difficult……

TCO

What if you could create and

Experts only scale MongoDB compatible
database clusters in minutes?

Time to scale

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon DocumentDB
Fast, scalable, highly available, fully managed MongoDB-compatible database
service

Millions of requests per Same code, drivers, and Simple

second, millisecond latency tools you use with MongoDB and fully managed

2x throughput of Deeply integrated

Secure and compliant
managed MongoDB services with AWS services

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
DocumentDB Architecture
Separate compute and storage provide 2x throughput of current MongoDB
managed services
AWS Region

Availability Zone 1 Availability Zone 2 Availability Zone 3

Instance Instance Instance

(primary)
writes (replica) (replica)
reads

reads
wri

reads
tes writes

Data copies Data copies Data copies

Cluster volume
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Fast
Fast, reliable, and fully-managed MongoDB-compatible database service

Automatic
Fast More throughput Analytics
storage scaling

Millions of requests per Separation of storage and DocumentDB will Launch instances in minutes
second with millisecond compute offloads replication, automatically grow the size for analytical queries and
latency; scale-out up to providing 2x the throughput of of your storage volume as shut them down at the end
15 read replicas current MongoDB managed your cluster storage needs of the day
services grow.

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Reliable
Fast, reliable, and fully-managed MongoDB-compatible database service

Automatic failure Automatic Simplified Durable

recovery failover backups

Failing instances are Replicas are Continuous backups with Data is replicated six-
automatically detected and automatically promoted to point in time recovery. ways across three AZs
recovered; no cache warm- primary Scheduled snapshots.
up needed No performance impact.

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Fully-managed
Fast, reliable, and fully-managed MongoDB-compatible database service

Automatic patching Quick start Monitoring Integrated

Up-to-date with the latest Provision production- Over 20 key operational Deeply integrated with AWS
patches ready clusters in minutes metrics for your clusters services such as
at no extra charge CloudFormation, CloudTrail,
CloudWatch, DMS, IAM, VPC,
and more.

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
MongoDB-compatible
Fast, reliable, fully-managed MongoDB-compatible database

Same drivers, Migration with

MongoDB 3.6 Replica sets
tools DMS

Compatible with MongoDB Use the same MongoDB Live migrations with Read scaling is easy with
Community Edition 3.6 drivers and tools with DMS; free for 6-months automatic replica set
DocumentDB; as simple as configurations
changing an application
connection string

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Airbnb uses different databases based
on the purpose
User search history: Amazon DynamoDB
• Massive data volume
• Need quick lookups for personalized search
Session state: Amazon ElastiCache
• In-memory store for submillisecond site rendering
Relational data: Amazon RDS
• Referential integrity
• Primary transactional database

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 300M total users
7B exercises per month

CHALLENGE
Wanted to enable anyone to learn a
language for free.

SOLUTION
Purpose-built databases from AWS:
• DynamoDB: 31B items tracking
which language exercises completed
• Aurora: primary transactional
database for user data
• ElastiCache: instant access to
common words and phrases

Result:
More people learning a language on
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Duolingo than entire US school system
AWS database services
Purpose-built databases, the right tool for the right job

Aurora RDS DynamoDB ElastiCache Neptune

Key value In-memory Graph

DocumentDB Timestream QLDB

Document Time series Ledger

Database Migration Service

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
To learn more about AWS Databases…

1. For more information on AWS Databases and how to get started, visit:
aws.amazon.com/databases
2. For more information on migrating you database to AWS visit:
aws.amazon.com/database-migrations
3. For any other questions, contact your local representative

Werner Vogel’s blog: A one size fits all database

doesn't fit anyone
https://www.allthingsdistributed.com/2018/06/purpose-built-databases-in-aws.html

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 It’s all about
choice Performance-oriented
Cost-oriented

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Any questions?

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Thank you!

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.