Amazon Dynamodb
Amazon Dynamodb
#amazon-
dynamodb
Table of Contents
About 1
Remarks 2
Examples 2
Installation or Setup 2
Introduction 2
Introduction 3
Remarks 3
Examples 3
Introduction 6
Remarks 6
Examples 6
cleanUpOldData 6
Remarks 8
Examples 8
Examples 12
Import a CSV file into a DynamoDB table using boto (Python package) 12
Remarks 14
The Models 14
Examples 15
Credits 17
About
You can share this PDF with anyone you feel could benefit from it, downloaded the latest version
from: amazon-dynamodb
It is an unofficial and free amazon-dynamodb ebook created for educational purposes. All the
content is extracted from Stack Overflow Documentation, which is written by many hardworking
individuals at Stack Overflow. It is neither affiliated with Stack Overflow nor official amazon-
dynamodb.
The content is released under Creative Commons BY-SA, and the list of contributors to each
chapter are provided in the credits section at the end of this book. Images may be copyright of
their respective owners unless otherwise specified. All trademarks and registered trademarks are
the property of their respective company owners.
Use the content presented in this book at your own risk; it is not guaranteed to be correct nor
accurate, please send your feedback and corrections to info@zzzprojects.com
https://riptutorial.com/ 1
Chapter 1: Getting started with amazon-
dynamodb
Remarks
This section provides an overview of what amazon-dynamodb is, and why a developer might want
to use it.
It should also mention any large subjects within amazon-dynamodb, and link out to the related
topics. Since the Documentation for amazon-dynamodb is new, you may need to create initial
versions of those related topics.
Examples
Installation or Setup
DynamoDB is a fully managed service provided by AWS. It does not need to be installed or
configured. AWS is responsible for all administrative burdens of operating, scalling and
backup/restore of the distributed database.
Introduction
https://riptutorial.com/ 2
Chapter 2: Batch Operations: Things to know
Introduction
Database is an integral part of any application and performance and persistance are real
challenges faced by any web application. NoSql databases are no different in this matter and need
to be dealt carefully. DynamoDB being one of the NoSQL database that is provided by Amazon
Web Services support batch operations in addition to the CRUD operations. Lets start with Batch
Operations. In this example we will learn how we can make use of Dynamo DB's JAVA SDK to
perform Batch Inserts.
Remarks
Good to know about Batch operations
1. Batch operation doesn't minimize the HTTP request count, rather it has more features to
handle when we receive a throttling error. To put it simple, each record that we insert into
dynamo db will consume one http request.
2. Sound implementation of batch operation is to first to cache data temporarily and once we
have the threshold number, ie 25, is the correct time to fire a batch request.
3. Any request that fails due to throttling will be retried between(500-999 ) milliseconds, as
recommended by Amazon best practices.
Examples
How to code the BatchWriteItemRequest and save data
try {
https://riptutorial.com/ 3
batchWriteItemResult.getUnprocessedItems();
if (unprocessedItems.size() == 0) {
System.out.println("No unprocessed items found");
} else {
System.out.println("Sleeping for: " +
ThreadLocalRandom.current().nextInt(500, 999 + 1));
Thread.sleep(ThreadLocalRandom.current().nextInt(500, 999 + 1));
wrList = unprocessedItems.get(forumTableName);
System.out.println("Retrieving the unprocessed items");
}
} catch (Exception e) {
System.err.println("Failed to retrieve items: ");
e.printStackTrace(System.err);
}
} catch (Exception e) {
}
}
This is what we need to know if we want to make use of batch operations to put data to a dynamo
db table. Lets see the steps that need to be followed to accomplish this. In this example, we are
trying to persist a list of EventTracker data, which is my POJO.
if (event != null) {
Map<String, AttributeValue> attributeMap = new HashMap<String, AttributeValue>();
https://riptutorial.com/ 4
item.put(attributeName, attributeValue);
}
https://riptutorial.com/ 5
Chapter 3: Dynamodb delete data over time
Introduction
Removing old data from dynamodb using a date attribute.
Remarks
My use case: removing old data from dynamodb using a date attribute.
• You can't query a table with using only range key attribute (date for example).
• You can only query a table using hash or hash+range key.
• You can't query a table using a hash key with '<' / '>' operations, only '='.
Possible Solutions:
Notes: Building the entity I was using the amazon dynamo annotations. I was using
DynamoDBQueryExpression to query, getting the result page with the defined Class object.
Examples
cleanUpOldData
https://riptutorial.com/ 6
calendar.get(Calendar.DAY_OF_YEAR)-1);
https://riptutorial.com/ 7
Chapter 4: How to create a DynamoDB Table
Remarks
When creating tables make sure to pay attention to the choice of attributes for the partition and
sort keys. See the published guidelines for working with tables.
Examples
Create Table in Java using Document API
In the following example we will be creating a table called Membership using the AWS Java SDK for
DynamoDB. The table will consist of items that represent team assignments. The table will be
partitioned by TeamID. Each team will have multiple members, identified by the MemberID (as a
sort key).
try {
// every DynamoDB table must have basic schema that determines
// which attributes are to be used as partition key (and optionally sort key)
List<KeySchemaElement> keySchema = new ArrayList<~>();
// required: specify the partition key (also called hash key)
keySchema.add(new KeySchemaElement()
.withAttributeName("TeamID")
.withKeyType(KeyType.HASH));
// optionally: add a sort key (also called a range key)
keySchema.add(new KeySchemaElement()
.withAttributeName("MemberID")
.withKeyType(KeyType.RANGE));
// after defining the key schema - the attributes that will be used as partition and
range key
// we need to specify these attributes' type
List<AttributeDefinition> attrDefinitions = new ArrayList<~>();
// we must specify the type for the TeamID attribute; suppose it's a string
attrDefinitions.add(new AttributeDefinition()
.withAttributeName("TeamID")
.withAttributeType("S"));
// if using a sort key we need to specify its type; suppose that it's numeric
attrDefinitions.add(new AttributeDefinition()
.withAttributeName("MemberID")
.withAttributeType("N"));
// after defining the attributes making up the schema and their type
// we build a request, specifying the table name and the provisioned capacity
CreateTableRequest request = new CreateTableRequest()
.withTableName("Membership")
.withKeySchema(keySchema)
.withAttributeDefinitions(attrDefinitions)
.withProvisionedThroughput(new ProvisionedThroughput()
https://riptutorial.com/ 8
.withReadCapacityUnits(30L)
.withWriteCapacityUnits(10L));
} catch (Exception e) {
// the table creation failed.
// the reason for failure may be determined from the exception
}
In following example we will create table movies with AWS Ruby SDK v2. Here, each Movie as one
unique Partition Key as id, and Range Key year. Apart from this we want to be able to query
movies with their name, hence we will create a Global Secondary Index (GSI) name-year-index with
name as Hash Key and year as Range Key. Movie can have other attributes such as released,
created_at, actor and actress. Schema for the table is shown below:
# array of attributes name and their type that describe schema for the Table and Indexes
attribute_definitions: [
{
attribute_name: "id",
attribute_type: "N"
}
{
attribute_name: "name",
attribute_type: "S",
},
{
attribute_name: "year",
attribute_type: "N",
https://riptutorial.com/ 9
}
],
# key_schema specifies the attributes that make up the primary key for a table
# HASH - specifies Partition Key
# RANGE - specifies Range Key
# key_type can be either HASH or RANGE
key_schema: [
{
attribute_name: "id",
key_type: "HASH",
},
{
attribute_name: "year",
key_type: "RANGE",
}
],
# global_secondary_indexes array specifies one or more keys that makes up index, with name
of index and provisioned throughput for global secondary indexes
global_secondary_indexes: [
index_name: "name-year-index",
key_schema: [
{
attribute_name: "name",
key_type: "HASH"
},
{
attribute_name: "year",
key_type: "RANGE"
}
],
# Projection - Specifies attributes that are copied (projected) from the table into the
index.
# Allowed values are - ALL, INCLUDE, KEYS_ONLY
# KEYS_ONLY - only the index and primary keys are projected into the index.
# ALL - All of the table attributes are projected into the index.
# INCLUDE - Only the specified table attributes are projected into the index. The list of
projected attributes are then needs to be specified in non_key_attributes array
projection: {
projection_type: "ALL"
},
https://riptutorial.com/ 10
$ddb.wait_until(:table_exists, {table_name: "movies"})
https://riptutorial.com/ 11
Chapter 5: How to insert data into table using
DynamoDb?
Examples
Import a CSV file into a DynamoDB table using boto (Python package)
import boto
items = []
count = 0
csv_file = open(csv_file_name, 'r')
for cur_line in csv_file:
https://riptutorial.com/ 12
count += 1
cur_line = cur_line.strip().split(',')
row = {}
for colunm_number, colunm_name in enumerate(colunm_names):
row[colunm_name] = column_types[colunm_number](cur_line[colunm_number])
item = dynamodb_table.new_item(
attrs=row
)
items.append(item)
if count % BATCH_COUNT == 0:
print 'batch write start ... ',
do_batch_write(items, table_name, dynamodb_table, dynamodb_conn)
items = []
print 'batch done! (row number: ' + str(count) + ')'
csv_file.close()
def main():
'''
Demonstration of the use of import_csv_to_dynamodb()
We assume the existence of a table named `test_persons`, with
- Last_name as primary hash key (type: string)
- First_name as primary range key (type: string)
'''
colunm_names = 'Last_name First_name'.split()
table_name = 'test_persons'
csv_file_name = 'test.csv'
column_types = [str, str]
import_csv_to_dynamodb(table_name, csv_file_name, colunm_names, column_types)
if __name__ == "__main__":
main()
#cProfile.run('main()') # if you want to do some profiling
test.csv's content (must be located in the same folder as the Python script):
John,Doe
Bob,Smith
Alice,Lee
Foo,Bar
a,b
c,d
e,f
g,h
i,j
j,l
Read How to insert data into table using DynamoDb? online: https://riptutorial.com/amazon-
dynamodb/topic/6354/how-to-insert-data-into-table-using-dynamodb-
https://riptutorial.com/ 13
Chapter 6: Using AWS DynamoDb with the
AWS .NET SDK
Remarks
Amazon DynamoDB is a fast NoSQL database service offered by Amazon Web Services (AWS).
DynamoDB can be invoked from .NET applications by using the AWS SDK for .NET. The SDK
provides three different models for communicating with DynamoDB. This topic is introduces the
various APIs in each model.
The Models
The SDK provides three ways of communicating with DynamoDB. Each one offers tradeoffs
between control and ease of use. See the AWS .NET SDK Reference for details on the APIs
below.
The three models provide different approaches to working with the service. While the low-level
approach requires more client-side code — the user must convert .NET types such as numbers
and dates to DynamoDB-supported strings — it provides access to all service features. By
comparison, the Object Persistence Model approach makes it easier to use the service—since the
user is for the most part working with familiar .NET objects—but does not provide all the
functionality. For example, it is not possible to make conditional Put calls with the Object
Persistence Model.
Learn more about working AWS using the .NET SDK in the .NET SDK Developer Guide.
Note: This topic was adapted with permission from a blog post originally published on the AWS
.NET SDK blog.
https://riptutorial.com/ 14
Examples
Low Level API Example
// Store item
client.PutItem(new PutItemRequest
{
TableName = "Books",
Item = new Dictionary<string, AttributeValue>
{
{ "Title", new AttributeValue { S = "Cryptonomicon" } },
{ "Id", new AttributeValue { N = "42" } },
{ "Authors", new AttributeValue {
SS = new List<string> { "Neal Stephenson" } } },
{ "Price", new AttributeValue { N = "12.95" } }
}
});
// Get item
Dictionary<string, AttributeValue> book = client.GetItem(new GetItemRequest
{
TableName = "Books",
Key = new Dictionary<string, AttributeValue>
{
{ "Id", new AttributeValue { N = "42" } }
}
}).Item;
// Store item
Document book = new Document();
book["Title"] = "Cryptonomicon";
book["Id"] = 42;
book["Authors"] = new List<string> { "Neal Stephenson" };
book["Price"] = 12.95;
booksTable.PutItem(book);
// Get item
book = booksTable.GetItem(42);
Console.WriteLine("Id = {0}", book["Id"]);
Console.WriteLine("Title = {0}", book["Title"]);
Console.WriteLine("Authors = {0}",
string.Join(", ", book["Authors"].AsListOfString()));
https://riptutorial.com/ 15
This example consists of two parts: first, we must define our Book type; second, we use it with
DynamoDBContext.
[DynamoDBTable("Books")]
class Book
{
[DynamoDBHashKey]
public int Id { get; set; }
public string Title { get; set; }
public List<string> Authors { get; set; }
public double Price { get; set; }
}
// Store item
Book book = new Book
{
Title = "Cryptonomicon",
Id = 42,
Authors = new List<string> { "Neal Stephenson" },
Price = 12.95
};
context.Save(book);
// Get item
book = context.Load<Book>(42);
Console.WriteLine("Id = {0}", book.Id);
Console.WriteLine("Title = {0}", book.Title);
Console.WriteLine("Authors = {0}", string.Join(", ", book.Authors));
Read Using AWS DynamoDb with the AWS .NET SDK online: https://riptutorial.com/amazon-
dynamodb/topic/5275/using-aws-dynamodb-with-the-aws--net-sdk
https://riptutorial.com/ 16
Credits
S.
Chapters Contributors
No
Batch Operations:
2 SACHESH A C
Things to know
Dynamodb delete
3 Lior
data over time
How to create a
4 Mike Dinescu, Parth Modi
DynamoDB Table
Using AWS
6 DynamoDb with the David Hale
AWS .NET SDK
https://riptutorial.com/ 17