B
usiness requirements for data,
such as compliance regulations,
are often uid, stringent and
challenging. In the communications sector,
for example, most countries dictate that
call detail records, short and multimedia
message services (like text messages), and
wireless application protocol content
(from mobile devices) should be retained
for several years. Other industries, includ-
ing nance and healthcare, face similar,
externally governed regulations. Regardless
of the sector, data retention and ease of
scale have become key issues that need to
be addressed from both a technology and
cost perspective.
Storing older data in the integrated data
warehouse is not always the most efcient
or economical answer. An alternative solu-
tion for housing less frequently accessed
historical data is RainStor database software.
RainStor offers specialized data compression
and reduction capabilities that can reduce
2 petabytes of raw data to approximately
100TB, giving organizations signicant
storage and management savings.
Save Money, Time and Space
The RainStor database software, running
on a customers commodity hardware,
efciently stores large volumes of historical
multi-structured data at a much lower cost
per terabyte than other methods. With a
small storage footprintand value, pattern
and algorithmic data reduction techniques
compressing data by 95% or moreit
provides cost savings across the storage,
hardware and administrative resources
required for retaining information.
RainStor is the rst product of its kind
to offer such massive data reduction while
allowing the information to be easily and
quickly accessible with SQL queries and
business intelligence (BI) tools. If a user
requires more complex analysis, the data can
be imported to the Teradata Database using
the jointly developed FastConnect software
module. While other methods of data
storage can be cost and space prohibitive,
RainStor is efcient. If a companys data
compresses 20 times, up to 1.2 petabytes
of raw data can be stored in a single rack
of commodity servers. Thats equivalent to
1,500 LTO-4 tape cartridges.
Additionally, unlike a traditional
database in which data can be changed,
or a tape archive that allows data to be
overwritten or corrupted, the informa-
tion contained in the RainStor software
is completely tamperproof and secure.
That means its perfect for a clean audit
PAGE 1 l Teradata Magazine l Q2/2012 l 2012 Teradata Corporation l AR-6586
APPLIED SOLUTIONS
Compress to Impress
A fast, easy-to-deploy data retention solution from RainStor
comes at a fraction of the cost.
by Betsy Huntingdon and Deirdre Mahon
PAGE 2 l Teradata Magazine l Q2/2012 l 2012 Teradata Corporation l AR-6586
trailauditors can be sure that what
theyre seeing is the actual data and it has
not been changed.
Users can set auto-expire policies to
ensure that only the amount of data
required by law is retained. It is also
scalable to meet future growth needs by
simply appending more data into the same
RainStor environment. The RainStor solu-
tion (see gure) provides the ability to:
> Ingest. Multi-structured data can be
loaded quickly into the RainStor data-
base software from a range of sources,
including any Teradata platform or
machine log via delimited-text les.
Data is loaded into a table in blocks
of approximately 1 million records.
> Reduce. Data size and the required
storage can be reduced by a factor
of up to 40 to 1. By storing only the
unique value or patterns of values,
data is deduplicated and stored in
optimized tree structures. Algorithmic
and byte-level compression further
reduce the footprint.
> Comply. Metadata and content data
values are encapsulated separately, and
version and schema changes can be
managed as information is continu-
ally appended from changing source
systems. Data can be auto-purged
down to the record level, based on
congurable retention rules.
> Query. Users can query RainStor
database software with standard SQL
or BI tools. Algorithms learn the data
patterns during ingest. When a query
comes in, its parsed and processed to
identify the subset of partitions that
contains the query result; then parti-
tions can be eliminated. The query is
run over this reduced set of data for a
more efcient query response.
> Scale. Data storage and query capabili-
ties can be scaled in parallel to many
servers and to data volumes in the
multi-petabyte range. This delivers
high levels of compression and direct
access to data via SQL.
> Manage. Very little administration is
required. During setup, users deter-
mine source system les and exact data
sets, then choose the data movement
method, which can be standard ETL or
the Teradata parallel-load utility.
Serious Cost Reduction
RainStor provides a low-cost online
repository for the long-term retention of
data for compliance and to satisfy users
who want larger historical data sets. Its
data reduction and compression technol-
ogy exceeds other solutions, which trans-
lates into more efcient multi-structured
data storage and has a serious impact on
the overall cost per terabyte. T
Betsy Huntingdon is a product market-
ing manager at Teradata. She has been at
Teradata for three years.
Deirdre Mahon is the vice president of
marketing at RainStor. She has more than
20 years of marketing experience.
Changing the Economics of Retaining Data FIGURE
The RainStor solution
is uniquely capable of
compressing massive
amounts of data into
a very small footprint
that remains online
and query-able.
Increased Storage Demands
As the amount of data increases, so does the need for cost-effective storage. The 2011 Informa-
tionWeek Analytics State of Storage Survey revealed that 66% of IT organizations are very
concerned about storage costs and most are faced with doubling storage capacity require-
ments every two to three years. And according to last years Cisco Visual Networking Index:
Global Mobile Data Trafc Forecast Update, 20102015, mobile data trafc will increase 26-
fold between 2010 and 2015, reaching 6.3 exabytes per month by 2015.
INGEST
Billions of
records/day
REDUCE
MANAGE
SCALE
~20-40 : 1
(95% + less)
COMPLY
Keep/purge
(retain)
QUERY
Billions of
records
(in seconds)
