Chapter 13: Data Storage Structures
File Organization
The database is stored as a collection of files. Each file is a sequence of
records. A record is a sequence of fields.
One approach
• Assume record size is fixed
• Each file has records of one particular type only
• Different files are used for different relations
This case is easiest to implement; will consider variable length records
later
We assume that records are smaller than a disk block
Simple approach
•Instead of allocating variable length bytes, we allocate maximum number
of bytes that each attribute can hold (Eg. 53 bytes for attributes ID, name,
dept_name, salary)
13.2
Fixed-Length Records
Two problems with the simple approach:
Record access is simple but records may cross blocks
Difficult to delete record – space occupied by record to be deleted
must be filled with some other record
Modification: do not allow records to cross block boundaries and
leave remaining blocks unused
13.3
Fixed-Length Records
Deletion of record i: alternatives:
move records i + 1, . . ., n to i, . . . , n – 1
move record n to i
do not move records, but link all free records on a free list
Record 3 deleted
13.4
Fixed-Length Records
Deletion of record i: alternatives:
move records i + 1, . . ., n to i, . . . , n – 1
move record n to i
do not move records, but link all free records on a free list
Record 3 deleted and replaced by record 11
13.5
Fixed-Length Records
Deletion of record i: alternatives:
move records i + 1, . . ., n to i, . . . , n – 1
move record n to i
do not move records, but link all free records on a free list
13.6
Variable-Length Records
Variable-length records arise in database systems in several ways:
Storage of multiple record types in a file.
Record types that allow variable lengths for one or more fields such
as strings (varchar)
Record types that allow repeating fields like arrays or multisets (used
in some older data models).
Attributes are stored in order
Fixed length attributes (numeric, dates, etc) are allocated bytes required
to store the value
Variable length attributes represented by fixed size (offset, length), with
actual data stored after all fixed length attributes
Null-value bitmap indicates attributes of records that have Null values
Representation of variable-length record
13.7
Variable-Length Records: Slotted Page Structure
Slotted page structure – to address the problem of storing variable –
length records in a block
Slotted page header contains:
number of record entries in the header
end of free space in the block
An array with location and size of each record
Records are allocated contiguously in the block, starting from end of the
block
Header ------Free space ----------records corresponding to the headers
13.8
Variable-Length Records: Slotted Page Structure
If record is deleted, space is freed
Records in block before deleted record are moved and free space is
created by deletion
Records can be moved around within a page to keep them contiguous
with no empty space between them; entry in the header must be updated.
Pointers should not point directly to record — instead they should point to
the entry for the record in header
13.9
Storing Large Objects
E.g., blob/clob types
Records must be smaller than pages
Alternatives:
• Store as files in file systems
• Store as files managed by database
• Break into pieces and store in multiple tuples in separate relation
PostgreSQL TOAST
13.10
Organization of Records in Files
Heap file organization – record can be placed anywhere in the file where
there is space
Sequential file organization – store records in sequential order, based on
the value of the search key of each record
In a multitable clustering file organization records of several different
relations can be stored in the same file
Motivation: store related records on the same block to minimize I/O
B+-tree file organization
Ordered storage even with inserts/deletes
Hashing file organization – a hash function computed on search key; the
result specifies in which block of the file the record should be placed
13.11
Heap File Organization
Records can be placed anywhere in the file where there is free space
Records usually do not move once allocated
Important to be able to efficiently find free space within file
13.12
Sequential File Organization
Suitable for applications that require sequential processing of
the entire file
The records in the file are ordered by a search-key
13.13
Sequential File Organization (Cont.)
Deletion – use pointer chains
Insertion –locate the position where the record is to be inserted
if there is free space insert there
if no free space, insert the record in an overflow block
In either case, pointer chain must be updated
Need to reorganize the file
from time to time to restore
sequential order
13.14
Multitable Clustering File Organization
Store several relations in one file using a multitable clustering
file organization
department
instructor
multitable clustering
of department and
instructor
13.15
Multitable Clustering File Organization (cont.)
good for queries involving department ⨝ instructor, and for queries
involving one single department and its instructors
bad for queries involving only department
results in variable size records
Can add pointer chains to link records of a particular relation
13.16
Partitioning
Table partitioning: Records in a relation can be partitioned into smaller
relations that are stored separately
E.g., transaction relation may be partitioned into
transaction_2018, transaction_2019, etc.
Queries written on transaction must access records in all partitions
• Unless query has a selection such as year=2019, in which case only
one partition in needed
Partitioning
• Reduces costs of some operations such as free space management
• Allows different partitions to be stored on different storage devices
E.g., transaction partition for current year on SSD, for older years
on magnetic disk
13.17
Data Dictionary Storage
The Data dictionary (also called system catalog) stores
metadata; that is, data about data, such as
Information about relations
names of relations
names, types and lengths of attributes of each relation
names and definitions of views
integrity constraints
Names of authorized Users and accounting information about users,
including passwords
Statistical and descriptive data
number of tuples in each relation
Method of storage for each relation (clustered/non-clustered)
Physical file organization information
How relation is stored (sequential/hash/…)
Physical location of relation
Information about indices
Name of index, relation being indexed, attributes on which index is
defined, type of index
13.18
Relational Representation of System Metadata
Miniature database
Relational
representation on
disk
Usage of
specialized data
structures designed
for efficient access,
in memory
Relational schema representing system
metadata.
13.19
Storage Access
Blocks are units of both storage allocation and data transfer.
Database system seeks to minimize the number of block transfers
between the disk and memory. We can reduce the number of disk
accesses by keeping as many blocks as possible in main memory.
Buffer – portion of main memory available to store copies of disk blocks.
Buffer manager – subsystem responsible for allocating buffer space in
main memory.
13.20
Buffer Manager
Programs call on the buffer manager when they need a block from disk.
• If the block is already in the buffer, buffer manager returns the
address of the block in main memory
• If the block is not in the buffer, the buffer manager
Allocates space in the buffer for the block
• Replacing (throwing out) some other block, if required, to make
space for the new block.
• Replaced block written back to disk only if it was modified
since the most recent time that it was written to/fetched from
the disk.
Reads the block from the disk to the buffer, and returns the
address of the block in main memory to requester.
13.21
Buffer Manager
Buffer replacement strategy: LRU – Least Recently used scheme
Pinned block: memory block that is not allowed to be written back to disk
• Pin done before reading/writing data from a block
• Unpin done when read /write is complete
• Multiple concurrent pin/unpin operations possible
Keep a pin count, buffer block can be evicted only if pin count = 0
Forced output of blocks:
• Situation – to write back block to disk even if not needed
13.22
Buffer-Replacement Policies
Most operating systems replace the block least recently used (LRU
strategy)
• Idea behind LRU – use past pattern of block references as a predictor
of future references
• LRU can be bad for some queries
Queries have well-defined access patterns (such as sequential scans),
and a database system can use the information in a user’s query to predict
future references
Mixed strategy with hints on replacement strategy provided
by the query optimizer is preferable
Example of bad access pattern for LRU: when computing the join of 2
relations r and s by a nested loops
for each tuple tr of r do
for each tuple ts of s do
if the tuples tr and ts match …
13.23
Buffer-Replacement Policies (Cont.)
Toss-immediate strategy – frees the space occupied by a block as soon
as the final tuple of that block has been processed
Most recently used (MRU) strategy – system must pin the block
currently being processed. After the final tuple of that block has been
processed, the block is unpinned, and it becomes the most recently used
block.
Buffer manager can use statistical information regarding the probability
that a request will reference a particular relation
• E.g., the data dictionary is frequently accessed. Heuristic: keep
data-dictionary blocks in main memory buffer
Operating system or buffer manager may reorder writes
• Can lead to corruption of data structures on disk
E.g., linked list of blocks with missing block on disk
File systems perform consistency check to detect such situations
• Careful ordering of writes can avoid many such problems
13.24
Optimization of Disk Block Access (Cont.)
Buffer managers support forced output of blocks for the purpose of recovery
Nonvolatile write buffers speed up disk writes by writing blocks to a non-
volatile RAM or flash buffer immediately
• Writes can be reordered to minimize disk arm movement
Log disk – a disk devoted to writing a sequential log of block updates
• Used exactly like nonvolatile RAM
Write to log disk is very fast since no seeks are required
Journaling file systems write data in-order to NV-RAM or log disk
• Reordering without journaling: risk of corruption of file system data
13.25
