An Expanded View of the Memory System

ECE4680
Computer Organization and Architecture Processor

Memory Hierarchy Control

Memory
Memory

Memory

Memory
Memory
Datapath

Speed: Fastest Slowest

Size: Smallest Biggest
Cost: Highest Lowest

ECE4680 Memory.1 2002-4-21 ECE4680 Memory.4 2002-4-21

The Big Picture: Where are We Now? Memory Hierarchy: Principles of Operation

°At any given time, data is copied between only 2 adjacent levels:
°The Five Classic Components of a Computer • Upper Level: the one closer to the processor
- Smaller, faster, and uses more expensive technology
Processor • Lower Level: the one further away from the processor
Input
- Bigger, slower, and uses less expensive technology
Control
Memory °Block:
• The minimum unit of information that can either be present or not
Datapath Output present in the two level hierarchy

Lower Level
To Processor Upper Level Memory
°Today’s Topic: Memory System Memory
Blk X
From Processor Blk Y

ECE4680 Memory.2 2002-4-21 ECE4680 Memory.5 2002-4-21

Who Cares About the Memory Hierarchy? Memory Hierarchy: Terminology

Processor-DRAM Memory Gap (latency) °Hit: data appears in some block in the upper level (example: Block X)
• Hit Rate: the fraction of memory access found in the upper level
µProc • Hit Time: Time to access the upper level which consists of
1000 60%/yr.
CPU RAM access time + Time to determine hit/miss
(2X/1.5yr)
“Moore’s Law” °Miss: data needs to be retrieve from a block in the lower level (Block Y)
Performance

100 Processor-Memory • Miss Rate = 1 - (Hit Rate)

Performance Gap: • Miss Penalty: Time to replace a block in the upper level +
(grows 50% / year) Time to deliver the block to the processor
10
DRAM °Hit Time << Miss Penalty
9%/yr.
DRAM
(2X/10 yrs) Lower Level
1 To Processor Upper Level Memory
Memory
2000
2001
2002
1999
1985

1988

1998
1981
1982
1983
1984

1986
1987

1989
1990
1991
1992
1993
1994
1995
1996
1997

Block X
From Processor Block Y

Time
ECE4680 Memory.3 2002-4-21 ECE4680 Memory.6 2002-4-21
 Memory Hierarchy: How Does it Work? Technology Trends

°Temporal Locality (Locality in Time): If an item is referenced, it will tend Capacity Speed (latency)
to be referenced again soon.
Logic: 2x in 3 years 2x in 3 years
• Keep more recently accessed data items closer to the processor
DRAM: 4x in 3 years 2x in 10 years
°Spatial Locality (Locality in Space): If an item is referenced, items
whose addresses are close by tend to be referenced soon.
Disk: 4x in 3 years 2x in 10 years
• Move blocks consists of contiguous words to the upper levels

DRAM
Year Size Cycle Time
1980 1000:1! 64 Kb 2:1! 250 ns
Lower Level 1983 256 Kb 220 ns
To Processor Upper Level Memory
Memory 1986 1 Mb 190 ns
Blk X 1989 4 Mb 165 ns
From Processor Blk Y 1992 16 Mb 145 ns
1995 64 Mb 120 ns

ECE4680 Memory.7 2002-4-21 ECE4680 Memory.10 2002-4-21

Memory Hierarchy of a Modern Computer System SPARCstation 20’s Memory System Overview

°By taking advantage of the principle of locality:

• Present the user with as much memory as is available in the Memory
cheapest technology. Controller Memory Bus (SIMM Bus) 128-bit wide datapath

Memory Module 7

Memory Module 5

Memory Module 4

Memory Module 3

Memory Module 1
• Provide access at the speed offered by the fastest technology.

Memory Module 2
Memory Module 6

Memory Module 0
Processor Processor Bus (Mbus) 64-bit wide

Control
Secondary
Storage Processor Module (Mbus Module)
Second Main
(Disk) SuperSPARC Processor
On-Chip
Registers

Level Memory
Cache

Datapath Cache (DRAM) Instruction

(SRAM) External Cache
Cache Register
File
Data
Cache
Speed (ns): 1s 10s 100s 10,000,000s (10s ms)
Size (bytes): 100s Ks Ms Gs

ECE4680 Memory.8 2002-4-21 ECE4680 Memory.11 2002-4-21

Memory Hierarchy Technology SPARCstation 20’s Memory Module

°Supports a wide range of sizes:
°Random Access:
• Smallest: 4 MB = 16 2Mb DRAM chips, 8 KB of Page Mode SRAM
• “Random” is good: access time is the same for all locations
• Biggest: 64 MB = 32 16Mb chips, 16 KB of Page Mode SRAM
• DRAM: Dynamic Random Access Memory
- High density, low power, cheap, slow
DRAM Chip 15
- Dynamic: need to be “refreshed” regularly
512 cols
• SRAM: Static Random Access Memory
256K x 8
- Low density, high power, expensive, fast
= 2 MB
- Static: content will last “forever” DRAM Chip 0 One page
512 rows

°“Non-so-random” Access Technology: 256K x 8 512 × 8 SRAM

• Access time varies from location to location and from time to time = 2 Mb
8 bits bits<127:0>
• Examples: Disk, tape drive, CDROM

°The next two lectures will concentrate on random access technology 512 × 8 SRAM
• The Main Memory: DRAMs bits<7:0> Memory Bus<127:0>
• Caches: SRAMs

ECE4680 Memory.9 2002-4-21 ECE4680 Memory.12 2002-4-21

 Summary:

°Two Different Types of Locality:

• Temporal Locality (Locality in Time): If an item is referenced, it will
tend to be referenced again soon.
• Spatial Locality (Locality in Space): If an item is referenced, items
whose addresses are close by tend to be referenced soon.

°By taking advantage of the principle of locality:

• Present the user with as much memory as is available in the
cheapest technology.
• Provide access at the speed offered by the fastest technology.

°DRAM is slow but cheap and dense:

• Good choice for presenting the user with a BIG memory system

°SRAM is fast but expensive and not very dense:

• Good choice for providing the user FAST access time.

ECE4680 Memory.13 2002-4-21