Parallel Computing
Landscape
(CS 526)
Department of Computer Science,
The University of Lahore,
The cache coherence
problem
• Since, we have private caches:
How to keep the data consistent across caches?
• Each core should perceive the memory as a
monolithic array, shared by all the cores
The cache coherence
problem
Suppose variable x initially contains
15213
Core 1 Core 2 Core 3 Core 4
One or more One or more One or more One or more
levels of levels of levels of levels of
cache cache cache cache
multi-core chip
Main memory
x=15213
35
The cache coherence
problem
Core 1 reads
x
Core 1 Core 2 Core 3 Core 4
One or more One or more One or more One or more
levels of levels of levels of levels of
cache cache cache cache
x=15213
multi-core chip
Main memory
x=15213
36
The cache coherence problem
Core 2 reads
x
Core 1 Core 2 Core 3 Core 4
One or more One or more One or more One or more
levels of levels of levels of levels of
cache cache cache cache
x=15213 x=15213
multi-core chip
Main memory
x=15213
The cache coherence
problem
Core 1 writes to x, setting it to
21660
Core 1 Core 2 Core 3 Core 4
One or more One or more One or more One or more
levels of levels of levels of levels of
cache cache cache cache
x=21660 x=15213
multi-core chip
Main memory
x= 15213
The cache coherence
problem
Core 3 attempts to read x… gets a stale
copy
Both caches
Core 1 Core 2 Core 3 Core 4 contain
inconsistent data.
Unpredictable
One or more One or more One or more One or more Behavior
levels of levels of levels of levels of
cache cache cache cache
x=21660 x=15213
multi-core chip
Cache Coherence Protocols
Has many solutions: Main memory
x=15213
The cache coherence problem
To address the cache coherence problem, various cache coherence
protocols have been developed.
Two common cache coherence protocols are :
• MESI Protocol: MESI stands for Modified, Exclusive, Shared, and
Invalid.
• MOESI Protocol: MOESI extends the MESI protocol with an "Owned"
state
The cache coherence problem
• MESI Protocol: It is a widely used cache coherence protocol that
defines several states for each cache line (data block) in a cache,
allowing caches to coordinate reads and writes to maintain data
consistency.
• MOESI Protocol: The Owned state helps improve performance by
allowing a processor to read data in the Exclusive state without
sending requests to other caches, assuming that it is the only owner
of the data
Programming for Multi-
core
• Programmers must use threads or processes
• Spread the workload across multiple cores
• Write parallel algorithms
• OS will map threads/processes to cores
Thread safety very
important
• Pre-emptive context-switching:
context switch can happen AT ANY TIME
• True concurrency, not just uniprocessor time-
slicing
• Concurrency bugs exposed much faster with
multi-core
Assigning threads to
the cores
• Each thread/process has an affinity mask
• Affinity mask specifies what cores the thread is
allowed to run on
• Different threads can have different masks
• Affinities are inherited across fork( )
Affinity masks are bit
vectors
• Example: 4-way multi-core, without SMT
1 1 0 1
core 3 core 2 core 1 core 0
• Process/thread is allowed to run on
cores 0,2,3, but not on core 1
Affinity masks when multi-core and SM
combined
1 1 0 0 1 0 1 1
core 3 core 2 core 1 core 0
thread thread thread thread thread thread thread thread
1 0 1 0 1 0 1 0
• Core 2 can’t run the process
• Core 1 can only use one simultaneous thread
Default
Affinities
• Default affinity mask is all 1s:
all threads can run on all processors
• Then, the OS scheduler decides what threads run
on what core
• OS scheduler detects skewed workloads,
migrating threads to less busy
processors
Process migration is costly
• Need to restart the execution pipeline
• Cached data is invalidated
• OS scheduler tries to avoid migration as much as
possible:
it tends to keeps a thread on the same core
• This is called soft affinity
46
Hard
Affinities
• The programmer can prescribe her own affinities
(hard affinities)
• Rule of thumb: use the default scheduler unless a
good reason not to
When to set your own
affinities
• Two (or more) threads share data-structures
in memory
–map to same core so that can share cache
• Real-time threads:
Example: a thread running
a robot controller:
- must not be context
Source: Sensable.com
switched,
or else robot can go unstable
- dedicate an entire core just to this thread
Flynn’s
Taxonomy
• Michael Flynn (from Stanford)
– Made a characterization of computer systems
which became known as Flynn’s Taxonomy
Comput
er
Instructio Dat
ns a
Multiple Processor Organization
Flynn’s Taxonomy:
1.Single Instruction, Single Data stream - SISD
2.Single Instruction, Multiple Data stream - SIMD
3.Multiple Instruction, Single Data stream - MISD
4.Multiple Instruction, Multiple Data stream- MIMD
1. Single Instruction, Single Data Stream – SISD
• Single processor
• Single instruction stream
• Data stored in single memory
• Example: Uni-processor Systems
SI SISD SD
2. Single Instruction, Multiple Data Stream - SIMD
• Single machine instruction
• Controls simultaneous execution
• Large Number of processing elements
• Each processing element has associated memory
• Each instruction executed on different set of data
by different processors
• Examples: GPUs
SISD SD
SI SISD SD
SISD SD
3. Multiple Instruction, Single Data Stream - MISD
• Sequence of data
• Transmitted to set of processors
• Each processor executes different instruction
sequence, using same Data
• Few examples: Systolic array Processors
SI SISD
SI SISD SD
SI SISD
4. Multiple Instruction, Multiple Data Stream- MIMD
• Set of processors
• Simultaneously execute different instruction
sequences
• Different sets of data
• Examples: Multi-cores, SMPs, Clusters
SI SISD SD
SI SISD SD
SI SISD SD
MIMD - Overview
• General purpose processors
• Each can process all instructions necessary
• Further classified by method of processor
communication:
1. Via Shared Memory
2. Message Passing (Distributed Memory)
Taxonomy of Processor Architectures
Tightly Coupled -
SMP
• Processors share memory
• Communicate via that shared memory
• Symmetric Multiprocessor (SMP)
– Single shared memory
– Shared bus to access memory
– Memory access time to given area of memory is
approximately the same for each processor
Symmetric Multiprocessors
(SMPs)
– Two or more similar processors
– Processors share same memory and I/O
– Processors are connected by a bus or other internal
connection
– Memory access time is approximately the same for
each processor
– All processors share access to I/O
SMP
Advantages
• Performance
– If some work can be done in parallel
• Availability
– Failure of a single processor does not halt the system
• Incremental growth
– User can enhance performance by adding additional
processors
• Scaling
– Vendors can offer range of products based on number
of processors
Block Diagram of Tightly Coupled
Multiprocessor (SMP)
Symmetric Multiprocessor
Organization
Multithreading and Chip
Multiprocessors
• Instruction stream divided into smaller
streams called “threads”
• Executed in parallel
Definitions of Threads and
Processes
• Process:
– An instance of program running on computer
– A unit of resource ownership:
• virtual address space to hold process image
– Process switch
• Thread: dispatchable unit of work within process
– Includes processor context (which includes the PC register
and stack pointer) and data area for stack
– Interruptible: processor can turn to another thread
• Thread switch
– Switching processor between threads within same process
– Typically less costly than process switch
Implicit and Explicit
Multithreading
• All commercial processors use explicit
multithreading:
– Concurrently execute instructions from
different
explicit threads
– Interleave instructions from different threads on shared
pipelines OR parallel execution on parallel pipelines
• Implicit multithreading: Concurrent execution of
multiple threads extracted from single sequential
program:
– Implicit threads defined statically by
compiler or
