Lecture # 2

CSE 423
Parallel And Distributed
Systems
 Types of Parallelism:

• Bit-level parallelism: It is the form of parallel computing which

is based on the increasing processor’s size. It reduces the
number of instructions that the system must execute in order
to perform a task on large-sized data.
• Example: Consider a scenario where an 8-bit processor must
compute the sum of two 16-bit integers. It must first sum up
the 8 lower-order bits, then add the 8 higher-order bits, thus
requiring two instructions to perform the operation. A 16-bit
processor can perform the operation with just one instruction.
• Instruction-level parallelism: A processor can only address less
than one instruction for each clock cycle phase. These
instructions can be re-ordered and grouped which are later on
executed concurrently without affecting the result of the
program. This is called instruction-level parallelism.
• Task Parallelism: Task parallelism employs the decomposition of
a task into subtasks and then allocating each of the subtasks for
execution. The processors perform execution of sub tasks
concurrently. Eg. Adding music at same time.
 Distributed Systems
• DEFINATION: A distributed system is a collection of
independent computers, interconnected via a
network, capable of collaborating a task.

• A distributed system can be characterized as

collection of multiple autonomous computers that
communicate over a communication network and
having following features:
 Distributed Systems cont..
• No common physical clock
• Enhanced reliability
• Increased performance
• Access to geographically remote data and
resources
• Scalability
A Distributed System
 Examples of Distributed System
• Telephone Networks and Cellular Networks
• Computer Networks such as internet or intranet
• ATM Machines
• Distributed database and database management system
• Network of workstations
• Mobile computing etc.
Example of Distributed System cont..
Automated Banking (Teller Machine) System
Example of Distributed System cont..
WEB SERVERS AND WEB BROWSERS
 Primary requirements for distributed
systems:
 Security and Reliability
 Consistency of replicated data
 Concurrent transactions
 Fault tolerance
 Advantages of Distributed System
• Information sharing among distributed users
• Resource sharing
• Extensibility and incremental growth(It is possible to extend
the power and functionality of a distributed computing
system be adding additional resources to the system)
• Shorter response time and higher output
 Advantages of Distributed System Cont..
• High reliability
• Better flexibility’s in meeting user’s needs
• Better price/performance ratio
• Scalability
• Transparency (transparency is defined as the masking from
the user and the application programmer regarding the
separation of components, so that the whole system seems
to be like a single entity rather than individual components)
 Disadvantages of Distributed System
• Difficulties of developing distributed software
• Networking problem (congestions and collisions)
• Security problems
• Performance
• Reliability and fault tolerance
Parallel Vs. Distributed System
Differences and Similarities among different
types of computing
The different types of Computing Environments are: 
 Personal Computing Environment
• In personal computing environment there is a stand-alone
machine.
• Complete program resides on computer and executed there.
• Example: Personal computing environment are laptops,
mobiles, printers, computer systems, scanners etc. That we use
at our homes and offices.
 Time-Sharing Computing Environment
• In this environment multiple users share system simultaneously.
• Different users (different processes) are allotted different time
slice and processor switches rapidly among users according to it.
• For example: student listening to music while coding something
in an IDE. Windows 95 and later versions, Unix, IOS, Linux
operating systems are the examples of this environment.
 Client Server Computing Environment
• In client server computing environment two machines are
involved i.e., client machine and server machine, sometime same
machine also serve as client and server.
• In this computing environment client requests resource/service
and server provides that respective resource/service.
• A server can provide service to multiple clients at a time and here
mainly communication happens through computer network.
 Distributed Computing Environment
• In a distributed computing environment multiple nodes are
connected together using network but physically they are
separated.
• A single task is performed by different functional units of
different nodes of distributed unit.
• Here different programs of an application run simultaneously on
different nodes, and communication happens in between
different nodes of this system over network to solve task.
 Grid Computing Environment
• In grid computing environment, multiple computers from
different locations works on single problem.
• In this system set of computer nodes running in cluster jointly
perform a given task by applying resources of multiple
computers/nodes.
• It is network of computing environment where several
scattered resources provide running environment for single
task.
 Cloud Computing Environment 
• In cloud computing environment on demand availability of
computer system resources like processing and storage are
availed.
• Here computing is not done in individual technology or
computer rather it is computed in cloud of computers where all
required resources are provided by cloud vendor.
• This environment primarily comprised of three services
i.e Software-as-a-service (SaaS), Infrastructure- as-a-service(IaaS)
and platform-as-a-service (PaaS).
 Cluster Computing Environment
• In cluster computing environment cluster performs task where
cluster is a set of loosely or tightly connected computers that work
together.
• It is viewed as single system and performs task parallelly that’s
why also it is similar to parallel computing environment.
• Cluster aware applications are especially used in cluster computing
environment.
 THANK YOU
&
ANY QUERIES?