SHRI VISHNU ENGINEERING COLLEGE FOR WOMEN: BHIMAVARAM

(AUTONOMOUS)
DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING
CLOUD COMPUTING
UNIT – III

1. Virtualization
2. Characteristics of virtualized environments
3. Taxonomy of virtualization techniques
4. Virtualization and cloud Computing
5. Pros and Cons of virtualization
6. Technology examples (XEN, VMware)
7. Containers & Building blocks of containers
8. Container platforms (LXC, Docker)
9. Container orchestration
10. Docker Swarm and Kubernetes
11. Public cloud Container Platforms (AWS Container Service, Azure Container
Service & Google Container Service)

Virtualization
Virtualization is a broad umbrella of technologies and concepts designed to provide an abstract
environment—whether virtual hardware or an operating system—for running applications. The
term virtualization is often used synonymously with hardware virtualization, which plays a
fundamental role in efficiently delivering Infrastructure-as-a-Service (IaaS) solutions in cloud
computing.

Virtualization has a long history in computer science, appearing in many forms, including:

 Operating system–level virtualization

 Programming language–level virtualization
 Application-level virtualization

Beyond running applications, virtualization also enables virtual environments for storage, memory,
and networking. Although it was sporadically explored in earlier decades, recent years have seen a
strong and growing trend in adopting virtualization due to several factors:

Key Drivers of Virtualization Adoption

1. Increased Performance and Computing Capacity

Modern desktop PCs are powerful enough to handle everyday computing needs with plenty
of unused capacity, often sufficient to run a virtual machine with acceptable performance.
High-end systems, such as supercomputers, can host hundreds or even thousands of virtual
machines.

DEPT OF CSE / SVECW/MNR Page 1

 2. Underutilized Hardware and Software Resources
o Many computing resources remain idle due to sporadic usage.
o Enterprise PCs, particularly those used for office automation, often remain unused
outside working hours.
o Virtualization enables transparent deployment of separate environments,
allowing such resources to be repurposed during idle times.
3. Lack of Physical Space

The constant demand for more computing power and storage makes data centers grow
rapidly. While tech giants like Google and Microsoft can build massive data centers, most
enterprises cannot.

o Server consolidation, enabled by virtualization, allows better utilization of existing

resources and reduces the need for physical expansion.
4. Greening Initiatives

Data centers consume significant amounts of energy—not just for running servers, but also
for cooling them.

o Consolidating servers through virtualization reduces both power consumption and

cooling requirements, thereby lowering a company’s carbon footprint.
5. Rising Administrative Costs

Power and cooling costs now often exceed the cost of the IT hardware itself. More servers
require more system administration, including hardware monitoring, repairs, setup, updates,
resource monitoring, and backups.

o Virtualization reduces the number of physical servers, lowering the associated

administrative workload and costs.

These factors can be considered the major causes for the widespread adoption of hardware
virtualization solutions, as well as other forms of virtualization.

The first significant step toward consistent adoption of virtualization technologies came with the
widespread use of virtual machine–based programming languages.

1995 – Java by Sun Microsystems

 Sun released Java, which quickly became popular among developers. Its ability to integrate
small Java applications, called applets, made it a highly successful platform. By the early
2000s, Java played a major role in the application server market, demonstrating that the
technology was ready to support the execution of managed code for enterprise-class
applications.

2002 – Microsoft’s .NET Framework

 Microsoft introduced the first version of the .NET Framework, an alternative to Java.
Based on similar principles, it could support multiple programming languages and offered
complete integration with other Microsoft technologies. It soon became the primary
development platform in the Microsoft ecosystem, gaining popularity among developers.

DEPT OF CSE / SVECW/MNR Page 2

 2006 – Google’s Official Languages

 By 2006, two of Google’s three official programming languages—Java and Python—were

based on the virtual machine model.

This steady shift toward virtualization from a programming language perspective highlighted an
important fact: the technology was mature enough to deliver virtualized solutions without
significant performance overhead.

This development paved the way for an even more radical form of virtualization, which has now
become a fundamental requirement for data center management infrastructures.

Characteristics of Virtualized Environments

Virtualization refers to creating a virtual version of a resource—this can be hardware, a software
environment, storage, or a network. In a virtualized environment, there are three major
components:

1. Guest – The system component that interacts with the virtualization layer instead of
directly with the host.
2. Host – The original environment in which the guest would normally operate, responsible for
providing the underlying physical resources.
3. Virtualization Layer – The layer that recreates the same or a different environment where
the guest can operate.

DEPT OF CSE / SVECW/MNR Page 3

Guest Layer:

 Contains the virtual image (operating system + applications) and the applications
themselves.

Virtualization Layer:

 Comprises virtualized versions of hardware, storage, and networking. This layer is

controlled by a virtual machine manager or other virtualization management software. It
uses software emulation to map guest operations to physical resources.

Host Layer:

 Consists of the actual physical hardware, physical storage, and physical networking
infrastructure on which the virtualization layer runs.

Examples of Virtualization Types

1. Hardware Virtualization
o The guest is a system image containing an operating system and applications.
o Managed by a virtual machine manager (VMM) running on the host’s physical
hardware (and sometimes its OS).
2. Virtual Storage
o The guest is a client application or user interacting with virtual storage
management software deployed on top of the real storage system.
3. Virtual Networking
o The guest is an application or user connecting to a virtual network (e.g., VPN).
o Managed by specific software (VPN client) using the physical network.
o Allows users to appear as if they are on a different physical network to access
otherwise restricted resources.

The common characteristic shared by all these virtualization implementations is that the virtual
environment is created through software.

The ability to use software to emulate such a wide range of environments opens up many
opportunities that were once less appealing due to the excessive overhead introduced by earlier
virtualization layers.

Modern technologies have greatly reduced this overhead, enabling the profitable and efficient use
of virtualization. This makes it possible to fully leverage the advantages that have always been
inherent in virtualized solutions.

1. Increased Security
2. Managed Execution
3. Portability

1. Increased Security:
One major advantage of virtualization is enhanced security. The ability to control the execution of
a guest system in a completely transparent manner creates new possibilities for delivering a secure,
controlled execution environment.

DEPT OF CSE / SVECW/MNR Page 4

A virtual machine provides an emulated environment in which the guest is executed. All guest
operations are typically performed against the virtual machine, which then translates and applies
them to the host.

This level of indirection allows the virtual machine manager (VMM) to:

 Control and filter guest activities.

 Prevent harmful operations.
 Hide or protect host resources from the guest.
 Shield sensitive host information without requiring complex security policies.

Security for untrusted code is a critical need. For example:

 Java applets downloaded from the Internet run inside a sandboxed version of the Java
Virtual Machine (JVM), limiting access to the host operating system’s resources.
 Both the JVM and the .NET runtime provide extensive security policies to customize an
application’s execution environment.

Hardware virtualization solutions such as VMware Desktop, VirtualBox, and Parallels allow
the creation of a virtual computer with customized virtual hardware, on which a separate operating
system can be installed.

 By default, the virtual machine’s file system is completely separate from that of the host.
 This makes it an ideal environment for running applications without affecting other users or
the host system.

2. Managed Execution:

Virtualizing the execution environment not only increases security but also enables a wide range of
features. The most relevant are sharing, aggregation, emulation, and isolation

1. Sharing
Virtualization allows the creation of separate computing environments within the same host.

 This enables full utilization of a powerful system that might otherwise be underused.

DEPT OF CSE / SVECW/MNR Page 5

  In virtualized data centers, sharing is used to reduce the number of active servers and
lower power consumption.

2. Aggregation
Virtualization also supports the opposite of sharing: combining multiple physical resources into
one virtual resource.

 A set of separate hosts can be grouped and represented as a single virtual host to guests.
 Common in distributed computing, for example in cluster management software, which
pools resources from multiple machines into one logical system.

3. Emulation
The virtualization layer can control and tune the environment exposed to the guest, making it
possible to:

 Emulate different hardware or system architectures from the physical host.

 Run guest programs that require specific features not present in the host.
 Facilitate testing across multiple platforms without physically changing hardware.
 Support legacy software by emulating older hardware or OS environments (e.g., MS-DOS
mode in Windows 95/98).
 Example: Emulating a SCSI storage device even if the host lacks the hardware.
 Example: Running arcade game emulators on standard PCs.

4. Isolation
Virtualization provides guests with a completely separate execution environment, where:

 Multiple guests can run on the same host without interfering with each other.
 Host and guest are separated, preventing harmful guest operations from affecting the host.
 The virtualization layer filters guest activity and protects underlying resources.

Additional Capabilities

Performance Tuning
Advances in virtualization technology make it possible to:

 Allocate only a fraction of host memory to a guest.

 Limit CPU frequency for a virtual machine.
 Implement Quality of Service (QoS) to meet Service-Level Agreements (SLAs).

Virtual Machine Migration

 The state of a guest can be captured, saved, and resumed on another physical machine.
 For example, Xen Hypervisor can stop a guest OS, transfer its virtual image to another
host, and resume execution transparently.
 This is a key feature for data center efficiency and load balancing.

DEPT OF CSE / SVECW/MNR Page 6

3. Portability

The concept of portability takes different forms depending on the type of virtualization being used.

 Hardware Virtualization
o The guest is packaged into a virtual image that can typically be moved and executed
on different virtual machines with ease.
o Aside from the file size, this process can be as simple as opening a picture on
different computers.
o Virtual images are usually stored in proprietary formats and require a compatible
virtual machine manager to run.
 Programming-Level Virtualization
o Examples: Java Virtual Machine (JVM) and .NET runtime.
o Application components (e.g., JAR files for Java or assemblies for .NET) can run
without recompilation on any implementation of the corresponding virtual
machine.
o This makes the application development cycle more flexible and deployment
straightforward—one version of an application can run on multiple platforms
without modification.
 User Benefits
o Portability means you can carry your entire system with you and use it wherever a
compatible virtual machine manager is available.
o This requirement is generally less restrictive than ensuring all your applications and
services are available everywhere you go.

Taxonomy of virtualization techniques

Virtualization is a broad concept covering many emulation techniques applied in different areas of
computing. Classifying these techniques helps in understanding their characteristics and use cases.

1. Classification by Service or Entity Being Emulated

 Execution Environment
 Storage
 Network
 (and others)

Among these, execution virtualization is:

 The oldest
 The most popular
 The most developed

→ Therefore, it gets a deeper classification.

DEPT OF CSE / SVECW/MNR Page 7

Execution Virtualization Categories
Execution virtualization can be split based on the type of host required:

1. Process-Level Virtualization
o Runs on top of an existing Operating System (OS controls the hardware fully).
o Examples:
 Emulation (application-level)
 High-Level Virtual Machines (e.g., JVM for programming languages)
 Multiprogramming (OS-level sharing of resources)
2. System-Level Virtualization
o Runs directly on hardware (no OS required or minimal OS support).
o Examples:
 Hardware-Assisted Virtualization
 Full Virtualization
 Paravirtualization
 Partial Virtualization

Type of Virtual Environment Offered

Depending on technique, the virtual environment may emulate:

 Bare hardware
 Operating system resources
 Low-level programming language
 Application libraries

DEPT OF CSE / SVECW/MNR Page 8

Execution virtualization:
Execution virtualization includes all techniques that aim to emulate an execution environment that
is separate from the one hosting the virtualization layer. All these techniques concentrate their
interest on providing support for the execution of programs, whether these are the operating system,
a binary specification of a program compiled against an abstract machine model, or an application.
Therefore, execution virtualization can be implemented directly on top of the hardware by the
operating system, an application, or libraries dynamically or statically linked to an application
image.

Machine Reference Model

This model explains the layered abstraction of a computing system, from hardware up to
applications, and the interfaces between these layers.

Layers and Interfaces

 Hardware – The physical CPU, memory, and I/O devices.
 ISA (Instruction Set Architecture) –
o The interface between hardware and software.
o Defines processor instructions, registers, memory handling, and interrupts.
o Split into:
 System ISA – Used by the OS to manage hardware.
 User ISA – Used by applications that directly manage hardware.
 ABI (Application Binary Interface) –
o Sits between the OS and applications/libraries.
o Defines low-level data types, calling conventions, executable format, and system
calls.
o Ensures applications can run on any OS that supports the same ABI.
 API (Application Programming Interface) –
o Interface between applications and libraries/OS functions.
o High-level interface hiding low-level implementation details.

Flow of Execution
Any high-level API call → passes through ABI → translated into ISA instructions → executed by
hardware.

DEPT OF CSE / SVECW/MNR Page 9

Why This Matters

 Simplifies system development.

 Makes multitasking and virtualized environments easier to implement.
 Allows security models to be built by controlling access at each layer.

Privilege Levels and Security Rings

Modern CPUs enforce privilege levels to protect the system and manage access to sensitive
resources.

Instruction Categories
 Non-privileged Instructions – Can be executed without affecting other processes (e.g.,
arithmetic operations).
 Privileged Instructions –
o Behavior-sensitive – Interact with I/O devices.
o Control-sensitive – Change CPU state or configuration.

Ring-Based Security Model

 Ring 0 – Most privileged (Supervisor mode), used by OS kernel.
 Ring 1 & 2 – Often used for OS-level services (rare in modern systems).
 Ring 3 – Least privileged (User mode), used by applications.
 Many modern CPUs simplify to two levels:
o Ring 0 → Supervisor mode
o Ring 3 → User mode

Purpose
 Prevents user applications from directly executing dangerous instructions.
 Protects system stability and security by restricting access to shared hardware resources.

DEPT OF CSE / SVECW/MNR Page 10

Execution Modes

Modern CPUs support at least two execution modes:

1. Supervisor Mode
o Also called kernel mode or master mode.
o Can execute all instructions (privileged + non-privileged) without restrictions.
o Used by the operating system kernel or hypervisor to manage hardware resources.
2. User Mode
o Restricted mode — cannot directly execute privileged instructions.
o If a user-mode program tries to run a privileged instruction:
 A hardware interrupt (trap) occurs.
 The CPU switches to supervisor mode so the OS can handle the request
safely.
o Some instructions may be conditionally privileged (privileged in some contexts,
non-privileged in others).

Hardware-Level Virtualization (System Virtualization)

Hardware-level virtualization creates a virtual version of a computer’s hardware environment,

allowing a guest operating system to run as if it were installed on a real physical machine.

Main Components

1. Host
o The actual physical computer hardware (CPU, memory, storage, I/O devices).
o The base layer on which everything else runs.
2. Virtual Machine Monitor (VMM) / Hypervisor
o Sits between the hardware (host) and the guest OS.
o Responsible for:
 Abstracting hardware for the guest OS.
 Managing resources (CPU, memory, I/O) between multiple guests.
 Isolating guests so they cannot interfere with each other.
o Can be:
 Type 1 (bare-metal) → runs directly on the hardware.
 Type 2 (hosted) → runs as software on top of a host OS.

DEPT OF CSE / SVECW/MNR Page 11

 3. Virtual Machine (VM)
o An emulated hardware environment in which the guest OS runs.
o Created and controlled by the hypervisor.
o Can be implemented using:
 Binary translation
 Instruction mapping
 Interpretation
 Or hardware-assisted virtualization (Intel VT-x, AMD-V).
4. Guest
o The operating system running inside the virtual machine.
o Has an in-memory representation of its state.
o Sees the ISA (Instruction Set Architecture) presented by the hypervisor as if it
were real hardware.
5. Storage / Virtual Image
o The guest OS and applications are stored in a virtual disk image file.
o The hypervisor loads this into memory to run the VM.

System Virtualization vs. Process Virtualization

 System Virtualization (Hardware-level)
o Exposes the ISA to the VM.
o Example: Running Windows inside VMware or VirtualBox.
 Process Virtualization
o Exposes the ABI (Application Binary Interface).
o Example: The Java Virtual Machine (JVM) or .NET CLR.

Hypervisors

Type I Hypervisor (Native / Bare-Metal)

Position in Stack:
 Runs directly on the hardware, replacing the operating system.
Interacts with the ISA (Instruction Set Architecture) exposed by the hardware.
How it works:
 Acts as the “OS” for the physical machine.
DEPT OF CSE / SVECW/MNR Page 12
  Manages hardware resources and allocates them to multiple virtual machines.
 Emulates the ISA for each guest OS so they believe they are running on real hardware.
Advantages:
 Better performance (no intermediate OS layer).
 More secure and efficient.

Examples: VMware ESXi, Microsoft Hyper-V (bare-metal mode), Xen.

2. Type II Hypervisor (Hosted)

Position in Stack:
 Runs on top of an existing operating system.
 Interacts with the OS through the ABI (Application Binary Interface).
How it works:
 The existing OS controls the hardware.
 The hypervisor is just another application running on that OS.
 It still emulates the ISA for each VM.
Advantages:
 Easier to set up (you just install it like software).
 Can run on an already configured OS.
Disadvantages:
 Slightly slower due to extra OS layer.
 Less direct control over hardware.
Examples: VirtualBox, VMware Workstation, Parallels Desktop.

Hypervisor Reference Architecture

A VMM is broken into three main modules:

1. Dispatcher
 Role: Entry point for all instructions from the Virtual Machine Instance (VM).
 Function: Routes each incoming instruction to either:
o Allocator → if it affects resource allocation.
o Interpreter → if it’s a privileged instruction that needs special handling.

DEPT OF CSE / SVECW/MNR Page 13

2. Allocator

 Role: Manages and controls system resources assigned to each VM.

 When Invoked: Whenever an instruction attempts to change resources (e.g., CPU, memory,
I/O devices).
 Purpose: Ensures the VMM has full control over resources to enforce isolation and
fairness.

3. Interpreter
 Role: Handles privileged instructions that cannot be executed directly by the VM.
 Mechanism:
o The VM triggers a trap when trying to run such an instruction.
o The VMM intercepts the trap and executes the correct interpreter routine to
emulate hardware behavior.

The combined action of these modules ensures:

1. Equivalence – The VM behaves the same as if it ran on real hardware.

2. Resource Control – The VMM retains complete authority over resources.
3. Efficiency – Most instructions execute without heavy overhead; only privileged ones
require trapping and emulation.

Hardware Virtualization Techniques

1. Hardware-Assisted Virtualization

 Definition: Hardware provides architectural support for building a Virtual Machine

Manager (VMM) capable of running guest OS in complete isolation.
 History:
o First introduced in IBM System/370.
o Modern examples: Intel VT (Vanderpool) and AMD-V (Pacifica).
 Purpose: Reduces the performance penalties of software-only emulation of x86 hardware.
 Before introduction:
o x86 architecture didn’t meet Popek and Goldberg’s virtualization requirements.
o Early products (e.g., VMware Virtual Platform in 1999) used binary translation to
handle sensitive instructions.
 After hardware support: Widely used in KVM, VirtualBox, Xen, VMware, Hyper-V,
Parallels, etc.

2. Full Virtualization

 Definition: Runs an OS directly on a virtual machine without any modification, emulating

the entire underlying hardware.
 Advantages:
o Complete isolation
o High security
o Ability to emulate different architectures
o Run multiple OSes on one platform
 Challenges:
o Handling privileged instructions (e.g., I/O) requires trapping and managing them.

DEPT OF CSE / SVECW/MNR Page 14

 o Can cause performance overhead.
 Solution: Combination of hardware + software to prevent harmful instructions from
executing directly on the host.

3. Paravirtualization

 Definition: Not fully transparent—guest OS is modified to work with the hypervisor

through a special software interface.
 Goal: Allow performance-critical operations to execute directly on the host to reduce
performance loss.
 Requirements: Guest OS source code access (easier in open-source environments).
 History & Use:
o First explored in IBM VM OS families.
o Term introduced in Denali project (University of Washington).
o Used in Xen for Linux OSes.
o Non-portable OSes (e.g., Windows) can still use it via special device drivers.
o Used in VMware, Parallels, TRANGO, Wind River, XtratuM.

4. Partial Virtualization

 Definition: Only partially emulates the underlying hardware—guest OS cannot execute in

full isolation.
 Effect: Many applications run transparently, but some OS features are unsupported.
 Example:
o Address space virtualization in time-sharing systems (separate memory space for
processes but shared CPU, disk, network).
o Implemented historically in IBM M44/44X.
 Importance: Served as a milestone toward achieving full virtualization; still common in
modern OS memory management.

Operating System-Level Virtualization

 Provides multiple isolated execution environments (containers) for applications within a
single operating system.
 No hypervisor or Virtual Machine Manager (VMM) is required.
 The OS kernel manages:
o Sharing of system resources.
o Limiting the impact of one instance on another.

How It Works

 Each user-space instance has:

o Its own isolated file system view.
o Separate IP addresses, software configurations, and device access.
 Based on an evolved form of chroot in Unix:
o chroot changes the root directory for a process, isolating it from the rest of the system.
o Devices in Unix are exposed as part of the file system, so this isolation can be
extended to devices.
 Essentially creates separate execution containers for applications.

DEPT OF CSE / SVECW/MNR Page 15

Advantages

 Low or no overhead:
o Applications directly use OS system calls (no hardware emulation).
 No need to:
o Modify applications.
o Modify hardware.
 Highly efficient for server consolidation when servers share the same OS and frameworks.

Limitations
 All user-space instances must share the same operating system (less flexible than
hardware virtualization).
 Cannot run completely different OS types in separate containers.

Use Cases
 Server consolidation: multiple application servers running on one physical server, each in its
own isolated user space.

Examples
 FreeBSD Jails
 IBM Logical Partition (LPAR)
 Solaris Zones and Containers
 Parallels Virtuozzo Containers
 OpenVZ
 iCore Virtual Accounts
 FreeVPS
 Some (like Solaris and OpenVZ) allow different versions of the same OS to run
concurrently.

Programming Language–Level Virtualization

 Provides ease of deployment, portability across platforms, and managed execution of
applications.
 It Uses a virtual machine (VM) to execute bytecode (compiled code for an abstract
architecture).
o At runtime, bytecode can be interpreted or just-in-time compiled (JIT) into native
machine instructions.
 History & Examples –
o First used in 1966 for BCPL (ancestor of C).
o Later in UCSD Pascal, Smalltalk, and became mainstream with Java (1996) and
.NET’s Common Language Infrastructure (CLI).
 Architecture Types –
o Stack-based VMs (e.g., Java Virtual Machine, CLI) – use an execution stack for
operations; easy to interpret and portable.
o Register-based VMs (e.g., Parrot) – closer to real CPU design, can be more efficient
for certain workloads.
 Advantages –

DEPT OF CSE / SVECW/MNR Page 16

 o Platform Independence – Same bytecode runs on any platform with a compatible
VM.
o Simplified Development – No need to compile separate versions for each OS.
o Security & Sandboxing – VM can control memory access and I/O, enabling
pluggable security policies.
 Trade-Offs –
o Performance is generally lower than native execution, but the gap is narrowing with
modern hardware and optimized JIT compilers.

Application-Level Virtualization

Definition:
A technique that allows applications to run in runtime environments that lack all the features those
applications require. Instead of being installed normally, the applications run as if they were in their
native environment.

Key Features:
 Focuses on partial emulation of:
o Filesystems
o Libraries
o Operating system components
 Implemented by a thin layer (a program or OS component) that executes the application.

Emulation Strategies:

1. Interpretation
o Each source instruction is interpreted into native instructions.
o Pros: Minimal startup cost.
o Cons: High overhead → slow performance.
2. Binary Translation
o Converts each source instruction into equivalent native instructions.
o Translated blocks are cached for reuse.
o Pros: Better performance over time.
o Cons: Large initial overhead.

Difference from Hardware-Level Virtualization:

 Application-level virtualization: Only emulates what’s needed for the application’s
runtime.
 Hardware-level virtualization: Emulates an entire hardware environment for installing a
complete OS.

Advantages:
 Handles missing libraries by:
o Linking with replacement libraries.
o Remapping calls to existing host functions.
 Lighter than hardware virtualization.
 Enables incompatible applications to run together.

DEPT OF CSE / SVECW/MNR Page 17

  Targets applications for a specific environment, unlike programming-level virtualization
(which works across all apps for a given VM).

Examples:
 Wine – Runs Windows apps on Unix-like OSes via:
o Guest app container
o Winelib (library for compiling apps for Unix)
 WABI – Win16 API implementation on Solaris.
 CrossOver – Runs Windows apps on macOS.
 VMware ThinApp – Captures app setups into isolated executable images.

Other Types of Virtualization

Aside from execution virtualization, there are other virtualization techniques that provide an
abstract environment for interaction, mainly in storage, networking, and desktop access.

1. Storage Virtualization

 Definition – Separates the physical organization of storage hardware from its logical
representation.
 Key Idea – Users access data via a logical path, without worrying about where it is
physically stored.
 Benefits –
o Unified access to multiple storage resources under a single logical file system.
o Flexibility in storage management.
 Common Technique –
o Network-based storage virtualization using Storage Area Networks (SANs),
which provide large-bandwidth, network-accessible storage devices.

2. Network Virtualization

 Definition – Uses hardware appliances and software to create/manage a virtual network.

 Types –
1. External Network Virtualization – Aggregates multiple physical networks into one
logical network (e.g., Virtual LAN / VLAN).
2. Internal Network Virtualization – Provides network-like functionality within an
OS partition (common in OS/hardware-level virtualization).
 Implementation Options –
o Sharing host’s network interface via NAT.
o Emulating a separate network device on the host.
o Creating a private network between guest systems.

3. Desktop Virtualization

 Definition – Abstracts the desktop environment so it can be accessed remotely via a

client/server model.
 How It Works –
o Desktop OS is hosted remotely (server or data center).
o Delivered to the user’s device over a network.

DEPT OF CSE / SVECW/MNR Page 18

 o Often stored as a virtual machine image that is loaded on demand.
 Benefits – High availability, persistence, remote accessibility, ease of management.
 Use Case – Cloud computing scenarios where users access their personalized desktop from
anywhere.
 Examples – Windows Remote Services, VNC, X Server, Sun VDI, Parallels VDI, Citrix
XenDesktop.
 Caution – Security concerns can limit adoption.

4. Application Server Virtualization

 Definition – Abstracts a collection of application servers into what appears as a single

virtual application server.
 How It Works –
o Uses load balancing to distribute requests among multiple physical servers.
o Ensures high availability of hosted services.
 Purpose – Similar to storage virtualization, the goal is to improve quality of service rather
than emulate a different environment.
 Key Benefit – Users experience the system as if it were a single, powerful application
server, even though it is actually a cluster of multiple servers working together.

Virtualization and Cloud Computing

Virtualization is a key enabler for cloud computing because it provides the customization,
security, isolation, and manageability required to deliver IT services on demand. It allows
providers to create configurable computing environments and virtualized storage systems, while
network virtualization plays a complementary role in building virtual computing systems.

Two main virtualization types used in cloud computing are:

1. Hardware Virtualization – Common in Infrastructure-as-a-Service (IaaS); allows

multiple virtual machines to run on the same physical hardware.
2. Programming Language Virtualization – Used in Platform-as-a-Service (PaaS); creates
sandboxed environments for application execution.

Benefits
 Resource Isolation & Control – Each virtual machine (VM) operates independently
without interfering with others.
 Service Leasing & Accountability – Easier to track usage and bill customers.
 Efficient Resource Utilization – Through server consolidation, underutilized resources are
combined, reducing the number of active physical servers.

Server Consolidation & VM Migration

 Server Consolidation – Running multiple VMs on fewer physical servers to improve
utilization and reduce costs.
 Virtual Machine Migration – Moving VMs from one physical server to another.
o Cold Migration – VM is stopped, moved, and restarted.
o Live Migration – VM is moved while running with no noticeable downtime (more
complex but efficient).
DEPT OF CSE / SVECW/MNR Page 19
Before Migration – VMs are spread across Server A and Server B. Both are active.

Migration Process – Virtual Machine Manager transfers VMs from Server B to Server A.

After Migration – All VMs run on Server A, and Server B becomes inactive, freeing resources or
enabling maintenance.

This demonstrates live migration used for server consolidation, improving efficiency without
disrupting services.

Pros and Cons of Virtualization

Advantages of Virtualization
1. Managed Execution & Isolation

 Secure and controllable environments (sandboxing prevents harmful operations from

affecting the host).
 Easier resource allocation and partitioning for server consolidation.
 Improves Quality of Service.

2. Portability & Self-Containment

 VMs stored as files → easy to move and replicate.

 Self-contained (minimal dependencies other than VM manager).
 Examples:
o Java programs: “Compile once, run anywhere.”
o Hardware virtualization: Carry a fully configured OS anywhere.
 Enables VM migration.

DEPT OF CSE / SVECW/MNR Page 20

3. Lower Maintenance Costs

 Fewer physical hosts to manage.

 Less hardware damage risk.
 Fewer VM managers than VM instances.

4. Efficient Resource Use

 Multiple systems share hardware securely without interference.

 Dynamic adjustment of physical resources to match load.
 Saves energy and reduces environmental impact.

Disadvantages of Virtualization
1. Performance Degradation

 Cause: Abstraction layer between guest and host.

 Sources in hardware virtualization:
o Maintaining VM CPU state.
o Handling privileged instructions.
o Paging inside the VM.
o Console operations.
 Extra overhead if VM manager runs as a regular program on host OS.
 Sources in programming-language VMs:
o Binary translation & interpretation slowdowns.
o Filtered access to memory/hardware.
 Mitigation: Use paravirtualization or native code compilation.

2. Inefficiency & Degraded User Experience

 Some host features may be inaccessible due to abstraction layer.

o Example: Limited GPU features in virtualized graphics cards.
 Early Java had poor GUI capabilities (fixed later with Swing and OpenGL support).

3. Security Holes & New Threats

 Malicious virtualization layers can mimic a host to steal sensitive data.

 Examples:
o BluePill – moves OS into a VM (AMD-targeted).
o SubVirt – infects OS, then gains host control.
 Risk is higher if:
o Hardware was not designed for virtualization.
o Host CPU lacks virtualization-specific instruction sets.
 Modern CPUs: Intel VT, AMD Pacifica add virtualization security.
 Programming-level VMs can also be exploited if runtime environment is replaced by a
malicious version.

DEPT OF CSE / SVECW/MNR Page 21

 Technology examples (XEN, VMware)

Xen: Paravirtualization

 Overview
o Xen is an open-source virtualization platform initially developed at the University
of Cambridge.
o It is supported by a large community and also offered commercially by Citrix as
XenSource.
o Used for desktop virtualization, server virtualization, and cloud computing (e.g.,
Xen Cloud Platform, XCP).
o Core technology: Xen Hypervisor.
 Paravirtualization vs. Full Virtualization
o Paravirtualization allows high-performance execution of guest operating systems
by modifying their code to handle certain privileged instructions via hypercalls.
o This reduces performance loss but is not fully transparent because OS code must
be adapted.
o Full virtualization (with hardware support) does not require OS modification.
 Architecture
o Xen Hypervisor runs at the highest privilege level (Ring 0).
o Domains:
 Domain 0 (Dom0): Special, privileged VM with access to the host,
responsible for creating, configuring, and terminating other VMs.
 Domain U (DomU): Unprivileged guest VMs.
o Typical mapping on x86 privilege levels:
 Hypervisor: Ring 0
 Guest OS (DomU): Ring 1
 Applications: Ring 3
 Technical Details
o x86 has four privilege levels (Rings 0–3), but most OSes use only Ring 0 (kernel)
and Ring 3 (user space).
o Running guest OSes in Ring 1 can cause certain sensitive instructions to fail.
o Xen replaces these with hypercalls so the hypervisor can intercept and manage them
safely.
 Limitations
o Requires OS modification (paravirtualization) → not all OSes are supported.

DEPT OF CSE / SVECW/MNR Page 22

 o Legacy hardware and legacy OSes that cannot be modified may not work unless
hardware-assisted virtualization is available.
o Open-source OSes like Linux are fully supported; Windows needs hardware-assisted
virtualization.
 Trend
o The limitation is less significant today because modern OSes are virtualization-
aware and modern hardware supports x86 virtualization extensions.

VMware: Full Virtualization

VMware’s technology is based on the concept of full virtualization, where the underlying
hardware is replicated and made available to the guest operating system. The guest OS runs
unaware of the abstraction layers and does not need to be modified.

VMware implements full virtualization in:

 Desktop environments – using Type II hypervisors

 Server environments – using Type I hypervisors

Full virtualization is achieved through:

 Direct execution for non-sensitive instructions

 Binary translation for sensitive instructions

This enables virtualization of architectures such as x86.

Besides these core solutions, VMware also provides additional tools and software to:

 Improve integration of virtual guests with hosts (desktop)

 Build and manage virtual computing infrastructures (server)

Full Virtualization and Binary Translation

VMware is known for its ability to virtualize x86 architectures, running unmodified on top of its
hypervisors. Before 2006, when hardware-assisted virtualization (Intel VT-x and AMD-V)
became available, VMware relied on dynamic binary translation as the only way to run
unmodified x86 guest operating systems in a virtualized environment.

DEPT OF CSE / SVECW/MNR Page 23

The challenge:

 x86 architecture violates the first theorem of virtualization because sensitive instructions
are not a subset of privileged instructions.
 When run outside Ring 0, these instructions behave differently and often trigger traps.

Binary translation process:

1. A trap occurs when a sensitive instruction is executed.

2. The offending instruction is translated into an equivalent set that achieves the same effect
without triggering exceptions.
3. The translated version is cached for future use, avoiding repeated translation.

Advantages:

 Runs unmodified guest OS (important for closed-source OS like Windows)

 Portable across different x86 environments

Disadvantages:

 Runtime translation introduces overhead (less efficient than paravirtualization or hardware-

assisted virtualization)
 Overhead is reduced since only a subset of instructions is translated; others run directly on
hardware

Other Virtualization Components in VMware

Memory Virtualization:

 Memory Management Unit (MMU) must be emulated.

 Hosted hypervisors (Type II) face performance impact due to nested MMU translations.
 VMware uses Translation Lookaside Buffer (TLB) mapping to avoid unnecessary nested
translations; slowdown happens only on a TLB miss.

I/O Virtualization:

 VMware fully virtualizes devices like:

o Network controllers
o Keyboards, mice
o Disks
o USB controllers

Containers
Cloud containers are software code packages that contain an application’s code, its libraries, and
other dependencies that it needs to run in the cloud. Any software application code requires
additional files called libraries and dependencies before it can run. Traditionally, software had to be
packaged in multiple formats to run in different environments such as Windows, Linux, Mac, and
mobile. However, a container packages the software and all of its dependencies into a single file
that can run anywhere. Running the container in the cloud provides additional flexibility and
performance benefits at scale.
DEPT OF CSE / SVECW/MNR Page 24
Self-Contained Units: A container is a streamlined, self-contained unit of software that bundles
application code, libraries, and dependencies.

Portability: Containers are designed to run quickly and reliably on various platforms, from laptops
and desktops to traditional systems and cloud environments.

Resource Isolation: They use operating system features like Linux namespaces and control groups
(Cgroups) to isolate processes and manage their access to CPU, memory, and disk resources.

Container Images: Developers create a container image, which is a standalone, executable bundle
of software. These images are shared among teams and communities, often through public or
private registries.

Container Runtime Engines: Runtime engines are the tools responsible for executing container
images as running containers. They manage the container's lifecycle, handle communication with
the OS kernel, and secure access to resources. Popular examples include containerd, Docker,
LXC, and Podman.

The most prominent use cases for containers are:

 DevOps: Containers are a fundamental part of the DevOps ecosystem. They're used in
combination with microservices to design, deploy, and test applications. This allows teams
to build, ship, and run software efficiently.
 Hybrid and Multi-Cloud: Containers enable organizations to run and test applications
seamlessly across different environments, including laptops, on-premises data centers, and
various public cloud providers. This flexibility makes them ideal for supporting hybrid and
multi-cloud scenarios.

Differences between Virtual Machines and Containers

Virtual machines (VMs) and containers are both forms of virtualization, but they differ significantly
in their architecture and how they achieve resource isolation. Containers offer several advantages
over VMs due to their more lightweight design.

DEPT OF CSE / SVECW/MNR Page 25

How They Work

 Virtual Machines (VMs): A VM runs on top of a hypervisor, a layer of software that

emulates the hardware of a physical server. Each VM is a complete, standalone system with
its own guest operating system (OS), binaries, and libraries. This full copy of an OS results
in a large storage footprint, typically measured in gigabytes (GBs), and a slower boot time.
 Containers: A container runs on top of a container runtime, which shares the host
machine's operating system kernel. Instead of having its own OS, a container only includes
the application and its dependencies, isolated at the application layer. This shared-kernel
approach makes containers much smaller, with image sizes typically measured in megabytes
(MBs), leading to more efficient and quicker deployment.

Key Differences

 Size and Speed: Containers are smaller and faster to deploy and boot up because they don't
have to carry a separate OS. VMs, with their full OS, are larger and require more time to
start.
 Resource Usage: Since containers share the host's OS kernel, they use fewer resources like
CPU and memory. Each VM requires its own dedicated OS, making them more resource-
intensive.
 Isolation: While both provide isolation, they do so differently. VMs offer a higher level of
isolation because they are completely separate operating systems. Containers achieve
isolation by relying on OS kernel features, making them more lightweight but with a slightly
lower degree of separation than VMs.
 Portability: Containers are highly portable. An application packaged in a container can be
deployed reliably across different environments (from a developer's laptop to a cloud server)
because it includes all necessary dependencies and doesn't rely on a specific guest OS. VM
migration can be more complex due to compatibility issues with different OSes or
hypervisors.

Building blocks of containers

Fundamental building blocks of containers, which are Cgroups and namespaces.

1. Cgroups (Control Groups)

DEPT OF CSE / SVECW/MNR Page 26

  Cgroups are a Linux kernel concept that isolates and governs the usage of system resources
like CPU time, memory, and bandwidth for a group of processes.
 Originally contributed by Google, Cgroups were introduced as a kernel feature to restrict,
monitor, and isolate the resource consumption of a group of processes.
 They provide system administrators with fine-tuned control for allocating, assigning
priorities, denying, managing, and monitoring system resources.
 Cgroups also help in the efficient use of hardware resources by dividing them among
different tasks or processes.
 Key features of Cgroups include:
o Resource limits: The ability to allocate and limit specific resources like memory,
CPU, network, and disk I/O to a process.
o Fine-grained control over processes: Processes within a Cgroup can be controlled
with a single command (e.g., start, stop, freeze).
o Resource allocation categorization: Allows for the comparison of resource
allocation for multiple processes in different Cgroups during resource contention.
o Security and accounting: Cgroups offer security by limiting users from spawning a
high number of processes, which can prevent a "fork bomb" attack. They also
monitor and report resource usage and limits at the Cgroup level.
 In the context of containers, Cgroups enable developers to allocate fine-grained resource
limits to each container, preventing them from overutilizing system resources.

2. Namespaces

 Namespaces are another key Linux kernel feature that partitions kernel resources, allowing
one set of processes to see one set of resources while another set sees a different set of
resources.
 This feature was originally developed by IBM.
 Basic examples of namespaces include PID (process ID), MNT (mount file/folder), IPC, and
NET (network).
 Network Namespace:

o Provides an isolated network environment within the operating system.

o It allows for the creation of a virtualized network stack with its own interfaces, IP
ranges, routing tables, and ports.
o Multiple containers can maintain distinct network stacks while sharing the host's
network resources.

DEPT OF CSE / SVECW/MNR Page 27

  Filesystem Namespace:

o Virtualizes and isolates the filesystem tree.

o It maps a node in the filesystem tree to a virtual root inside the namespace.
o Users within a container's filesystem namespace can only see what is inside that
namespace and cannot go beyond the virtualized root.
 Process ID (PID) Namespace:

o Confines the assignment of process IDs to a specific process and its related
subprocesses.
o Within a PID namespace, a process perceives itself as having PID 1, while only
being aware of its own descendant processes.
o Each PID namespace has a distinct view of the process table, and processes cannot
send signals from one PID namespace to another, which prevents containers from
interacting with the host or other PID namespaces.

LXC
 Introduced in 2008, Linux Containers (LXC) provide OS-level virtualization on Linux.
 They use Cgroups (Control Groups) and Namespaces to run multiple isolated Linux
systems (containers) on a single host.
 Unlike earlier container approaches, LXC did not require kernel patches, making it the
first complete runtime engine for Linux containers.
 It comes with tools, templates, and language bindings, making it developer-friendly.
DEPT OF CSE / SVECW/MNR Page 28
Kernel Features Used in LXC for Isolation

1. Kernel Namespaces
o Provide process and resource isolation between containers.
o Types include:
 IPC → Isolates inter-process communication.
 UTS → Allows containers to have unique host/domain names.
 Mount → Separate file system mount points per container.
 PID → Isolated process IDs (containers have their own PID 1).
 Network → Isolates networking (interfaces, routing tables, etc.).
 User → Maps user IDs inside the container to different UIDs outside.

✅ Example: Running ps on the host shows host + container processes, but running it
inside the container shows only that container’s processes.

2. AppArmor & SELinux Profiles

o Enhance security by restricting what processes inside a container can do.
o AppArmor → Defines security policies to confine apps.
o SELinux → Enforces mandatory access control (MAC) at the OS level.
3. Seccomp Policies (Secure Computing Mode)
o Filters system calls (syscalls) available to a process.
o Reduces attack surface by blocking dangerous or unnecessary syscalls.
4. Chroot & pivot_root
o Chroot changes the root directory of a process, isolating its filesystem view.
o pivot_root replaces the root filesystem of a process, giving containers their own
root directory (e.g., /bin, /usr, etc.), independent of the host.

Memory Management

 Linux memory model:

o Kernel space → Reserved for OS core (managing CPU, memory, and devices).
o User space → Runs user applications and processes.
 LXC leverages kernel features to:
o Create isolated processes in user space.
o Provide separate filesystems using namespaces + chroot.

Docker
 Docker is an open-source platform for developing, shipping, and running applications.
 Its ecosystem accelerates the Dev → Test → Deploy cycle by reducing the time from
writing code to production deployment.
 Origin:
o Started in 2013 as an internal project at dotCloud by Solomon Hykes.
o Released as open source (Moby Project) in March 2013.
o Quickly became one of the most popular open-source community projects,
attracting 3400+ contributors.
o 2016 → Docker Enterprise Edition (EE) launched as a commercial offering.

DEPT OF CSE / SVECW/MNR Page 29

Docker Provides

1. Containerization Platform
o Packages applications and their dependencies into lightweight, portable containers.
o Containers can run on any environment consistently—developer laptop, test
servers, or cloud.
2. Isolation & Security
o Uses Linux kernel features:
 Namespaces → Process, network, and filesystem isolation.
 Cgroups (Control Groups) → Resource limits (CPU, memory, I/O).
o Ensures multiple containers run securely on a single host.
3. Consistency Across Environments
o Docker containers are shareable artifacts.
o Developers and teams can share images, ensuring “works on my machine”
problems are eliminated.
o Same container behaves identically across development, testing, and production.

Why Docker is Popular

 Lightweight → Runs directly on the OS kernel (no heavy hypervisors like VMs).
 Fast deployment → Containers start in seconds.
 Portability → Run anywhere (Linux, Windows, Mac, cloud, on-prem).
 Scalable → Ideal for microservices architecture.
 Ecosystem tools → Docker Hub (image sharing), Docker Compose (multi-container apps),
Docker Swarm/Kubernetes (orchestration).

Docker Architecture

Docker follows a client-server architecture, consisting of the Docker Client, Docker Daemon,
Docker Host, and Docker Registry.

(a) Docker Daemon (dockerd)

 Runs as a background service on the Docker host.

 Listens to requests via the Docker API (through REST, UNIX sockets, or TCP).
 Manages all Docker objects:
DEPT OF CSE / SVECW/MNR Page 30
 o Images
o Containers
o Volumes
o Networks
 Responsible for container lifecycle management → building, running, stopping, and
deleting containers.

(b) Docker Client

 Provides the Command-Line Interface (CLI) via the docker command.

 Used by users to interact with Docker (e.g., docker build, docker pull, docker run).
 Example workflow:
1. User runs docker run <image>.
2. The command is sent to the Docker Daemon via API.
3. The daemon executes it → creates/runs the container.

(c) Docker Host

 The machine where Docker Daemon runs.

 Can be:
o Personal computers (Linux, Windows, Mac)
o Virtual Machines
o Enterprise servers
 Provides the runtime environment for containers.

(d) Docker Registry

 A repository of Docker images.

 Default registry = Docker Hub (public).
 Supports public and private registries.
 Commands:
o docker pull <image> → downloads image from registry.
o docker push <image> → uploads image to registry.
 Requires a Docker Hub account for pushing/sharing custom images.

Docker Image Layers:

DEPT OF CSE / SVECW/MNR Page 31

Docker Objects

1. Images
o Read-only templates used to create containers.
o Built using a Dockerfile, with each instruction adding a new layer.
o Union filesystem makes images lightweight:
 Only changed layers are rebuilt.
 Layers are shared among containers, saving space.
o Example:
 A base image (Ubuntu) + additional layers (Python, app code).
2. Containers
o A runtime instance of an image.
o Includes the app + dependencies.
o Isolated using namespaces and cgroups.
o Containers are ephemeral → can be created, stopped, moved, or deleted quickly.
3. Volumes
o Provide persistent storage beyond container lifecycle.
4. Networks
o Enable communication between containers and external systems.

A Docker image is built in layers, and when a container runs, a read/write layer is added on top of
these layers.

1. Kernel
o The base of the system (Linux kernel).
o Provides process, memory, and hardware management.
2. bootfs (Boot Filesystem)
o Similar to Linux/UNIX boot filesystem.
o Handles system boot-up processes.
3. rootfs (Root Filesystem)
o Contains the operating system files (e.g., Ubuntu, Debian).
o Mounted as read-only inside Docker images.
4. Image Layers (Read-Only)

DEPT OF CSE / SVECW/MNR Page 32

 oExample:
 Base image → Ubuntu
 Added image → Emacs
 Added image → Apache
o Each added software package or modification creates a new read-only layer.
o Union filesystem (OverlayFS) allows these multiple layers to appear as one single
filesystem.
5. Writable Layer (Container Layer)
o Added on top when a container is created.
o Any changes (writes, file edits, deletions) happen here.
o Uses Copy-on-Write (CoW):
 If you modify a file, Docker copies it from the read-only layer to the writable
layer and then edits it.
 The original stays intact in the read-only layer.

✅ Advantage: This layered approach makes Docker images lightweight, modular, and efficient,
since unchanged layers can be reused across multiple containers.

Activities in Creating a Docker Container

This figure explains the workflow of docker run.

Steps:

1. Docker CLI
o User issues a command like:
o docker run -i -t ubuntu /bin/bash
oCLI sends request to Docker Daemon.
2. Docker Daemon
o Checks if the requested image (e.g., Ubuntu) is available locally.
3. Image Retrieval
o If image not found locally → Docker Daemon pulls it from Docker Registry (e.g.,
Docker Hub).
4. Docker Registry
o Stores public and private images.
o Provides the required image to the daemon.

DEPT OF CSE / SVECW/MNR Page 33

 5. Image to Container
o Docker Daemon creates a container from the image.
o A writable layer is added on top of the image’s read-only layers.
6. Container Execution
o The requested application (/bin/bash in this case) starts running inside the new
container.

Examples of Docker Storage

By default, files created in a container exist only in its writable layer. This means data is lost when
the container is stopped or removed. To solve this, Docker provides three storage mechanisms:

1. Volumes

 Managed by Docker.
 Stored in a special area of the host filesystem (often /var/lib/docker/volumes/).
 Independent of the container’s lifecycle → persists even if the container is deleted.
 Best practice for persistent data (e.g., databases).
 In the above Figure Container connects to the Docker Area in filesystem using Volumes.

2. Bind Mounts

 Directly links a host machine directory or file to a container path.

 Whatever changes are made inside the container are reflected on the host, and vice versa.
 Useful in development, where you need real-time updates (e.g., editing code locally while
running in a container).
 In the above Figure Container maps directly to host filesystem via Bind Mount.

3. tmpfs Mounts

 Stores data in host system memory (RAM), not on disk.

 Data is not persisted after the container stops.
 Useful for sensitive data (e.g., API keys, credentials) that shouldn’t be written to disk.
 On Linux → tmpfs mount
 On Windows → named pipe.
 In the above Figure Container uses host Memory via tmpfs mount.

DEPT OF CSE / SVECW/MNR Page 34

Docker vs. LXC

 Docker initially used LXC (before version 1.0) as its default execution environment.
 Later replaced it with libcontainer (now part of runc), giving Docker more control.
 Differences that made Docker more popular than plain LXC:
o Rich tooling (images, registries, networking).
o Focus on application containers (one process per container).
o Built-in network, storage, and logging namespaces.
o Strong developer ecosystem (Docker Hub, Compose, Swarm, later Kubernetes
integration).

Key Benefits of Containers

(a) Resource Sharing and Isolation

 Uses cgroups and namespaces to provide fine-grained isolation of CPU, memory, and other
resources.
 Multiple containers can run on the same host without interfering with each other.
 Improves resource utilization and increases density of services on a single machine.
 Example: Limit a container to 2 CPUs out of 8.

(b) Portability

 Containers follow open standards (OCI – Open Container Initiative).

 Can be moved easily between registries and hosts (public/private).
 Supports multi-architecture images (e.g., x86, ARMv8).
 Uses copy-on-write layers to enhance portability and efficiency.
 “Build once, run anywhere.”

(c) Lightweight and Increased Density

 Containers don’t need a full OS like VMs → only a few MBs instead of GBs.
 More containers can run on the same hardware → better density.
 Reduced resource overhead = cost savings in cloud/data centers.

(d) Deployment Speed

 Containers package everything needed: OS components, libraries, code.

 Can be created, tested, and deployed within seconds.
 Patches, updates, and new features can be rolled out faster.
 Supports CI/CD pipelines and faster time-to-market.

(e) Scalability

 Containers scale horizontally by adding multiple replicas of the same app instantly.
 Faster boot time than VMs.
 Container orchestration tools (e.g., Kubernetes, Docker Swarm) handle load balancing,
scaling, and resource allocation automatically.

DEPT OF CSE / SVECW/MNR Page 35

(f) Improved Developer Productivity

 Developers can isolate their app but still add custom code/libraries without affecting others.
 Ensures consistency across development → testing → production.
 Faster debugging and testing cycles.
 Shortens time-to-market for applications.

(g) Vendor Agnostic and Multi-Cloud Support

 Containers are vendor-neutral thanks to OCI standards.

 Same container image can run on AWS, Azure, GCP, or private clouds.
 Freedom from vendor lock-in → easier migration and hybrid cloud support.

Container Orchestration
Container orchestration is the process of automating deployment, management, networking,
scaling, and availability of containers.

It is particularly important for managing large sets of containerized applications in cloud or on-
premise environments.

Need for Container Orchestration:

1. High Scale Management

o In production, organizations may run hundreds or thousands of containers.
o Manual deployment, scaling, and monitoring becomes impractical.
2. High Availability & Reliability
o If a container fails, orchestration tools can automatically restart or replace it.
o Ensures zero/minimal downtime for business-critical applications.
3. Efficient Scaling
o Allows applications to scale up/down automatically based on demand.
o Saves cost and optimizes resource utilization.
4. Automation of Repetitive Tasks
o Updates, patching, rollbacks, and enforcing security policies can be automated.
o Reduces manual errors and ensures consistency.
5. Multi-Cloud / Hybrid Flexibility
o Simplifies container movement across servers, clusters, or cloud providers.
o Helps manage multi-tenant environments with diverse apps and programming
languages.

Key Features

 Automated Deployment – Instantly launch containers from images.

 Scaling – Add or remove containers based on demand.
 Load Balancing – Distribute workload across containers.
 Self-Healing – Restart failed containers automatically.
 Service Discovery – Allow containers to find and communicate with each other.
 Security & Policy Enforcement – Manage network policies, secrets, and access.

DEPT OF CSE / SVECW/MNR Page 36

Popular Container Orchestration Tools

 Docker Swarm – Native Docker orchestration tool, simple but limited.

 Kubernetes – Most widely used, feature-rich, supports large-scale deployments.
 OpenShift – Enterprise-ready platform built on top of Kubernetes with extra tools.

Docker Swarm

 Swarm is Docker’s native container orchestration tool.

 It organizes multiple Docker hosts (physical or virtual machines running Docker) into a
cluster.
 Manager Nodes:
o Maintain the cluster state using Raft Consensus (Distributed State Store).
o Handle scheduling, load balancing, and cluster management.
o Expose the Swarm HTTP API.
o Should be in odd numbers (e.g., 3, 5) to ensure consensus and fault tolerance.
 Worker Nodes:
o Actually run the containers (tasks).
o Communicate with each other using the Gossip Protocol for fast updates (e.g., new
tasks, container status).
o Can be promoted to Managers if needed.
 Join Mechanism:
o Workers join a Swarm using a join token provided by a Manager.
 Managers keep the desired state, workers execute the tasks.

Swarm Scheduling of Nginx Service:

DEPT OF CSE / SVECW/MNR Page 37

  A Service is the smallest deployment unit in Docker Swarm.
 Example: Running an Nginx service with 3 replicas.
 The Swarm Manager:
o Accepts the service definition (desired state).
o Breaks it into tasks (each task = 1 container).
o Schedules tasks across available Swarm Worker nodes.
 Execution Flow:
o Service defined (nginx replicas: 3).
o Manager schedules 3 tasks across worker nodes.
o Each task runs an Nginx container (nginx.1, nginx.2, nginx.3).
 Health Management:
o Tasks go through states (assigned → prepared → running).
o If a task/container fails on one node, Swarm re-creates it on another node to maintain
the desired state (self-healing).

Kubernetes (K8s)
 An open-source container orchestration engine for automated deployment, scaling, and
management of containerized workloads and services.
 Origin:
o Built on Google’s 15 years of production experience.
o Open-sourced in 2014, later donated to the Cloud Native Computing Foundation
(CNCF).
 Adoption:
o Used by 60% of organizations worldwide.
o Estimated market value: $944 million by 2024.

Key Features of Kubernetes

 Declarative configuration with YAML files.

 Self-healing: Automatically restarts, reschedules, and replaces containers.
 Scalability: Horizontal scaling of applications.
 Failover: If a node fails, workloads are rescheduled on healthy nodes.
 Extensible ecosystem: Many third-party tools and plugins.

Kubernetes Terminologies

(a) Nodes

 Worker machines (physical/virtual) in a cluster.

 Run containers via container runtime (e.g., Docker, containerd).

(b) Namespaces

 Divide cluster resources among multiple users/projects.

 Provide resource isolation and quotas.

(c) Pods

 Smallest deployable unit in Kubernetes.

DEPT OF CSE / SVECW/MNR Page 38

  A Pod may run one or multiple tightly coupled containers sharing storage, networking,
and lifecycle.

(d) ReplicaSets

 Ensure a fixed number of Pod replicas are running.

 Auto-replace Pods if nodes fail or Pods become unhealthy.

(e) Deployments

 Declarative way (via YAML) to manage Pods & ReplicaSets.

 Used for updates, rollbacks, scaling, and version control.

(f) Services

 Provide network access to Pods.

 Act as a stable endpoint (load-balancer) even if Pods/Nodes change.

(g) Volumes

 Provide storage for Pods.

 Can be ephemeral (lifecycle = Pod) or persistent (outlives Pod).

(h) Secrets

 Store sensitive data (passwords, tokens, keys) securely.

 Mounted into Pods without hardcoding in code.

(i) ConfigMaps

 Store non-sensitive configuration data as key-value pairs.

 Pods can use ConfigMaps as environment variables, CLI args, or config files.

Core Functionalities of Kubernetes (K8s)

(a) Self-Service Discovery & Load Balancing

 Services can be exposed via DNS names or unique IP addresses.

 Kubernetes automatically distributes traffic across Pods to handle high loads and ensure
application availability.

(b) Efficient Storage Management

 Supports multiple storage backends:

o Local storage
o Cloud storage (AWS S3, Azure File Share, GCP Persistent Disks, etc.)
o Network-attached storage (NAS)
 Flexible mounting of volumes into Pods.

DEPT OF CSE / SVECW/MNR Page 39

(c) Automated Rollouts & Rollbacks

 Rollouts → Gradually deploy new configurations/versions without downtime.

 Rollbacks → Revert to previous stable state if issues occur.
 Ensures zero-downtime updates.

(d) Bin Packing & Scheduling

 Developers can define fine-grained resource requests/limits (CPU, RAM) for containers.
 Kubernetes scheduler places Pods intelligently across nodes for optimal resource
utilization.
 Supports affinity/anti-affinity rules → developers can control which workloads run
together or separately.

(e) Self-Healing & Failover Recovery

 Automatically restarts failed containers.

 Terminates unhealthy Pods (based on health checks).
 Reschedules workloads on healthy nodes during failures.

(f) Secrets & Configuration Management

 Secure storage for sensitive information (passwords, tokens, SSH keys).

 Configurations can be managed via ConfigMaps & Secrets.
 Updates configurations dynamically without downtime.

Kubernetes Architecture Components:

Kubernetes architecture consists of two main parts:

1. Control Plane (manages the cluster)

2. Worker Nodes (run the actual workloads/containers)

DEPT OF CSE / SVECW/MNR Page 40

1. Control Plane Components

These manage and control the entire cluster.

(a) API Server (kube-apiserver)

 The front-end of Kubernetes control plane.

 Exposes the Kubernetes API to users, CLI (kubectl), and other components.
 Handles requests (create pod, scale service, etc.) → validates → updates cluster state.
 Scales horizontally (multiple API server instances can run with load balancing).

(b) etcd

 A distributed key-value store.

 Stores all cluster data (configurations, secrets, Pod definitions, current state, desired state).
 Ensures consistency & reliability across the cluster.

(c) Scheduler (kube-scheduler)

 Assigns newly created Pods to appropriate Nodes.

 Scheduling decisions depend on:
o Resource requirements (CPU, RAM)
o Node constraints/affinity rules
o Data locality (storage/network proximity)
o Workload deadlines and policies
 Ensures efficient bin-packing across nodes.

(d) Controller Manager (kube-controller-manager)

 Runs multiple controller processes as a single service.

 Examples:
o Node Controller → monitors node health.
o Job Controller → ensures jobs complete successfully.
o EndpointSlice Controller → manages service-to-pod mappings.
o Service Account Controller → manages service accounts for namespaces.
 Maintains desired state vs. actual state of cluster.

(e) Cloud Controller Manager

 Integrates Kubernetes with cloud service providers (AWS, Azure, GCP).

 Manages cloud-specific tasks (load balancers, storage volumes, networking).
 Not required for on-premise or local clusters.

2. Worker Node Components

Worker nodes run the application workloads in Pods.

 Kubelet:
o Agent running on each worker node.
o Ensures containers (in Pods) are running correctly as instructed by the control plane.
 Kube-proxy:

DEPT OF CSE / SVECW/MNR Page 41

 o Handles network routing on each node.
o Implements service discovery & load balancing by forwarding traffic to Pods.
 Pods:
o Smallest deployable units in Kubernetes.
o Contain one or more containers (application workloads).

The following are the controllers that can have Cloud provider dependencies:

1. Node Controller
o Checks with the cloud provider if a node that has stopped responding is deleted or
still exists.
o Ensures cluster stays consistent with cloud infrastructure.
2. Route Controller
o Configures network routes in the cloud provider environment.
o Ensures Pods across nodes can communicate.
3. Service Controller
o Manages cloud load balancers.
o Creates, updates, deletes load balancers to expose services externally.
4. Kubelet
o Runs on every node like a daemon service.
o Ensures containers inside Pods are running and healthy.
o Watches for PodSpecs (desired state) from the API server → enforces them.
o Reports node & Pod status back to control plane.
5. Kube-Proxy
o Runs on each node.
o Implements Kubernetes Service networking.
o Maintains network rules for communication:
1. Pod-to-Pod within cluster.
2. External-to-Service access.
o Can work with iptables, IPVS, or eBPF rules.
6. Container Runtime
o Responsible for running containers.
o Manages full lifecycle of containers (start, stop, delete, monitor).
o Kubernetes supports multiple runtimes via Container Runtime Interface (CRI):
1. containerd
2. CRI-O
3. (Docker was earlier supported but now deprecated in favor of
containerd/CRI-O)

Public Cloud Container Platforms

In the earlier section, we discussed container runtimes (LXC, Docker, CRI-O, containerd) and
container orchestration engines (Docker Swarm, Kubernetes). These tools allow enterprises to set
up on-premises container environments to support developers.

However, managing such infrastructure on-premises can be complex and resource-intensive. To

address this, major public cloud Infrastructure-as-a-Service (IaaS) providers—such as Amazon
Web Services (AWS), Google Cloud Platform (GCP), and Microsoft Azure—offer fully
managed container services.

DEPT OF CSE / SVECW/MNR Page 42

These services provide enterprises with:

 Abstracted infrastructure management (no need to provision/maintain servers).

 Scalability and elasticity (containers scale up/down with demand).
 Integration with cloud-native services (networking, storage, monitoring, CI/CD).
 Security and compliance features built into the cloud environment.

AWS Elastic Container Service (ECS):

Amazon ECS is a fully managed container orchestration service that enables developers to run,
manage, and scale containerized applications without managing the underlying control plane. It
integrates tightly with other AWS services (e.g., ECR, IAM, CloudWatch) and supports both
EC2-based clusters and serverless container execution (Fargate).

ECS has a layered architecture, shown in Fig. 13.16, consisting of three main layers:

(a) Capacity / Infrastructure Layer

This is the compute foundation where ECS tasks (containers) run. Developers have multiple
options:

 Amazon EC2 Instances → Users choose instance types, sizes, and cluster configurations to
host containers.
 AWS Fargate (serverless engine) → No need to manage servers; pay-per-use compute for
containers.
 On-premises Compute (ECS Anywhere) → External servers/VMs can be registered with
ECS clusters for hybrid workloads.

DEPT OF CSE / SVECW/MNR Page 43

(b) Controller Layer

This layer is responsible for scheduling tasks onto the chosen infrastructure.

 Amazon ECS Scheduler manages the placement of tasks (based on resource requirements,
availability, and policies).
 Handles task definitions (CPU, memory, networking, IAM roles, volumes, etc.) and
ensures they run on the assigned cluster.

(c) Container Provisioning Layer

This layer provides multiple tools for interacting with ECS and deploying applications:

 AWS Management Console – Web interface for managing ECS resources.

 AWS CLI – Command-line tool for provisioning and managing containers.
 AWS SDKs – Programmatic APIs in multiple programming languages for automation.
 AWS Copilot – Open-source CLI tool to simplify container app deployment (CI/CD, load
balancers, monitoring).
 AWS CDK (Cloud Development Kit) – Infrastructure-as-code framework using familiar
programming languages, integrates with CloudFormation.

Application Life Cycle in Amazon ECS:

The process of deploying and managing a containerized application in Amazon ECS goes through
several stages:

1. Build & Store Container Image

 The developer packages the application code, runtime, libraries, and dependencies into a
container image using a Dockerfile.
 The built image is stored in a container registry:
o Amazon Elastic Container Registry (ECR) (recommended)
o or Docker Hub (or another registry).

DEPT OF CSE / SVECW/MNR Page 44

2. Define Task Definition

 A Task Definition is a blueprint in JSON format that describes how containers should
run.
 It includes:
o Container image name (from ECR/Docker Hub)
o CPU & memory requirements
o Networking details (ports, VPC, subnets, security groups)
o Volumes to mount
o IAM roles and environment variables
 This Task Definition acts as the recipe for running containers in ECS.

3. Deploy on Infrastructure (Cluster)

 Using provisioning tools (CLI, Console, SDK, Copilot, or CDK), developers launch the
Task Definition as Tasks or Services.
 ECS runs these tasks on registered compute infrastructure:
o Amazon EC2 instances (user-managed clusters)
o AWS Fargate (serverless execution, no servers to manage)
o On-premises servers (via ECS Anywhere for hybrid deployment)

4. Scheduling & Auto-Healing

 The ECS Service Scheduler ensures that the desired number of tasks is always running.
 If a container or task fails, ECS automatically restarts it to maintain the defined count.
 The ECS Container Agent (running on each EC2 instance or Fargate) reports resource
utilization and executes start/stop commands from ECS.

5. Monitoring & Logging

 Amazon CloudWatch integrates with ECS for:

o Resource utilization (CPU, memory, network, disk)
o Logs from running containers
o Alarms for failure detection
 Helps in troubleshooting and ensuring application reliability.

Azure Container Service (ACS):

Azure provides multiple ways to run containers depending on use cases, ranging from simple
single-container runs to complex orchestrated workloads.

Azure Container Instances (ACI):

Azure Container Instances is the simplest way to run Docker containers without managing servers
or orchestration.

Key Characteristics

Startup Time:

o Containers can start in seconds without provisioning VMs.

DEPT OF CSE / SVECW/MNR Page 45

 o Supports Linux and Windows images from Docker Hub, Azure Container Registry
(ACR), or private registries.

Container Access:

o Exposed to the internet with IP address and DNS name (users can pick DNS).
o Interactive shell with secure TLS communication for debugging and
troubleshooting.

Customized Deployment & Billing:

o Run a single application per container.

o Resource requirements (CPU, memory) defined by the developer.
o Pay-as-you-go billing (per second), with discounted Spot Containers (up to 70%
cheaper).
o Support for multi-container groups, allowing co-scheduling of containers on the
same host (e.g., main app + logging service).

Customer Data & Storage:

o No storage of customer data directly, but can use Azure File Shares for persistent
storage.

Considerations / Limitations

 Image size limit: ≤ 15 GB. Larger images may require Azure File Share.
 Avoid hard-coded IPs → IP can change when container restarts (use static IP via Azure
Application Gateway).
 Certain reserved ports cannot be used.
 Default quota limits (e.g., 20 volumes per container group) → can be increased if needed.

Extended Azure Container-Based Services:

Beyond ACI, Azure provides specialized container services:

(a) Azure Container Apps

 Serverless microservices platform built on containers.

 Abstracts away container orchestration complexity.
 Ideal for event-driven applications and scale-to-zero workloads.

(b) Azure App Service

 Fully managed web application hosting platform.

 Can deploy apps from code or container images.
 Supports websites, REST APIs, and mobile backends.
 Optimized for web workloads.

(c) Azure Kubernetes Service (AKS)

 Fully managed Kubernetes service.

DEPT OF CSE / SVECW/MNR Page 46

  Enables deployment of cloud-native apps across Azure, on-premises, edge, and
multicloud.
 Provides integrations with Azure security, IAM, cost management, and migration services.
 Best suited for large-scale orchestrated workloads.

(d) Azure Red Hat OpenShift

 Fully managed OpenShift clusters (co-managed by Microsoft + Red Hat).

 Built on Kubernetes, but provides developer-friendly PaaS features (CI/CD, logging,
monitoring, etc.).
 Designed for enterprises that want ready-to-use container PaaS with Red Hat’s ecosystem.

(e) Azure Spring Apps

 Fully managed platform for Spring Boot applications.

 Developers focus on business logic, while Azure manages scaling, patching, and
infrastructure.

Google Container Services:

Google is the pioneer of containerization—it not only runs billions of containers per week but
also invented Kubernetes (K8s), based on its internal cluster management system Borg. Google
contributes 10x more Kubernetes code than any other provider.

Google’s container services stand out for:

 Container-native networking → load balancing, routing, DNS, firewall, and observability.

 Zero-trust security architecture → security-first design.
 Multi-cluster and multi-cloud support → high resilience and availability.

Key Services for Running Containers on Google Cloud

(a) Google Kubernetes Engine (GKE)

 Most scalable, fully automated Kubernetes service.

 Available in two editions:
o GKE Standard → automated cluster lifecycle, autoscaling (pods & clusters), cost
visibility, cost optimization.
DEPT OF CSE / SVECW/MNR Page 47
 o GKE Enterprise → adds governance, multi-team/multi-cluster management,
advanced security, service mesh, and GitOps-based config.
 Autopilot mode → developers don’t need Kubernetes expertise; infrastructure is fully
managed.
 Anthos Integration → manage K8s clusters across Google Cloud, other clouds, and on-
premises → true hybrid & multi-cloud orchestration.

(b) Google Compute Engine (GCE) with Containers

 Lets users run containers on VMs with container-optimized OS (COS) (based on

Chromium OS).
 Supports runtimes: Docker, Podman, Kubernetes (preconfigured).
 Flexibility: users can pick container runtime & orchestration tool.
 Supports LXD (Linux containers) → build from Docker images and run on Ubuntu-based
images.
 Can run both public images (from Docker Hub) and custom images.
 Good balance of control (VM-based) + container efficiency.

(c) Google Cloud Run (Serverless Containers / FaaS)

 Fully managed → No infrastructure management.

 Run containers in any language (Go, Python, Java, .NET, Ruby, etc.).
 Bring your own OS libraries & package code + dependencies.
 Integration with Google DevTools (Cloud Build, Cloud Code, Artifact Registry, Docker).
 Billing → pay only when code executes (per 100 ms).
 Free tier: $300 credits + 2M free requests/month.

DEPT OF CSE / SVECW/MNR Page 48