UNIT-3

Cloud Platforms in Industry (Chapter -1)

Cloud platforms provide, businesses and developers the tools they need to store, manage, and
process data online, without the need for physical hardware. Here are some popular cloud
platforms

AWS (Amazon Web Services) is a platform that helps you create flexible applications by offering
solutions for scalability, messaging, and data storage. It provides a web-based console to manage
resources, and it charges users based on what they use ("pay-as-you-go").

AWS offers a robust set of services to help developers build, deploy, and manage applications in the
cloud. Its flexibility and scalability make it an essential tool for modern application development.
Compute Services

1. Amazon Elastic Compute Cloud (EC2): EC2 provides virtual servers (computers)
that can run applications.
a) AMI: AMIs (Amazon Machine Images): An Amazon Machine Image (AMI) is a
template that you use to launch EC2 instances. It provides all the information needed
to create a virtual machine, so you can quickly deploy an instance with the desired
configuration.

b) EC2 Instances: These are virtual machines that run on AWS. You can create them
using templates called VMI.

We can identify Five major categories:

Types of EC2 Instances:

1. Standard Instances: Good for most general applications.

2. Micro Instances: Use low resources, good for small apps with occasional CPU
spikes.
3. High-Memory Instances: Ideal for apps that need a lot of memory, like
websites with high traffic.
4. High-CPU Instances: Suited for tasks that need a lot of processing power.
5. Cluster GPU Instances: These include GPUs and are used for tasks like
rendering graphics.
 C) EC2 Environment:

Amazon EC2 provides virtual machines, called instances that run in a virtual
environment with key services like storage and network configuration.

Key concepts:

• IP Addresses: Instances get internal IPs to communicate within the EC2

network. You can assign Elastic IPs for public access, allowing instances to serve
as servers.
• Domains: Each instance has a domain name based on its IP
• Zones: EC2 has multiple zones across the U.S., Europe, and Asia, allowing control
over where instances are deployed.
• Security: EC2 uses security groups (firewalls) to control traffic by setting rules
for IP addresses, ports, and protocols (TCP, UDP, ICMP).

2. Amazon Elastic MapReduce (EMR): EMR helps process large amounts of data quickly
using tools like Hadoop.

3. AWS Elastic Beanstalk: A service for quickly deploying and managing applications
without worrying about the underlying infrastructure.

4. AWS CloudFormation: Automates the creation and management of AWS resources

using templates.

5. Autoscaling: Automatically adjusts the number of servers (EC2 instances) based on

demand.

Storage Services:

1. Amazon Simple Storage Service (S3): S3 (Simple Storage Service) is designed for simple
storage accessible through a REST interface, similar to a distributed file system but with
key differences. It involves

a) Buckets: A bucket is a top-level container for storing objects in S3, similar to a virtual
drive. Buckets cannot contain sub-buckets. Buckets are created in specific geographic
locations and are replicated for redundancy.

b) Objects and Metadata: These are the files stored in S3, identified by unique names
within their buckets. Objects can be up to 5GB in size. And Metadata can attach
metadata to objects, which are key-value pairs that provide additional information
about the objects.
 c) Resource Naming

• Canonical Form: [Link]

• Subdomain Form: [Link] Virtual

Hosting Form: [Link]

d) Access Control and Security: These policies define permissions for buckets and
objects, allowing users to control who can access or modify their resources.

Permissions:

• READ: Allows reading the object and its metadata.

• WRITE: Allows adding or modifying the object.
• READ_ACP: Allows reading the access control policy.
• WRITE_ACP: Allows modifying the access control policy.
• FULL_CONTROL: Grants all the above permissions.
• Grantees: Access can be granted to individual users or groups (e.g., all users,
authenticated users).

2. Amazon Elastic Block Store (EBS): Provides storage for EC2 instances, like attaching a
hard drive to a computer.

3. Amazon ElastiCache: A caching service that stores frequently accessed data in memory
for quick access.

4. Amazon SimpleDB: A simple, scalable database service for smaller applications.

5. Amazon Relational Database Service (RDS): Managed database service that supports
databases like MySQL and PostgreSQL.

6. Amazon CloudFront: A content delivery network (CDN) that delivers content to users
faster by caching it in multiple locations worldwide.

7. Amazon Import/Export: A service for physically transferring large amounts of data in

and out of AWS using portable storage devices.

Communication Services:
1. Amazon Simple Queue Service (SQS): A messaging service that helps applications
communicate by sending and receiving messages between components.

2. Amazon Simple Notification Service (SNS): A service that sends notifications (emails,
text messages, or app alerts) to users or systems.

3. Amazon Simple Email Service (SES): A scalable email service for sending and receiving
emails.

4. Amazon Route 53: A scalable Domain Name System (DNS) service to route users to
different servers.

5. Amazon Virtual Private Cloud (VPC): A service that lets you create a private network
within AWS, isolated from the internet.

6. Amazon Direct Connect: A service that provides a dedicated network connection from
your data center to AWS.

7. Amazon Elastic Load Balancing: Automatically distributes incoming traffic across

multiple EC2 servers to ensure no one server is overwhelmed.

Additional Services:
1. Amazon GovCloud: A secure and compliant AWS cloud region for government use.

2. Amazon CloudWatch: A monitoring service for AWS resources and applications,

providing real-time insights.

3. Amazon Flexible Payment Service (FPS): A payment processing service for handling
online transactions.

4. Amazon DevPay: A service that allows you to sell applications and services built on
AWS.

5. Amazon Fulfillment Web Service (FWS): A service for managing the logistics of
shipping and inventory for e-commerce businesses.

6. Amazon Mechanical Turk: A crowdsourcing marketplace for getting small tasks done
by humans.
 7. Alexa Web Information Service: A service that provides web traffic and data insights
about websites.

8. Alexa Top Sites: A ranking service that lists the most popular websites globally.

GAE (Google App Engine):

Google App Engine is a PaaS implementation that provides services for developing and
hosting Scalable Web applications. It provides the Runtime environment for the application
developers.

It was developed by the google company and announced in the year 2008. Developers can
develop applications in Java, Python.

App Engine provides a free service with limited quotas that get reset every 24 hours. Once
the application has been tested and tuned for App Engine, it is possible to setup a billing
account and obtain more allowance and be charged on a pay-per-use basis.

Architecture: App Engine is a platform for developing/hosting scalable applications

accessible through the Web.

Google App Engine platform architecture.

Development Environment
 • Local Machine: This is where the developer, write and test the application

• Python SDK: This is a Software Development Kit (SDK) that provides tools and
libraries for developing and testing Python applications locally. It helps simulate how
the app will run in Google App Engine's environment.

• Sandboxed Runtime Environment / Gcloud CLI Environment (Local): This is a

controlled environment on local machine where your app is executed. And it
preventing access to your full computer system.

• Web App (Local): This represents the web application that you are developing. It
runs within the sandbox, interacting only with specific services.

Runtime Environment (Cloud)

• Google App Engine Infrastructure: This represents Google’s cloud infrastructure,

which handles the deployment and running of your web application.

• Sandboxed Runtime Environment (Cloud) / Gcloud CLI/ Active Shell

Environment: Once deployed, your web app runs in the cloud in a secure, isolated
environment (sandbox). This limits access to system resources and ensures security.

• Web App Instances: These are multiple copies of your web app running in the cloud.
Google App Engine can run several instances to handle more users at the same time
(scaling).

• Data Store: This is a cloud-based database service that stores your app’s data, such
as user information, settings, or other data required by the web app.

• URL Fetch: This service allows your app to make HTTP/HTTPS requests to external
websites. It is useful for connecting to other web services or APIs from within your
app.

• Image Manipulation: This service provides tools for editing and managing images,
such as resizing or cropping, directly within your app.

• Task Queue: This feature allows your app to handle background tasks and it queues
tasks to be executed asynchronously.
 • Cron(Scheduler) Jobs: This is a service that lets you schedule regular tasks, like
performing backups or sending reports, at specified intervals.

Application Lifecycle:

GAE Management Console: This is the interface for managing, upgrading, and monitoring
the application once it is deployed.

GAE SDK: This tool helps developers during the build, test, and deploy phases by
simulating the cloud environment locally and managing the application's lifecycle.

Building: The initial phase where developers write and construct the application. The
applications can be built by using JAVA/PYTHON

Java SDK:

➢ Developers can build apps using Java 7 or Java 8.

➢ Eclipse (a popular development tool), the Google App Engine plugin integrates
with it to help you build, test, and deploy apps directly from the tool.
➢ The SDK (Software Development Kit) allows you to develop with common tools
like servlets, HTMLs and provides additional tools for building web apps.
 Python SDK:

➢ Developers can use Python 3.12 to build web apps.

➢ The Google App Engine Launcher is a tool that helps manage apps locally and
deploy them to App Engine.
➢ The launcher allows you to control apps, monitor logs, and profile resources
while the app is running.
➢ The webapp framework (a simplified web framework) is integrated into the
Python SDK.

Testing: Once the application is built, it is tested locally or in the cloud environment to
ensure it functions as intended.

Deploying: After testing, the application is deployed onto the Google App Engine for actual
use by users. Each application gets a unique identifier (like a URL), for example:
[Link] You can also map it to a custom domain name.

Once an application identifier has been created, it is possible to deploy an application on

App Engine. This task can be done using either the respective development environment
(Google App Engine Launcher and Google App Engine plug-in) or the command-line tools.

Managing: After deployment, Developers can manage the application by using the
administrative console. It is also possible to manage multiple versions of a single
application.

Upgrade: If there are new features or improvements, developers can update and upgrade
the application.

Microsoft Azure

Microsoft Azure is a cloud operating system built on Microsoft’s data center infrastructure,
offering developers a wide array of services for cloud application development. It offers
services like computing power, storage, networking, and tools for data analysis.

Below diagram provides an overview of services provided by Azure. Here services can be
managed and controlled through the Windows Azure Management Portal, which acts as an
administrative console for all the services offered by the Azure platform.
 Microsoft Windows Azure Platform Architecture.

Local Development Environment

• Development Tools: These are tools for developers to build and test their
applications locally on their own machines.
• Windows Azure: Refers to the main Azure platform itself where applications are
deployed and managed.
• AppFabric SDK: A set of tools (SDK) for developers to build cloud applications
using Azure services like messaging, caching, and integration.

Windows Azure Platform (Cloud Components)

Management Portal: A web-based interface where users can manage all their Azure
resources, such as virtual machines, databases, and networking services.

Compute:
Windows Azure Compute: This is where Azure provides virtual machines and other
services to run applications in the cloud. Essentially, it handles the processing or
"computing" power needed to run applications.

Compute services are managed through roles:

Web Role:

➢ Used for hosting scalable web applications.

➢ Managed by IIS (Internet Information Services) and automatically scaled during
high demand.
➢ Supports .NET technologies like [Link] and WCF, and can run PHP applications
using FastCGI.
➢ Load balancing ensures traffic is distributed when demand is high.

Worker Role:

➢ Designed for background processing and tasks that don’t need to interact with
HTTP (such as web browsers).
➢ Runs continuously until stopped, providing compute power for non-web tasks.
➢ Can be used for background jobs that support web roles (e.g., handling complex
computations).
➢ Can host any services supported by Windows Server.

Virtual Machine (VM) Role:

➢ Provides the most control over the system, allowing developers to create a
custom Windows Server image with their required applications.
➢ Based on Hyper-V virtualization technology, enabling developers to deploy
virtualized environments.
➢ Requires more administrative effort compared to Web and Worker roles, as
developers manage the configuration and maintenance of the environment.
Storage:

Windows Azure Storage: Provides scalable cloud storage solutions for different types
of data (files, databases, etc.).

Azure offers Blob Storage to store large amounts of data as binary large objects
(BLOBs), such as big text files or media files. There are two types of blobs:

Block Blobs:
➢ Made up of blocks.
➢ Optimized for sequential access, ideal for media streaming.
➢ Maximum size is 200 GB.

Page Blobs:
➢ Made up of pages identified by an offset.
➢ Optimized for random access, suitable for different types of data.
➢ Maximum size is 1 TB.

This service is great for storing large text or binary files efficiently. The ways are,

1. Azure Drive:

➢ Azure allows page blobs to store an entire file system as a Virtual Hard Drive
(VHD).
➢ This VHD can be mounted as part of the NTFS file system in Azure, providing
durable storage.
➢ A page blob used in this way is called an Azure Drive.
2. Tables:

➢ Azure Tables provide semi-structured storage (like spreadsheets).

➢ Data is stored as entities (rows), and each entity has properties (columns).
➢ No strict schema or relationships like in SQL tables.
➢ Tables can store huge amounts of data (up to 100 TB) and allow for quick access
using keys.
➢ They support load balancing by dividing data into partitions.

3. Queues:

➢ Queue storage lets applications communicate by sending messages into queues.

➢ Messages are processed in First-In, First-Out (FIFO) order but can be made
available again if not fully processed.
➢ Clients can read or "peek" at messages, ensuring they are not lost.
➢ All data is geo-replicated three times for disaster recovery, ensuring availability
across different locations.

SQL Azure:

SQL Azure is a relational database service hosted on Microsoft Azure. It provides

scalable, highly available, and fault-tolerant databases.

➢ It is fully compatible with SQL Server, meaning applications built for SQL Server
can easily migrate to SQL Azure.
➢ You can manage SQL Azure using REST APIs or through the Windows Azure
Management Portal.
➢ It ensures that multiple copies of each server are maintained and synchronized
for data reliability.
➢ SQL Azure offers two editions:
➢ Web Edition: For small web applications (supports databases up to 5 GB).
➢ Business Edition: For larger business or enterprise applications (supports
databases up to 50 GB).
 SQL Azure Architecture.

➢ Applications (e.g., PHP, SQL Server tools) interact with SQL Azure via data access
technologies like ODBC and [Link]. These communications use the TDS
protocol.
➢ The Services Layer manages essential features such as provisioning, billing, and
routing of connections.
➢ The Platform Layer ensures database reliability through SQL Azure Fabric, which
handles data replication and management.
 ➢ At the core, SQL Azure operates on physical servers in Microsoft’s data centers
(Infrastructure Layer), ensuring that the databases are replicated and available
in case of failures.

Networking:
Windows Azure Connect: A service to create secure connections between
onpremises systems and Azure resources in the cloud.

AppFabric Service Bus: Allows applications and services to communicate with each
other across different environments, providing messaging and event handling
capabilities.

AppFabric Integration: Integrates different on-premise applications and cloud

services, making sure they work together seamlessly.

Content Delivery Network (CDN): A network of distributed servers that deliver web
content (like videos, images) to users based on their geographic location to make it
faster.

AppFabric Caching: A service that temporarily stores data in memory to make data
access faster and reduce the load on the primary storage systems.

Performance: Refers to various optimization techniques within Azure to improve

application speed and responsiveness.

Identity:

AppFabric Access Control: Manages access to Azure resources by handling

authentication and authorization. It allows users to securely access applications using
their credentials (e.g., username/password).

Marketplace:

DataMarket: A place to buy and sell datasets and information services for use in
Azure applications.

Applications: Refers to a marketplace where users can find, purchase, or deploy

prebuilt applications for Azure.

Enterprise Level Infrastructure

This bottom layer represents the backbone infrastructure that supports all these services,
providing high availability, security, and scalability for enterprise-level applications.
Aneka- Integration of private and public cloud:

Introduction: Aneka is a cloud computing platform and framework developed by

Manjrasoft. It allows developers to build, deploy, and manage applications on different
cloud infrastructures. Aneka provides a platform for creating scalable and flexible cloud
applications.

• Platform Independence: It supports different types of cloud environments,

including private, public, and hybrid clouds.
• Multiple Programming Models: Supports models like task, thread, and MapReduce,
which can be used to parallelize applications.
• Resource Management: It provides resource provisioning, scheduling, and
management to ensure optimized usage of cloud resources.
• Application Development: Helps developers in creating cloud applications by
providing APIs and tools.

Aneka is widely used for research, education, Scientific Applications and enterprise
solutions in cloud computing.

Aneka’s capabilities at a glance:

One of Aneka’s key advantages is its extensible set of application programming interfaces
(APIs) associated with different types of programming models—such as Task, Thread, and
MapReduce-used for developing distributed applications, integrating new capabilities into
the cloud, and supporting different types of cloud deployment models: public, private, and
hybrid. These features differentiate Aneka from infrastructure management software and
characterize it as a platform for developing, deploying, and managing execution of
applications on various types of clouds.
Aneka Architecture:
The core infrastructure of the system is based on the .NET Technology and allows the Aneka
container to be portable over different platforms and operating systems.

Infrastructure Layer:
 • The base of the system consists of different types of hardware infrastructure like
Enterprise Desktop Grid, Data Centers, Clusters, and Public Cloud. This is where all
the actual computing resources are located.
• These resources include physical or virtual machines, storage systems, and
networking components.
Platform Abstraction Layer (PAL):

The Aneka container constitutes the building blocks of Aneka Clouds and represents the
runtime machinery available to services and applications
• This layer abstracts the hardware infrastructure. It acts as a bridge between the
physical hardware and higher-level services, making it easier to manage and use
resources without worrying about the specific hardware details.
• It supports different environments such as .NET, Mono, and various operating
systems like Windows, Linux, and Mac.

Fabric Services:
Fabric Services define the lowest level of the software stack representing the Aneka
Container.
➢ High Availability: Ensures the system is reliable and runs continuously.
➢ Resource Provisioning: Allocates and manages resources like CPU, memory,
and storage.
➢ Hardware Profiling: Identifies and optimizes hardware capabilities.
➢ Membership: Manages which machines and resources are part of the cloud
network.

Foundation Services:
This layer provides essential services required to manage cloud applications:
➢ Storage: Manages where and how data is stored.
➢ Resource Reservation: Reserves resources in advance to ensure they are
available when needed.
• Basic Reservation: This allows users to reserve space on cloud nodes (machines)
for their tasks.
• Libra Reservation: Similar to Basic Reservation, but it adds the ability to charge
different prices for different nodes based on their hardware power or capabilities.
• Relay Reservation: A lightweight option that allows a resource broker to reserve
nodes in Aneka Cloud and control how these nodes are reserved.
➢ Billing & Reporting: Keeps track of resource usage and generates bills
accordingly.
➢ Licensing & Accounting: Handles permissions, licenses, and cost
management.
 Application Services:
These are the core services that developers use to create applications:
These services can be divided in to two types
1. Scheduling Service: Scheduling Services are in charge of planning the execution
of distributed application
2. Execution Service: Execution Services control the execution of single jobs that
compose applications

➢ Task Model: Supports independent tasks where tasks can be run in any order.
➢ Thread Model: Extends multithreaded programming to distributed systems by
executing methods on remote machines.
➢ MapReduce Model: Implements Google’s MapReduce model on the Aneka
platform for processing large datasets.

Application Development & Management Layer:

This layer provides developers with tools, APIs (Application Programming Interfaces), and
SDKs (Software Development Kits) to build, manage, and deploy applications on the
Aneka platform.

Integration of Private and Public clouds: Aneka supports public, private, and hybrid
cloud setups for flexible cloud management.

Private cloud deployment mode:

A private deployment mode in Aneka uses local physical or virtualized resources, like
desktop machines or clusters, to form a cloud. It integrates virtual nodes from systems such
as XenServer, Eucalyptus, and OpenStack using the Resource Provisioning Service. This
setup is ideal for predictable workloads, where resources can be optimized for different
needs, such as using office machines after hours for cloud tasks.
 Master Node: Manages application scheduling and resource allocation.
Application Management & Scheduling: Decides which resources are needed and when.
Resource Reservation: Reserves resources in advance for tasks.
Resource Provisioning: Allocates reserved resources for tasks.
Virtual Clusters (Eucalyptus Systems): Resources are provided to virtual machines
managed by Eucalyptus. Clusters: Physical servers that host virtual machines.
Workstations & Desktop Machines: End-users access resources and run applications
from their desktops or workstations.

Public cloud deployment mode:

In a Public Cloud deployment mode, Aneka master and worker nodes are installed on a
virtualized infrastructure hosted by provider like Amazon EC2. This is included in Aneka's
architecture to leverage its cloud computing capabilities as it has Hybrid Cloud Support.
 • Master Node: Controls and manages the overall system.
• Application Management & Scheduling: Decides which applications run and
allocates resources for them.
• Resource Provisioning: Provides the necessary resources to run applications.
• Amazon EC2 Instance: Allocates and deploys virtual machines in the cloud (Amazon
EC2).
• Reporting, Billing, And Accounting: Tracks usage, billing, and other management
tasks.
• Slave Nodes: Executes tasks on Amazon EC2 Instances which are part of the cloud
infrastructure within defined EC2 boundaries.