Overview of Software Development Life Cycle
Overview of Software Development Life Cycle
INTRODUCTION
SDLC:
SDLC stands for Software Development Life Cycle.
It represents the process of developing quality software as per customer requirements.
It consists of several stages, each with specific activities that lead to the next stage.
Phase 1: Requirements Gathering and Analysis:
1. Phase conducted by Project Manager, Business Analyst or senior member in the team
2. To understand the aim, purpose and exact requirements of the customer.
3. Software Requirement Specification(SRS)/ Business Requirement Specification / User
Requirement Specification document is prepared.
4. Identify the risk assessment in the project.
5. Planning of project schedule , cost estimation and other resourses.
Phase 2: Design
Phase conducted by System Analyst.
Design Document Specification / Software Design Document (SDD)document is
prepared.
The Design phase is crucial as it involves creating the blueprint for the software. This
phase can be broken down into two main categories: High-Level Design (HLD) and
Low-Level Design (LLD).
High-Level Design (HLD):
1. Architecture Design: This involves deciding on the overall structure of the
system. It includes choosing the architecture pattern (e.g., client-server,
microservices, monolithic) and defining the major modules or components and
their interactions.
2. Data Flow Design: Specifies how data will move through the system, including
how data is input, processed, and output. This could involve designing data flow
diagrams (DFDs) or Entity-Relationship Diagrams (ERDs).
3. Technology Stack: Choosing the technologies, programming languages,
databases, frameworks, and tools that will be used to build the system.
4. Interface Design: Outlining how different modules or systems will interact with
each other, often through APIs or messaging systems.
Low-Level Design (LLD):
1. Detailed Module Design: Designing each module or component in detail. This
includes specifying the classes, methods, functions, and data structures that will
be used.
2. Algorithm Design: Developing the algorithms that will be implemented to solve
specific problems within the software, such as sorting, searching, or data
manipulation algorithms.
3. Database Design: Defining the database schema, including tables, relationships,
keys, indexes, and constraints. This also includes writing SQL queries or defining
how data will be stored and retrieved.
Phase 3: Develop
The actual code is written based on the design documents. Developers start coding the various
components of the application.
Phase 4: Testing:
The software is tested to find and fix any bugs. This includes unit testing (testing individual
parts), integration testing (testing how parts work together), and system testing (testing the whole
application).
Phase 5: Deployment / Release
The software is released to users.
Phase 6: Maintenance
After the software is deployed, it enters the maintenance phase where it is monitored, and
updates are made to fix bugs or add new features.
There will be good communication between customer , Business Analysts , developers and
testers
When to use Agile Model
1. When project size is large.
2. When frequent changes are required.
3. When a customer is ready to have a meeting with a software team all the time.
Advantages:
1. Requirement changes are allowed in any stage of development or we can accommodate
requirement changes in the middle of the development.
2. Release will be very fast (weekly).
3. Good communication between team members.
4. It is a very easy model to adopt.
5. Customers do not need to wait for a long period of time.
6. Little planning required.
7. Updated versions of functioning software are released every week.
Disadvantages:
1. Less focus on design and documentation since we deliver software very faster.
For any software development / product , two groups of engineers will work
1. Development group
2. Operations Group / Non - Development / Administrators Group.
Note: Again this classification can be divided into smaller groups.
1. Development Group:
The people who involve in
1. Planning
2. Coding
3. Building
4. Testing
Are considered as Development Group
Example:
Business Analyst (BA)
System Analyst (SA)
Design Architect (DA)
Developer / coders
Build Engineers
Test Engineers (QA)
2. Operations Group:
The people who involve in
1. Release
2. Deployment
3. Operations
4. Monitoring
Are considered as Operations Group
Example:
Release Engineer
Configuration Engineer
System Admin
Database Admin
Network Admin etc
Project Owner:
1. Define the features of the project.
2. They decide which tasks are the most important and should be done first.
3. They make and manage a list of tasks (called the product backlog) that the team needs to
complete.
4. They check the completed tasks to make sure they meet the requirements and provide
feedback.
5. They make important decisions about the product features and release plans.
6. Accept or reject work results.
Scrum master:
The main role is facilitating and driving the Agile process.
Scrum Terminology:
User Story: A feature / module in a software product.(Smaller requirement)
Epic: Collection of user stories.(larger requirement)
Product backlog : Contains list of user stories. Prepared by the Product Owner.
Sprint / Iteration : Period of time to complete the user stories, decided by the product owner
and [Link] 2 - 4 weeks of time.
Sprint planning meeting: Meeting conducts with the team to define what can be delivered in
the sprint and duration.
Sprint backlog: List of committed stories by Dev / QA for specific sprint.
Scrum meeting: Meeting conducted by the scrum master everyday 10 to 15 min.
Also called scrum call or daily standup meetings.
Some of the questions in the meeting are
1. What did you do yesterday?
2. What will you do today?
3. Are there any impediments in you way?
Sprint retrospective meeting: conducts the meeting after the completion of the [Link] entire
team, including both the Scrum master and the product owner should participate.
Main purpose is
1. What went wrong during the sprint
2. What went well during the sprint
3. Are there any improvements for the upcoming sprints.
Story point: Rough estimation of user stories, will be given by Dev and QA in the form of
fibonacci series.
1 story point = 1 hour / 1 day (6 Hours).
Login —-> Dev 5 QA - 3 = 8 Hours / 1 day
Burndown chart: shows how much work remains in the sprint. Maintained by scrum master
daily.
source : internet
Roles:
Product owner
Scrum master
Team
Artifacts:
Product backlog
Sprint backlog
Burndown chat
Ceremonies:
Sprint Planning
Daily scrum
Sprint Review
Kanban
Kanban is a visual workflow management method used to optimize the flow of work and
improve efficiency in various processes. It is widely used in software development, project
management, and manufacturing. The primary components of Kanban are the Kanban board,
cards, and columns.
Basic Concepts of Kanban
1. Kanban Board: A visual representation of the workflow. It consists of columns that
represent different stages of the process.
2. Kanban Cards: Items or tasks that move through the workflow. Each card represents a
single work item.
3. Columns: Different stages of the workflow, such as "To Do", "In Progress", and "Done".
Example:
Let's create a simple Kanban board for a software development project.
Kanban Board
User Login Implement API Design database Review login Deploy website
Explanation
1. Backlog: This column contains tasks that are planned but not yet ready to be worked on.
2. To Do: This column contains tasks that are ready to be started.
3. In Progress: This column contains tasks that are currently being worked on.
4. Code Review: This column contains tasks that are completed but need to be reviewed
before moving to "Done".
5. Done: This column contains tasks that have been completed and reviewed.
Benefits of Kanban
1. Visual Management: Provides a clear visual representation of the workflow, making it
easy to see the status of tasks.
2. Flexibility: Allows for easy adjustments to the workflow as tasks move through different
stages.
3. Improved Efficiency: Helps identify bottlenecks and areas for improvement, leading to a
more efficient process.
4. Limit Work in Progress (WIP): By limiting the number of tasks in each column,
Kanban helps prevent overload and maintain a steady workflow.
Scrum kanban
Scrum has defined roles: Scrum Master, Kanban doesn’t prescribe specific roles.
Product Owner, and the Development Team. Teams can adapt roles based on their needs.
Scrum includes specific ceremonies like Kanban does not have mandatory events,
Sprint Planning, Daily Stand-ups, Sprint though teams might use regular meetings for
Review, and Sprint Retrospective. coordination.
Key artifacts in Scrum include the Product The primary artifact in Kanban is the Kanban
Backlog, Sprint Backlog, and Increment. board with columns representing different
stages of the process (e.g., To Do, In Progress,
Done).
Scrum is less flexible in terms of changing the Kanban is more flexible, allowing changes in
scope during a sprint. Once a sprint starts, the task priorities at any time. Teams can pull
tasks should remain fixed until the sprint ends. new tasks into the workflow as they finish
current ones.
WIP is managed through sprint planning, WIP is explicitly limited by setting WIP limits
where the team commits to completing a on each stage of the workflow. This helps
specific set of tasks within the sprint. prevent bottlenecks and ensures a steady flow
of work.
Scrum is typically used by smaller, cross- Kanban can be used by teams of any size and
functional teams (usually 3-9 members). is particularly effective for teams working on
maintenance or support tasks.
The DevOps process and Continuous Delivery
Here, we will discuss the different parts of the Continuous Delivery pipeline.
The developers:
❖ Write and test code.
❖ Developers should have environments on their computers that closely resemble the
production environment where the software will eventually run.
❖ However, it's often challenging to replicate the entire production environment. Therefore,
developers often simulate or "mock" parts of it, like external payment systems or specific
hardware.
❖ If a developer has a strong background, they might prefer and frequently use prepackaged
environments. These environments save time and provide consistency, including
necessary tools like specific versions of Java or Python and development software like
Eclipse.
❖ Includes all tools needed for development, such as specific versions of programming
languages and development environments. This environment is not used for testing or
production.
❖ Developers need a way to deploy and test their code in an environment similar to
production. This could be done using virtual machines, cloud instances, or Docker
containers.
The revision control system:
Revision Control System is mainly used to store and track changes to the software code. But it
can also store other important files.
Types of Files Stored
❖ Code: The main software code that makes the software work.
❖ Configurations for Infrastructure: These are files that tell the software how to run on
different servers or environments. For example:
➢ How to set up a database.
➢ Which server settings to use.
➢ Network configurations.
❖ Hardware Designs: If the software interacts with specific hardware, the designs or
Package managers:
Package Formats
❖ When we talk about "packages," we mean files that contain software.
❖ Different types of Linux systems use different package formats:
➢ Red Hat-like systems use RPM (Red Hat Package Manager) files.
➢ Debian-like systems use .deb files.
Binary Repository
❖ A binary repository is a storage location where these package files are kept.
❖ It's like an online store where your server can go to download the software packages it
needs.
Package Managers
❖ Package managers are tools that help install, update, and manage software on your server.
❖ They fetch packages from the binary repository and install them on your server.
❖ Different Linux systems use different package managers:
➢ On Red Hat systems, the package managers are yum or dnf.
➢ On Debian systems, the package managers are apt-get or dpkg.
How Package Managers Work
1. Installation: You tell the package manager to install a piece of software.
2. Updating: You tell the package manager to update software to the latest version.
3. Handling Dependencies: If the software you want needs other software to work (called
dependencies), the package manager will automatically install those too.
Test environments:
After the build server finishes creating the software files (called artifacts), it saves them in a
special storage place called a binary repository.
These stored files can then be installed in test environments. Test environments are places where
you test the software to make sure it works correctly.
Staging:
A staging environment is a type of test environment that closely mimics the production
environment.
Before deploying the software to the production environment, it is deployed to the staging
environment. The staging environment is set up to be almost identical to the production
environment.
It serves as the final testing ground to catch any issues that might occur in the production
[Link] the staging environment is similar to the production environment, it allows
for realistic testing of how the software will perform once it’s live.
In the staging environment, the software should be installed and configured using the same
methods as in the production environment. This ensures that any issues related to deployment are
identified and resolved before going live.
Production Environment:
This is the live environment where real users interact with the application.
It handles actual traffic, data, and business operations.
Once software is deployed to production, it becomes publicly available to users.
Release Management:
In an ideal DevOps setup, the release process would be fully automated, allowing changes to
flow from development to production without human intervention. However, in reality, it's hard
to achieve this level of automation. One reason is that it's challenging to have enough automated
testing to trust that everything works perfectly. Another reason is that business priorities often
don't align with the pace of technical changes. Because of these challenges, human involvement
is needed during the release process. A "faucet" is used as a symbol to represent the role of a
release manager who steps in to control when and how updates are deployed. While automation
helps with most of the process, humans still decide which software versions should be used in
different environments, such as testing, staging, or production, based on testing results and
business needs.
identifying bottlenecks:
In an ideal situation, whenever a developer makes changes to the code, these changes should be
tested and ready for use within 15 minutes. However, when things go wrong, this process can
take much longer, sometimes even days. Some common reasons for these delays include:
Changes to the database: If the structure of the database is modified, it can cause issues.
Test data mismatch: The test data might not align with what is expected, leading to errors.
Dependence on specific people: Sometimes, only a particular person knows how to deploy, and
if they're unavailable, the process is delayed.
Too much bureaucracy: Unnecessary rules or approvals may slow down the process.
Large changes: If the changes are too big, they require extra effort to ensure they are deployed
safely. This often happens when the system's architecture is a single large unit (monolith) instead
of smaller, independent parts.
UNIT - II
Business agility: It refers to an organization's ability to rapidly adapt and respond to changes in
the market, customer needs, and technological advancements. It’s the capacity of a business to
stay flexible and responsive in a fast-paced, often unpredictable environment.
The DevOps lifecycle aims to enhance business agility by integrating development (Dev) and
operations (Ops) teams to deliver faster, more reliable software and services. Here's how each
stage of the DevOps lifecycle contributes to business agility:
Plan
Involves defining project goals, feature sets, and timelines collaboratively across [Link]
collaboration between development, operations, and business teams ensures that customer needs
are understood and can be quickly integrated into the development process.
Code
Developers write and manage the code for new features or [Link] shared repositories,
modular code structures, and collaboration tools speeds up the development process, allowing for
quick updates.
Build
The code is compiled into executable applications or services.
Automated build tools (like Jenkins, GitLab CI) ensure fast feedback on code quality, helping
teams detect and resolve issues early, reducing delays.
Test
Automated testing is performed to verify the code and ensure functionality.
Continuous testing with automation allows teams to deliver bug-free code faster and more
frequently, reducing risks and enabling rapid releases.
Release
The application is packaged and deployed to production or staging environments.
Automating the release process with tools like Jenkins, Bamboo, or Azure DevOps ensures faster
and more reliable deployments with minimal downtime.
Deploy
The application or service is deployed to production and becomes available to users.
Continuous Deployment (CD) pipelines enable businesses to push features, updates, and patches
to production quickly, allowing businesses to adapt to market needs swiftly.
Operate
Ongoing monitoring and management of the software in a live environment.
By using real-time monitoring (e.g., Prometheus, Grafana), teams can detect issues quickly, fix
problems before customers notice, and continuously improve performance.
Example of Operation:
❖ The operations team ensures the website is up and running 24/7, so customers can browse
and buy products.
❖ When new features (like a product recommendation system) are ready, the team deploys
them to the live site.
❖ If a sudden surge in traffic happens during a big sale (like Black Friday), the team
ensures that the site scales up its resources (more servers) to handle the load without
crashing.
❖ Regular updates and security patches are applied without causing downtime or
disruptions for users.
Monitor
Continuous monitoring of application performance and user experience.
Continuous feedback from users and systems helps teams quickly iterate on the product, ensuring
faster adaptation to user needs and issues.
Monitoring tools continuously check how the site is performing. They track metrics like how
long it takes for product pages to load, how many users are adding items to the cart, and how
many successful transactions occur.
If a problem arises, such as checkout errors increasing to 10% within a few minutes, the
monitoring system will send an alert so the team can fix the issue before it impacts more
customers.
Benefits:
Early Bug detection
Faster feedback loop
Improved Release Confidence
Cost and time consuming
How DevOps Affects Architecture
Introducing software architecture
Software Architecture and Non-Functional Requirements
Software Architecture: Think of this as the blueprint or plan for building software. It shows how
different parts of the software fit together.
Functional Requirements: These are about what the software does. For example, "Our system
should handle credit card transactions."
Non-Functional Requirements: These are about how well the software performs its tasks. For
example, "Our system should handle several credit card transactions per second."
How DevOps Affects Software Architecture
When using DevOps, we need our software to meet two key non-functional requirements:
1. Deploy Small Changes Often: We need to make and deploy small updates to the software
frequently.
2. High Confidence in Quality: We need to be very sure that these updates won't break the
software.
1. User Interaction: A user interacts with the web page (presentation tier) by filling out a
form to add a new customer.
2. Processing: The form data is sent to the server (logic tier), which processes and verifies
the data.
3. Data Storage: The verified data is sent to the PostgreSQL database (data tier) to be
stored.
4. Retrieval: When the user wants to view or edit customer information, the logic tier
retrieves the data from the database and sends it back to the presentation tier to be
displayed on the web page.
Handling Database Migrations Explained
When we change a database's structure (like adding a new column to a table), it's called a
database migration. This process needs special care because databases store data and structure.
Database State: Unlike application binaries (program files), databases contain lots of data (state).
When we update a database, we must ensure the data remains correct and intact.
1. Tracking Versions: We need a way to keep track of different versions of the database as
it changes over time.
Steps for Database Migration
1. Version Table: Add a special table in the database to store the current version of the
database schema.
2. Changesets: Group database change commands into versioned sets called changesets.
These changesets describe what changes need to be made to move from one version of
the database to the next.
3. Migration Tools: Use tools like Liquibase or Flyway to manage and apply these
changesets.
Liquibase is a tool that helps manage and track changes to your database schema over time. It
integrates well into DevOps processes, making database changes (or "migrations") as easy and
automated as application code changes. Liquibase allows you to define database changes in a
version-controlled, easy-to-read format, like XML, YAML, JSON, or SQL.
Example
Let's say we have a simple database for a library system, and we need to add a new column to
store the publication year of books.
Before Migration
id title author
Changeset
We create a changeset that describes adding a new column, publication_year, to the Books table.
Changeset Example (in SQL):
ALTER TABLE Books ADD COLUMN publication_year INT;
Changeset Example (in XML for Liquibase):
<changeSet id="1" author="admin">
<addColumn tableName="Books">
<column name="publication_year" type="INT"/>
</addColumn>
</changeSet>
After Migration
Books Table (with new column):
A rolling upgrade is a deployment strategy that aims to minimize downtime when making
changes to a system, such as updating a database schema or server configuration. This method
allows parts of the system to be updated gradually, while the rest of the system continues to
operate.
Example:
In a scenario where you have two servers, a single database, and a load balancer distributing
requests, you need to perform a schema change that splits a "name" field into "first name" and
"surname" fields, all while minimizing downtime. First, you modify the database by adding the
new fields and populate them by splitting the existing "name" field. However, you keep the
original "name" field to ensure compatibility with the existing system, avoiding any immediate
downtime. Next, you take one server (Server 2) offline by reconfiguring the load balancer to
direct all traffic to Server 1, which continues to operate using the old schema. With Server 2
offline, you upgrade it to use the new fields, then bring it back online and verify that it's
functioning with the new schema. The load balancer is then switched to route traffic to Server 2
while you upgrade Server 1. Once both servers are upgraded to use the new fields, the load
balancer resumes distributing traffic to both servers, now running the updated code. Finally, after
confirming that the system no longer uses the old "name" field, you can remove it from the
database. This process allows a smooth, phased upgrade with minimal disruption to users.
Benefits of Rolling Upgrade:
❖ Minimal downtime: At least one server is always online to handle user requests.
❖ Backward compatibility: The system continues to function even while part of it is being
upgraded.
❖ Gradual changes: Each server is upgraded one at a time, reducing the risk of a complete
system failure.
Microservices:
Microservices is an architectural style where an application is built as a collection of small,
independent services. Each service performs a specific function and communicates with others
through lightweight protocols (like HTTP APIs or messaging queues).
Resilience
Resilience in software systems refers to the ability of a system to handle and recover from
failures gracefully, without affecting the overall functionality or user experience. In the context
of modern software architecture, particularly in distributed systems like microservices, resilience
ensures that the system can continue operating under adverse conditions, such as hardware
failures, network issues, or unexpected load spikes.
UNIT – III
Historical Versions of Source Code:Archiving: The earliest form of source code management
involved storing historical versions of code in separate archives. This is still relevant today,
especially in free software projects that provide downloadable archives of older releases (e.g., tar
files). This method is simple but lacks sophisticated management features.
Centralized Source Code Management:
Check-In and Check-Out: In centralized systems, developers check files in and out, often
locking them for exclusive use. This means that while one developer is working on a file, others
cannot modify it.
Examples
RCS (Revision Control System): One of the earliest tools for managing source code.
SCCS (Source Code Control System): Another early tool focused on individual file
management.
Centralized Workflow: Later systems, like CVS (Concurrent Versions System) and Subversion
(SVN), introduced a centralized repository where developers merge changes before committing.
This allows for better coordination among team members, but can lead to bottlenecks as all
operations depend on the central server. Subversion remains popular for organizations that prefer
centralized workflows.
Decentralized Source Code Management:
Flexibility and Concurrency: As the need for collaboration increased, decentralized systems
emerged, offering more flexibility. Each developer can work on their own copy of the codebase
without affecting others until they are ready to share their changes.
Popular Tools: Git is the most widely used tool in this category, known for its robust branching
and merging capabilities. Other tools like Bazaar and Mercurial also offer decentralized
management, but Git has become the dominant player.
Roles and code
SCM tools serve as a central hub for collaboration among various roles in an organization. They
facilitate version control and maintain the history of changes in the codebase, making it easier for
teams to work together.
Different Roles and Their Use of SCM
Developers: For developers, SCM is essential. They rely on it for version control, collaboration,
and tracking changes in the source code. It is fundamental to their daily tasks.
Operations Personnel: Operations teams also benefit from SCM by managing infrastructure
descriptions as code. This includes scripts and configurations that define how infrastructure
components (like servers, networks, etc.) should be set up. This practice is often referred to as
"Infrastructure as Code" (IaC).
Quality Assurance (QA): QA personnel can utilize SCM to store automated tests and testing
frameworks (like Selenium or JUnit). This allows for better integration of testing with the
development process, ensuring that testing artifacts are version-controlled and easily accessible.
Centralized Documentation: From a DevOps perspective, there is a call to action for
organizations to centralize documentation. By storing all documentation in a wiki format within
the SCM repository, all roles can access consistent and organized information. This enhances
collaboration and ensures that everyone has the information they need, reducing the barriers
created by fragmented documentation practices.
Source code management system migrations
SCM migration is when a team moves their code and its history (all the changes and versions)
from one SCM system to another.
Teams usually migrate to get better tools, improved speed, or features that their old system lacks.
Reason for Migration:
The old system might be outdated, slow, or lack new features that developers need.
Newer systems (like Git) have better tools, work faster, or fit better with modern development
practices.
Trying to move everything, including every little change ever made. Sometimes, this is important
(like for open-source projects), but it can be time-consuming and complex.
For some companies, keeping all the old versions isn’t always worth the time and hassle. They
might decide just to start fresh in the new system, or keep the old system around just in case they
need it.
Types of SCM Migrations:
Simple Migrations: Some migrations, like moving from Subversion to Git, are easier because
the systems are similar enough that history can be transferred without much trouble.
Complex Migrations: Moving from really old systems (like Visual SourceSafe) can be much
harder because they work differently, and trying to move everything perfectly can take a lot of
effort.
Shared authentication
In many organizations, a central system is used to manage who can log in (authenticate) to
various applications or services. A common choice for this central system is something called an
LDAP server (Lightweight Directory Access Protocol). It acts like a shared directory where user
accounts and their credentials (e.g., usernames and passwords) are stored and managed in one
place.
Instead of managing separate login systems for each application (which can get messy), LDAP
makes it easier to centralize this process. Applications can "ask" the LDAP server to check if
someone’s username and password are correct.
While many organizations already have LDAP servers set up for their operations, you might
need a test LDAP server to learn how it works or test your setups. A popular choice for such
purposes is the 389 Server, which gets its name from the default port (389) used for LDAP. You
can use this server along with a tool called phpLDAPadmin to make managing and testing LDAP
easier. phpLDAPadmin provides a web-based interface for you to manage users and settings on
the LDAP server without needing to type complex commands.
Hosted Git servers
Organizations need a way to manage their code securely and efficiently. When it comes to using
Git, there are two main options: hosting your own Git server or using a hosted service like
GitHub or GitLab.
Certain organizations, like government agencies or companies in banking, insurance, or gaming,
often prefer hosting their Git servers internally. This is because:
1. Legal Requirements: Laws might prevent them from storing sensitive data outside their
premises.
2. Security Concerns: They might feel uneasy about storing critical code on someone
else’s servers.
For such cases, hosting a Git server within the organization ensures that all code stays in their
control.
If there are no strict rules or security concerns, using hosted services like GitHub or GitLab can
be a great choice. These services offer private accounts, allowing teams to collaborate on code
securely. They are also user-friendly and a good way to learn Git and explore its features. Both
platforms offer free accounts to help you get started.
These hosted services provide many additional tools that go beyond what plain Git offers, such
as:
Web Interfaces: Easy-to-use graphical tools for viewing and managing code.
Git’s distributed nature allows flexibility in using different server implementations for managing
code repositories. Whether it’s a public hosting platform like GitHub or GitLab, or a private
server hosted internally by an organization, the client-side workflow remains consistent across
all setups. Public Git servers are ideal for collaboration and learning Git, offering user-friendly
interfaces and additional features such as wikis, issue tracking, and pull request workflows. In
contrast, private Git servers are used by organizations that prioritize security and compliance,
especially for sensitive or regulated projects. Git also supports the use of multiple remotes within
a single repository, enabling developers to connect to various servers for collaboration, backup,
or other purposes. This setup ensures flexibility in choosing the best hosting solution, maintains
backup copies for security, and simplifies code management while adhering to consistent
workflows.
Docker
Virtualization:
Virtualization means run multiple Operating System on single physical system
1. In virtualization every VM is independent.
2. To set up a VM hypervisor is required.
3. Space is wasted and high cost.
Containerization:
It is a thin virtualization .here containers mean VM’s .container has only one [Link] will
take the required files from host OS
1. It doesn't have any OS
2. They share Host OS
3. Portable
What is Docker ?
Docker is a containerization platform that provides an easy way to containerize your
applications, which means, using Docker you can build container images, run the images to
create containers and also push these containers to container registries such as DockerHub,
[Link] and so on.
In simple words, you can understand as containerization is a concept or technology and Docker
Implements Containerization.
Or
Docker is a containerization platform that packages your applications and all its dependencies
together in the form of a docker container to ensure that your application works seamlessly in
any environment.
What is a container ?
A container is a standard unit of software that packages up code and all its dependencies so the
application runs quickly and reliably from one computing environment to another. A Docker
container image is a lightweight, standalone, executable package of software that includes
everything needed to run an application: code, runtime, system tools, system libraries and
settings.
A container is a bundle of Application, Application libraries required to run your application and
the minimum system dependencies.
Dockerfile
Dockerfile is a file where you provide the steps to build your Docker Image.
Images
An image is a read-only template with instructions for creating a Docker container. Often, an
image is based on another image, with some additional customization. For example, you may
build an image which is based on the ubuntu image, but installs the Apache web server and your
application, as well as the configuration details needed to make your application run.
You might create your own images or you might only use those created by others and published
in a registry. To build your own image, you create a Dockerfile with a simple syntax for defining
the steps needed to create the image and run it. Each instruction in a Dockerfile creates a layer in
the image. When you change the Dockerfile and rebuild the image, only those layers which have
changed are rebuilt. This is part of what makes images so lightweight, small, and fast, when
compared to other virtualization technologies.
Docker offers several key benefits, especially in software development and deployment:
1. Consistency Across Environments
2. Lightweight & Efficient
3. Improved Scalability
4. Isolation
5. Version Control & Rollbacks
6. Simplifies DevOps & CI/CD
7. Portability
8. Resource Efficiency
9. Faster Deployment & Start-up
10. Simplified Dependency Management
11. Easier Collaboration
12. Security
Gerrit
Gerrit is an essential tool for teams that require a structured approach to code reviews. It is
designed to enhance code quality by requiring that changes be reviewed and approved before
merging into the main codebase. This process not only helps catch errors early but also fosters
accountability among developers. For junior developers, Gerrit provides a learning opportunity
by allowing them to receive feedback and guidance from senior team members. It is particularly
useful for critical systems where even small errors can have significant consequences and for
large teams where managing contributions effectively is a priority.
The workflow in Gerrit is straightforward. Developers push their changes to Gerrit instead of
directly to the main branch. These changes are then reviewed by another team member or a
group of reviewers. Once the review is complete and the changes are approved, Gerrit merges
them into the main codebase. This centralized review process ensures that all contributions are
thoroughly evaluated, improving collaboration and maintaining a high standard of code quality.
Gerrit offers several technical features that make it suitable for enterprise-level projects. It
supports integration with various relational databases such as MySQL and PostgreSQL and
includes a built-in H2 database for quick evaluations. Gerrit is scalable and can handle large
repositories and high traffic, making it a robust choice for both small teams and large
organizations. It also integrates seamlessly with other tools like CI/CD pipelines, enabling
automatic testing and deployment of reviewed code.
One of the simplest ways to try Gerrit is by using Docker. With pre-built images like
openfrontier/gerrit, users can quickly set up a Gerrit instance and begin exploring its features.
Once deployed, the web interface provides an easy way to review and manage code changes.
This makes it accessible for teams looking to evaluate its potential benefits without committing
to a full production setup initially.
While Gerrit is powerful, it does come with a few challenges. It has a learning curve, especially
for new users who may need time to understand its interface and workflows. Full production
configurations can be complex despite the simplicity offered by Docker for initial setups.
Additionally, Gerrit relies heavily on timely reviews, which means delays from reviewers can
slow down development progress. Despite these challenges, Gerrit remains a valuable tool for
projects requiring precise control over code contributions and a systematic review process.
Pull request model
The pull request model is a common workflow used in version control systems, especially in Git
and platforms like GitHub. It facilitates collaboration on software development projects by
enabling developers to contribute code in an organized and controlled manner. Here's how it
works:
1. Branch Creation: A developer creates a new branch from the main (or master) branch in
the repository. This branch is used to develop a new feature, fix a bug, or make some
changes.
2. Commit Changes: The developer works on this branch locally and commits the code
changes as they progress.
3. Push Changes: Once the developer is satisfied with the changes, they push the branch to
the remote repository (like GitHub).
4. Pull Request Creation: The developer then creates a pull request (PR) from their branch
to the main branch (or another target branch). A pull request is essentially a request to
merge the changes from one branch into another.
5. Code Review: Other team members (or maintainers) review the pull request. They can
leave comments, request changes, or approve the code.
6. Testing: Automated tests can be triggered during this process, ensuring the code doesn't
introduce new issues.
7. Merge: Once the pull request is approved, it is merged into the target branch (often the
main branch). The branch with the changes can then be deleted, if necessary.
Example:
Open github and create a new repository and name it CalculatorProject
When we create a child branch , we get all the files will be inherited from the main branch
Open add-square-feature and add the square function
GitLab is a web-based DevOps platform that provides a complete set of tools for managing the
software development lifecycle. It integrates source code management (SCM) with Git,
continuous integration/continuous deployment (CI/CD), and project management features,
making it a one-stop solution for developers and DevOps teams. GitLab allows teams to
collaborate efficiently by providing a centralized platform for hosting repositories, tracking
issues, automating testing and deployments, and monitoring application performance.
Features
1. Source Code Management (SCM): Supports Git repositories with features like
branching, merging, and pull requests (called merge requests in GitLab).
2. Integrated CI/CD: Automates building, testing, and deploying applications using
GitLab’s built-in CI/CD pipelines.
3. Issue Tracking: Provides a system to track bugs, enhancements, and tasks directly linked
to the codebase.
4. Wiki and Documentation: Offers in-built wikis for project documentation.
5. Self-Hosted Option: Allows organizations to host GitLab on their servers for greater
control and security.
6. Container Registry: Includes a built-in container registry for managing Docker images.
7. Security and Compliance: Offers features like vulnerability scanning and code quality
checks.
UNIT - IV
Build Systems:
Building code involves transforming it from its original form (source code) into a form that a
machine can execute, typically through compilation or interpretation.
Compilation: This is the process of translating source code into native machine code or virtual
machine bytecode. For example, languages like C and C++ compile directly into machine code,
while Java compiles into bytecode that runs on the Java Virtual Machine (JVM). The
compilation step ensures that the code can be executed efficiently on the target platform.
Linting: Linting is like doing a thorough check of your code before running it. It doesn’t actually
execute the code, but it looks at the code and tries to spot mistakes or "bad" patterns. Think of it
like proofreading an essay for typos or grammar issues before submitting it.
The name "Lint" comes from a tool that was used in old Unix systems to find problems in
C programs. Now, many programming languages have their own linting tools to check
for issues.
Example: For java the lint tool is CheckStyle, C & C++ Cppcheck
Unit Testing: This step runs individual pieces of code (called units) in isolation to ensure they
behave as expected. Unit testing helps catch bugs early by verifying that each part of the code
works correctly before combining them into a full system.
Artifact Generation: Once the code has passed through compilation, linting, and testing, it is
packaged into deployable units called artifacts. These could be binary files, libraries, or Docker
containers, depending on how the application will be deployed.
Build systems automate the process of compiling code, running tests, and generating executable
files or deployable artifacts. Depending on the programming language and the complexity of the
software, different build tools are preferred:
Java: Uses tools like Maven, Gradle, and Ant.
C/C++: Uses tools like Make.
JavaScript: Uses tools like Grunt.
Scala: Uses sbt.
Ruby: Uses Rake.
Python:Wheel
Each of these tools helps automate tasks such as compiling, packaging, and testing code. For
example, Maven is a popular build tool for Java because it helps manage dependencies and
define project configurations in a declarative way (using XML).
Sometimes an organization might use more than one programming language (like Java, C++, or
JavaScript), and each language might have its own build system. Sometimes, this can lead to
complications because different teams might use different build tools, which could create
confusion or inefficiencies when trying to standardize builds.
Despite the variety of build systems, organizations often try to standardize builds so that
developers can easily check out the code and build it without hassle. However, this can be tricky,
especially when working with multiple languages and systems.
Example:
A company building a Java desktop application uses Maven to compile the Java code and
package it.
Once the Java part is done, Maven can automatically call another tool, like NSIS (a
Windows installer tool), to package the Java application into an executable installer for
Windows.
The command mvn clean install is an example of a Maven command that cleans previous
build outputs and installs the new build.
The Jenkins build server
The Jenkins build server is a crucial part of any Continuous Integration (CI) and Continuous
Delivery (CD) pipeline. It automates the process of building, testing, and deploying code,
allowing teams to work more efficiently.
What is the Jenkins Build Server?
The Jenkins build server is a software tool that acts as a central hub for automating various
stages of software development. It takes source code from version control (like Git), compiles it,
runs tests, and deploys the software, all without manual intervention. This automation improves
code quality and speeds up the software development process.
Core Features of the Jenkins Build Server:
Continuous Integration: Automatically triggers builds when changes are made in the code
repository (e.g., after a git push), ensuring that code is regularly compiled, tested, and validated.
Continuous Delivery: Automates the deployment process, pushing code to different
environments (development, staging, production) once it passes certain tests.
Extensibility with Plugins: Jenkins supports over 1,800 plugins, which allow it to integrate with
various tools (like Docker, Maven, GitHub, Kubernetes) to handle different parts of the
development lifecycle.
How Jenkins Build Server Works:
Step 1: Source Code Management Integration (Git)
Jenkins integrates with version control systems like Git, GitHub, or Bitbucket.
Jenkins is configured to monitor a repository for changes. Every time code is pushed,
Jenkins automatically detects the change.
Step 2: Building the Code
Jenkins then triggers a build. For example, in the case of a Java project:
Jenkins can use Maven to compile the Java source code into bytecode and package it into a
deployable .jar file.
If it’s a [Link] project, Jenkins can use npm to install dependencies and bundle the code.
Step 3: Running Tests
Jenkins automatically runs tests to ensure that the code works as expected. These tests could
include:
Unit tests: Testing individual components of the code.
Integration tests: Ensuring that different parts of the system work well together.
Automated UI tests: Using tools like Selenium for frontend [Link] tests fail, Jenkins can halt
the process and notify the team.
Step 4: Deploying the Application
After a successful build and passing tests, Jenkins can automatically deploy the application to:
A development environment for further testing.
A staging server for testing in an environment similar to production.
The production environment itself, if everything is in order.
Jenkins can also trigger Docker or Kubernetes for containerized deployments.
4. Pipeline as Code in Jenkins
Jenkins allows the entire build process to be defined as code using Jenkins Pipelines. Pipelines
are written in a Groovy-based domain-specific language (DSL), and you can define the steps of
the build in a Jenkinsfile.
Managing Build Dependencies:
When you build software, your program often depends on external libraries, frameworks, or
other tools to function properly. These external elements are called dependencies, and managing
them efficiently is critical, especially in CI systems like Jenkins.
Jenkins plugins are like add-ons or extensions that enhance Jenkins's functionality, allowing you
to customize it according to your project's needs. Plugins can provide support for various tools,
integrations, and features, enabling Jenkins to work with a wide range of technologies and
workflows.
Integrate Jenkins with other tools such as version control systems (Git, GitHub), build tools
(Maven, Gradle), testing tools (Selenium), and deployment platforms (Kubernetes, Docker).
You can search for, install, and manage plugins directly from the Manage Jenkins > Plugin
Manager section.
Most plugins can be installed and used immediately without needing to restart Jenkins, which
reduces downtime.
Example: Installing a Plugin
1. Go to Manage Jenkins → Plugin Manager.
2. Click the Available tab to browse available plugins.
3. Search for the desired plugin (e.g., Git Plugin).
4. Select it and click Install without restart.
5. Once installed, it will be ready for use.
file system layout
In Jenkins, jobs are stored in the filesystem under directories like /var/lib/jenkins/jobs, with each
job having its own folder containing its configuration ([Link]) and build workspace. Backing
up these XML files is essential for restoring jobs in case of server failure, and plugins like
ThinBackup can automate this process. As builds can consume significant disk space, it's
recommended to configure Jenkins to retain only a specific number of builds or use tools to clean
up old builds automatically. Manual cleanup might be needed in emergencies. Additionally,
debugging failed builds often requires checking the workspace or dependencies, especially for
tools like Maven, where corrupted local repositories can cause issues.
The host server
The build server plays a crucial role in software development, serving as the central hub where
code from various developers integrates for the first time. Given that building software is
resource-intensive—requiring significant processing power, ample memory (RAM), and
sufficient disk space—it's vital to invest in a high-performance server. More processor cores
enable faster, parallel processing, while adequate RAM ensures smooth operation of multiple
tasks. A well-equipped build server can significantly speed up the development cycle, providing
quicker feedback and enhancing team productivity. Therefore, organizations should prioritize
robust specifications for their build server, as this investment can lead to long-term efficiency
gains, outweighing the initial costs.
Build Slaves (Agents) in Jenkins
Build slaves (also referred to as agents) are an essential part of Jenkins' architecture, designed to
offload tasks from the main Jenkins server, known as the master. The master is responsible for
managing and scheduling Jenkins jobs, while the slaves actually execute the jobs. This
separation of duties helps scale Jenkins by distributing the workload, ensuring that build and test
processes do not overwhelm the master server and are instead distributed across multiple
machines.
In a Jenkins setup, the master node acts as the central control point, managing the queue of tasks
and assigning them to available slave nodes. When a job is triggered, the master node checks the
available agents, assigns the job to one, and the slave executes the task. Once the job is
completed, the slave sends the results back to the master.
1. Scalability: With the ability to add multiple slaves, Jenkins can scale horizontally to
handle an increasing number of jobs, making it ideal for large projects and teams.
2. Parallel Execution: Multiple slaves allow Jenkins to run multiple builds in parallel,
reducing the overall time required for continuous integration and deployment (CI/CD).
3. Resource Allocation: Slaves can be allocated for specific tasks depending on the
hardware or software requirements. For instance, a build slave with more CPU power can
handle complex builds, while another can be dedicated to testing.
4. Isolation: Each slave can be configured to use different environments, ensuring that
builds and tests are conducted in the appropriate configuration, such as different OS
platforms, versions of dependencies, or software tools.
5. Reduced Load on Master: Offloading builds and tests to slaves prevents the master
server from becoming a bottleneck, which improves Jenkins’ overall performance and
reliability.
Types of Build Slaves
Jenkins acts as an orchestrator for automating build processes but doesn't perform the actual
builds itself. Instead, it delegates this task to specialized build tools like Maven, Gradle, Make, or
npm, which must be installed on the server where Jenkins is running. For example, if you're
building a Java project, Jenkins will trigger Maven commands, but Maven handles tasks like
compiling the code, resolving dependencies, and packaging the application. This approach
allows Jenkins to remain lightweight and versatile, relying on these external tools optimized for
specific programming languages or build systems. Therefore, the host system must have all
necessary build tools and dependencies installed for Jenkins to execute builds successfully.
Triggers
In Jenkins, triggers are used to automatically start builds based on certain events. There are
several ways to trigger builds:
1. Polling the Code Repository: Jenkins can check for changes in the code repository (et)
at regular intervals. If there are any changes, it triggers a build.
2. Timer Triggers: You can set builds to run on a schedule, like nightly builds. These builds are
typically more resource-intensive and can be run during off-hours when nobody is working.
3. Upstream/Downstream Triggers: One build (upstream) can trigger another (downstream)
ance it successfully completes. This is useful for creating dependent build pipelines.
Job chaining and build pipelines
Job chaining in Jenkins allows you to trigger one job after another. When an upstream job
finishes successfully, it can trigger a downstream job. This simple chain is often enough for
many workflows. However, for more control and visualization of the build process, Jenkins
offers build pipelines. These pipelines provide a more structured way to manage complex job
sequences. The multijob plugin and the workflow plugin are two popular tools for creating
advanced pipelines. The workflow plugin, in particular, allows you to define pipelines using a
Groovy DSL, providing more flexibility and control over your build process.
Build servers and infrastructure as code
In Jenkins, the web interface (GUI) is commonly used to configure build jobs, making it easy to
set up and manage projects without needing deep technical knowledge. However, this approach
has some limitations. While Jenkins jobs are saved as configuration files (like [Link]), these
files are not usually edited directly. Instead, most changes are made through the GUI. This can
be efficient for simple projects but becomes difficult to manage as the number of jobs grows or if
the configuration becomes more complex.
The GUI lacks advanced programming features like inheritance and reusability, which are
common in code. For example, if you want to reuse a set of build steps across multiple jobs, it
can be challenging. As a result, managing large numbers of jobs or making widespread changes
across many configurations can become inefficient.
In contrast, tools like GitLab CI treat the build process as code from the beginning. With Gitlab
CI, the entire build pipeline is defined in a configuration file (.[Link]), which can be
version- controlled and managed like any other source code. This approach is more in line with
the Infrastructure as Code (laC) philosophy, making it easier to scale and automate workflows in
a more flexible and maintainable way. While Jenkins offers powerful features, using the GUI can
make it harder to manage and scale as projects become more complex.
Building by dependency order
In build tools, tasks often depend on other tasks, and the build process needs to follow a specific
order to ensure everything is built correctly. This is called building by dependency order. In tools
like Make, you explicitly define dependencies. For example, to build a out, you first need to
build b.o and co because they are dependencies. The rules look like this:
[Link]: b.o c.o
b.o:b.c
c.o:c.c
This means that b.o and c.o must be built before [Link].
In tools like Maven and Gradle, the build order is automatically determined based on the defined
dependencies between components. Maven, for instance, builds a dependency graph of your
project before starting the build process. Jenkins can visualize this build order for Maven
projects, showing a view called the reactor. However, Jenkins does not provide a similar
visualization for Make-style builds
Build phases
Maven is a widely used build tool that offers a significant advantage by standardizing the build
process. This standardization is especially beneficial for large organizations, as it eliminates the
need to create custom build standards, ensuring consistency across projects. Maven enforces a
predefined lifecycle consisting of phases such as compile, test, package, and deploy, making it
easier to maintain order and discipline in the build process. For instance, every Maven project
follows the same sequence, starting from compiling the code to running tests and, finally,
deploying the build artifact. This rigidity ensures uniformity but may feel restrictive to
developers accustomed to flexible tools like Ant, which allow more freedom to define custom
workflows.
While other tools, such as Ant or Make, can also implement these build phases, they do not
enforce the standard order, making it harder to maintain consistency across teams. For example,
with Ant, a developer might forget to run tests before deploying, potentially leading to errors in
production. In contrast, Maven requires the testing phase to complete successfully before moving
to packaging or deployment, reducing the chances of introducing bugs.
The testing phase in Maven is particularly critical for ensuring software quality. In a Continuous
Integration (CI) environment, the CI server uses automated testing during this phase to catch
errors early in the development lifecycle. For instance, if a developer submits code that breaks a
unit test, Maven will halt the build process, preventing the deployment of faulty code. This
automated enforcement of testing not only ensures better code quality but also helps maintain the
reliability of the CI pipeline.
Alternative build servers
Jenkins is a dominant tool in the build server space, but it's not the only option. Alternatives like
Travis CI (a hosted solution popular for open-source projects), Buildbot (Python-based and
highly customizable), GoCD (by ThoughtWorks), Bamboo (by Atlassian), and GitLab CI/CD
(integrated with GitLab) are also available. Each tool has its own strengths, so it's important to
evaluate them based on your project's needs. When choosing a build server, be cautious of
vendor lock-in, where a tool ties you to its ecosystem, making migration difficult. Remember, a
build server complements but doesn’t replace the need for code to build and work reliably on
developers' local machines. Additionally, prioritize tools that support configuration via files (like
YAML or JSON), as this makes automation and version control easier compared to tools that
rely solely on graphical interfaces, which developers and DevOps teams often find restrictive.
Collating quality measures
Collating software quality measures is a key function of a build server like Jenkins, as it helps
monitor and improve the quality of the codebase. Jenkins provides built-in support for executing
and visualizing Java unit tests. These test results can be viewed directly on the job page, giving
developers immediate feedback about the success or failure of their code changes and the health
of their application.
For more advanced quality analysis, tools like SonarQube (referred to as "Sonar") can be
integrated. SonarQube performs a variety of static code analysis checks during the build phase,
evaluating aspects like code complexity, duplication, adherence to coding standards, and
potential bugs. These results are sent to the Sonar server, where they are stored and presented
through detailed dashboards and reports. Developers can use these visualizations to track the
progress of their efforts to improve code quality, fix technical debt, and ensure best practices are
followed.
However, running SonarQube during every build can significantly slow down the process,
especially for large projects. To address this, a common recommendation is to integrate
SonarQube analysis into nightly builds, which are scheduled once a day when immediate
feedback is less critical. This approach balances the need for code quality insights with the
efficiency of regular builds, ensuring that developers can focus on speed during the day while
gathering detailed quality metrics overnight.
UNIT - V
Introduction:
Software: Software is a collection of computer programs that helps us to perform a task.
Software testing is the process of finding defects in a software.
The goal is to release quality software to the customer / client.
Quality:
1. Bug free
2. Within the budget
3. Deliver in time
4. Meets client expectations / requirements
5. Maintainable
Project
If a software application is developed for a specific customer based on the requirements then it is
called Project.
Product:
If a software application is developed for multiple customers based on the market requirements
then it is called a product.
Error:
An error is a mistake or flaw in the code or logic of a program made by a developer during the
software development process.
Example:
A developer mistakenly writes = (assignment) instead of == (comparison) in a conditional
statement, causing incorrect results.
Bug:
A bug is an issue in the software application caused by an error in the code, leading to
unexpected or incorrect behavior during execution. Detected during testing or use.
Example:
A shopping cart application adds the wrong item to the cart due to incorrect logic.
Failure:
A failure occurs when the software does not perform as expected or does not meet the specified
requirements during execution. Detected during runtime (when the application is being used).
Example:
A payment gateway crashes while processing a transaction, preventing the user from completing
the payment.
Static testing:
The documentation is tested.
Done during verification process i.e before development
As we know 85% errors found in design phase
Dynamic testing:
Dynamic testing is the process of validating the software by executing its code. It checks the
system's behavior under different conditions to ensure it works as intended.
Verification:
Verification is the process of ensuring that the product is being built correctly according to the
specified requirements and design.
Reviewing a requirement document to ensure all customer needs are addressed.
Verification → done before development → check we are doing correct?
Validation:
Validation is the process of ensuring that the software product meets the user's needs and
expectations by testing its functionality during or after execution.
Validation → Actually testing software → done after software ready → done Right?
Whitebox testing:
White box testing, also known as clear box testing, involves testing the internal structure, logic, and
implementation of the software. Performed by Developers.
Unit testing and integration testing comes under this.
Black box testing:
Black box testing focuses on testing the functionality of the application without any knowledge of its
internal code or logic. The tester validates input and output based on the requirements.
Testers do not need to know the source code or implementation details.
Testers or end-users.
System testing and User Acceptance testing comes under this.
Unit Testing :- in unit testing individual components of software tested.
The purpose of this testing is that each module is working properly
It focuses on the smallest unit of software design (done by developer by using sample input and
observing its sample output)
Eg. In a program we are checking if loop, method or function is working fine
Integration Testing: in integration Testing individual units are combined and tested as group
(developer)
System Testing : in this testing we can test the whole application (complete / integrated
software is tested) done by tester.
Acceptance Testing : a level of software Testing in which software is tested for
user acceptance UAT done at client location where software is actually used
→ Alpha Testing : done by tester in company in presence of customer
→ Beta Testing : done by customer to check software is ok , satisfy requirement
Functional testing
is the process through which QAs determine if a piece of software is acting in accordance with
predetermined requirements. It uses black-box testing techniques, in which the tester has no
knowledge of the internal system logic
Ensures that the software's features work as expected.
Testing what the system does.
Based on business requirements and use cases.
Verifies specific actions and functionalities like data input, processing, and output.
Example:
Testing a login form to ensure:
It accepts valid credentials.
It rejects invalid credentials.
A password reset link works
Non Functional testing:
Non-functional testing evaluates the software's performance, usability, reliability, and other
quality attributes that do not relate directly to specific functionalities.
Key Features:
Retesting focuses on verifying defect fixes, while regression testing ensures that the
application remains stable after updates. Both are essential to maintaining the quality and
reliability of software systems.
2. Automated
Performing the testing with the help of tools is known as Automation testing.
Automation of testing Pros and cons
Selenium Features:
❖ Selenium is an open source and portable Web testing Framework.
❖ Selenium IDE provides a playback and record feature for authoring tests without the need
to learn a test scripting language.
❖ It can be considered as the leading cloud-based testing platform which helps testers to
record their actions and export them as a reusable script with a simple-to-understand and
easy-to-use interface.
❖ Selenium supports various operating systems, browsers and programming languages.
Following is the list:
➢ Programming Languages: C#, Java, Python, PHP, Ruby, Perl, and JavaScript
➢ Operating Systems: Android, iOS, Windows, Linux, Mac, Solaris.
➢ Browsers: Google Chrome, Mozilla Firefox, Internet Explorer, Edge, Opera,
Safari, etc.
❖ It also supports parallel test execution which reduces time and increases the efficiency of
tests.
❖ Selenium can be integrated with frameworks like Ant and Maven for source code
compilation.
❖ Selenium can also be integrated with testing frameworks like TestNG for application
testing and generating reports.
❖ Selenium requires fewer resources as compared to other automation test tools.
❖ WebDriver API has been indulged in selenium whichis one of the most important
modifications done to selenium.
❖ Selenium web driver does not require server installation, test scripts interact directly with
the browser.
❖ Selenium commands are categorized in terms of different classes which make it easier to
understand and implement.
❖ Selenium Remote Control (RC) in conjunction with WebDriver API is known as
Selenium 2.0. This version was built to support the vibrant web pages and Ajax.
Testing backend integration points
Backend integration testing focuses on verifying the functionality of backend systems, such as
SOAP and REST APIs, which act as communication points between software systems. Testing
these APIs ensures that the systems interact correctly and reliably. One significant advantage of
testing backend systems is that it is often cost-effective compared to testing graphical user
interfaces (GUIs). Since backend systems tend to remain stable over time, the tests written for
them require less maintenance.
Tools like soapUI make writing and executing tests for backend systems straightforward. Testers
can create and run these tests interactively, while developers and DevOps teams can integrate
them into their build processes using command-line tools or Maven plugins. This integration is
particularly beneficial in a Continuous Integration (CI) environment, where automation plays a
crucial role in ensuring smooth and reliable builds.
SoapUI is versatile and caters to various roles. Testers benefit from its user-friendly interface,
which allows them to incrementally build and run test cases in a structured manner. Developers,
on the other hand, can use the tool’s command-line features to include tests in their builds
without interacting with the GUI. DevOps engineers managing build servers also find these
features useful, as they enable seamless automation.
The tool’s open-source licensing makes it accessible and reduces the stress of managing licenses,
a common issue with proprietary tools. The open-source version is reliable, with the option to
access additional features in a paid version. This reliability is crucial in DevOps, where
unexpected license expirations or floating license limitations can disrupt workflows.
SoapUI has a simple and intuitive interface. On the left, a tree view displays test cases and test
suites, allowing users to select and run individual tests or entire suites. Test results are shown on
the right side of the interface. Additionally, test cases in soapUI are stored as XML files, which
can be version-controlled in repositories like Git. This feature allows teams to manage tests like
code, enabling easy editing when updates, such as a global search and replace for a renamed
identifier, are needed.
While soapUI has some limitations, its flexibility and usefulness make it a popular choice for
testing backend systems. It supports the essential practices of DevOps by enabling efficient,
reliable, and maintainable testing workflows.
Test-driven development
Test-Driven Development (TDD) is a way of developing software where tests are written before
the actual code. It became popular in the 1990s, thanks to a programming approach called
Extreme Programming. TDD works in a cycle with the following steps:
REPL-driven development is a way of writing and testing code that works well with interpreted
programming languages like Python, JavaScript, Ruby, and Lisp. It revolves around using a tool
called a REPL (Read-Eval-Print Loop), which allows you to write, execute, and test code
interactively, one small piece at a time.
Step by step explanation:
1. Write Small, Independent Functions:
You break your code into small, simple functions that don't depend on other parts of the
program or a global state. This makes them easier to test and understand.
2. Test Functions as You Write Them:
In a REPL, you can immediately test each function after writing it. For example, if you
write a function to add two numbers, you can run it right away in the REPL to make sure
it works.
3. Focus on Simplicity and Clarity:
Unlike TDD (Test-Driven Development), where you write tests first, REPL-driven
development focuses on writing straightforward, side-effect-free code first. This makes
your code easier to read and maintain.
4. Combine with Unit Testing:
You can also use REPL-driven development to create and test unit tests interactively.
This means you can combine this approach with traditional testing methods for even
better results.
Virtualization stacks
Puppet is a configuration management tool that helps you manage servers in a systematic way.
Instead of manually configuring servers one by one, Puppet automates this process, ensuring that
all your servers are consistent and follow the rules you define. Think of it like a script that tells
your servers how to behave and what software or settings they should have.
Example:
Imagine you have 100 servers, and you want all of them to:
1. Install a web server (like Apache or Nginx).
2. Ensure the same version of software is running everywhere.
3. Apply security settings like closing unused ports.
Instead of logging into each server and doing this manually, you write instructions (called
manifests) in Puppet, and Puppet applies these instructions to all your servers automatically.
Components:
1. Puppet Master: This is the brain of Puppet. It’s a central server that stores all the
configuration details. It decides what each machine (node) should look like.
2. Puppet Agent: This is software that runs on each server (node). It connects to the Puppet
Master to fetch its configuration and applies it to the server.
3. Manifest: This is a file where you write the instructions for how a server should be
configured. It’s written in a language called Puppet DSL (Domain Specific Language),
which is easy to learn.
4. Puppet Catalogue: When a Puppet agent connects to the Puppet Master, the Master
sends it a catalogue—a detailed plan of what changes or settings should be applied to that
specific server.
Example
Define the Configuration: You write a manifest that describes how your servers should be
configured.
For example:
package { 'nginx':
ensure => installed,
}
service { 'nginx':
ensure => running,
}
This tells Puppet to install and start Nginx on your servers.
Puppet Agent Checks In: Every Puppet agent (on a server) periodically contacts the Puppet
Master to check for updates.
Configuration Applied: The Puppet agent applies the instructions sent by the Puppet Master. If
the server already matches the desired configuration, nothing changes. If something is different,
Puppet fixes it.
Repeat: Puppet regularly checks to ensure all servers remain in the desired state. If someone
manually changes something, Puppet will automatically fix it during the next check-in.
Advantages:
Automation: No need to configure servers manually.
Consistency: All servers follow the same configuration rules.
Time-Saving: Makes managing large numbers of servers fast and efficient.
Ansible
Ansible is a tool that helps you automate repetitive tasks on your computers or servers. Imagine
you need to install software, update settings, or restart services on many machines instead of
doing it manually on each one, Ansible does it for you, all at once.
Components:
1. Control Machine: You install Ansible on one computer (called the control machine).
2. Managed Machines: These are the computers or servers you want to control. Ansible
connects to them using SSH (for Linux) or WinRM (for Windows), so no special
software is needed on those machines.
3. Inventory File: You make a list of the computers you want to manage in a simple file,
called the inventory.
4. Playbooks: You write instructions in an easy-to-read format (YAML files) telling
Ansible what to do — like install software or copy files.
Capabilities:
Install software (e.g., web servers like Nginx or databases like MySQL).
Update settings (e.g., change configuration files on your servers).
Run commands (e.g., check the status of all servers).
Deploy applications (e.g., set up a website on multiple servers).
Advantages:
No Extra Software: You don’t need to install anything on the servers you’re managing.
Easy to Understand: Ansible uses plain text and simple instructions.
Save Time: You can automate tasks on hundreds of servers with a single command.
Example: Installing Nginx with Ansible
Inventory File: List your servers in a file:
[Link]
[web_servers]
[Link]
[Link]
Playbook File: Write what Ansible should do
- name: Install Nginx
hosts: web_servers
become: true
tasks:
- name: Install Nginx
apt:
name: nginx
state: present
Run the Playbook: Use one command to install Nginx on all server
ansible-playbook -i inventory install_nginx.yml
Chef
Chef is another powerful tool used for automating server management and configuration. It
helps you manage infrastructure, ensuring that your systems are configured correctly and
consistently. It works by defining the desired state of your systems and automating the process of
bringing them to that state.
Components:
1. Chef Server: This is the central hub where all the configurations (recipes, cookbooks)
are stored. It communicates with the Chef Clients running on your managed servers.
2. Chef Clients: These are the servers or machines you manage using Chef. They contact
the Chef Server regularly to check if any updates or changes are needed.
3. Cookbooks and Recipes: A cookbook is a collection of recipes (instructions) that define
how software should be installed, configured, and managed on your servers.
4. Node: A node is simply a server or machine being managed by Chef.
Features:
Functionality
1. Write Recipes: You define a “recipe” that describes the exact configuration you want on
your servers, such as installing a web server or setting up a database.
2. Chef Server: You upload these recipes to the Chef Server.
3. Chef Client: Each server (Chef Client) fetches the recipes and applies them to ensure the
system matches the desired configuration.
Advantages:
Consistency: Chef ensures that all servers are set up exactly the same, making it easier to
scale and maintain systems.
Flexibility: Chef is very flexible and can handle complex configurations.
Integration: Chef integrates well with other DevOps tools and platforms.
Salt Stack
Salt Master:
The central control server where configurations are defined and commands are issued.
Salt Minions:
The nodes (servers or devices) that are managed by the Salt Master.
Minions receive instructions from the Master and execute them.
States:
YAML files describing the desired state of systems. For example
apache2:
[Link]: []
[Link]:
- require:
- pkg: apache2
Ensures Apache2 is installed and running.
Pillars:
Secure data storage for sensitive information like passwords or API keys.
Grains:
Static information about a system, such as its OS, IP address, or hostname.
Salt Modules:
Predefined functions to execute tasks, such as managing files, users, or packages.
Benefits:
1. Speed: Real-time communication makes SaltStack one of the fastest tools for executing
commands and applying configurations.
2. Scalability: Ideal for managing thousands of nodes.
3. Flexibility: Can be used for configuration management, remote execution, and
orchestration.
4. Community and Enterprise Support: Offers both open-source and enterprise versions
for added features like UI dashboards and advanced security.