AI for Cyber Security

M5010018/M5020018

Lecture 14

School of Computer Science and Engineering

(SoCSE)
Digital University Kerala ,Thiruvananthapuram
Neural Networks (NN)
Why Neural Networks?

• Traditional machine learning models (like linear regression, decision

trees, SVMs) work well for small datasets and manually engineered
features.
• But for complex problems like : Image recognition (cats vs dogs),
Speech recognition, Natural language understanding, Autonomous
driving
• We need models that can automatically learn features from raw
data.
• Neural Networks are powerful function approximators — they can
learn non-linear mappings between inputs and outputs.
Biological Inspiration

Neural Networks are inspired by the structure of the human

brain:
• The brain is made up of billions of neurons interconnected via
synapses.
• Each neuron receives input signals, processes them, and passes
output to other neurons.
• Learning happens by strengthening or weakening connections
(analogous to adjusting weights in a neural network).
Biological Neuron vs Artificial Neuron
Biological
Artificial Neuron
Neuron

Dendrites Inputs (features)

Synapse strength Weights

Soma (cell body) Summation + Activation function

Axon Output
What are Neural Networks?

• Neural networks are set of algorithms inspired by the

functioning of human brain.
• Generally, when you open your eyes, what you see is called
data and is processed by the Neurons(data processing cells) in
your brain, and recognizes what is around you.
• That’s how similar the Neural Networks works.
• They takes a large set of data, process the data(draws out the
patterns from data), and outputs what it is.
What they do?
• Neural networks sometimes called as Artificial Neural networks
(ANN’s), because they are not natural like neurons in your brain.
• They artificially mimic the nature and functioning of Neural network.
• ANN’s are composed of a large number of highly interconnected
processing elements (neurons) working in unison to solve specific
problems.
• ANNs, like people, like child, they even learn by example.
• An ANN is configured for a specific application, such as pattern
recognition or data classification, Image recognition, voice recognition
through a learning process.
• Neural networks (NN) are universal function approximators so that means
neural networks can learn an approximation of any function f(x) such that.
y=f(x)
NN trying to predict the image(data) that
given to it. it predicts that the no is 2 here
Why use Neural networks?

• Neural networks, with their remarkable ability to derive meaning from

complicated or imprecise data, can be used to extract patterns and
detect trends that are too complex to be noticed by either humans or
other computer techniques.
• A trained neural network can be thought of as an “expert” in the
category of information it has been given to analyse.
• This expert can then be used to provide projections given new
situations of interest and answer “what if” questions.
Other advantages include:
• Adaptive learning: An ability to learn how to do tasks based on the
data given for training or initial experience.
• Self- Organization : An ANN can create its own organization or
representation of the information it receives during learning time.
Artificial Neuron

• An artificial neuron takes multiple inputs, applies weights

and bias, and outputs an activation value.
• Mathematical model:
Artificial Neuron
Layers in Neural Network Architecture
Contd…

• Input Layer: This is where the network receives its input

data. Each input neuron in the layer corresponds to a
feature in the input data.
• Hidden Layers: These layers perform most of the
computational heavy lifting. A neural network can have
one or multiple hidden layers. Each layer consists of units
(neurons) that transform the inputs into something that the
output layer can use.
• Output Layer: The final layer produces the output of the
model. The format of these outputs varies depending on
the specific task like classification, regression.
Working of Neural Networks

1. Forward Propagation
• When data is input into the network, it passes through the
network in the forward direction, from the input layer through
the hidden layers to the output layer. This process is known as
forward propagation. Here’s what happens during this phase:
• Linear Transformation: Each neuron in a layer receives
inputs which are multiplied by the weights associated with the
connections. These products are summed together and a bias is
added to the sum. This can be represented mathematically as:
Where
• represents the weights

• represents the inputs

• is the bias
Contd…

• Activation: The result of the linear transformation

(denoted as ) is then passed through an activation function.
• The activation function is crucial because it introduces non-
linearity into the system, enabling the network to learn more
complex patterns.
• Popular activation functions include ReLU, sigmoid ,
softmax and tanh.
Common Activation Functions
Back Propagation

• After forward propagation, the network evaluates its

performance using a loss function which measures the
difference between the actual output and the predicted output.
• The goal of training is to minimize this loss.
• This is where backpropagation comes into play:
Steps
• Loss Calculation: The network calculates the loss which provides a measure of error in

the predictions. The loss function could vary; common choices are mean squared

error for regression tasks or cross-entropy loss for classification.

• Gradient Calculation: The network computes the gradients of the loss function with

respect to each weight and bias in the network. This involves applying the chain rule of

calculus to find out how much each part of the output error can be attributed to each

weight and bias.

• Weight Update: Once the gradients are calculated, the weights and biases are updated

using an optimization algorithm like stochastic gradient descent (SGD). The weights

are adjusted in the opposite direction of the gradient to minimize the loss. The size of

the step taken in each update is determined by the learning rate.

Iteration

• This process of forward propagation, loss calculation,

backpropagation and weight update is repeated for many iterations
over the dataset.
• Over time, this iterative process reduces the loss and the network's
predictions become more accurate.
• Through these steps, neural networks can adapt their parameters to
better approximate the relationships in the data, thereby improving
their performance on tasks such as classification, regression or any
other predictive modeling.
Example of Email Classification

• Let's consider a record of an email dataset:

• To classify this email, we will create a feature vector based on the analysis
of keywords such as "free" "win" and "offer"
• The feature vector of the record can be presented as:
• "free": Present (1)
• "win": Absent (0)
• "offer": Present (1)
How Neurons Process Data in a Neural Network

• In a neural network, input data is passed through multiple

layers, including one or more hidden layers.
• Each neuron in these hidden layers performs several
operations, transforming the input into a usable output.
• 1. Input Layer: The input layer contains 3 nodes that
indicates the presence of each keyword.
• 2. Hidden Layer: The input vector is passed through the
hidden layer. Each neuron in the hidden layer performs two
primary operations: a weighted sum followed by an activation
function.
Weights:
• Neuron H1: [0.5,−0.2,0.3]
• Neuron H2: [0.4,0.1,−0.5]
Input Vector: [1,0,1]
• Weighted Sum Calculation
• For H1: (1×0.5)+(0×−0.2)+(1×0.3)=0.5+0+0.3=0.8
• For H2: (1×0.4)+(0×0.1)+(1×−0.5)=0.4+0−0.5=−0.1
Activation Function
• Here we will use ReLU
• H1 Output: ReLU(0.8)= 0.8
• H2 Output: ReLu(-0.1) = 0
Output Layer

• The activated values from the hidden neurons are sent to the
output neuron where they are again processed using a
weighted sum and an activation function.
• Output Weights: [0.7, 0.2]
• Input from Hidden Layer: [0.8, 0]
• Weighted Sum: (0.8×0.7)+(0×0.2)=0.56+0=0.56
• Activation (Sigmoid):
Final Classification

• The output value of approximately 0.636 indicates the

probability of the email being spam.
• Since this value is greater than 0.5, the neural network
classifies the email as spam (1).
Learning of a Neural Network

Learning with Supervised Learning

• In supervised learning, a neural network learns from labeled input-output pairs provided
by a teacher. The network generates outputs based on inputs and by comparing these
outputs to the known desired outputs, an error signal is created. The network iteratively
adjusts its parameters to minimize errors until it reaches an acceptable performance level.
Learning with Unsupervised Learning
• Unsupervised learning involves data without labeled output variables. The primary goal
is to understand the underlying structure of the input data (X). Unlike supervised
learning, there is no instructor to guide the process. Instead, the focus is on modeling
data patterns and relationships, with techniques like clustering and association
commonly used.
Learning with Reinforcement Learning
• Reinforcement learning enables a neural network to learn through interaction with its
environment. The network receives feedback in the form of rewards or penalties, guiding
it to find an optimal policy or strategy that maximizes cumulative rewards over time.
This approach is widely used in applications like gaming and decision-making.