Low-Level Design (LLD) for Image

Forgery Detection and Explainability

Submitted by:
Aashish Sharma
Aayush Mishra
Abhiram Prasanna
Aditya Vikram Singh
Under the guidance of Dr. Surabhi Narayan

This document provides a detailed low-level design for the

implementation of the image forgery detection project. Each component
is explained in detail, followed by the corresponding implementation
details.

1. Dataset Preparation

Functions and Steps

1.1 Data Loading

● Purpose: Load the dataset into a structured format for processing.

● Implementation: The CASIA v2 dataset is organized into folders
(Au for authentic images and Tp for tampered images).
● Libraries Used: torchvision.datasets.ImageFolder.
● Output: A dataset object that provides image-label pairs.

1.2 Data Transformation

● Purpose: Normalize and resize images for compatibility with

MobileNetV2.
● Steps:
○ Resize images to 224x224 pixels.
○ Convert images to tensors.
○ Normalize using ImageNet mean and standard deviation
values.
 ● Libraries Used: torchvision.transforms.

1.3 Data Splitting

● Purpose: Split the dataset into training and testing sets for model
evaluation.
● Steps:
○ Use torch.utils.data.DataLoader to create separate
DataLoaders for training and testing.

2. Model Architecture
Functions and Steps

2.1 Base Model

● Purpose: Use a pre-trained MobileNetV2 model as the feature

extractor.
● Steps:
○ Load the MobileNetV2 architecture pre-trained on ImageNet.
○ Freeze feature extraction layers (optional for fine-tuning).

2.2 Custom Classifier Head

● Purpose: Replace the default classifier with a custom fully

connected layer for binary classification.
● Steps:
○ Add a linear layer with two output nodes for Authentic and
Tampered classes.

2.3 Weight Initialization

● Purpose: Initialize the weights of the custom classifier for stable

training.
● Method: Use PyTorch's default initialization.

3. Training Pipeline
Functions and Steps
3.1 Loss Function

● Purpose: Compute the error between predictions and actual

labels.
● Method: CrossEntropyLoss for multi-class classification.

3.2 Optimizer

● Purpose: Update model weights to minimize loss.

● Method: Adam optimizer with a learning rate of 0.001.

3.3 Training Loop

● Purpose: Train the model iteratively over the dataset.

● Steps:
1. Forward pass: Compute predictions.
2. Compute loss.
3. Backward pass: Calculate gradients.
4. Update weights using the optimizer.

3.4 Model Checkpoint Saving

● Purpose: Save the trained model's weights for reuse.

● Method: Use torch.save to store model state.

4. Evaluation
Functions and Steps

4.1 Accuracy Calculation

● Purpose: Measure the proportion of correct predictions.

● Method: Use sklearn.metrics.accuracy_score.

4.2 Precision, Recall, and F1 Score Calculation

● Purpose: Evaluate the model's performance on classifying

tampered and authentic images.
● Method: Use sklearn.metrics for these calculations.
4.3 Confusion Matrix

● Purpose: Display the distribution of predictions across classes.

● Method: Use sklearn.metrics.confusion_matrix.

4.4 Classification Report

● Purpose: Provide a detailed summary of precision, recall, and F1

score for each class.
● Method: Use sklearn.metrics.classification_report.

5. Explainability
Functions and Steps

5.1 LIME Explanation

● Purpose: Provide interpretable visual explanations for the model's

predictions.
● Steps:
1. Divide the input image into superpixels using SLIC.
2. Perturb superpixels to generate multiple versions of the
image.
3. Compute the contribution of each superpixel to the final
prediction.

5.2 Visualization

● Purpose: Highlight important regions of the image that influenced

the model's decision.
● Method: Use matplotlib for displaying superpixel-based
explanations.
6. Testing
Functions and Steps

6.1 Test Image Preprocessing

● Purpose: Prepare the test image for the model.

● Steps: Resize, normalize, and convert to a tensor.

6.2 Model Prediction

● Purpose: Predict whether the test image is tampered or authentic.

● Method: Perform a forward pass through the trained model.

6.3 Explanation on Test Image

● Purpose: Use LIME to explain the prediction on the test image.

● Output: Visualize important regions contributing to the
classification.
Implementation Details

1. Dataset Preparation

1.1 Data Loading

● Implementation Details:
○ The ImageFolder class from
torchvision.datasets is used to load images from
the CASIA2 dataset.
○ The dataset is structured into two folders: Au for
authentic images and Tp for tampered images.
○ The root parameter points to the dataset directory on
Google Drive, mounted via drive.mount.

from google.colab import drive

import os

# Mount Google Drive

drive.mount('/content/drive')

# Define dataset paths

base_dir = '/content/drive/MyDrive/CASIA2'
au_dir = os.path.join(base_dir, 'Au')
tp_dir = os.path.join(base_dir, 'Tp')

1.2 Data Transformation

● Implementation Details:
○ Images are resized to 224x224 pixels to match
MobileNetV2 input requirements.
○ Pixel values are normalized to align with ImageNet’s
mean and standard deviation.
from torchvision import transforms

transform = transforms.Compose([
transforms.Resize((224, 224)),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229,
0.224, 0.225])
])

1.3 Data Splitting

● Implementation Details:
○ The entire dataset is loaded as a single dataset using
ImageFolder.
○ A DataLoader is used to batch and shuffle the dataset.

from torchvision import datasets

from torch.utils.data import DataLoader

dataset_path = '/content/drive/MyDrive/CASIA2'
train_dataset = datasets.ImageFolder(root=dataset_path,
transform=transform)
train_loader = DataLoader(train_dataset, batch_size=32,
shuffle=True)

# Print class indices

print("Classes in the dataset:", train_dataset.class_to_idx)

2. Model Architecture

2.1 Base Model: MobileNetV2

● Implementation Details:
 ○ A pre-trained MobileNetV2 model is loaded using
torchvision.models.
○ Pretrained weights are loaded to benefit from ImageNet
training.

from torchvision import models

class ForgeryDetectionMobileNet(nn.Module):
def __init__(self, num_classes=2):
super(ForgeryDetectionMobileNet, self).__init__()
self.model = models.mobilenet_v2(pretrained=True)
self.model.classifier = nn.Sequential(
nn.Linear(self.model.last_channel, num_classes)
)

def forward(self, x):

return self.model(x)

# Instantiate the model

device = torch.device("cuda" if torch.cuda.is_available() else
"cpu")
model =
ForgeryDetectionMobileNet(num_classes=2).to(device)

2.2 Custom Classifier Head

● Implementation Details:
○ The original classification head of MobileNetV2 is
replaced with a fully connected layer.
○ The new layer maps to two output classes
(authentic/tampered).
3. Training Pipeline

3.1 Loss Function

● Implementation Details:
○ CrossEntropyLoss is used to compute the classification
loss.

criterion = nn.CrossEntropyLoss()

3.2 Optimizer

● Implementation Details:
○ Adam optimizer is used with a learning rate of 0.001.

optimizer = optim.Adam(model.parameters(), lr=0.001)

3.3 Training Loop

● Implementation Details:
○ The model is trained for 5 epochs.
○ Each iteration involves a forward pass, loss computation,
backward pass, and parameter update.

def train(model, criterion, optimizer, loader, epochs=5):

model.train()
for epoch in range(epochs):
running_loss = 0.0
for images, labels in loader:
images, labels = images.to(device), labels.to(device)

optimizer.zero_grad()
outputs = model(images)
loss = criterion(outputs, labels)
loss.backward()
optimizer.step()
 running_loss += loss.item()
print(f"Epoch [{epoch+1}/{epochs}], Loss:
{running_loss/len(loader):.4f}")

train(model, criterion, optimizer, train_loader)

3.4 Model Checkpoint Saving

● Implementation Details:
○ Model weights are saved using torch.save.

torch.save(model.state_dict(),
'/content/drive/MyDrive/casia2_mobilenet_model.pth')

4. Evaluation

4.1 Metrics Calculation

● Implementation Details:
○ Accuracy, precision, recall, F1-score, and confusion
matrix are calculated using scikit-learn metrics.

from sklearn.metrics import accuracy_score, precision_score,

recall_score, f1_score, confusion_matrix, classification_report

def evaluate_metrics(model, loader, class_names):

model.eval()
all_preds, all_labels = [], []

with torch.no_grad():
for images, labels in loader:
images, labels = images.to(device), labels.to(device)
 outputs = model(images)
_, preds = torch.max(outputs, 1)
all_preds.extend(preds.cpu().numpy())
all_labels.extend(labels.cpu().numpy())

print("Confusion Matrix:")
print(confusion_matrix(all_labels, all_preds))
print("Classification Report:")
print(classification_report(all_labels, all_preds,
target_names=class_names))

evaluate_metrics(model, train_loader,
list(train_dataset.class_to_idx.keys()))

5. Explainability

5.1 Explanation Generation

● Implementation Details:
○ LIME is used to perturb input images and explain
predictions.

from lime import lime_image

from skimage.segmentation import slic

def explain_with_lime(model, image_tensor):

def transform_to_numpy(tensor):
img = tensor.squeeze().detach().cpu().numpy()
img = np.transpose(img, (1, 2, 0))
return img

explainer = lime_image.LimeImageExplainer()
 explanation = explainer.explain_instance(
transform_to_numpy(image_tensor),
predict_fn=lambda x:
model(torch.stack([torch.Tensor(i).permute(2, 0, 1) for i in
x]).to(device)).cpu().detach().numpy(),
segmentation_fn=lambda x: slic(x, n_segments=100,
compactness=10, sigma=1)
)
return explanation

5.2 Visualization

● Implementation Details:
○ The top 5 influential segments are visualized using
Matplotlib.

def visualize_explanation(explanation):
temp, mask = explanation.get_image_and_mask(
explanation.top_labels[0],
positive_only=False,
num_features=5
)
plt.imshow(temp)
plt.title("LIME Explanation")
plt.axis('off')
plt.show()

6. Testing

6.1 Test Image Preprocessing

● Implementation Details:
○ Images are resized, normalized, and converted into
tensors for inference.
test_transform = transforms.Compose([
transforms.Resize((224, 224)),
transforms.ToTensor(),
])

def preprocess_test_image(image_path):
from PIL import Image
image = Image.open(image_path).convert('RGB')
image_tensor =
test_transform(image).unsqueeze(0).to(device)
return image_tensor

6.2 Model Prediction

● Implementation Details:
○ The model outputs probabilities for each class, which are
converted into predicted labels.