DENT AND SCRATCH DETECTION USING

COMPUTER VISION
Goutham S1, Dhanushya D2, Manu Raju3
1,3
Electronics and communication Enginerring,2 Information Technology
Bannari Amman Institute of Technology, India
1
goutham.ec20@bitsathy.ac.in
2
dhanushya.it20@bitsathy.ac.in
3
manuraju@bitsathy.ac.in

Abstract— This paper gives insights on identification and feature 4. Serious dents could damage the car's structural
extraction of vehicle damage caused by numerous obstacles and integrity, which could jeopardise its safety in an
other impending cars. In the past, automotive specialists mostly accident.
used physical examinations to find automobile defects and
scratches. This procedure entailed visually inspecting the whole
vehicle's surface under lighting conditions to find flaws. These II. LITERATURE REVIEW
systems can analyse enormous volumes of data quickly and find In order to identify and discriminate faults, defect
flaws with higher precision and consistency by utilising the power detection on vehicle bodies often uses an inspection program
of computer vision. We used Yolov8 architecture to identify the flow that can execute image acquisition, and merging by
dents and scratches of vehicles and achieved an mean average gathering pictures of car and their structures, image
precision of 0.579 with threshold value of 0.5.
processing, and classification. In order to find surface flaws,
many vehicle inspection methods and methodologies have
Keywords—Car dent, scratch, Yolo v8, vehicle damage, been developed.
inspection Phyu and Kuntpong examined several deep learning
methods and used transfer learning in pre-trained VGG
I. INTRODUCTION
models, to identify the damaged section of an automobile and
Common occurrences like car dents and scratches can determine its position and severity [2]. With the help of
change how a car looks and lower its resale value. These flaws transfer learning and L2 regularisation, they were able to
may be annoying and expensive for automobile owners to fix, achieve accuracy of 95.22% of VGG19 and 94.56% of
whether it be a little parking lot accident or a more serious VGG16 in damaged detection, 76.48% of VGG19 and
collision. In comparison to manual inspection techniques, 74.39% of VGG16 in damage localisation, and 58.48% of
automated technologies can analyze a car's exterior in just a VGG19 and 54.8% of VGG16 in damage severity. They
couple of seconds. Computer vision algorithms remove human concluded that VGG19 performed better than VGG16. Their
subjectivity, producing findings that are more trustworthy and models addressed overfitting issues.
consistent when spotting flaws [1]. Early discovery of dents Najmeddine et al., examination of the industry. They
and scratches allows for early repair, stopping small concerns suggested a technique that employed fully connected neural
from growing into bigger, more expensive difficulties. A more networks after an Inception-Resnet pre-trained model as a
unbiased and open inspection method benefits car owners by features extractor to identify and categorise the damages. The
increasing their trust in and contentment with automotive damage site was marked with a mask R-CNN. They
service providers. Fortunately, technological advancements concluded that the suggested pre-trained model, which has a
have sparked the creation of creative approaches for precisely precision of 86.8% as opposed to 75.4%, is more effective in
and effectively identifying and analysing automotive dents and localising damage. In comparison to the 80% accuracy of the
scratches. Car dents are depressions or deformities in a car's Inception-ResnetV2 model, they discovered that the VGG16
bodywork brought on by collisions with other cars, objects, model performs worse, with just 69% precision in
shopping carts, hail, or other moving items. On the other hand, determining the severity of damage. So, using the Inception-
scratches are superficial scrapes or abrasions in the paintwork ResnetV2 model, they discovered a solution highlighting a
of the automobile that are frequently caused by contact with crucial resource for the insurance sector to combat problems
abrasive objects like tree branches or by acts of vandalism. with claims leakage.
The importance of detecting dent and scratches are listed as
follows: In the study and implementation of a model utilising R-
1. For automobile owners who take pleasure in their CNN, Cesar et al., achieved a test performance accuracy rate of
vehicles, the outside design is crucial. An automobile 96.89 [3]. The errors in these cases were caused by reflections
with dents and scratches may appear old, uncared for, that some cars had in their windows or lateral zones, so one
and unattractive. way to reduce them might be to add more images of vehicles
2. A car outside condition has a big impact on its ability with comparable characteristics to the database. This is
to be sold. A car with a spotless appearance will because the colour of the scratches can cause them to be
probably sell for more money than one with obvious mistaken for the surface and go undetected. It is challenging to
dings and scrapes. use this method in the automobile sector and in real-world
3. Scratches may reveal the metal that is covered by applications because of its average execution time, which is
paint, exposing it to corrosion. Quickly identifying close to 3 seconds.
and fixing scratches can stop the body of the car from A model utilising a two-route CNN model has been put out
suffering additional harm. by Mahboub and Majid in order to extract features from the
 Figure 1. YOLOv8 Architecture

Picture [4]. For each route, they employed the new mReLU
YOLOv1, was originally characterised as having a "refreshingly
and inception blocks, which oversee extracting high-level
simple" approach with the publication of "You Only Look Once:
visual information. In order to reduce the model's effectiveness
Unified, Real-Time Object Detection" first suggested by Joseph
at segmenting pieces, they employed a pre-trained ResNet-50
Redmon et al., in 2015. YOLOv1's initial processing speed was
model. To address the overfitting, a 33 mReLU block and an
45 frames per second, whereas fast YOLO, a version, could
inception block were proposed. They discovered that using a
analyse pictures at speeds of up to 155 frames per second. In
pretrained CNN model to each route produced superior
comparison to other object identification algorithms at the time,
segmentation results compared to other models including
it also obtained high mAP. The advantage of using YOLO is
FFNN, YOLOv3, HTC, and VGG. They used the
given as follows:
recommended methodology to a personal database of an
(1) YOLO moves quickly. Since we see detection as a
automobile collection. Their conclusion was 93% accurate.
regression problem, a complicated pipeline is
III. MODEL ARCHITECTURE unnecessary. We only execute our neural network on a
fresh picture at test time to forecast detections.
One of the most well-known object detection algorithms,
 On a Titan X GPU, our base network operates at 45
frames per second without batch processing, while a
fast version operates at more than 150 frames per
second. This indicates that we can handle videos that
stream in real-time with a latency of under 25
milliseconds. Additionally, YOLO achieves a mean
overall precision comparable to real-time systems.
(2) Second, while generating predictions, YOLO thinks
broadly about the image. Because YOLO views the
full image throughout training and testing, as opposed
to sliding window and area proposal-based
approaches, it implicitly stores contextual information
regarding classes in addition to their appearance.
A. Yolov8 Architecture
The most recent iteration of the YOLO object
detection model is called YOLOv8. This most recent
version of YOLO shares the same architecture as its
forerunners 6, but it also adds many improvements Figure 3. Sample Input data
over those found in the earlier iterations, including a
new neural network architecture that makes use of B. Pre-processing
both the Feature Pyramid Network (FPN) and Path Before using images for purposes like training machine
Aggregation Network (PAN), as well as a new learning models, an image dataset must be cleaned, improved,
labelling tool that makes the annotation process easier. and organized. To increase the effectiveness and performance
Numerous helpful features, like automatic labelling, of the model, this may involve scaling, normalizing the pixel
labelling shortcuts, and programmable hotkeys, are values, eliminating noise, and enhancing the data [6].
included in this labelling tool. It is simpler to annotate
photos for model training thanks to the combination of C. Network Architecture
these properties. Yolo architecture has two components, Head with self-
attention mechanism and Backbone with CSPDarknet53. The
IV. METHODOLOGY
self-attention method enables the model to concentrate on
various portions of the picture, giving those areas additional
significance for object recognition. In situations when objects
may be close together or partially obscured, this can assist to
increase the accuracy of object localisation and categorization.
The addition of a self-attention mechanism to the YOLO
architecture's head improves the model's capacity to recognise
intricate spatial connections in the input picture, producing
predictions for object recognition that are more reliable and
accurate [7]. The beginning layers of the neural network that
handle the raw input picture use this architecture since it uses
CSPDarknet53 as its backbone. These layers oversee taking the
hierarchical characteristics from the image and passing them
on to later layers (like the head with a self-attention
mechanism) so they can forecast what objects will be detected.
Figure 2. Proposed Methodology
D. Training
A. Image acquisition
Training the model involves
Choosing and labelling pictures for predetermined ● Initialize the YOLO model with the updated
goals is a necessary step in building a dataset for settings and, if available, pre-trained weights.
computer vision applications. Using tools, manual or ● Employ a loss function that includes classification
semi-automatic labelling can add class labels and loss and localization loss.
create object boundaries. Through transformations, ● Utilise labelled pictures and comments to train the
data augmentation increases variety, while quality model using the training set.
checks guarantee label correctness [5]. It is necessary ● To avoid overfitting, keep an eye on the loss and
to separate the data into training, validation, and test other metrics on the validation set.
sets. In our case we have collected the dataset from
Kaggle and separated the dataset in the ratio 7: 2: 1 for E. Evaluation
training, testing and validation respectively. We have Metrics like mean Average Precision (mAP), precision,
used 465 training images, 133 validation images and recall are frequently used to evaluate YOLO [8]. A general
67 testing images. Sample image is shown in figure 3. indicator of detection performance across different confidence
threshold levels is provided by mAP.
 dents and scratches, even if doing so results in finding some false
Table 1. Trade-offs between different YOLOv8 modelsize positives. The curves shown from Figure 5 to Figure 7 illustrates
the performance of the algorithm for detecting dents and scratches
for the training and validation phases. Figure 5 shows that recall
Model Size mAP val Speed Speed param FLOP achieves a maximum value of 0.73 at confidence 0 and precision
(Pix CPU A100 s(M) s(B) reaches 1 for detecting for all classes at confidence 0.930. A higher
els) 50-95 ONNX Tenso
(ms) rRt
confidence threshold might be used if it is crucial to reduce the
(ms) amount of false positives, even if doing so implies missing some
actual positives. Precision Recall curve gives the mAP score of
YOLOv8n 640 37.3 80.4 0.99 3.2 8.7 0.460 at 0.5 threshold value. The dent detection is giving a higher
value of 0.543 but scratch detection is difficult due to which the
YOLOv8s 640 44.9 128.4 1.20 11.2 28.6
value is 0.378 at threshold value of 0.5. To assess the effectiveness
YOLOv8 640 50.2 234.7 1.83 25.9 78.9 of YOLOv8 models for dent and scratch detection, we utilize the
m F1 confidence curve. Figure 8 shows the F1 confidence curve
which gives a maximum F1 score of 0.54 at threshold of 0.62. The
YOLOv8l 640 52.9 375.2 2.39 43.7 165.2
suitable confidence threshold for the YOLOv8 model can then be
YOLOv8x 640 53.9 479.1 3.53 68.2 257.8 selected using the F1 confidence curve. For instance, we may wish
to select a low confidence threshold if we must detect all dents and
scratches, even if doing so results in the detection of some false
positives. If we can only afford to look at a small number of
F. Recognition photos, we might want to choose a higher confidence level. It is
Recognition entails spending time reviewing the necessary to apply the F1 probability curve to comprehend and
Document of our work, which describes the model's design, enhance the performance of YOLOv8 models for dent and scratch
setup, training procedure, and usage guidelines. This offers detection.
information on the architecture, upgrades, performance
indicators, and assessment.

G. Web Interface
Our concept is accessible to end users who may lack
technical competence thanks to a well-designed user interface
(UI), which abstracts complicated technological elements. So
we have decided to design an web interface using ReactJS
and Flask

H. Deployment
Our model is deployed to make sure it is usable,
trustworthy, and useful in practical situations. To guarantee
that our model functions successfully and efficiently, this
procedure necessitates careful consideration of several
different elements like optimization, scalability, and security. Figure 5: Recall-confidence curve

V. RESULTS AND DISCUSSIONS

Dent and scratch detection is an challenging task and it helps the

inspection process by automation. We have developed a model
using YOLOv8. The detection boxes with level of confidence
detected using YOLOv8 architecture is shown in figure 4

Figure 6: Precision-confidence curve

Figure 4: Sample output of YOLOv8 architecture.

For a certain YOLOv8 model, recall and precision confidence

curves can be used to determine where there is a trade-off
between the two. Using this knowledge, one can select the ideal
confidence level for a particular application. For instance, a
lower confidence level can be selected if it is crucial to find all
 References

[1] Bergmann, P., Batzner, K., Fauser, M., Sattlegger, D., Steger, C. (2022).
Beyond Dents and Scratches: Logical Constraints in Unsupervised
Anomaly Detection and Localization. International Journal of Computer
Vision, 130:947-969, 956-956.
https://doi.org/10.1007/s11263-022-01578-9
[2] Kyu, P.M. and Woraratpanya, K., (2020). Car damage detection and
classification. In Proceedings of the 11th international conference on
advances in information technology (pp. 1-6).
[3] Pachón-Suescún, C. G., Pinzón-Arenas, J. O., & Jiménez-Moreno, R.
Figure 7 : Precision-Recall curve (2019). Detection of Scratches on Cars by Means of CNN and R-CNN.
Int. J. Adv. Sci. Eng. Inf. Technol., 9(3), 745-752.
[4] Parhizkar, M., & Amirfakhrian, M. (2022). Recognizing the Damaged
Surface Parts of Cars in the Real Scene Using a Deep Learning
Framework. Mathematical Problems in Engineering, 2022.
[5] Suescun, C. G. P., Arenas, J. O. P., Moreno, R. J. (2019). Detection of
Scratches on Cars by Means of CNN and R-CNN. International Journal
on Advanced Science Engineering and Information Technology,
9(3),746-749.
https://www.reasearchgate.net/publication/334744794
[6] Bergmann, P., Batzner, K., Fauser, M., Sattlegger, D., Steger, C. (2022).
Beyond Dents and Scratches: Logical Constraints in Unsupervised
Anomaly Detection and Localization. International Journal of Computer
Vision, 130:947-969, 956-956.
https://doi.org/10.1007/s11263-022-01578-9
[7] Suescun, C. G. P., Arenas, J. O. P., Moreno, R. J. (2019). Detection of
Scratches on Cars by Means of CNN and R-CNN. International Journal
on Advanced Science Engineering and Information Technology,
9(3),746-749.
https://www.reasearchgate.net/publication/334744794
[8] Chen, Z., Cao, L., Wang, Q. (2022). YOLOv5-Based Vehicle Detection
Figure 8 : F1-Confidence curve Method for High-Resolution UAV Images, Mobile Information Systems,
2022(1828848), 2-6.
https://doi.org/10.1155/2022/1828848
The Table2 shows the evaluation performance on validation [9] Parhizkar, M., Amirfakhrian, M. (2022). Recognizing the Damage
data for detecting dents and scratches for different epochs. It is Surface Parts of Cars in the Real Scene Using a Deep Learning
observed that mAP value comes to maximum 0.579 at Framework. Mathematical problems in Engineering, 2022(5004129),
3-5.
threshold value of 0.5 for epoch 300. It signifies the capability
https://doi.org/10.1155/2022/5004129
of algorithm to detect dents and scratches.

Table2: Performance evaluation of algorithm

Epochs Precision Recall mAP_0.5

225 0.642 0.563 0.512

250 0.527 0.411 0.439

300 0.631 0.612 0.579

VI. CONCLUSIONS

In this paper, we have developed a model that detects dent

and scratch present in the exterior region of our vehicles. Yolo
v8 has eliminated grid sensitivity, improved training
strategies, compute, and balance losses. Biggest advantage of
using YOLO v8 is its speed. It can process 155 frames per
second. The process of recognizing and finding automotive
dents and scratches has been improved to an unparalleled
degree of precision by utilizing the YOLOv8 model's
excellent object identification skills. As a result, there will be
fewer possibilities for human mistakes, shorter inspection
times, and ultimately higher customer satisfaction.