See discussions, stats, and author profiles for this publication at: [Link]

net/publication/323166370

Hardware Implementation of IoT-Based Image Processing Filters

Chapter · July 2018

DOI: 10.1007/978-981-10-8228-3_63

CITATION READS
1 1,232

4 authors, including:

Ajay Rupani Pawan Whig

JIET Group of Institutions Vivekananda Institute Of Professional Studies
15 PUBLICATIONS   24 CITATIONS    68 PUBLICATIONS   199 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Very large scale integration View project

VLSI Design View project

All content following this page was uploaded by Ajay Rupani on 14 February 2018.

The user has requested enhancement of the downloaded file.

 Hardware Implementation of IoT based Image Processing
Filters
Ajay Rupani1, Pawan Whig2, Gajendra Sujediya3, Piyush Vyas4
RIET, Jaipur1, 3, VIPS Delhi2, JIET Jodhpur4 India
(ajayrupani1991@[Link])1, (pawanwhig@[Link])2
(gajendrafromsawar@[Link])3, ([Link]@[Link])4
…………………………………………………………………………................
Abstract – The image processing based on interfacing of FPGA and Raspberry
Pi using Internet of Things that is recently introduced technique is a hot topic in
present scenario. The unimaginable interconnection of smart devices, smart
cities, smart vehicles, and smart people throughout the globe is made possible
by Internet of things. The hardware implementation of various filters used in
image processing using Internet of things on an FPGA platform is presented in
this dissertation. The Raspberry Pi and FPGA interfacing based implementation
of image processing filters using Internet of Things have gotten huge
consideration from the exploration group in the previous couple of years. A
powerful, low cost, user friendly way of 24 hours real time image processing
system based on Internet of things has been offered by this technique. In this
paper, we highlight that how one can access the design resources based on
FPGA from anyplace. The primary point of this research is to highlight how the
clients can get to the FPGA based outline resources from anyplace. In this
manner we exhibit an idea that abbreviates the utilization of immediately
unused resources for executing different assignments automatically.
------------------------------------------------------------------------------------------------

Key Words — FPGA, Internet of Things, Image Processing.

1 Introduction
Some factors introduce brightness variation in an image when there is no detail is
available for that image. There often occurs random variation and no particular pattern of
variation is present [1], [2]. The quality of an image is reduced in many cases due to the
variation.

1.1 Purpose of Image Processing

a) Visualization: Observe the objects that are not visible.
b) Image Sharpening and Restoration: To create a better image.
c) Image Recognition: Distinguish the objects in an image.
d) Image Retrieval: Seek for the image of interest.
e) Measurement of pattern: Measure various objects in an image.

1.2 Objectives of this research

Not all image processing filtering methods are appropriate for a particular application. An
objective and quantitative dimension on the appropriateness of each method for a
specified application is of great consequence and usefulness. The following are the main
objective of this research work:
 To design the IoT based architecture of various Image processing filters, where
user can select the type of filter operation just by clicking on appropriate option.
 To implement the different image processing filters using the interfacing of
Raspberry-Pi and ZedBoard Zynq 7000 APSoC.
 To obtain the output of various image processing filters viz. Gaussian filter,
average filter and sharpen filter for the Lena’s image having size 256x256 along
with the on chip power components report.

2 Related Work
In this section, review of literature has been carried out. Research papers, books and
articles have been used in preparing this research work. Various researches have been
carried out in the field of image processing; some of them are described below:
D. M. Wu et al (1995) displayed the outline and usage of an IBM-PC based circulated
image processing framework. Traditional image processing operations, for example,
spatial sifting and improvement are likewise implemented on the Computer. Thus, an
optional technique is accommodated ongoing image processing which was accessible
only for more costly frameworks [3].
Zhu Bochun et al (1996) presented a constant image processing framework in view of
two-chip DSP TMS320C40 of Texas Instruments. There was frame store available that
could be able to configure again whenever required. This, consolidated with the adaptable
interface and powerful correspondence capacity of TMS320C40, makes the framework
be effectively designed as MIMD, SIMD pipeline and parallel structures. The distortion
present in the image processed by few components is smartly maintained a strategic
distance from without extra circuits [4].
In 2000, Jie Jiang et al presented the execution and design of ongoing image processing
system for Electronic Endoscope. The real time image processing has been utilized as a
part of Electronic endoscope effectively. The picture quality and the precision rate of
finding are made strides. FPGA is embraced as the center of the framework for its
alterable and reconfigurable element, so the framework can be effortlessly adjusted, in
addition the execution of the framework can be enhanced in the future, without changing
the equipment. With a lot of logic gates in a solitary gadget, FPGA gives the framework
high combination and magnificent execution [5].
In 2003, T. Sugimura et al proposed a reconfigurable parallel image processing using
FPGA for the superior real time image processing framework. Parallel setup of logic
units and interconnection network has effectively diminished arrangement information. A
test chip was manufactured for this image processing using FPGA utilizing 0.35 p m
CMOS technology [6].
In 2005, Pei-Yung Hsiao et al proposed edge detection method for image processing
system by means of an FPGA architecture design. There are two edge detection
algorithms available which are appropriate for realization on hardware and not sensitive
to noise. These two algorithms can create distinctive yields appropriate for various
applications. The altered LGT algorithm presented in this paper preserves the original
edge detection performance as well as extraordinarily diminishes the utilization of
hardware asset [7].
In 2008, Jun Wang et al presented the DSP and FPGA based real time image processing
system having all the features viz. extendibility, modularization and small in size. The
framework with correspondingly suitable broadening can meet an assortment of
necessities in different image processing fields. The framework has favorable
circumstances of client configurable interface with sensor, capable abilities of image
preprocessing and displayed the processed image in the PAL in video format [8].
In 2010, Chunfu Gao et al presented the SOPC based real-time image processing system.
For the algorithm, the image was collected by this system and that image was processed
by using the embedded Nios II soft-core processor, which has the elements of
effortlessness and incredible information amount and parallelism. The implementation of
this system was done using FPGA. Along these lines, the processing time of image was
diminished that enhanced the continuous execution of framework adequately and
understands the optimizing of cutting direction progressively. The system has various
advantages viz. small volume, reconfiguration of software and hardware, flexibility of
programming, portability, strong generality et cetera [9].
In 2011, Hao Wei proposed multiple parallel processing of multiple video image data
using the combination of both DSP and FPGA. Configuration takes full preferred
standpoint of programmable gadgets, for example, FPGA and FPGA equipment assets
accessible inside the enhanced framework reconciliation and unwavering quality,
additionally streamlines the framework equipment outline. The framework has been
connected to a pseudo-review instrument ventures, has gotten attractive outcomes. After
the genuine venture approval, the framework configuration is effective [10].
In 2012, Hayato Hagiwara proposed the FPGA implementation based real time image
processing system for robot vision system. A moving object is easily identified and
tracked by the robot vision system. Additionally this robot vision framework
distinguishes the passageway limits amongst floors and dividers, and judges its position
and introduction in the entry condition. This robot vision framework is sufficiently small
and needs low power utilization for different robot applications. The Hardware-software
integration for real time image processing system could be appropriate to different goals
[11].
In 2014, Sunmin Song et al proposed a window-based ongoing Gaussian Filter technique
for the pre-processing of a continuous stereo vision framework. The mean filter and the
low pass filter are used to blur an image. In the mean filter, the value of each pixel is
replaced by the average value of all neighboring pixel intensity values; however it causes
side effects, blurring the edge. As the weight average value provided by low pass filter, a
few pixels are more critical. Rather, the significance of other pixels is relinquished. The
Gaussian Filter having a place with the low pass filter has been of enthusiasm for the PC
vision field and digital image processing. The Gaussian Filter is for the most part utilized
as a part of the pre-processing of edge recognition [12].
In 2015, Jinalkumar Doshi highlighted how clients can get to FPGA plan assets from
anyplace in blend with a potential remote FPGA lab. The design of the cloud base stage
is depicted with the analysis of a load for the server. The authors presented the cloud
based approach in this paper and a near investigation is talked about in view of the
outcomes acquired. The Python and Hypertext Preprocessor (PHP) scripting languages
are used to create the remote environment on the Ubuntu i.e. an open source operating
system [13].
In 2015, Zalak Dave et al proposed a Network on Chip based image processing system. A
Torus topology having 6 nodes is used to design the architecture of NoC [14]. This paper
introduced display interface a memory interface, various IP algorithms and computer
interface (utilizing Raspberry Pi). A versatile plot for remote applications has been
provides by the Raspberry Pi.
As indicated by Shivank Dhote (2016), the system architecture which comprises of
numerous logic blocks that perform point operations or neighborhood operations on the
pixel information. This information is recovered from the innovative Buffer RAM. The
RAM has its input perused from the UART buffer register. Client application transfers
the image over an FTP server onto the Raspberry-pi. The R-pi at that point sends the
bitmap of the image file to the Spartan 6 utilizing the UART protocol. Decoder logic is
composed to choose which type of filter is to be connected over the gotten image. The
processed image is put away on a RAM and transmitted back to the R-pi on ask for,
utilizing the UART protocol [15].

3 Proposed Methodology

The system architecture consists of numerous logic blocks that execute point operations
or neighborhood operations on the pixel data. This data is retrieved from the innovative
Block RAM. The input image is loaded to ZedBoard Zynq 7000 as .coe file. Decoder
logic is written to select which type of filter is to be applied over the acknowledged
image from Raspberry pi to ZedBoard Zynq 7000. The processed image is stored on a
Block RAM and fed to VGA monitor. A decoder is designed in the FPGA to make
possible the selection of filter to be applied as per the user commands. The decoder
generates a signal to make active the required logic block. FPGA being a parallel device
in nature, the application of multiple special effects on pixels at the same time does not
have need of much on-chip resources. Raspberry Pi 3 MODEL B is an upgraded version
of enhanced network with Bluetooth Low Energy (BLE) and BCM43143 Wi-Fi on board
[16]. ZedBoard Zynq 7000 FPGA development kit is an ease advancement board for AP
SoC (All Programmable SoC) [17]. The VGA driver is used for implementing the
horizontal and vertical synchronization of the monitor with the FPGA. Data from the
RAM is sent as input to the VGA driver. The obtained output of various image
processing filters has been displayed on 640x480 VGA display. On horizontal section,
the image of size 256x256 has been displayed from pixel number 200 to pixel number
456 while on vertical section; the image is displayed from pixel number 110 to pixel
number 366 pixels on VGA Screen.
 Figure 1: Proposed System Architecture

The hardware implementation of proposed IoT based image processing system i.e.
the interfacing of Raspberry-Pi and FPGA1 is shown in figure 2:

Figure 2: Hardware Implementation of Proposed Image Processing System

4 Experimental Results
The code written in Verilog Hardware Description Language is simulated using
Xilinx Vivado software to obtain the output of various filters viz. Gaussian filter,
Sharpening filter and Average filter. The ZedBoard Zynq 7000 APSoC is used to
implement the design i.e. used in this paper. The total power utilized for the whole
system is 40.580 Watts. The tabular representation of power utilization for the system is
shown below:
TABLE 1. Power Utilization Report of On-Chip Components

Power Utilization
On-Chip Used Available
(W) (%)

Slice Logic
18.119 38069 --- ----
LUT as
15.292 9879 53200 18.56
Logic

Register 2.593 18557 106400 17.44

CARRY4 0.225 118 13300 0.88

BUFG .009 2 32 6.25

Signals 12.541 19526 --- ---

Block
7.935 72 140 51.42
RAM

DSPs 0.410 3 220 1.36

I/O 0.486 22 200 11.00

Static 1.089
Power

40.580
Total
The input image and output images captured from VGA monitor are shown below:

Figure 3: Input Image Figure 4: Gaussian Filter Output Figure 5: Sharpen Filter Output

Figure 6: Grayscale Output Figure 7: Edge Detection Output Figure 8: Average Filter Output

Figure 3 shows the input image on which various filtering operations have been
performed. From the Figure 4, it is clear that the Gaussian filter gave gentler smoothing
and the edges have been preserved superior to anything a comparably sized mean filter.
Figure 5 displays the output of sharpening operation and the Figure 6 shows the output
after Greyscale operation. From the Figure 7, it is clear that the effect of the edge
detection indicates how easily or unexpectedly the values of image pixel change at each
point in the whole image and in this way we can say how likely the segment of the
picture denotes to an edge. We have observed that the harder edges have been appeared
as brighter (higher values) in the results. Figure 8 shows the effect of applying a 3×3
mean (average) filter. From the Figure, it is clear that the noise is less apparent, but the
image has been softened. We observed that the pixels will be nearer to each other in
value; accordingly hard edge has been eliminated in this situation.
5 Conclusion
The Various best suitable image processing filters are available to enhance a noisy
image. The decision of applying which specific filter strongly depends upon various
levels of noise present at various pixel locations. In this paper, all inclusive filter
architecture used for image processing has been presented. This gives the solution for
multi-practical image processing for various applications. The design and implementation
for various image processing filters has been presented. All of the modules have been
designed and implemented on a ZedBoard Zynq 7000 APSoC. It has been concluded how
Internet of Things and Image Processing can be combined and implanted how exactly
they need to be combined in order to get satisfactory results.

6 Future Scope
To make the system architecture more comprehensive, we can add more filters to our
system architecture presented in this paper. The existing filter modules can also be
streamlined as advancements are made in individual filter architectures and as more
shared modules are created. As IoT explores the interconnecting of devices, cloud
computing concepts can be established with them for a complete consolidated system.

REFERENCES

[1] Trimberger S. M, “Field – Programmable Gate Array Technology”, Kluwer

Academic Publishers, 1995.
[2] R. C. Gonzalez and P. Wintz, Digital Image Processing – Second Edition,Addison-
Pearson Education Publishing Company, 2005.
[3] D. M. Wu, L. Guan, G. Lau and D. Rahija “ Design And Implementation of a
Distributed Real-Time Image Processing System”, Engineering of Complex
Computer Systems, 1995 held jointly with 5th CSESAW, 3rd IEEE RTAW and 20th
IFAC/IFIP WRTP, Proceedings, pp- 266-269, 1995.
[4] Zhu Bochun, Guo Chengming, Wang Zhaohua “A Parallel Real-time Image
Processing System Based on TMS320C40” 3rd International Conference on Signal
Processing Proceedings , Vol. 1, pp-653-656, 1996.
[5] Jie Jiang, Daoyin Yu, Zheng Sun, “Real-time Image Processing System Based on
FPGA for Electronic Endoscope”, IEEE APCCAS- IEEE Asia-Pacific Conference on
Circuits and Systems proceedings, pp-682-685, 2000.
[6] T. Sugimura, Jeoung Chill Shim, H. Kurino, M. Koyanagi, “Parallel Image
Processing Field Programmable Gate Array for Real Time Image Processing
System”, IEEE International Conference on Field-Programmable Technology (FPT)
Proceedings, pp- 372-374, 2003.
[7] Pei-Yung Hsiao, Le-Tien Li, Chia-Hsiung Chen, Szi-Wen Chen, Sao-Jie Chen, “An
FPGA Architecture Design of Parameter- Adaptive Real-Time Image Processing
System for Edge Detection” Conference on Emerging Information Technology, pp-
1-3, 2005.
 [8] Jun Wang, Lei Peng, Li Wei, “Design and Implementation of a Real-Time Image
Processing System with Modularization and Extendibility”, International
Conference on Audio, Language and Image Processing, pp- 798-802, 2008.
[9] Chunfu Gao, Zhiyong Luo, Xinsheng He, ―Study on The Real-time Image
Processing System of Cutting Robot Based on SOPC”, International Conference on
E-Product E-Service and E-Entertainment, pp- 1-4, 2010.
[10] Hao Wei, “Designing and Development of Multi-DSP real-time image processing
system based on FPGA” Proceedings of 2011 International Conference on
Computer Science and Network Technology, vol. 2, pp- 1263-1265, 2011.
[11] Hayato Hagiwara, Kenichi Asami, Mochimitsu Komori, “Real-time Image
Processing System by Using FPGA for Service Robots” , The 1st IEEE Global
Conference on Consumer Electronics 2012, pp- 720-723, 2012.
[12] Sunmin Song, SangJun Lee, Jae Pil Ko, Jae Wook Jeon, “A Hardware Architecture
Design for Real-time Gaussian Filter”, IEEE International Conference on Industrial
Technology (ICIT) pp- 626-629, 2014.
[13] Jinalkumar Doshi, Pratiksha Patil, Zalak Dave, Ganesh Gore, Jonathan Joshi, Reena
Sonkusare, Surendra Rathod, “Implementing a Cloud Based Xilinx ISE FPGA
design Platform for Integrated Remote Labs”, International Conference on Advances
in Computing, Communications and Informatics (ICACCI),pp- 533-537, 2015.
[14] Zalak Dave, Shivank Dhote, Jonathan Joshi, Abhay Tambe, Sachin Gengaje,
“Network on Chip Based Multi-function Image Processing System using FPGA”,
International Conference on Advances in Computing, Communications and
Informatics (ICACCI),pp- 488-492, 2015.
[15] M. Shivank Dhote, Pranav Charjan, Aditya Phansekar, Aniket Hegde, Sangeeta
Joshi, Jonathan Joshi, “Using FPGA-SoC Interface for Low Cost IoT Based Image
Processing”, International Conference on Advances in Computing, Communications
and Informatics (ICACCI), pp- 1963-1968, 2016.
[16] [Link]
[17] [Link]

View publication stats