The Second International Symposium on Electrical and Electronics Engineering ISEEE-2008, Galati, Romania

Remote Video Tracking System Based

On Computer Vision
AbstractThis paper describes a vision-based face detection and tracking system which can
be controlled over the internet. The contribution is the fusion of the characteristics of two
known systems: IP cameras and vision-based machine-user interfaces. The proposed system
can run in real-time achieving good results.

I I!TRO"#CTIO!
T$e aim o% t$is paper is to de&e'op a system t$at a''o(s detection and tracking o% $uman %aces) (it$
t$e possi*i'ity o% remote contro' t$e tracking de&ice &ia t$e internet T$e e+istence o% comp'e+
sur&ei''ance systems *ased on %acia' recognition) as t$ose %rom ,ace-Tek Systems or --. Identity
So'utions) is due %or t$e moment to t$e demand p'aced *y go&ernmenta' or mi'itary organi/ation) and
'east *y t$e commercia' sector In t$is conte+t) e+perimenta' design (it$ minima' costs o% a tracking
system *ased on image processing is a c$a''enge and a prior stage to designing a comp'e+ security
system
II ,0CE "ETECTIO! 0!" TR0C1I!2 3ET4O"S
A. Human face detection
"esigning automated system t$at ana'y/es t$e in%ormation comprised in images asks %or ro*ust and
e%%icient detection a'gorit$ms
T$e %ace detection met$od used in t$is paper is main'y *ased on t$e met$od proposed *y 5au'
Vio'a and 3ic$ae' 6ones 7.8) 728 and on t$e impro&ements to t$e met$od *y Rainer -ien$art) and
6oc$en 3aydt in 798) 0'e+ander 1urano&) Rainer -ien$art) and Vadim 5isare&sky in 7:8) and ;oa&
,reund and Ro*ert Sc$apire in 7<8 T$is approac$ o% detection is *y using statistica' c'assi%iers
T$ese samp'es may *e o*tained *y ana'y/ing training images set ($ic$ (i'' 'ater *e used %or
detection Suc$ a samp'e presupposes t$at training *e done on mu'tip'e instances o% t$e o*=ect to *e
detected and a'so on mu'tip'e negati&e samp'es >images t$at do not comprise t$e o*=ect under %ocus?
"i%%erent >4aar? %eatures are e+tracted %rom t$e training set) and a%ter(ards are se'ected t$ose
distincti&e %eatures t$at may c'assi%y t$e o*=ect
4aar %eatures and a (ide set o% &ery simp'e >@(eakA? c'assi%iers t$at use a sing'e %eature to c'assi%y
t$e ana'y/ed image as %ace or non-%ace) $a&e *een used in order to e+tract t$e %eatures c$aracteristics
o% t$e %aces
T$is approac$ is imp'emented (it$in t$e OpenCV - t$e open source programming 'i*rary
de&e'oped *y Inte' %or use *y researc$ers and pro%essiona's a'ike #sing t$is 'i*rary and t$e c'assi%iers
cascades produced *y OpenCV de&e'opers) accurate detection resu'ts can *e o*tained
. !ace trac"ing process
In order to track t$e detected %ace is necessary to sa&e t(o consecuti&e states o% t$e detected
image and t$en mo&e t$e camera a distance proportiona' to t$e di%%erence *et(een t$e a*so'ute
positions o% t$e t(o mentioned states One important issue re%ers to mu'tip'e detections) *ecause it is
possi*'e to $a&e simu'taneous di%%erent %aces (it$in t$e same %rame 0s t$is paper does not approac$
t$e %ace recognition issue) ($ic$ (ou'd $a&e a''o(ed %or c$oosing on'y one %ace) t(o $ypot$eses
(i'' *e ad&anced 'eading to a simi'ar resu't >,ig .?
Hypothesis #. It is proposed t$at t$e tracking process to *e per%ormed t$roug$ %oreground %ace
detection T$is supposes t$at t$e %ace area) 'ocated in t$e %oreground) (i'' *e *igger t$an t$e ot$er
ones ,or appro+imation) instead o% %ace area it (i'' *e considered t$e area o% t$e circ'e
circumscri*ed to t$e main e'ements o% t$e %ace) and %or a %aster ca'cu'ation) it (i'' not *e need to
ca'cu'ate t$e circ'e area *ut on'y its radius) t$e %ina' se'ection taking into account t$e radius r
r $ ma%&r
#
, r
'
, ... , r
n
(
The Second International Symposium on Electrical and Electronics Engineering ISEEE-2008, Galati, Romania
Hypothesis '. 0 second se'ection criterion (i'' *e t$e minimum distance *et(een t$e mass point o%
t$e %ace detected (it$in t$e pre&ious stage >t$e center o% t$e circ'e circumscri*ed to t$e main
e'ements o% t$e %ace? and t$e mass points o% t$e %aces detected in t$e current stage
d $ min&d
#
, d
'
, ... , d
n
(
T$us) assuming t$at 5
0
is t$e center o% t$e %ace detected (it$in t$e pre&ious stage >,ig .? and t$at
r
0
is t$e radius o% t$e circ'e t$at circumscri*es t$e decisi&e e'ements o% t$e %ace detected (it$in t$e
same stage) 5
.
) 5
2
))5
n
are t$e circ'es centers t$at circumscri*e t$e e'ements o% t$e %aces detected at
t$e current stage
,ig . T(o criterion %or se'ecting mu'tip'e detectionsB t$e ma+imum radius o% t$e detected image sur%ace and t$e
minima' distance *et(een t$e image detected at t$e pre&ious stage and t$e ones detected in t$e current stage
Once %ace se'ection %inis$ed) kno(ing t$e coordinates o% t$e %ace centre) as (e'' as t$e coordinates
o% t$e image centre) t$e command (i'' *e gi&en to t$e dri&ing mec$anism to mo&e t$e (e* camera is
suc$ a manner in ($ic$ t$e t(o centers to superpose
). )ommunication
T$e system proposed in our paper is aimed at detecting and tracking %aces %rom distance) %rom t$e
Internet Suc$ a system is endo(ed (it$ t$e %eatures o% t(o systems a'ready kno(n) t$at isB $uman-
mac$ine inter%aces) and t$e I5 cameras) t$e %irst $a&ing communication inter%aces %or s$ort distance
>#SB port) seria' port) in%rared port) B'uetoot$?) ($ereas t$e 'atest ones are 'ong distance internet
communication systems par e+ce''ence "ue to t$e %act t$at t$e proposed system is o% type c'ient-
ser&er) ensuring secured data transmission o&er t$e Internet) using a secured communication
protoco' >SS-? is a necessity >,ig 9?
,ig 9 T$e %ace tracking system communication and data %'o( diagram
III S;STE3 STR#CT#RE
A. Hard*are
0s (e stated in t$e *eginning o% t$is paper) t$e system (i'' *e imp'emented (it$ eCuipments t$at
are no(adays considered usua' T$e system (i'' *e made o% a persona' computer acting as a (e*
ser&er) a ser&o contro''er >3ini SSC II?) and a (e* camera $a&ing attac$ed a dri&ing mec$anism
x
r
2
P
2
(x
2
,y
2
)
r
n
P
n
(x
n
,y
n
)
r
1
r
0
P
1
(x
1
,y
1
)
y
d
n
d
2
d
1
P
0
(x
0
,y
0
)
Selection of a detected face using previous frame
data, in case of multiple face detections
r
n
main
top-e%t
midd'e-e%t
3enu
Commands
midd'e
Rig$t
Vertica'
Command
top3idd'e topRig$t
midd'e
midd'e3idd'e
Video Content
midd'eBottom
4ori/onta' Commands
Video Capture D Tracking
Command 3odu'e
Seria' Command 3odu'e
The Second International Symposium on Electrical and Electronics Engineering ISEEE-2008, Galati, Romania
operated *y t(o ser&omotors >,ig :?
,ig: 2enera' diagram o% t$e remote &ideo tracking system
. Soft*are
Starting %rom t$e %act t$at t$e proposed tracking system must o*ser&e t$e condition o% distance
&isua'i/ation and command) (e assume t$at t$e user inter%ace must a''o( %or continuous disp'aying
t$e image captured *y t$e ser&er T$is is t$e reason ($y t$e e%%icient so'ution is to di&ide t$e
inter%ace in t(o partsB command and) respecti&e'y) &isua'i/ation Because o% t$is reason) and
considering t$e %act t$at ser&er main (indo( re%res$ is not economica' and may resu't in distur*ing
%'ickering) t$e most e%%icient (ay to design t$e user inter%ace is to use t$e 4T3- %rames
T$is is $o( (e can separate t$e content t$at must *e continuous'y re%res$ed >at a con&enient
%reCuency?) apart %rom t$e static content
0s one o% t$e reCuests o% t$e system is to ensure t$e secured access) t$is supposes t$e ser&er may
o%%er t$e possi*i'ity o% creating (e*pages t$at (ork on sessions Because it is kno(n t$at t$e 4TT5
protoco' is state'ess) eac$ connection ending a%ter t$e c'ient recei&ed an ans(er %rom t$e ser&er) it is
necessary to use dynamic (e* pages) 545 tec$no'ogy *eing a po(er%u' and easy to use so'ution
T$ere%ore) a'' documents t$at (i'' *e do(n'oaded into %rames (i'' *e o% type 545 scripts 3oreo&er)
in order to o%%er t$e possi*i'ity o% distance manua' operating o% t$e (e*cam) it (i'' *e necessary %or
t$e system to dispose o% contro' e'ements in &ertica' and $ori/onta' p'ans o% t$e (e* camera
4o(e&er) as automated operating presupposes operatorEs 'ack on inter&ention) t$e contro' e'ements
s$ou'd *e inacti&e in t$e case o% manua' operating Taking into account t$ese e'ements) t$e user
grap$ic inter%ace (i'' 'ook 'ike in ,ig <
T$e &ideo content to *e disp'ayed in t$e dedicated %rame (i'' *e o% t(o typesB image %i'e >%rom t$e
&ideo acCuisition *y t$e (e*cam?) or a &ideo %i'e T$e %irst type o% %i'e represents data o% a static
nature) ($ereas t$e second type o% %i'e represents a data %'o( t$at (i'' *e disp'ayed in a dynamic
manner To gi&e a sensation o% mo&ement continuity %or t$e %irst type o% %i'e itEs needed t$at t$e
images captured *y t$e (e*cam to *e disp'ayed one *y one) continuous'y) at a con&enient re%res$
rate) as %ast as it does not %'icker) *ut not t$at %ast so t$at to unnecessari'y use t$e resources o% t$e
'oca' station or to %'ood t$e ser&er (it$ access reCuests T$is content re%res$ (i'' *e done *y using
6a&aScript 'anguage
The Second International Symposium on Electrical and Electronics Engineering ISEEE-2008, Galati, Romania
,ig < #ser inter%ace and t$e &ideo capture and tracking modu'e) as (e'' as microcontro''er command modu'e

IV 5ER,OR30!CE 0!0-;SIS
To assess t$e per%ormance o% t$e system) (e (anted to assess t$e *e$a&ior o% t$e system ($en t$e
mo&ement speed o% a %ace (it$in t$e &isua' range goes past certain 'imits
0s it is di%%icu't to measure t$e mo&ement speed o% a %ace (it$in t$e &isua' range o% a (e*cam) (e
can consider t$e image as *eing sti'') and (it$ a sta*'e *ackground) and to mo&e t$e camera
o*=ecti&e) o*taining t$e same e%%ect T$is is $o( (e s$a'' $a&e a c'ear contro' on t$e mo&ement and
t$e data (i'' *e accurate'y determined
3oreo&er) t$e %act t$at t$e %ace (i'' $a&e a %i+ed position on a determined *ackground (i'' a''o(
%or de%ining idea' e+perimenta' conditions) (it$out &ariations o% 'ig$t) position) or *ackground
To o*tain di%%erent angu'ar speeds ($en operating t$e motor) (e can add &ia t$e program a de'ay
@+tA *et(een t(o consecuti&e steps o% t$e motor and) conseCuent'y) t$is is $o( (e s$a'' $a&e c'ear
contro' on t$e speed
Out o% t$e tec$nica' speci%ications o% t$e ser&omotor) (e can use t$e %o''o(ing dataB speed Fs and
angu'ar step G Hit$ t$ese data it can *e determined t$e angu'ar speed o% t$e motor during a certain
time period) in accordance (it$ t$e app'ied de'ayB
t t +
=

(it$
s
t

=
Once t$e angu'ar speed %or t$e (e*cam mo&ement is de%ined) t$e ne+t step is to esta*'is$ t$e
ma+imum ang'e %or t$e (e*cam to detect a %ace
In order to ac$ie&e t$is) t$e &isua' range o% t$e (e*cam must *e proportiona''y diminis$ed (it$ t$e
si/e taken *y t$e %ace in t$e image >,ig I? T$e ang'e 0 in t$e %igure is gi&en *y t$e (e*cam
producer as *eing t$e cameraEs @,ie'd o% &ie(A So %ar (e $a&e admitted t$at it is accepta*'e to
appro+imate t$e si/e o% a detected %ace (it$ t$e sur%ace o% t$e circ'e $a&ing radius @rA) t$at
circumscri*es t$e main e'ements o% t$e %ace >,ig 8?) and (e s$a'' %urt$er use t$e same
appro+imation
,ig I "imension o% t$e %ace reduces t$e (e* cameraEs %ie'd o% &ie( >during t$e mo&ement?
,ig 8 T$e detected %ace and t$e circ'e t$at circumscri*es t$e main e'ements o% t$e %ace
T$e %ace can *e detected on'y in t$e @' J --2rA range
T$e space diminis$ed *y t$e same %ace in t$e image) in t$e mo&ement time) isB
l $ ,-'r &!ig. ..'(
and) conseCuent'y) t$e proportiona' reduction o% t$e &isua' %ie'd >in ($ic$ detection it is possi*'e? is
gi&en *y t$e re'ationB
r
' J - K 2r
-
r
a
0
r
r
--2r
The Second International Symposium on Electrical and Electronics Engineering ISEEE-2008, Galati, Romania
,
A
r , a ? 2 > =
($ere @aA is t$e corresponding ang'e) diminis$ed as compared (it$ t$e ma+imum ang'e 0 >,ie'd O%
Vie( o% t$e camera? >,ig L?

,ig L Reduction o% t$e ang'e o% &ie( >during t$e mo&ement? due to dimension o% t$e %ace
0s t$e angu'ar speed is t$e same (it$ t$e angu'ar speed registered during t$e period determined
a*o&e) (e can noteB
/
t
a
=
T$us) one can (rite t$e re'ations$ip *et(een t$e parameters o% t$e &ideo camera) t$e si/e o% t$e
%ace to detect to >measure in pi+e's?) and t$e angu'ar speedB
? >
.
? 2 >
.
,
Ad
A
t ,
A
r ,
t t
a
= = =
M
? >
.
,
Ad
A
t
=
($ere @dA is t$e diameter o% t$e circ'e t$at circumscri*es t$e main e'ements o% t$e detected %ace
V EN5ERI3E!T0- RES#-TS
To assess t$e per%ormance o% t$e system) %e( sets o% iterati&e tests $a&e *een carried out It (as
use a station type 5C (it$ t$e %o''o(ing c$aracteristicsB processor 03" "#RO! .O 24/ C5#)
IO8 3B ""R2 R03 T$e (e* camera used) (as a -ogitec$ Puick Cam 3essenger) and t$e
microcontro''er %or t$e steppers command (as a 3ini SSC II
,or t$e (e*cam mo&ement at a preset angu'ar speed >o*tained *y introducing t$e de'ay? (e $a&e
(ritten a program t$at commands t$e ser&omotor to 'e%t - rig$t *a'ance in $ori/onta' p'ane T$e
&a'ue o% t$e ang'e %or a comp'ete mo&ement >'e%t K rig$t? (as c$osen at L0 degrees) and t$e duration
o% a test (as set to 20 seconds T$e %igures >,ig .0 - .2? s$o( t$e o*tained data sets >.O? *y
&arying t$e angu'ar speed in 098L radQs K <<.. radQs range
,ig .0 2rap$ica' representation o% t$e %rame rate
,ig .. 2rap$ica' representation o% t$e num*er o%
tota' detections and o% t$e num*er o% %a'se detections
The Second International Symposium on Electrical and Electronics Engineering ISEEE-2008, Galati, Romania
,ig .2 2rap$ica' representation o% t$e &a'ues o*tained %or a test >20 sec? %or t$e angu'ar speed o% 098L radQs
>app'ied de'ayB .< mi''iseconds Q period?
VI CO!C-#SIO!
T$e desideratum o% =oining speci%ic used tec$niCues %or t(o di%%erent %ie'ds o% data processing)
computer &ision and 'ong distance data transmission) (as met *ut t$ere are needed impro&ements
*ot$ at t$e 'e&e' o% $ard(are imp'ementation and at t$e so%t(are 'e&e' 0'so ac$ie&ing %ace
recognition (i'' resu't in sta*i'i/ing camera mo&ements and in t$e possi*i'ity o% tracking on'y one
certain %ace) t$us gi&ing up on using t$e a'gorit$m current'y imp'emented %or tracking %oreground
%aces
Setting up o% suc$ a compact tracking system) (it$ possi*i'ities o% communication in 'oca'
net(ork) may 'ead to t$e idea o% insta''ing same type systems t$at are 'ogica''y interconnected T$ere
are tec$no'ogies >060N and 545 tec$no'ogy *y #R- C'ient -i*rary *eing t(o o% t$em?) ($ic$
a''o( %or di%%erent systems to *e connected and to communicate using &arious protoco's