Good afternoon, everyone, today we are going to talk about the interactive design with
Kinect.
In today’s lesson, you will understand which application areas the Kinect can be used. Then,
identify the key components of it and how it works with processing code. And finally, you
will appreciate how to design a system with Kinect.
So what is Kinect? Does anyone heard of it before? Excellent. Here, let me give you guys an
overview of how Kinect is being used. In entertainment, Kinect is popular in gaming, where it
can track player movements. And it has also been used in theaters to perceive the movements
and changes in the show. In healthcare, Kinect can be used for exercise therapy. It can track
patients' movements in real-time and provide visual feedback to adjust exercises based on
their performance. In shopping, it has been used in virtual try-on to help people make
shopping decisions.
Here I give the pics about two versions of the Kinect. Now let’s take the older version as an
example to understand each component of it. It has three eyeballs. The one is an infrared
projector. It can send out infrared light into the room then it has a camera to read. The middle
one is a RGB camera, which to obtain the color images. You can see their difference among
these images.
So now we are going to talk about how to obtain depth image information using Kinect.
First, we need to import Kinect library and define an object of type to declare Kinect. Next,
we start at the "setup" function to initialize some basic settings.
Here, we set the canvas size to match the Kinect capture size;
We then initialize the Kinect object and enable depth information.
The "draw" function is called repeatedly to draw the image on the canvas. And we start by
setting the background color to the black.
Next, we need to get the depth image as the PImage object in cam and display on the canvas.
And this can be further extended for some other applications such as motion detection or
gesture recognition.
So how to perform hand tracking with Kinect. Let's start by the "drawHandState" function.
We begin by hiding the stroke using the code: noStroke();
Next, we invoke the "handState" function based on the current hand state.
After that, we use the "pushMatrix" function to save the current matrix, and translate to the
joint position.
We then draw an ellipse with noFill().
Next, we draw the stroke with it related features, and to create the sign text "Hello!" with
offset.
With this code, we can create interactive design based on hand states.
Now let’s look at another sample application to create particle effects. The particle system can
be used as visual element, responding to the movements and gestures. For example, audience
could use their body movements to interact with particles in art exhibition. And in music
performances, the particles can respond to the beats of the music. When music speeds up or
slows down, the particles can follow suit, creating a dynamic experience.
So now, before we start to design an interactive system using Kinect, there are several design
principles which should be considered.
One principle to follow is simplicity and don’t require a lot of explanation. That means the
mapping of the user's movement should be easy to understand.
Another principle is feedback. The feedback should indicate the user interaction is captured in
a meaningful way. For example, the particles could change color, size when the user moves
their hand from the Kinect. By providing feedback, the user is more likely to engage and feel
like their actions are impacting the surrounding environment.
And third one is aesthetics. The system should have a visually appealing design that is
consistent with the overall context in which it is used.
Lastly, the principle of efficiency should be considered. This means making the code run
faster and reduce resources. And the animation should be smooth with a suitable frame rate.
By following these principles, I hope you guys could create your own design successfully.
And next seminar, we will further talk about your own project. Well, that’s all of today’s
lesson. Have a nice day!
