XFacetrackerLK

Component Type: Sensor (Subcategory: Computer Vision)
OS: Windows, Linux, Mac OSX

This component provides a cross-platform face tracking computer vision algorithm which puts out estimated movement of a users’ nose and chin in x and y coordinates. The underlying mechanism builds upon the JavaCV (OpenCV) library (in particular a trained cascade of haar-like features in combination with a Lukas Kanade optical flow algorithm is used to track a face and its movement). The x- and y- coordinates can be used in camera-mouse configurations or to enable selection or control tasks. The values are only provided if a face can be tracked by the algorithm. Note that the x- and y- values represent relative movement in pixels and have to be accumulated (e.g. via the integrate component) to generate e.g. absolute mouse positions.

Requirements

A camera has to be available (this can be any webcam or a camera which is available as image acquisition device via the operating system). The picture below shows the Logitech Webcam 9000 Pro. Also the camera device of a Kinect sensor or PS3Eye should work. Furthermore, a video stream of an IP camera can be used for tracking.

Logitech Webcam 9000 Pro

Output Port Description

noseX [integer]: This value specifies the relative change in the x coordinate of the user's nose with respect to the previous image frame.
noseY [integer]: This value specifies the relative change in the y coordinate of the user's nose with respect to the previous image frame.
chinX [integer]: This value specifies the relative change in the x coordinate of the user's chin with respect to the previous image frame.
chinY [integer]: This value specifies the relative change in the y coordinate of the user's chin with respect to the previous image frame.

Event Listener Description

init: if this event is triggered, the face recognition procedure is initiated. This can be useful if the correct face position has been lost due to drifting of the LK algorithm.
showCameraSettings an incoming event displays the settings window for the camera device (only on Windows), where parameters like image brightness or contrast can be adjusted.

Properties

frameGrabber[string, combobox selection]:Name of FrameGrabber to use (Default, VideoInput, OpenCV, FFmpeg, OpenKinect, PS3Eye, IPCamera).
frameGrabberFormat[string]:Format for grabber, e.g. FFmpeg: 'dshow' (default), 'vfwcap', 'gdigrab'.
deviceList [string, combobox selection]:List of available devices, if supported by grabber
cameraSelection [string]: The camera device to be used - use camdIdx e.g. '0' (VideoInput, OpenCV, OpenKinect, PS3Eye), or camName e.g. 'video=Integrated Camera' or 'desktop' (FFmpeg), stream-url for IPCamera.
cameraResolution [string, combobox selection]: This selection box provides several standard camera resolutions. Changing the resolution affects accuracy and performance (CPU load of the runtime system). Provided selections include “160x120”, “320x240”, “640x480”, “800x600”, “1024x768” and “1600x1200”.
titleVideoFrameWindow [string]:The title of the window showing the video frame.

Example Configuraitons

Default

Let the plugin probe available framegrabbers and choose the first available one. The following grabber are tried in the following order:

OpenCV
VideoInput
FFmpeg

Configuration

frameGrabber:Default
cameraSelection:0

OpenCV

frameGrabber:OpenCV
cameraSelection:0

VideoInput

frameGrabber:VideoInput
cameraSelection:0

FFmpeg

frameGrabber:FFmpeg
frameGrabberFormat (or vfwcap):dshow
cameraSelection (device name as of device manager):video=Integrated Camera

MJPEG stream of SmartPhone camera

Install the following Android app on your smartphone and start streaming the front camera: Camera Stream - WiFi IP Webcam (Web Host LLC)

Enter the http url displayed at your smartphone and add /video to the url

e.g.

http://192.168.1.100:8080/video

Set the cameraSelection property of the XFacetrackerLK plugin to this url.