Difference between revisions of "ReacTable"

From Hackteria Wiki
Jump to: navigation, search
Line 10: Line 10:
 
In a nutshell the system works like this: reacTIVision tracks specially designed fiducial markers in a real time video stream. The source image frame is first converted to a black&white image with an adaptive thresholding algorithm. Then this image is segmented into a tree of alternating black and white regions (region adjacency graph). This graph is then searched for unique left heavy depth sequences, which have been encoded into the fiducial symbol. Finally the found tree sequences are matched to a dictionary to retrieve an unique ID number. The fiducial design allows the efficient calculation of the marker's center point as well as its orientation. OSC messages implementing the TUIO protocol encode the fiducials' presence, location, orientation and identity and transmit this data to the client applications.  
 
In a nutshell the system works like this: reacTIVision tracks specially designed fiducial markers in a real time video stream. The source image frame is first converted to a black&white image with an adaptive thresholding algorithm. Then this image is segmented into a tree of alternating black and white regions (region adjacency graph). This graph is then searched for unique left heavy depth sequences, which have been encoded into the fiducial symbol. Finally the found tree sequences are matched to a dictionary to retrieve an unique ID number. The fiducial design allows the efficient calculation of the marker's center point as well as its orientation. OSC messages implementing the TUIO protocol encode the fiducials' presence, location, orientation and identity and transmit this data to the client applications.  
 
Additionally reacTIVision uses the result of the image segmentation in order to retrieve and identify small round white blobs as finger tips on the surface. A quick and dirty shape matching algorithm selects the actual finger blobs from the possible region candidates. A complementary blob tracking algorithm is also taking advantage of the same data in order to track eventually not recognized fiducials, for example where fast movements destroy the actual fiducial structure in the image.
 
Additionally reacTIVision uses the result of the image segmentation in order to retrieve and identify small round white blobs as finger tips on the surface. A quick and dirty shape matching algorithm selects the actual finger blobs from the possible region candidates. A complementary blob tracking algorithm is also taking advantage of the same data in order to track eventually not recognized fiducials, for example where fast movements destroy the actual fiducial structure in the image.
 +
 +
 +
 +
 +
== building the table ==
 +
 +
 +
'''table & surface'''
 +
 +
A camera and a projector with wide-angle lenses need to be placed underneath the table, so they can both cover the entire surface. Alternatively a mirror can be used in order to achieve a larger projection distance. For the interactive surface itself a normal perspex board can be used in conjunction with some ordinary tracing paper on the top side for the projection. This material is completely transparent for objects and finger tips in direct contact with the surface. In order to avoid direct reflections of the light source and projector lamp, the lower side of the surface should have a matte finish, while maintaing the overall transparency.
 +
 +
 +
'''illumination'''
 +
 +
For the tracking, the objects need to be properly illuminated, so the camera and thus the computer vision application can see them correctly. For the projection onto a table, the surface needs to be dark though, so the user can see the projected image well enough. Since these two necessary steps logically exclude each other, the solution is to operate in two different spectra:
 +
The projection has to be visible to the user, so the computer vision component needs to operate in a different, invisible spectrum such as near infrared in the range of 850nm. Most CCD cameras are perfectly sensitive within the near IR spectrum, therefore infrared LED lamps can be used to illuminate the table. All light from the visible spectrum needs to be filtered in the camera, so the computer vision algorithm is not disturbed by the projection. Eventually an existing infrared blocker needs to be removed from the camera sensor.
 +
 +
 +
'''camera'''
 +
 +
You should make sure that the camera has an acceptable lens and sensor size. For lowest latency and best performance we recommend firewire cameras from the top range, such as industrial cameras with a high framerate, resolution and sensor size. These cameras usually also come with high quality C-mount lenses. Cheaper firewire cameras, such as the unibrain fire-i also allow optional wide-angle lenses. From the large range of available USB cameras we recommend to use high end models with a native resolution of at least 640x480 at a frame rate of 30Hz. A very affordable and relatively good camera for this purpose is the Sony PS3eye, which is also working well under Windows, Linux and Mac OSX. DV cameras supporting full-frame mode are suitable, while those with interlaced mode only, will not work at all.

Revision as of 04:00, 19 October 2010

This is a multi touch table that reads markers placed on the screen and does a corresponding action. Thus reading the object on the screen and functioning accordingly. The table uses reactivision software. This application was designed to track specially designed fiducial markers. By downloading the TUIO library you can cerate a program that operates on these markers. the table uses infra red light for finger tracking. a webcam takes in the feed and runs it through reactivision. It is then processed in processing and displayed again on the screen.

We used this to give information and protocals when a certian object was put on it.


How reactivision works

In a nutshell the system works like this: reacTIVision tracks specially designed fiducial markers in a real time video stream. The source image frame is first converted to a black&white image with an adaptive thresholding algorithm. Then this image is segmented into a tree of alternating black and white regions (region adjacency graph). This graph is then searched for unique left heavy depth sequences, which have been encoded into the fiducial symbol. Finally the found tree sequences are matched to a dictionary to retrieve an unique ID number. The fiducial design allows the efficient calculation of the marker's center point as well as its orientation. OSC messages implementing the TUIO protocol encode the fiducials' presence, location, orientation and identity and transmit this data to the client applications. Additionally reacTIVision uses the result of the image segmentation in order to retrieve and identify small round white blobs as finger tips on the surface. A quick and dirty shape matching algorithm selects the actual finger blobs from the possible region candidates. A complementary blob tracking algorithm is also taking advantage of the same data in order to track eventually not recognized fiducials, for example where fast movements destroy the actual fiducial structure in the image.



building the table

table & surface

A camera and a projector with wide-angle lenses need to be placed underneath the table, so they can both cover the entire surface. Alternatively a mirror can be used in order to achieve a larger projection distance. For the interactive surface itself a normal perspex board can be used in conjunction with some ordinary tracing paper on the top side for the projection. This material is completely transparent for objects and finger tips in direct contact with the surface. In order to avoid direct reflections of the light source and projector lamp, the lower side of the surface should have a matte finish, while maintaing the overall transparency.


illumination

For the tracking, the objects need to be properly illuminated, so the camera and thus the computer vision application can see them correctly. For the projection onto a table, the surface needs to be dark though, so the user can see the projected image well enough. Since these two necessary steps logically exclude each other, the solution is to operate in two different spectra: The projection has to be visible to the user, so the computer vision component needs to operate in a different, invisible spectrum such as near infrared in the range of 850nm. Most CCD cameras are perfectly sensitive within the near IR spectrum, therefore infrared LED lamps can be used to illuminate the table. All light from the visible spectrum needs to be filtered in the camera, so the computer vision algorithm is not disturbed by the projection. Eventually an existing infrared blocker needs to be removed from the camera sensor.


camera

You should make sure that the camera has an acceptable lens and sensor size. For lowest latency and best performance we recommend firewire cameras from the top range, such as industrial cameras with a high framerate, resolution and sensor size. These cameras usually also come with high quality C-mount lenses. Cheaper firewire cameras, such as the unibrain fire-i also allow optional wide-angle lenses. From the large range of available USB cameras we recommend to use high end models with a native resolution of at least 640x480 at a frame rate of 30Hz. A very affordable and relatively good camera for this purpose is the Sony PS3eye, which is also working well under Windows, Linux and Mac OSX. DV cameras supporting full-frame mode are suitable, while those with interlaced mode only, will not work at all.