Abstract
(Semi-)immersive displays like back projection walls have become widely used in various visual domains. Despite the superior image quality of ?through-the-windows VR? users are still not satisfied with the interaction capabilities — mostly due to the extra wiring for the interaction devices. This paper represents a hand gesture interaction system for a back-projection wall environment, supporting object manipulation (translation, rotation, and uniform scale) and selection tasks through the vision-tracking technique. We propose to use thimble-shaped fingertip markers made of white printing paper with a ?black light? source. Gesture parameters are calculated by 3D positions of the marked fingertips and their pinching states.