Interactive Surface Sensing Technologies Evolve, Surveying Resolution, Scalability, and Multimodal Integration

Interactive surface sensing technologies now underpin a vast range of devices, moving far beyond simple touchscreens to enable increasingly rich and expressive human-computer interaction. David Wang, Wilson Chen, and Tianju Wang, all from the University of Michigan, alongside Jiale Zhang et al., comprehensively survey this rapidly evolving field, tracing its development from early infrared and capacitive systems to contemporary vision and acoustic sensing techniques. This work examines the operating principles, resolution, and scalability of each modality, alongside a comparative analysis of their strengths and weaknesses, revealing the key technical and design factors influencing performance and user experience. By identifying persistent challenges such as sensing accuracy and privacy, this survey outlines a path towards truly ubiquitous and intelligent interactive environments, demonstrating the potential for future innovation in this critical area of technology. Sensing Modalities, Strengths and Trade-offs This document provides a comprehensive overview of various sensing technologies used in interactive surfaces, breaking down the characteristics of each modality and offering key insights. The research clearly demonstrates that no single sensing technology is perfect; each presents trade-offs, and the optimal choice depends heavily on the specific application. Accuracy often comes at the cost of complexity, cost, or robustness; for example, computer vision offers rich interaction but requires significant computational power and is sensitive to lighting conditions. Emerging technologies like millimeter wave radar and vibration sensing offer unique capabilities, such as through-object detection and force sensing. The inclusion of multimodal systems, combining multiple sensing modalities, suggests a trend towards overcoming the limitations of individual technologies, enabling more robust and versatile interactive experiences. Several modalities, including acoustic and vibration sensing, are heavily reliant on the material properties of the surface, a crucial consideration for design and implementation. Furthermore, the research highlights privacy concerns associated with computer vision and acoustic interfaces, positioning millimeter wave radar as a more privacy-preserving alternative.

Infrared Imaging Locates Touch Interactions Precisely This research details a thorough examination of interactive surface technologies, beginning with a comprehensive analysis of infrared-based optical systems and capacitive touch. Scientists meticulously investigated Frustrated Total Internal Reflection (FTIR) and Diffuse Illumination (DI), establishing their foundational role in the development of large-scale interactive surfaces. The study employed infrared cameras positioned beneath or beside acrylic panels to capture scattered light resulting from disruptions in Total Internal Reflection when a user touches the surface, enabling precise recovery of touch locations from captured imagery. Researchers demonstrated the capabilities of FTIR through the development of a system that supports smooth multi-finger interaction and responsive visual feedback, a landmark achievement widely recognized as a catalyst for the multi-touch movement. This work directly inspired commercial platforms such as Microsoft PixelSense and fostered continued exploration of low-cost optical touch systems. Leveraging the principles of optical propagation, scientists extended FTIR to create early interactive walls and immersive display systems, highlighting the scalability of optical interaction in public spaces.

The team further advanced multi-user interaction by developing DiamondTouch, a system that reliably supports simultaneous multi-point input and enables user differentiation on a single tabletop, facilitating influential interdisciplinary projects. However, the study also rigorously assessed the limitations of FTIR, identifying challenges related to detecting light touches, bulky form factors, and environmental sensitivity.

Millimeter Wave Radar Achieves Sub-Millimeter Accuracy This work details a comprehensive survey of interactive surface technologies, revealing significant advancements in sensing modalities and their potential for future applications. Researchers have meticulously examined infrared, capacitive, acoustic, and vision-based sensing, alongside emerging technologies like millimeter wave radar and vibration sensing, to understand their operating principles, resolution, scalability, and applications. The study demonstrates that millimeter wave radar, a technique utilizing high frequency radio waves, achieves sub-millimeter level accuracy in tracking gestures and movements at frame rates reaching 10,000 FPS, exceeding the precision of traditional capacitive touch interfaces. Experiments with millimeter wave radar reveal its ability to accurately track large-scale movements and multiple objects within an environment, achieving over 90% accuracy using techniques like Kalman filtering and point clouds. Furthermore, the research highlights that millimeter wave radar systems, unlike vision-based tracking, function effectively in all lighting conditions and do not require cameras, thereby addressing privacy concerns. Orientation detection using millimeter wave signals allows for precise reconstruction of object orientations, enabling accurate tracking of wearables and realistic simulation of real-world objects in virtual environments. Researchers demonstrate that modern millimeter wave radar systems are now compact enough for integration into mobile devices and smart spaces, establishing a foundation for the development of ubiquitous and intelligent interactive environments.

Multimodal Sensing Enables Rich Interactions Interactive surface technology has undergone significant evolution, progressing from early optical and infrared methods to a diverse range of sensing approaches including capacitive, acoustic, millimeter wave, and vibration-based techniques.

This research demonstrates that each sensing modality possesses unique strengths regarding precision, scalability, robustness, and ease of integration, while also presenting inherent trade-offs that influence user interaction with digital content. The development of vibration sensing, for example, allows for passively powered activity trackers capable of determining user actions without requiring charging or cameras. Advances in materials, embedded hardware, and signal processing are driving the emergence of multimodal sensing as a promising avenue for achieving more reliable, expressive, and context-aware surface interactions. However, several challenges remain, including the accurate differentiation of multiple simultaneous users, enabling robust non-contact interaction, and adapting to diverse materials and environments. Researchers acknowledge the need to balance sensing fidelity with user privacy as a critical consideration for future development, focusing on lightweight, scalable, and seamlessly integrated sensing technologies to create truly ubiquitous interactive environments. 👉 More information 🗞 The Evolving Landscape of Interactive Surface Sensing Technologies 🧠 ArXiv: https://arxiv.org/abs/2512.05071 Tags: