Scene Responsiveness for Visuotactile Illusions in Mixed Reality (Full Video)

Scene Responsiveness deepens the illusion of passthrough AR so that virtual actions and virtual characters seemingly affect the physical scene. We toggle the reality state of objects from physical to virtualized by building on situated digital twins and a novel graphics and shader pipeline. This allows the objects to be manipulated in all ways virtuality affords while maintaining visual coherence and visuotactile consistency. Mohamed Kari, Reinhard Schütte, and Raj Sodhi. Scene Responsiveness for Visuotactile Illusions in Mixed Reality. In: ACM Symposium on User Interface Software Technology (UIST '23). project page: Research conducted at Meta Reality Labs Research #mixedreality #augmentedreality #metaverse #mr #xr #ar #vr #spatialcomputing