{"type":"ScholarlyArticle","title":"Keep It Real: a Window to Real Reality in Virtual Reality","authors":[{"name":"Baihan Lin","slug":"baihan-lin","given":"Baihan","family":"Lin","affiliation":"Computational Visual Inference Lab , Columbia University , New York , NY 10027 , USA Department of Applied Mathematics ,; University of Washington , Seattle , WA 98195 , USA"}],"abstract":"This paper proposed a new interaction paradigm in the virtual reality (VR) environments, which consists of a virtual mirror or window projected onto a virtual surface, representing the correct perspective geometry of a mirror or window reflecting the real world. This technique can be applied to various videos, live streaming apps, augmented and virtual reality settings to provide an interactive and immersive user experience. To support such a perspective-accurate representation, we implemented computer vision algorithms for feature detection and correspondence matching. To constrain the solutions, we incorporated an automatically tuning scaling factor upon the homography transform matrix such that each image frame follows a smooth transition with the user in sight. The system is a real-time rendering framework where users can engage their real-life presence with the virtual space.","doi":"10.24963/ijcai.2020/766","url":"https://ijcai.org/proceedings/2020/766","pdf":"https://ijcai.org/proceedings/2020/0766.pdf","markdown":"https://ijcai.org/proceedings/2020/766.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/766","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Demos","track":"Demos","number":766,"pages":{"first":5270,"last":5272},"keywords":["Computer Vision: general","Human-Computer Interactive Systems: general"],"videos":[],"references":[{"position":1,"raw":"[Blum et al., 2012] Tobias Blum, Valerie Kleeberger, Christoph Bichlmeier, and Nassir Navab. mirracle: An augmented reality magic mirror system for anatomy education. In 2012 IEEE Virtual Reality Workshops (VRW), pages 115-116. IEEE, 2012.","title":"mirracle: An augmented reality magic mirror system for anatomy education","authors":"Blum","year":"2012","venue":"IEEE Virtual Reality Workshops (VRW)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bowman and McMahan, 2007] Doug A Bowman and Ryan P McMahan. Virtual reality: how much immersion is enough? Computer, 40(7):36-43, 2007.","title":"Virtual reality: how much immersion is enough?","authors":"Mcmahan Bowman, Doug A Bowman, Ryan P Mcmahan","year":"2007","venue":"Computer","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Brunelli, 2009] Roberto Brunelli. Template matching tech- niques in computer vision: theory and practice. John Wi- ley & Sons, 2009.","title":"Template matching techniques in computer vision: theory and practice","authors":"Roberto Brunelli","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Burdea and Coiffet, 2003] Grigore C Burdea and Philippe Coiffet. Virtual reality technology. John Wiley & Sons, 2003.","title":"Virtual reality technology","authors":"C Grigore, Philippe Burdea, Coiffet","year":"2003","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Coxeter, 2003] Harold Scott Macdonald Coxeter. Projective geometry. Springer Science & Business Media, 2003.","title":"Projective geometry","authors":"Harold Scott, Macdonald Coxeter","year":"2003","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Sato et al., 2010] Kenji Sato, Satoshi Fukumori, Takashi Matsusaki, Tomoko Maruo, Shinichi Ishikawa, Hiroyuki Nishie, Ken Takata, Hiroaki Mizuhara, Satoshi Mizobuchi, Hideki Nakatsuka, et al. Nonimmersive virtual reality mir- ror visual feedback therapy and its application for the treat- ment of complex regional pain syndrome: an open-label pilot study. Pain medicine, 11(4):622-629, 2010.","title":"Nonimmersive virtual reality mirror visual feedback therapy and its application for the treatment of complex regional pain syndrome: an open-label pilot study","authors":"Kenji Sato, Satoshi Fukumori, Takashi Matsusaki, Tomoko Maruo, Shinichi Ishikawa, Hiroyuki Nishie, Ken Takata, Hiroaki Mizuhara, Satoshi Mizobuchi, Hideki Nakatsuka","year":"2010","venue":"Pain medicine","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Spanlang et al., 2014] Bernhard Spanlang, Jean-Marie Nor- mand, David Borland, Konstantina Kilteni, Elias Gi- annopoulos, Ausiàs Pomés, Mar González-Franco, Daniel Perez-Marcos, Jorge Arroyo-Palacios, Xavi Navarro Mun- cunill, et al. How to build an embodiment lab: achieving body representation illusions in virtual reality. Frontiers in Robotics and AI, 1:9, 2014.","title":"How to build an embodiment lab: achieving body representation illusions in virtual reality","authors":"Bernhard Spanlang, Jean-Marie Normand, David Borland, Konstantina Kilteni, Elias Giannopoulos, Ausiàs Pomés, Mar González-Franco, Daniel Perez-Marcos, Jorge Arroyo-Palacios, Xavi Navarro Muncunill","year":"2014","venue":"Frontiers in Robotics and AI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}