JOIG Stereo Vision based Localization of Handheld Controller in Virtual Reality for 3D Painting Using Inertial System

Abstract

Google Tilt Brush is expensive for virtual drawing which needs further emphasize on the betterment of software rather than hardware expects addressed in this research. Several issues are addressed by this research in this context, i.e., noise removal from sensor data, double integration-based drift issues and cost. Currently, available smart phones do not have the ability to paint in virtual reality using cardboard and daydream of google without buying Oculus Rift and HTC Vive (Virtual Reality Headset) which are too costly for mass people. In addition, various external sensors, i.e., satellite positioning sensor, barometer, laser, ultra- wideband system, ultrasonic positioning system are not used for drawing in virtual reality. This research used extended Kalman filter and Butterworth filter to perform positioning using 6-DOF (Degrees of Freedom) MEMS (Microelectromechanical Systems) sensor data. A stereo visual method using SLAM (Simultaneous Localization and Mapping) is used to measure the distance for localization using smartphone camera for the controller application to estimate drift. This research utilizes Google Virtual Reality Software Development Kit with Unity3D engine. Experimentation validation states that proposed method can perform painting using virtual reality headset along with controller application using smartphone mobile instead of external controller device, i.e., Oculus Rift and HTC Vive with high precision rate.