On and off body sensor fusion for a 3d motion controller

Abstract

There are a variety of f engineering applications that would benefit from the development of a 3D motion controller including fields such as gaming, entertainment, sport and education. Companies such as Nintendo, Microsoft and Sony have recently developed a number of motion gaming controllers. These use both on-device motion sensors and off-device sensors to determine the controllers' position. In this final year project, signal processing and communication techniques are used to develop 3D orientation and position determining algorithms which can be used to realise an accurate 3D motion controller. This is achieved by fusing together information from on and off body sensors, with an Android smartphone providing the on-body sensors and a pair of cameras providing the off-body sensors. To develop the 3D orientation determination algorithm 3-axis gyroscope, accelerometer and magnetometer data are fused together to give orientation data. A complementary filter [ 6] is designed and implemented to achieve this as opposed to the slower, yet more traditional, Kalman filter. This filters out drift caused by the gyroscope and noise from the accelerometer and magnetometer to give accurate and robust orientation data. A revised complementary filter is then proposed to extend the solution to full 360° rotations. In the 3D position determination algorithm off-body cameras are used in conjunction with the Hough Circle Transform [32] to locate a spherical marker attached to a Smartphone in video feeds. Mono and stereo video position location techniques are then used to determine the 3D position relative to one camera. Three scenarios are examined using a single camera, a pair of parallel oriented cameras and a pair of perpendicular cameras. In the final setup the cameras are calibrated such that one is the reference whilst the other is unconstrained. The x, y, z coordinates of the marker are found from the closest distance between skew vectors emanating from the two camera models, with sub-centimeter accuracy. An algorithm is finally presented which can be used as the bases of a future improvement to automate the calibration process. The two algorithms are finally combined together leading to a successful 3D motion controller.