Augmented Reality over video stream acquired from UAVs for operations support
Susana Ruano Sainz, SCSS
4.30-5.30pm 1st Nov 2018
Augmented reality (AR) has become, due to recent technology developments, a fastgrowing discipline. The potential of AR supports its study not only for specific devices such as glasses or helmets, but for anything equipped with a camera. Following this idea, Airbus promoted an innovation project, Situational Awareness VIrtual EnviRonment (SAVIER), to incorporate AR in their ground control stations, thus allowing the enhancement of the video stream captured from Unmanned Aerial Vehicles (UAVs). In this talk, I will explain different approaches to improve the situational awareness of the UAV operators during a mission, framed in that project.
Initially, it will be focused on geo-registration, a strategy used for the localization of the UAV in GPS-denied environments. Two key systems for a geo-registration pipeline with different reference data will be described.
First, a multi-view stereo processing pipeline for building a dense terrain model from images of the UAV video feed. This is helpful when a reference terrain model is needed for geo-registration but it is unavailable, outdated, or it has low resolution.
Second, a joint geometric and photometric image registration method that can deal with generic types of distortion: parametric warpings (such as homographies) and non-linear photometric transformations. It is built on top of area-based registration methods to be able to operate in scenarios where feature-based geo-registration methods are not reliable.
Finally, the general case will be considered, in which every sensor measurement is known with enough accuracy and the focus is on displaying virtual elements over the video stream acquired by the UAV. An AR tool to improve the situational awareness of UAV operators during intelligence and surveillance missions will be presented.
I worked as a researcher in the Image Processing Group (GTI, Grupo de Tratamiento de Imágenes) at UPM and, currently, I am in the final stage of my Ph.D. My thesis is enclosed in the SAVIER Open Innovation project funded by AIRBUS with the aim of designing the Ground Control Station of the future. The main topics of my research are the study, design and implementation of image registration algorithms, dense 3D terrain reconstruction and Augmented Reality (AR) solutions to improve the situational awareness of the Unmanned Aerial Vehicles (UAVs) operators. Before starting the Ph.D, I studied a joint degree in Mathematics and Computer Science and a master degree in Computer Vision. I am particularly interested in research areas such as Computer Vision and the improvement of AR & VR experiences.