用于相机再定位的卷积神经网络深入研究

        如图1所示为两个姿态下的图片,已有A图对应的姿态,可以用B图近似等价于C图,得到C图的姿态,实现方位数据的增加。新方位数据与图像平移的转换关系为:

其中Vh,Vv为视场角。


        3、 多任务处理

图2 上部分为传统的PoseNet网络架构,下面为本文的多任务架构BranchNet,先共享网络,然后平移和方向分别由不同的网络给出。之所以分开,是因为仿真表明,方向和平移向量的自相关大于互相关。

主要结果

表1:不同模型的试验结果,数据为中位数误差

        数据集:7Scenes

        1、采用Euler6后,相对于PoseNet,方向误差下降13.6%,平移误差下降5.4%。

        2、增加数据合成后,性能有提升,PoseNet-Euler6-Aug相对于PoseNet-Euler6,方向误差下降12.7%,平移误差下降10.5%。BranchNet-Euler6-Aug相对于BranchNet -Euler6,方向误差下降15.4%,平移误差下降3.3%。

        3、采用BranchNet后,性能同样提升,比如BranchNet- Euler6相对于PoseNet-Euler6,平移误差下降21.1%。

       4、 其他实验结果:


        1)Fine-tuning。在ImageNet上训练并不能提升重定位的精度。

        2)使用Full convolution trick性能提升很小。

        3) 网络参数大小:46MB,在NVIDIA Titan X GPU上进行前向计算,单帧处理时间为6ms,BranchNet-Euler6 在Intel NUC移动平台 的GPU帧频为43。

        本文的局限性:只适合深度信息未知的情况,如果知道深度信息,有更好的方法。如何利用深度信息来提升当前网络的性能依然是个问题。


Abstract

Convolutional Neural Networks have been applied to camera relocalization, which is to infer the pose of the camera given a single monocular image. However, there are still many open problems for camera relocalization with CNNs. We delve into the CNNs for camera relocalization. First, a variant of Euler angles named Euler6 is proposed to represent orientation. Then a data augmentation method named pose synthesis is designed to reduce sparsity of poses in the whole pose space to cope with overfitting in training. Third, a multi-task CNN named BranchNet is proposed to deal with the complex coupling of orientation and translation. The network consists of several shared convolutional layers and splits into two branches which predict orientation and translation, respectively. Experiments on the 7Scenes dataset show that incorporating these techniques one by one into an existing model PoseNet always leads to better results. Together these techniques reduce the orientation error by 15.9% and the translation error by 38.3% compared to the state-of-the-art model Bayesian PoseNet. We implement BranchNet on an Intel NUC mobile platform and reach a speed of 43 fps, which meets the real-time requirement of many robotic applications.



如果你对本文感兴趣,想要下载完整文章进行阅读,可以关注公众号。