Speeded up detection of squared fiducial markers☆
Introduction
Pose estimation is a common task for many applications such as autonomous robots [[1], [2], [3]], unmanned vehicles [[4], [5], [6], [7], [8]] and virtual assistants [[9], [10], [11], [12]], among others.
Cameras are cheap sensors that can be effectively used for this task. In the ideal case, natural features such as keypoints and texture [[13], [14], [15], [16]] are be employed to create a map of the environment. Although some of the traditional problems of previous methods for this task have been solved in the last few years, other problems remain. For instance, they are subject to filter stability issues or significant computational requirements.
In any case, artificial landmarks are a popular approach for camera pose estimation. Square fiducial markers, comprised by an external squared black border and an internal identification code, are especially attractive because the camera pose can be estimated from the four corners of a single marker [[17], [18], [19], [20]]. The recent work of Muñoz-Salinas et al. [21] is a step forward in the use of this type of markers in large-scale problems. One only needs to print the set of markers with a regular printer, place them in the area under which the camera must move, and take a set of pictures of the markers. The pictures are then analyzed and the three-dimensional marker locations automatically obtained. Afterward, a single image spotting a marker is enough to estimate the camera pose.
Despite the recent advances, marker detection can be a time-consuming process. Considering that the systems requiring localization have in many cases limited resources, such as mobile phones and aerial vehicles, the computational effort of localization should be kept to a minimum. The computing time employed in marker detection is a function of the image size employed: the larger the images, the slower the process. On the other hand, high-resolution images are preferable since markers can be detected, even if they are far from the camera, with high accuracy. The continuous reduction in the cost of the cameras, along with the increase of their resolution, makes it necessary to develop methods able to reliably detect the markers in high-resolution images.
The main contribution of this paper is a novel method for detecting square fiducial markers in video sequences. The proposed method relies on the idea that markers can be detected in smaller versions of the image, and employs a multi-scale approach to speed up computation while maintaining the precision and accuracy. In addition, the system is able to dynamically adapt its parameters in order to achieve maximum performance in the analyzed video sequence. Our approach has been extensively tested and compared with the state-of-the-art methods for marker detection. The results show that our method is more than an order of magnitude faster than state-of-the-art approaches without compromising robustness or accuracy, and without requiring any type of parallelism.
The remainder of this paper is structured as follows. Section 2 explains the works most related to ours. Section 3 details our proposal for speeding up the detection of markers. Finally, Section 4 gives a exhaustive analysis of the proposed method and Section 5 draws some conclusions.
Section snippets
Related works
Fiducials marker systems are commonly used for camera localization and tracking when robustness, precision, and speed are required. In the simplest case, points are used as fiducial markers, such as LEDs, retroreflective spheres and planar dots [22,23]. However, their main drawback is the need of a method to solve the assignment problem, i.e., assigning a unique and consistent identifier to each element over time. In order to ease the problem, a common solution consists in adding an identifying
Speeded up marker detection
This section provides a detailed explanation of the method proposed for speeding up the detection of squared planar markers. First, Section 3.1 provides an overview of the pipeline employed in the previous work, ArUco [17], for marker detection and identification, highlighting the parts of the process susceptible to be accelerated. Then, Section 3.2 explains the proposed method to speed up the process.
Experiments and results
This section shows the results obtained to validate the methodology proposed for the detection of fiducial markers.
First, in Section 4.1, the computing times of our proposal are compared to the best alternatives found in the literature: AprilTags [18], ChiliTags [36], ArToolKit+ [31], as well as ArUco [17] which is included in the OpenCV library1. Then, Section 4.2 analyzes and compares the sensitivity of the proposed method with the above-mentioned methods. The main goal is
Conclusions and future work
This paper has proposed a novel approach for detecting fiducial markers aimed at maximizing speed while preserving accuracy and robustness. The proposed method is specially designed to take advantage of the increasing camera resolutions available nowadays. Instead of detecting markers in the original image, a smaller version of the image is employed, in which the detection can be done at higher speed. By wisely employing a multi-scale image representation, the proposed method is able to find
Acknowledgments
This project has been funded under projects TIN2016-75279-P and IFI16/00033 (ISCIII) of Spain Ministry of Economy, Industry and Competitiveness, and FEDER.
References (41)
- et al.
Autonomous vision-based robotic exploration and mapping using hybrid maps and particle filters
Image Vis. Comput.
(2009) - et al.
Towards shared autonomy for robotic tasks in manufacturing
Procedia Manuf.
(2017) - et al.
An intelligent framework for simulating robot-assisted surgical operations
Expert Syst. Appl.
(2005) - et al.
Applications of computer vision to intelligent vehicles
Image Vision Comput.
(2000) - et al.
Timely autonomous identification of UAV safe landing zones
Image Vis. Comput.
(2014) - et al.
Parametric-based path generation for automated vehicles at roundabouts
Expert Syst. Appl.
(2017) - et al.
An improved augmented reality system based on AndAR
J. Vis. Commun. Image Represent.
(2016) - et al.
Visual orientation inhomogeneity based scale-invariant feature transform
Expert Syst. Appl.
(2015) - et al.
Automatic generation and detection of highly reliable fiducial markers under occlusion
Pattern Recogn.
(2014) - et al.
Robust identification of fiducial markers in challenging conditions
Expert Syst. Appl.
(2018)
Mapping and localization from planar markers
Pattern Recogn.
Generation of fiducial marker dictionaries using mixed integer linear programming
Pattern Recogn.
Real-time positioning for augmented reality on a custom parallel machine
Image Vis. Comput.
A reliable open-source system architecture for the fast designing and prototyping of autonomous multi-UAV systems: simulation and experimentation
J. Intell. Robot. Syst.
Vision based fuzzy control autonomous landing with UAVs: from V-REP to real experiments
Augmented marker tracking for peri-acetabular osteotomy surgery
Markerless tracking system for augmented reality in the automotive industry
Expert Syst. Appl.
A real-time reconstructed 3D environment augmented with virtual objects rendered with correct occlusion
LSD-SLAM: Large-Scale Direct Monocular SLAM
ORB-SLAM: a versatile and accurate monocular SLAM system
IEEE Trans. Robot.
Cited by (0)
- ☆
This paper has been recommended for acceptance by Luis Merino.