Elsevier

Image and Vision Computing

Volume 76, August 2018, Pages 38-47
Image and Vision Computing

Speeded up detection of squared fiducial markers

https://doi.org/10.1016/j.imavis.2018.05.004Get rights and content

Highlights

  • A system that speed up the detection of squared planar markers

  • Employ a multiresolution approach to reduce computing time

  • Temporal information is also considered to reduce computation.

  • Up to 40 times faster than the state-of-the-art approaches without any parallelization

Abstract

Squared planar markers have become a popular method for pose estimation in applications such as autonomous robots, unmanned vehicles and virtual trainers. The markers allow estimating the position of a monocular camera with minimal cost, high robustness, and speed. One only needs to create markers with a regular printer, place them in the desired environment so as to cover the working area, and then registering their location from a set of images.

Nevertheless, marker detection is a time-consuming process, especially as the image dimensions grows. Modern cameras are able to acquire high resolutions images, but fiducial marker systems are not adapted in terms of computing speed. This paper proposes a multi-scale strategy for speeding up marker detection in video sequences by wisely selecting the most appropriate scale for detection, identification and corner estimation. The experiments conducted show that the proposed approach outperforms the state-of-the-art methods without sacrificing accuracy or robustness. Our method is up to 40 times faster than the state-of-the-art method, achieving over 1000 fps in 4 K images without any parallelization.

Introduction

Pose estimation is a common task for many applications such as autonomous robots [[1], [2], [3]], unmanned vehicles [[4], [5], [6], [7], [8]] and virtual assistants [[9], [10], [11], [12]], among others.

Cameras are cheap sensors that can be effectively used for this task. In the ideal case, natural features such as keypoints and texture [[13], [14], [15], [16]] are be employed to create a map of the environment. Although some of the traditional problems of previous methods for this task have been solved in the last few years, other problems remain. For instance, they are subject to filter stability issues or significant computational requirements.

In any case, artificial landmarks are a popular approach for camera pose estimation. Square fiducial markers, comprised by an external squared black border and an internal identification code, are especially attractive because the camera pose can be estimated from the four corners of a single marker [[17], [18], [19], [20]]. The recent work of Muñoz-Salinas et al. [21] is a step forward in the use of this type of markers in large-scale problems. One only needs to print the set of markers with a regular printer, place them in the area under which the camera must move, and take a set of pictures of the markers. The pictures are then analyzed and the three-dimensional marker locations automatically obtained. Afterward, a single image spotting a marker is enough to estimate the camera pose.

Despite the recent advances, marker detection can be a time-consuming process. Considering that the systems requiring localization have in many cases limited resources, such as mobile phones and aerial vehicles, the computational effort of localization should be kept to a minimum. The computing time employed in marker detection is a function of the image size employed: the larger the images, the slower the process. On the other hand, high-resolution images are preferable since markers can be detected, even if they are far from the camera, with high accuracy. The continuous reduction in the cost of the cameras, along with the increase of their resolution, makes it necessary to develop methods able to reliably detect the markers in high-resolution images.

The main contribution of this paper is a novel method for detecting square fiducial markers in video sequences. The proposed method relies on the idea that markers can be detected in smaller versions of the image, and employs a multi-scale approach to speed up computation while maintaining the precision and accuracy. In addition, the system is able to dynamically adapt its parameters in order to achieve maximum performance in the analyzed video sequence. Our approach has been extensively tested and compared with the state-of-the-art methods for marker detection. The results show that our method is more than an order of magnitude faster than state-of-the-art approaches without compromising robustness or accuracy, and without requiring any type of parallelism.

The remainder of this paper is structured as follows. Section 2 explains the works most related to ours. Section 3 details our proposal for speeding up the detection of markers. Finally, Section 4 gives a exhaustive analysis of the proposed method and Section 5 draws some conclusions.

Section snippets

Related works

Fiducials marker systems are commonly used for camera localization and tracking when robustness, precision, and speed are required. In the simplest case, points are used as fiducial markers, such as LEDs, retroreflective spheres and planar dots [22,23]. However, their main drawback is the need of a method to solve the assignment problem, i.e., assigning a unique and consistent identifier to each element over time. In order to ease the problem, a common solution consists in adding an identifying

Speeded up marker detection

This section provides a detailed explanation of the method proposed for speeding up the detection of squared planar markers. First, Section 3.1 provides an overview of the pipeline employed in the previous work, ArUco [17], for marker detection and identification, highlighting the parts of the process susceptible to be accelerated. Then, Section 3.2 explains the proposed method to speed up the process.

Experiments and results

This section shows the results obtained to validate the methodology proposed for the detection of fiducial markers.

First, in Section 4.1, the computing times of our proposal are compared to the best alternatives found in the literature: AprilTags [18], ChiliTags [36], ArToolKit+ [31], as well as ArUco [17] which is included in the OpenCV library1. Then, Section 4.2 analyzes and compares the sensitivity of the proposed method with the above-mentioned methods. The main goal is

Conclusions and future work

This paper has proposed a novel approach for detecting fiducial markers aimed at maximizing speed while preserving accuracy and robustness. The proposed method is specially designed to take advantage of the increasing camera resolutions available nowadays. Instead of detecting markers in the original image, a smaller version of the image is employed, in which the detection can be done at higher speed. By wisely employing a multi-scale image representation, the proposed method is able to find

Acknowledgments

This project has been funded under projects TIN2016-75279-P and IFI16/00033 (ISCIII) of Spain Ministry of Economy, Industry and Competitiveness, and FEDER.

References (41)

  • R. Muñoz-Salinas et al.

    Mapping and localization from planar markers

    Pattern Recogn.

    (January 2018)
  • S. Garrido-Jurado et al.

    Generation of fiducial marker dictionaries using mixed integer linear programming

    Pattern Recogn.

    (2016)
  • D. Johnston et al.

    Real-time positioning for augmented reality on a custom parallel machine

    Image Vis. Comput.

    (2005)
  • J.L. Sanchez-Lopez et al.

    A reliable open-source system architecture for the fast designing and prototyping of autonomous multi-UAV systems: simulation and experimentation

    J. Intell. Robot. Syst.

    (2015)
  • M. Olivares-Mendez et al.

    Vision based fuzzy control autonomous landing with UAVs: from V-REP to real experiments

  • S. Pflugi et al.

    Augmented marker tracking for peri-acetabular osteotomy surgery

  • J.P. Lima et al.

    Markerless tracking system for augmented reality in the automotive industry

    Expert Syst. Appl.

    (2017)
  • S. Khattak et al.

    A real-time reconstructed 3D environment augmented with virtual objects rendered with correct occlusion

  • J. Engel et al.

    LSD-SLAM: Large-Scale Direct Monocular SLAM

    (2014)
  • R. Mur-Artal et al.

    ORB-SLAM: a versatile and accurate monocular SLAM system

    IEEE Trans. Robot.

    (2015)
  • Cited by (0)

    This paper has been recommended for acceptance by Luis Merino.

    View full text