melodic noetic   Show EOL distros: 

Overview

This package wraps an object tracker provided by the ViSP library. The tracker provides the tracked object position with respect to the camera. It requires the object 3d model and the object initial pose.

The package is composed of one node called tracker that does the tracking and two additional nodes client (that allows to set the initial pose by user mouse click) and viewer that allows to visualize the result of the tracking.

The tracker node tries to track the object as fast as possible but needs to be initialized using the client. The viewer can be used to monitor the tracking result.

Use cases

• Optical Localization for Robot Arms, Initial Experiments (Edinburgh Hacklab)

• Augmented reality, object grasping by Romeo humanoid robot, satellite tracking moving on a geosynchronous orbit, ...

Reference

• A.I. Comport, E. Marchand, M. Pressigout, F. Chaumette. Real-time markerless tracking for augmented reality: the virtual visual servoing framework. IEEE Trans. on Visualization and Computer Graphics, 12(4):615-628, July 2006

Roadmap

The package implementation is finished. Minor changes in the API may happen in the future but impact should be limited.

These features would be valuable contribution to this package:

• Rely on AssImp to support COLLADA models (instead of ViSP based VRML loader).

• Use RViz markers and API to support display directly in RViz.

Installation

visp_tracker is part of vision_visp stack.

• To install visp_tracker package run

  sudo apt-get install ros-$ROS_DISTRO-visp-tracker

• Or to install the complete stack run

  sudo apt-get install ros-$ROS_DISTRO-vision-visp

Features

This package objective is to provide the pose of an object (i.e. position and orientation) in a sequence of images knowing its 3d model and its initial pose. It also provides the pose covariance.

ROS API

Note: data is only published if there is at least one listener.

tracker

Subscribed Topics

• Camera calibration information, see the camera_calibration package for more details.

• Rectified image, see the camera_calibration package for more details.

• Estimation of the tracked object position. This is optional and allows automatic tracker restart if the tracking fails.

Published Topics

• Tracking object pose (stamped).

• Tracking object pose with its associated covariance (stamped). You may want to use rviz_plugin_covariance to display this datatype in rviz (Fuerte+).

• Moving edge sites information (stamped). For debugging/monitoring purpose.

• Position and id of the keypoints (stamped). For debugging/monitoring purpose.

Services

• Initialize the tracking by providing the model name, model path, initial object pose and moving edge settings.

• Provides the current camera prefix, model name and model path.

Parameters

• Camera topics namespace ('image_rect' and 'camera_info' topics must be published in this namespace)

• What proportion of points should be valid to acccept an initial pose.

• Gain used to compute the control law.

• Mask size (in pixel) used to compute the image gradient and determine the object contour.

• Minimum allowed samplestep. Useful to specify a lower bound when the samplestep is changed.

• Model name (without the .wrl filename extension)

• Model database path. Uses the resource_retriever syntax to fetch remote models (i.e. package://, http://, etc.)

• Minimum image contrast allowed to detect a contour.

• Maximum image contrast allowed to detect a contour.

• Number of masks applied to determine the object contour.

• How many discretization points are used to track the feature.

• Maximum seek distance on both sides of the reference pixel.

• Minimum distance in pixel between two discretization points.

• How many pixels are ignored around the borders.

• Value used to determine if a moving edge is valid or not.

• Ignored.

• Ignored.

• If set to true, the robot movement will be used to help reliable tracking (experimental). For this feature to work, tf must provide the transformation between the world frame id and the camera.

• World frame name (see ~compensate_robot_motion).

Required tf Transforms

• Robot movement (optional, see ~compensate_robot_motion).

Provided tf Transforms

• Object position w.r.t. the camera.

client

Parameters

• Camera topics namespace ('image_rect' and 'camera_info' topics must be published in this namespace)

• Namespace in which the tracker has been launched.

• Model location on the filesystem.

• Model name. This parameter is mandatory when launching the client.

viewer

Parameters

• Namespace in which the tracker has been launched.

• image_transport plug-in to be used.

Nodelets

This package provides both nodes and nodelet versions of the tracker, client and viewer.

They are respectively named:

• visp_tracker/Tracker

• visp_tracker/TrackerClient

• visp_tracker/TrackerViewer

...and provide the same API than the equivalent nodes.

When the nodelet version is used, no-copy intraprocess publishing is used. See the nodelet documentation for more information.

An example launch file is also provided:

roslaunch visp_tracker tutorial-nodelet.launch

Note: even if a nodelet is available for the client and viewer, one should avoid running them in the same process than the vision pipeline to avoid crashing the whole pipeline if something goes wrong.

Launch file example

Here is a launch file which starts the tracker and client.

   1 <launch>
   2   <!-- you should first setup your camera grabbing/rectification nodes,
   3        see image_pipeline documentation. -->
   4 
   5   <!-- tracking node -->
   6   <node pkg="visp_tracker" type="tracker" name="tracker_mbt">
   7     <param name="camera_prefix" value="/wide_left/camera" />
   8   </node>
   9 
  10   <!-- starts the client while setting the camera input and the model
  11        which will be tracked -->
  12   <node pkg="visp_tracker" type="client" name="tracker_mbt_client">
  13     <param name="camera_prefix" value="/wide_left/camera" />
  14     <param name="model_name" value="laas-box" />
  15   </node>
  16 </launch>

   1 <launch>
   2   <!-- you should first setup your camera grabbing/rectification nodes,
   3        see image_pipeline documentation. -->
   4 
   5   <!-- tracking node -->
   6   <node pkg="visp_tracker" type="tracker" name="tracker_mbt">
   7     <param name="camera_prefix" value="/wide_left/camera" />
   8     <param name="tracker_type" value="mbt+klt" />
   9   </node>
  10 
  11   <!-- starts the client while setting the camera input and the model
  12        which will be tracked -->
  13   <node pkg="visp_tracker" type="visp_tracker_client" name="visp_tracker_mbt_client">
  14     <param name="camera_prefix" value="/wide_left/camera" />
  15     <param name="model_name" value="laas-box" />
  16     <param name="tracker_type" value="mbt+klt" />
  17   </node>
  18 </launch>

Report a bug

Use GitHub to report a bug or submit an enhancement.