cv_mech_turk Documentation

cv_mech_turk

A wrapper around the OpenCV HighGUI cv_mech_turk viewer.

Author(s): Alexander Sorokin,Gary Bradski,Alex Teichman
Review Status: unreviewed
Review Notes:
Homepage: None
Dependency Tree

Interface to mechanical turk for image annotation

The package contains tools to upload images in bulk to mechanical turk and get the annotations back. The annotations can be retrieved as messages or as XML files/segmentation masks. The images can be manually selected using interactive tool.

Code API

ROS API

use to document each node separately.

List of nodes:

snapper.py
select_images

snapper.py listens to an image topic and uploads the image to the annotation server

Usage

$   rosrun cv_mech_turk snapper.py [parameters/overrides]

Example

$ rosrun cv_mech_turk snapper.py image:=image_to_annotate

ROS topics

Subscribes to:

"image": [sensor_msgs/Image] the image to submit

ROS parameters

Reads the following parameters from the parameter server

"~server" : [string] the server name to submit to.
"~session" : [string] the session on the server.
"~submission_mode" : [string] the mode of the snapper ('submit' - to submit to the server, 'blocked' (default)- to block submission - for debug)
"~img_dir" : [string] Root folder to store temporary data. The images will be saved in img_dir/images/session folder.
"~image_mode" : [string] Grayscale ('gray') or color images (otherwise).
"~frame" : [string] The frame ID of the image. The annotations will be tagged with the frame ID to simplify subsequent lifting to 3D.

Registers remote publisher in the current master. The publisher information is obtained via HTTP and is updated periodically.

ROS parameters

Reads the following parameters from the parameter server

"~uri" : [string] The URL of the HTTP topic information server.

select_images

select_images listens to an image topic, shows the image and republishes the image when key (s/S/m/M) is pressed

Usage

$   rosrun cv_mech_turk select_images [parameters/overrides]

Example

$ rosrun cv_mech_turk select_images  _image_in:=/forearm/image_rect_color

ROS topics

Subscribes to:

"image_in": [sensor_msgs/Image] the images to display and republish

Publishes:

"image_out": [sensor_msgs/Image] republished images

ROS parameters

Reads the following parameters from the parameter server

"~image_in" : [string] the topic name to listen to. Defaults to /wide_stereo/left/image.
"~image_out" : [string] the topic name to publish selected frames. Defaults to image_to_annotate.
"~key_wait" : [int] How many milliseconds to wait for the key press (depends on required frame rate).

Command-line tools

The package contains tools to download the annotations from the server and convert them to useful types (message/files)

convert_session_results_to_mask

Usage

$ ./convert_session_results_to_mask_name <session_results_dir> <widthxheight> [scale_factor]

Converts XML annotations to segmentation masks. The following directories will be created to hold the results of the session:

<session_results_dir>/CvRects/ 
<session_results_dir>/masks/
<session_results_dir>/annotations/

Example

$ ./bin/convert_session_results_to_mask results/single-object-s 640x480

Downloads annotations from the server and published them as messages.

Usage

$ ./session_2_messages.py [args]

Example

$ ./session_2_messages.py foo bar

Downloads images and annotations from the server (jpg/xml).

Usage

$ ./session_results.py --session=session_name --server=server_name --saveto=path_to_save_data [--size=img_widthXimg_height]

Example

$ ./session_results.py --session=prf-jul-08-L1p --server=vm6.willowgarage.com --saveto=./results/

Publishes the annotations as messages cv_mech_turk/ExternalAnnotation. The messages are published on /annotations_2d topic.

Usage

$ ./session_2_messages.py --session=session_name --server=server_name

Example

$ ./session_2_messages.py --session=prf-jul-08-L1p --server=vm6.willowgarage.com

notes

//SET UP ROS (assuming you've previously downlaoded it)
0.1. svn up in .../ros/ros and .../ros/ros-pkg just to be sure.
0.2. in .../ros/ros, rosmake roscpp_demo (that will rebuild all kinds of crap hat you might need)

//TO GET THE NODE RUNNING:
// -- Do everything all at once:
roslaunch mt.launch [this will run roscore, start the camera with dc1394 and then run ./cv_mturk image:=dc
am/image

// -- Do it piece by piece:
1 RUNNING THE MASTER NDOE:
run roscore. You just open a window and type roscore. [This is the master node]
2 RUNNING A 1394 CAMERA:
Open another window.

2.1. Type "roscd dcam".
2.1a. If not made, then make it by typing: rosmake.
2.2. Then run camera capture by: ./bin/dcam [This reads from the camera and publishes images]
2.3 IF THE CAMERA BOMBS OUT WITH ERRORS, TRY SETTING PERMISSIONS
[ You might have to: sudo chmod 666 /dev and /dev/video1394 ]

3 RUNNING THE MTURK NODE:
roscd cv_mech_turk [If you svn up in ros-pkg, you will get this new package]

3.1. IF YOU NEED TO MAKE THE NODE: rosmake
3.1a. If things don't work, you might have to try "rosapt-get cv_view" and or struggle with loading any
number of dependencies and lots of make cleans in the appropriate packages. Once it builds:
3.2 RUN THE NODE: ./cv_mturk image:=dcam/image [This will run the camera viewer]

4 OTHER NEEDED FILES
When you type "m", cv_mturk will invoke: submit_img.py which in turn depends on:
* id_rsa_SIU [These are access keys]
* id_rsa_SIU.pub
* Needs an image storage subdirectory called: "/images"

5 put id_rsa_SIU and id_rsa_SIU.pub in src/. Note: id_rsa_SIU need to have permission 600 for scp and subse
quent http request to work properly.