Only released in EOL distros:   electric fuerte groovy

INSTALL

To be able to compile it, rosjava_jni is needed here (although rosjava can be considered in future due to the fact that rosjava_jni is unsupported in future).

Also, JAVA_HOME should be set. Put the line below in .bashrc (if under Ubuntu 11.04. For other systems, the path should be changed accordingly).

export JAVA_HOME=/usr/lib/jvm/java-6-openjdk

Environment variable of ROBOT_ENV must be set properly, as it will be read by the program to decide which rdf file to be loaded. With the default simulation

export ROBOT_ENV=ipa-kitchen

ROS API

Implementation of ros service for a semantic database (SDB) using Java (rosjava_jni) and Jena/pellet.

knowledge_srs_node

Services

• When there is a new task, send this request. A new session id will be generated used for following action generation

• To get the next action that the robot needs to execute

• To send a sparql query directly to the database and return the result in JSON format (mainly for testing purpose)

• To query in semantic Database (SDB) for all movable or graspable objects, such as MilkBox0. All object names are unique and case sensitive as defined in the owl file.

• To query in SDB for all workspace or furniture objects, such as table, fridge. All object names are unique and case sensitive as defined in the owl file

• To query in SDB for all rooms. All names are unique and case sensitive as defined in the owl file

• To query in SDB for all poses that are predefined, such as home, charging_position, etc.

• To query in SDB for all possible workspaces or furniture pieces that could store a particular object. E.g. book_shelf and desk could be the workspace for a book.

Retrieving information from knowledge base

Here are some examples of calling the above /get_things_info services.

The knowledge server reads environment information from a parameter server at /srs/language_short. The purpose is to let UIs have readable information in different languages, rather than the object names in the RDF files. Details can be referred to the service definitions, where readableNames[] contains corresponding readable information.

Example of service /get_objects_on_map

Set the language as English

$ roscd srs_environments/param
$ rosparam load dm_simple_english.yaml

To call the service,

$ rosservice call /get_objects_on_map ipa-kitchen false  
                                ## false to indicate that the geometry information is not returned 
objects: ['MilkBox0', 'Salt0', 'Pringle0', 'Bottle0']
classesOfObjects: ['Milkbox', 'Salt', 'Pringles', 'Bottle']
spatialRelation: ['spatiallyRelated', 'NA', 'spatiallyRelated', 'NA']
spatialRelatedObject: ['Dishwasher0', 'NA', 'Table0', 'NA']
houseHoldId: ['9', '11', '10', '12']
objectsInfo: []
readableNames: ['Milkbox', 'Salt', 'Crisps (Brand: Pringles)', 'Bottle']

When the language is Italian (translation is made with Google Translate, hence the accuracy is not guaranteed.):

$ rosparam load dm_simple_italian.yaml 
$ rosservice call /get_objects_on_map ipa-kitchen false
objects: ['MilkBox0', 'Salt0', 'Pringle0', 'Bottle0']
classesOfObjects: ['Milkbox', 'Salt', 'Pringles', 'Bottle']
spatialRelation: ['spatiallyRelated', 'NA', 'spatiallyRelated', 'NA']
spatialRelatedObject: ['Dishwasher0', 'NA', 'Table0', 'NA']
houseHoldId: ['9', '11', '10', '12']
objectsInfo: []
readableNames: ['Latte', 'Sale', 'patatine (Pringles)', 'bottiglia']

Example of /get_workspace_on_map

$ rosservice call /get_workspace_on_map ipa-kitchen false

objects: ['Table0', 'Oven0', 'Stove0', 'Fridge0', 'Sofa0', 'Sink0', 'Dishwasher0']
classesOfObjects: ['Table-PieceOfFurniture', 'Oven', 'StoveTop', 'Refrigerator-Freezer', 'Sofa-PieceOfFurniture', 'Sink', 'Dishwasher']
objectsInfo: []
houseHoldId: ['7', '1', '3', '4', '5', '2', '6']

    #the related id in HHDB (-1 as the default value showing there is no associated model in HHDB).

readableNames: ['Kitchen Table', 'Oven', 'Stove', 'Fridge', 'Sofa in Living room', 'Sink', 'Dishwasher']
json_properties: ['{"insideOf":"ipa-kitchen"}', '{"insideOf":"ipa-kitchen"}', '{"insideOf":"ipa-kitchen"}', '{"insideOf":"ipa-kitchen"}', '{"insideOf":"ipa-kitchen"}', '{"insideOf":"ipa-kitchen"}', '{"insideOf":"ipa-kitchen"}']

### {"insideOf":"ipa-kitchen"} is in the JSON format to indicate that the corresponding workspace is inside of the room ipa-kitchen. 

• Coordinates for furniture pieces from IPA kitchen can be also retrieved by setting ifGeometryInfo as true in the service call

    $ rosservice call /get_workspace_on_map ipa-kitchen true

• Additional information, such as 2D projection of the furnitures information should be retrieved from HHDB.

Example of /get_rooms_on_map

$ rosservice call /get_rooms_on_map ipa-kitchen false

rooms: ['LivingRoom0', 'ipa-kitchen']
roomsInfo: []

Example of /get_predefined_poses

To get all predefined positions and their readable/meaningful names

$ rosservice call /get_predefined_poses ipa-kitchen

locations: ['kitchen_backwards', 'ChargingStation0', 'order', 'new_kitchen', 'home', 'kitchen']
poses: 
  - 
    x: -2.03999996185
    y: -0.300000011921
    theta: 3.1400001049
  - 
    x: 1.0
    y: -1.60000002384
    theta: 1.57000005245
  - 
    x: 1.47000002861
    y: -0.699999988079
    theta: 0.75
  - 
    x: -2.1400001049
    y: 0.0
    theta: 0.0
  - 
    x: 0.0
    y: 0.0
    theta: 0.0
  - 
    x: -2.03999996185
    y: 0.300000011921
    theta: 0.0
readableNames: ['Kitchen (Backward)', 'Charging Station', 'User (Order Position)', 'Kitchen ', 'Robot Home Position', 'IPA Kitchen']
json_properties: ['{"insideOf":"ipa-kitchen"}', '{"insideOf":"ipa-kitchen"}', '{"insideOf":"ipa-kitchen"}', '{"insideOf":"ipa-kitchen"}', '{"insideOf":"ipa-kitchen"}', '{"insideOf":"ipa-kitchen"}']

Example of /get_workspace_for_object

Get all possible workspaces for a particular object.

Get the types of possible workspaces for a particular object, such as where a Milkbox could be located at.

$ rosservice call /get_workspace_for_object Milkbox 0

workspaces: ['Refrigerator-Freezer', 'Cupboard', 'Dishwasher', 'IkeaShelf', 'Table-PieceOfFurniture']

Or: Get the instances of all possible workspaces for a particular object, such as where a Milkbox could be located at.

$ rosservice call /get_workspace_for_object Milkbox 1

workspaces: ['Dishwasher0', 'Fridge0', 'Table0']

Types of objects

Two types of objects are defined here. Graspable or movable objects, and Workspace or furniture objects.

graspable or movable objects

possible actions:

• move - move to the furniture which holds the object
• grasp - grasp the object

Furniture objects

Similarly, furniture objects are considered not movable (though can be updated in the database as well, but having different actions accordingly)

• There is no direct action for furniture pieces, as currently fetch and carry tasks only focus on graspable objects, such as book, milkbox, etc.
• Workspaces are places where objects are located. The need to distinguish this and graspable objects is to use the semantic relationship between objects, e.g. bookshelf is a workspace of books, and table for milkbox etc.
• To execute a task such as get a milkbox, the robot would search for the object from possible locations, such as table, fridge, and even oven top.
• The user can also specify where the possible workspaces are, to reduce the search space or to improve the efficiency, if the user knows better the environment.

Possible actions

• environment_update - update the pose information of the targeted object
• workspace verification - verify if the workspace exists at the given pose.

Services /task_request and /plan_next_action

These two services are used for task planning to plan the next action the robot should execute.

A python script is prepared

srs_knowledge/src/demoplanaction.py

for the purpose of testing.

Here is a list of some unit tests, simulating different conditions of the robot task execution.

Task: Move

For demostration purpose, part of the test script is copied here.

   1 def test_move():
   2     res = requestNewTaskJSONMove()
   3     sessionId = res.sessionId
   4     acts = list()
   5 
   6     act = planNextActionServiceJSON(sessionId, 0, '')
   7     acts.append(act)
   8 
   9     act = planNextActionServiceJSON(sessionId, 0, '')
  10     acts.append(act)
  11 
  12     return acts
  13 
  14 def planNextActionServiceJSON(sessionId, result, jsonFeedback):
  15     print 'Plan next Action service'
  16     rospy.wait_for_service('plan_next_action')
  17     try:
  18         next_action = rospy.ServiceProxy('plan_next_action', PlanNextAction)
  19         req = PlanNextActionRequest()
  20         req.sessionId = sessionId
  21         req.resultLastAction = result
  22         req.jsonFeedback = jsonFeedback
  23         resp1 = next_action(req)
  24     
  25         return resp1.nextAction
  26     
  27     except rospy.ServiceException, e:
  28         print "Service call failed: %s"%e

To Run it:

$ rosrun srs_knowledge demoplanaction.py 

Request new task
   ### Task details are in JSON
send task request {"time_schedule":1263798000000,"task":"move","destination":{"predefined_pose":"home"}}


### First action to accomplish the task is to “move”
[status: 0

generic: 

  jsonActionInfo: {"action":"move","destination":{"pose2d":{"theta":0.0,"y":0.0,"x":0.0 } } }

actionType: generic, status: 1

## Next action to accomplish the task is “finish_success” (given the above "move" action is completed successfully)

generic: 

  jsonActionInfo: {"action":"finish_success"}

actionType: generic]

• Communication is mainly based on the JSON format, in order to transfer different data types. The protocol of JSON is specified in the following section.

If test_move() is changed to:

   1 def test_move():
   2     res = requestNewTaskJSONMove()
   3     sessionId = res.sessionId
   4     acts = list()
   5 
   6     act = planNextActionServiceJSON(sessionId, 0, '')
   7     acts.append(act)
   8 
   9     act = planNextActionServiceJSON(sessionId, 1, '')   ### last action is not completed successfully
  10     acts.append(act)
  11 
  12     return acts

To Run it:

$ python demoplanaction.py 

Request new task

send task request {"time_schedule":1263798000000,"task":"move","destination":{"predefined_pose":"home"}}

### First action to accomplish the task is to “move”

[status: 0

generic: 

  jsonActionInfo: {"action":"move","destination":{"pose2d":{"theta":0.0,"y":0.0,"x":0.0} } }

actionType: generic, status: -1

### Next action to accomplish the task is “finish_fail” (because the robot could not move to the target)
generic: 

  jsonActionInfo: {"action":"finish_fail"}

actionType: generic]

The test can be performed using the script by changing the parameters accordingly. Here, only successful actions are presented for demonstration purposes to show the action sequence of a Get task.

Task: Get/Fetch

$ python demoplanaction.py 

Test FETCH task

Request new task

send task request {"time_schedule":1263798000000,"task":"get","deliver_destination":{"predefined_pose":"order"},"object":{"object_type":"Milkbox"},"grasping_type":"Simple"}

[status: 0

### Move to target where the target object could be nearby
generic: 

  jsonActionInfo: {"action":"move","destination":{"pose2d":{"theta":0.0,"y":0.0591675355492316,"x":-2.2000000357627867} } }

actionType: generic, status: 0


### Detect the object
generic: 

  jsonActionInfo: {"action":"detect","object":{"object_type":"Milkbox","workspace":"Dishwasher0","object_id":9}}

actionType: generic, status: 0


### Move to a closer position after detected the object
generic: 

  jsonActionInfo: {"action":"move","destination":{"pose2d":{"theta":0.06921684521207505,"y":-0.044910181658103565,"x":-2.208831782585601} } }

actionType: generic, status: 0


### Grasp the object
generic: 

  jsonActionInfo: {"action":"grasp","object":{"object_type":"Milkbox","workspace":"Dishwasher0","object_id":9}}

actionType: generic]

Tasks Fetch and Search are similar to Get (refer to srs_decision_making), hence not listed here. The test clients allow using different combinations of the parameters as input to make sure the correctness of the result. So far, all tests provide correct result for task planning with this unit test approach.

Some missing examples/usages

Usually, this package can be run simply by launching the launch file in the launch file in package srs_scenario.

To start the service individually, run

rosrun srs_knowledge knowledgeEngine

and an example script how to test it

rosrun srs_knowledge demoplanaction.py

or

rosrun srs_knowledge testRosJavaService.py 

Mainly, this package is used with srs_decision_making together.

rosrun srs_decision_making srs_actions_server.py

References: JSON protocol