Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Object Hand Over Action #227

Merged
merged 42 commits into from
Oct 24, 2020
Merged
Show file tree
Hide file tree
Changes from 39 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
3b12738
[hand_over_action] Add initial action implementation and parameter files
af-a Jul 19, 2019
9c21e4d
Added a demo for context-aware item hand over
alex-mitrevski Jul 19, 2019
f023cb0
[demo_context_aware_hand_over/states/hand_over] Fixed incorrect actio…
alex-mitrevski Jul 19, 2019
0d012ac
[demo_context_aware_hand_over/sm] Updated the bb ratios for the diffe…
alex-mitrevski Jul 19, 2019
b753fcc
[hand_over_action] Correct dmp weight files (RPY vector sizes)
af-a Jul 20, 2019
1998d8b
[hand_over_action] Change policy param pickle files to python 2 format
af-a Jul 20, 2019
642c738
[hand_over_action] Fix action params: config file paths
af-a Jul 20, 2019
6fd4fc3
[hand_over_action] Change argument parsing in client test file
af-a Jul 20, 2019
bd583bf
[hand_over_action] Add force sensing for object reception (testing)
af-a Jul 20, 2019
6b9ca3b
Merge remote-tracking branch 'origin/feature/hand-over-action' into f…
af-a Jul 20, 2019
f607f43
[hand_over_action] Implement change detection for object reception
af-a Jul 20, 2019
6cf8b05
[demo_context_aware_hand_over] Add Alex's fixes to scenario files
af-a Jul 20, 2019
3ab4eec
[hand_over_action] Extend object reception to check both dims: x and z
af-a Jul 20, 2019
9cc5457
[hand_over_action] Add choice for context-awareness in action & scenario
af-a Jul 20, 2019
f6ac96d
[hand_over_action] Remove explicit vocalization of detected posture
af-a Jul 22, 2019
4b2e4a5
[hand_over_action] Minor modifications (debugging outputs, fixes, par…
mas-hsr Oct 11, 2019
1a99ec9
[context_aware_hand_over/identify_posture] Adapted the Person msg use…
mas-hsr Mar 12, 2020
e75cd6e
[context_aware_hand_over/identify_posture] The object is now handed o…
mas-hsr Mar 12, 2020
4fba100
[hand-over-action] Remove unnecessary learned policy parameter files
af-a Mar 12, 2020
9acbea2
[hand-over-action] Add boolean release detection parameter, and some …
af-a Mar 12, 2020
7162177
Merge branch 'feature/hand-over-action' of github.com:b-it-bots/mas_d…
af-a Mar 12, 2020
050c415
[hand-over-action] Fix incorrect release_detection variable naming
af-a Mar 12, 2020
b66882f
[hand-over-action] Fix incorrect result when object is released witho…
af-a Mar 12, 2020
ccbff68
[demo_context_aware_hand_over] Included the newly added 'release_dete…
mas-hsr Mar 12, 2020
7546bcb
[hand_over_action] Switch to robot-independent topics and packages
af-a Mar 27, 2020
575f8b7
[hand_over_action] Update README with more detailed description
af-a Apr 1, 2020
d0d406f
[hand_over_action] move arm to neutral position in init state (otherw…
sthoduka May 27, 2020
dc8f546
Merge branch 'devel' into feature/hand-over-action
alex-mitrevski Sep 10, 2020
793b07e
Add point cloud filtering to improve person pose estimation
af-a Sep 18, 2020
942c1e4
Utilize person pose for pre-approach (in action and scenario)
af-a Sep 25, 2020
6a36dae
[demo_hand_over] Add transition to detect_people if identify_posture …
af-a Sep 25, 2020
5588aae
[hand_over_action] Fixed a small bug due to which the arm wasn't movi…
alex-mitrevski Oct 2, 2020
a2ea35d
[hand_over_action] Add force sensor topic
af-a Oct 23, 2020
c516eba
[hand_over_action] Update action client script
af-a Oct 23, 2020
9cac5d5
[hand_over_action] Modify release detection strategy to use force rea…
af-a Oct 23, 2020
30b7b6b
[hand_over_action] Remove hsr-specific topics
af-a Oct 23, 2020
6f983fb
[hand_over_action] Remove specification of post message timestamp
af-a Oct 23, 2020
20bffbe
[hand_over_action] Remove unnecessary comments and add move_base goal…
af-a Oct 23, 2020
503fb7f
Merge branch 'feature/hand-over-action' of github.com:b-it-bots/mas_d…
af-a Oct 23, 2020
452c62d
Merge branch 'devel' of github.com:b-it-bots/mas_domestic_robotics in…
alex-mitrevski Oct 23, 2020
c06ed07
[hand_over_action/action_states] Resolved some Codacy complaints
alex-mitrevski Oct 23, 2020
6b83fe9
[find_people/src/find_people] Removed unnecessary line
alex-mitrevski Oct 23, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
cmake_minimum_required(VERSION 2.8.3)
project(mdr_hand_over_action)

find_package(catkin REQUIRED COMPONENTS
rospy
roslint
actionlib
actionlib_msgs
genmsg
message_generation
geometry_msgs
)

roslint_python()
catkin_python_setup()

add_action_files(DIRECTORY ros/action
FILES
HandOver.action
)

generate_messages(
DEPENDENCIES
actionlib_msgs
geometry_msgs
)

catkin_package(
CATKIN_DEPENDS
rospy
actionlib
actionlib_msgs
message_runtime
geometry_msgs
)

include_directories(
${catkin_INCLUDE_DIRS}
)

install(PROGRAMS
ros/scripts/hand_over_action
ros/scripts/hand_over_action_client_test
DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION}/scripts
)
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# mdr_hand_over_action

An action for handing a grasped object over to a person. It is intended to be adaptive to different contexts of the task, such as the receiver's posture, the presence of an intermediate obstacle, receiver attributes, etc.

Currently, the action is only adaptive to postural context, through a hand-over position selection policy acquired using reinforcement learning.

The context-aware object hand-over demo scenario demonstrates the intended workflow of the action, including:
* Detecting a person
* Determining their posture
* Handing a grasped object over

## Action definition

### Goal:

* ``string posture_type``: The receiver's posture: standing up, seated, or lying down
* ``bool obstacle``: Indicates whether an obstacle is present between the robot and the receiver
* ``bool release_detection``: Determines whether a force sensor-based object reception detection strategy is used
* ``bool context_aware``: Determines whether knowledge about context is utilized to adapt execution

### Result:

* ``bool success``

### Feedback:

* ``string current_state``
* ``string message``

## Directory structure

```
mdr_pickup_action
| CMakeLists.txt
| package.xml
| setup.py
| README.md
|____config
| |____learned_policy_parameters
| | |_____learned_position_policy_parameters.pkl
| | |
| |____trajectory_weights
| | grasp.yaml
| | learned_obstacle_avoiding_weights.yaml
| | learned_smoothed_corrected_obstacle_avoiding_weights.yaml
| | learned_smoothed_corrected_underhand_weights.yaml
| |_____learned_smoothed_obstacle_avoiding_weights
|____ros
|____action
| |_____HandOver.action
| |
|____launch
| |_____hand_over.launch
| |
|____scripts
| | hand_over_action
| |_____hand_over_client_test
| |
|____src
|____mdr_hand_over_action
| __init__.py
|____action_states.py
```

## Launch file parameters

### Action server

The following parameters may be passed when launching the action server:
* ``move_arm_server``: Name of the `move_arm` action server (default: 'move_arm_server')
* ``gripper_controller_pkg_name``: Name of a package that implements functionalities for controlling a robot's gripper (default: 'mdr_gripper_controller')
* ``init_config_name``: Name of the initial arm configuration for object hand-overs (default: 'neutral')
* ``hand_over_dmp``: Path to a YAML file containing the weights of a dynamic motion primitive used for object hand-overs (default: 'grasp.yaml')
* ``hand_over_position_policy_parameters_file``: Name of a YAML file containing the parameters of a learned context-dependent hand-over position policy (default: 'learned_position_policy_parameters.pkl')
* ``hand_over_policy_config_dir``: Path to a directory containing learned hand-over position policy parameters (default: '')
* ``hand_over_dmp_weights_dir``: Path to a directory containing learned hand-over trajectory shape policy parameters (default: '')
* ``dmp_tau``: The value of the temporal dynamic motion primitive parameter (default: 30)

## Action execution summary

It is assumed that the robot starts with the object to be handed over already in its grasp. The action proceeds through the following steps:
1. The end-effector position at which the object will be handed over is first determined:
- if the ``context_aware`` goal parameter is set to ``True``, a position is sampled from learned policy, given the value of ``posture_type``,
- otherwise, a pre-set, context-independent position is used.
2. The arm is moved to the ``neutral`` configuration.
3. The arm is then moved such that the end-effector ends in the position chosen in step 1 (currently the trajectory to be followed is set to a learned DMP trajectory).
4. If the ``release_detection`` action parameter is set to ``True``, the robot utilizes the wrist force sensor to determine when the person is pulling the object from its grasp with the intent of receiving it, using a signal change detection algorithm, and then releases the object. Otherwise, the robot waits for a short time and simply releases the object.
5. The arm is moved back to the ``neutral`` configuration.

## Dependencies

* ``smach``
* ``smach_ros``
* ``tf``
* ``actionlib``
* ``geometry_msgs``
* ``mdr_move_arm_action``
* ``mdr_move_base_action``

## Example usage

1. Run the ``move_arm`` action server: ``roslaunch mdr_move_arm_action move_arm.launch``
2. Run the action server: ``roslaunch mdr_hand_over_action hand_over.launch``
3. Run the client example: ``rosrun mdr_hand_over_action hand_over_action_client_test ['standing', 0, 1]``, providing a list of the action's parameters (context, obstacle, release_detection) as arguments
Binary file not shown.
Loading