WIP: Pointing Gesture Recognition (and Gesture Classification)

[af-a]

Summary of changes

This pull request addresses introduces a recognize_gesture_action action, which involves two stages:

• Gesture classification: classifying an observed gesture according to a set of known gesture examples (based on the method presented in this paper)
• Gesture processing: reasoning about the classified gesture; for example, determining the object being pointed to after observing a pointing gesture (Note: so far the processing of only this gesture has been implemented).

The primary inspiration for this action is the robocup "What is that" task, which is handled within the action by processing a pointing gesture. This processing involves finding the object most likely being pointed to by a person (see the functionalities in PointingGestureRecognizer). The approach relies on the openpose library for determining the pose of the pointing person, and the native fasterrcnn_resnet50_fpn Pytorch object detector. In addition, the gesture classifier relies on the fastdtw library.

Note: The GestureClassifier is particularly still a work in progress, and is subject to further improvements.

TODO before merging

• Test the pointing gesture recognizer with more people and objects

[Sushant-Chavan]

@AhmedFaisal95 Can you mark this as draft PR? I think just using a WIP doesn't help with skipping the CI.

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
6 Code Smells

0.0% Coverage
0.0% Duplication