Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Face Detection Depth Fix #130

Merged
merged 5 commits into from
Nov 14, 2023
Merged

Conversation

taylorkf
Copy link
Contributor

@taylorkf taylorkf commented Nov 14, 2023

Description

This PR addresses issue #67. Often, the fork and/or food is positioned in front of the user's mouth while face detection runs. This can lead to the camera detecting the depth of the fork rather than the user's mouth. This pull request updates the depth perception in the following ways:

  • Uses the octomap depth image, which already filters out (most) pixels that are fork/food.
  • Adds a fallback plane-fitting approach, that fits a plane to all the internal face points and then finds the depth of that plane at the stomion of the mouth. That way, even if mouth pixels are obscured, enough internal face points should exist to allow this approach to work.

Testing procedure

Test face detection in isolation:

  • ros2 launch ada_feeding_perception ada_feeding_perception.launch.py
  • ros2 service call /toggle_face_detection std_srvs/srv/SetBool "{data: true}"
  • ros2 topic echo /face_detection
  • Display face_detection_img in RVIZ.
  • Move the face around, including in locations where the mouth is obstructed by the foos/fork. Verify that the perceived (x,y,z) of the stomion is stable and makes sense.

Test face detection with the web app:

  • Launch the web app the real robot nodes as documented in the README.
  • Go through an entire bite, verify that everything works as expected.

Before opening a pull request

  • Format your code using black formatter python3 -m black .
  • Run your code through pylint and address all warnings/errors. The only warnings that are acceptable to not address is TODOs that should be addressed in a future PR. From the top-level ada_feeding directory, run: pylint --recursive=y --rcfile=.pylintrc ..

Before Merging

  • Squash & Merge

@amalnanavati amalnanavati changed the title Taylorkf/face detection fork fix Face Detection Depth Fix Nov 14, 2023
@amalnanavati amalnanavati merged commit 48aeced into ros2-devel Nov 14, 2023
@amalnanavati amalnanavati deleted the taylorkf/face-detection-fork-fix branch November 14, 2023 03:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants