Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HITL - Handle task metrics. #2024

Merged
merged 5 commits into from
Aug 21, 2024
Merged

HITL - Handle task metrics. #2024

merged 5 commits into from
Aug 21, 2024

Conversation

0mdc
Copy link
Contributor

@0mdc 0mdc commented Aug 20, 2024

Motivation and Context

This changeset adds handling of task metrics:

  • Add a thin wrapper around common metrics to access them safely.
  • Use task_percent_complete for defining episode success.
  • Log task_percent_complete and task_explanation.

How Has This Been Tested

Tested on HITL applications.

Types of changes

  • [Development]

Checklist

  • My code follows the code style of this project.
  • I have updated the documentation if required.
  • I have read the CONTRIBUTING document.
  • I have completed my CLA (see CONTRIBUTING)
  • I have added tests to cover my changes if required.

@facebook-github-bot facebook-github-bot added the CLA Signed Do not delete this pull request or issue due to inactivity. label Aug 20, 2024
eundersander
eundersander approved these changes Aug 21, 2024
View reviewed changes
Copy link
Contributor

@eundersander eundersander left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left some comments otherwise LGTM!

examples/hitl/rearrange_v2/metrics.py
from habitat_hitl.app_states.app_service import AppService


class Metrics:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO a plural word is a bad choice for a class name. E.g. what if one day you need a list of these, what would you call that variable? metricses?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can rename the class.

I will do this separately as a refactor pass.

examples/hitl/rearrange_v2/rearrange_v2.py Outdated Show resolved Hide resolved
examples/hitl/rearrange_v2/rearrange_v2.py Show resolved Hide resolved
@0mdc 0mdc merged commit 82448a7 into main Aug 21, 2024
1 of 3 checks passed
@0mdc 0mdc deleted the 0mdc/hitl_metrics branch August 21, 2024 18:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@eundersander eundersander eundersander approved these changes

@aclegg3 aclegg3 Awaiting requested review from aclegg3

@zephirefaith zephirefaith Awaiting requested review from zephirefaith

Assignees
No one assigned
Labels
CLA Signed Do not delete this pull request or issue due to inactivity.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@0mdc @eundersander @facebook-github-bot