Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add detection metrics #44

Open
wants to merge 33 commits into
base: master
Choose a base branch
from
Open

Conversation

jvlmdr
Copy link
Contributor

@jvlmdr jvlmdr commented Jun 28, 2021

This PR adds the detection metrics from the MOT Challenge. There are some differences between this implementation and the original implementation. I have outlined them here:
https://docs.google.com/document/d/1UTR4G8va_fe-KsNZhrQJNE5w_7F9fPBVy9G4dKWrBfk/preview

Jack Valmadre added 26 commits March 27, 2021 10:01
Rename to AP_coarse for MOT challenge integral.
Add det_averaged combine functions for detection.
Add MIN_VIS config option.
Add IGNORE_TRACK_IDS for predictions of detectors.
Remove DetLoc (contained only per-sequence AP).
Use pessimistic approach for legacy code.
@jvlmdr
Copy link
Contributor Author

jvlmdr commented Jun 30, 2021

I forgot to mention, the command that I used to compute detection metrics for the MOT17Det benchmark is:

python scripts/run_mot_challenge.py \
  --METRICS Det \
  --BENCHMARK MOT17Det \
  --REQUIRE_UNIQUE_IDS False \
  --MIN_VIS 0.5 \
  --DROP_OTHER_CLASSES True \
  --DROP_EXTRA_TIMESTEPS True

There are several new flags here: MIN_VIS sets the minimum visibility for the ground-truth, REQUIRE_UNIQUE_IDS disables the assertion that track IDs are unique (must be disabled for "Det" challenges), DROP_OTHER_CLASSES removes predictions with class ID > 1 and DROP_EXTRA_TIMESTEPS disables the assertion that frame IDs <= sequence length (ignoring any predictions in extra frames).

Edit: I meant --IGNORE_TRACK_IDS False instead of --REQUIRE_UNIQUE_IDS True as highlighted below in #44 (comment).

@youonlytrackonce
Copy link

Hello Jack,

in your branch: https://github.com/jvlmdr/TrackEval/tree/detection-merge
I get the following error message:
"run_mot_challenge.py: error: unrecognized arguments: --REQUIRE_UNIQUE_IDS False"

Would omitting this flag effect the results?

Thank you.

@jvlmdr
Copy link
Contributor Author

jvlmdr commented Apr 7, 2022

Hey! My mistake. I forgot that I inverted the sense of this flag from --REQUIRE_UNIQUE_IDS False to --IGNORE_TRACK_IDS True. (And it should only be necessary to do this for MOT17Det where the track IDs are unused; it's not required for MOT17.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants