This repository has been archived by the owner on Apr 11, 2023. It is now read-only.

NDCG computation #233

Open

Zamony opened this issue Feb 27, 2021 · 1 comment

Zamony commented Feb 27, 2021

Hi there! In the function corresponding to NDCG calculation I noticed that ignore_rank_of_non_annotated_urls flag is present.

CodeSearchNet/src/relevanceeval.py

Line 76 in 76a006f

ignore_rank_of_non_annotated_urls: bool=True) -> float:

The question is: are your results in the paper calculated with ignore_rank_of_non_annotated_urls=True?

The text was updated successfully, but these errors were encountered:

Alex-HaochenLi commented Sep 10, 2022

Both are reported.
In Table 4, NDCG-Within for True, NDCG-All for False.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.