Skip to content

ADD Recall metrics#4

Open
Tagaihahimazin wants to merge 3 commits into
mpkato:masterfrom
Tagaihahimazin:master
Open

ADD Recall metrics#4
Tagaihahimazin wants to merge 3 commits into
mpkato:masterfrom
Tagaihahimazin:master

Conversation

@Tagaihahimazin
Copy link
Copy Markdown

Recallを計算するプログラムとそのテストを追加しました.

よろしくお願いします.

Comment thread pyNTCIREVAL/metrics/recall.py Outdated
from .metric import Metric

class Recall(Metric):
def __init__(self, total_positive):
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to take xrelnum as input for __init__ and compute the total_positive in __init__. This would enable the users to take the same actions for all kinds of metrics.

Comment thread pyNTCIREVAL/metrics/recall.py Outdated
self.total_positives = total_positive

def compute(self, labeled_ranked_list):
true_positives = sum(item[1] if item[1] is not None else 0 for item in labeled_ranked_list)
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To clarify what item[1] is, better to write the for-loop as follows:
sum(grade if grade is not None else 0 for docid, grade in labeled_ranked_list)

In addition, grade can be 1 or larger (e.g., 2). Thus,
len(grade for docid, grade in labeled_ranked_list if grade is not None and grade > 0)

Comment thread tests/test_recall.py Outdated
assert total_positive == 3

result = metric.compute(labeled_ranked_list)
assert result == 1.0 No newline at end of file
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please consider the case where recall is not 1.0. Recall=1.0 is a special case.

@Tagaihahimazin
Copy link
Copy Markdown
Author

Tagaihahimazin commented Jul 31, 2023

I've made the requested modifications to the Recall calculation based on your comments.
Please let me know if you have any questions or need further assistance.
Thank you for your valuable feedback.

@Tagaihahimazin
Copy link
Copy Markdown
Author

Added comments to the recall function. Please review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants