Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fastRP results for the new EP task #1

Open
Knorreman opened this issue Aug 30, 2024 · 2 comments
Open

fastRP results for the new EP task #1

Knorreman opened this issue Aug 30, 2024 · 2 comments

Comments

@Knorreman
Copy link

I am back again!
Running the eval script from this repo using the same files from EQTPartners/CompanyKG#1 (comment)
Looks like they are overall competitive results except for eGraphMAE for simcse, ada2 and pause.

base weights ep_overall_auc et2 et3 et4 et5 et8 et9 et14 et15
msBERT [1.0] 83.78% 75.46%% 83.97% 79.58% 94.34% 93.77% 80.03% 80.17% 82.94%
msBERT [1.0, 1.0] 84.11% 78.68%% 83.25% 80.08% 94.33% 93.64% 79.29% 80.05% 83.56%
msBERT [1.0, 1.0, 0.25] 83.74% 76.99% 82.61% 79.59% 94.17% 93.69% 79.01% 79.54% 84.34%
ada2 [1.0] 83.96% 77.48% 83.8% 79.9% 94.29% 94.58% 79.19% 80.50% 81.9%
ada2 [1.0, 1.0] 84.26% 78.1% 85.2% 79.3% 93.7% 94.37% 80.1% 79.85% 83.3%
simcse [1.0] 82.96% 77.18% 82.85% 77.66% 93.76% 93.17% 77.91% 77.66% 83.47%
simcse [1.0, 1.0] 83.29% 78.12% 82.63% 78.39% 93.79% 93.20% 78.89% 77.45% 83.89%
pause [1.0] 78.24% 78.76% 63.06% 77.54% 91.54% 87.82% 70.44% 76.58% 80.17%
pause [1.0, 1.0] 78.58% 77.87%% 61.12% 78.49% 91.51% 89.20% 71.63% 77.54% 81.12%
@caolele
Copy link
Member

caolele commented Sep 23, 2024

Hi Thanks for the contribution! Yes, the results you provided seems quite competitive on the EP task! //Lele

@Knorreman
Copy link
Author

No problem! I hope it is of interest :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants