Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

I think the adjusted dprime formula is wrong. #113

Closed
ricardoV94 opened this issue Nov 7, 2019 · 2 comments
Closed

I think the adjusted dprime formula is wrong. #113

ricardoV94 opened this issue Nov 7, 2019 · 2 comments

Comments

@ricardoV94
Copy link

psycho.R/R/dprime.R

Lines 77 to 79 in 611e984

# Adjusted ratios
hit_rate_adjusted <- (n_hit + 0.5) / ((n_hit + 0.5) + n_miss + 1)
fa_rate_adjusted <- (n_fa + 0.5) / ((n_fa + 0.5) + n_cr + 1)

This potential issue has been raised before here: neuropsychology/neuropsychology.R#10

I think the correct formula is:

hit_rate_adjusted <- (n_hit + 0.5)/(n_hit + n_miss + 1)
fa_rate_adjusted <- (n_fa + 0.5)/(n_fa + n_cr + 1)

The confusion, I think, comes from the wording of the referenced paper (Hautus, 1995, link):

A method that has found much use in log-linear analysis requires the addition of 0.5 to each cell in the two-by-two contingency table that defines the performance of the observer (e.g., Fienberg, 1980; Goodman, 1970; Knoke & Burke, 1980). Row and column totals are increased by one.

What I think the author meant is that if you are using the row or column totals in your formula, then 1 should be add to those (and not in addition to the 0.5 to each cell)

hit_rate_adjusted <- (n_hit + 0.5)/(n_signal_trials + 1)

This I think would be the formula implied by (Stanislaw & Todorov, 1999, link):

In the discussion below, H is used to indicate the hit rate. This rate is found by dividing the number of hits by the total number of signal trials. Similarly, the false-alarm rate, F , is found by dividing the number of false alarms by the total number of noise trials.

A third approach, dubbed loglinear , involves adding 0.5 to both the number of hits and the number of false alarms and adding 1 to both the number of signal trials and the number of noise trials, before calculating the hit and false-alarm rates. This seems to work reasonably well (Hautus,
1995). Advocates of the loglinear approach recommend using it regardless of whether or not extreme rates are obtained.

I tracked down the first reference by Hautus, (Fienberg, 1980, p.64):

Still, ''too many" sampling zeros in the body of a table may create a problem where a marginal table to be fitted in the model contains zero cells. Two basic alternatives are possible: (1) add a small value to every cell in the body of the table, including those with nonzero frequencies. A value of .5 is often suggested (Goodman, 1970: 229). (This is a conservative procedure which will tend to underestimate effect parameters and their significance.) Or (2) arbitrarily define zero divided by zero to be zero (Fienberg, 1977: 109).

Note that they don't mention an extra 1 should be added (this is a mere consequence of adding .5 + .5 to the two cells in a row or column.

I am not totally confident of my interpretation, but I could not find a convincing source for the formula used in the library either...

@ricardoV94 ricardoV94 changed the title I think adjusted dprime formula is wrong. I think the adjusted dprime formula is wrong. Nov 7, 2019
@DominiqueMakowski
Copy link
Member

DominiqueMakowski commented Nov 11, 2019

Hey @ricardoV94,

yes, this is an issue I've been investigating since some time, as I've seen and had conflicting feedback about that formula. Unfortunately, Hautus' paper isn't crystal clear regarding it. But indeed I agree with your conclusion (thanks for the thorough investigation!), I'll update that formula soon :)

@DominiqueMakowski
Copy link
Member

psycho.R/R/dprime.R

Lines 79 to 81 in f3b614c

# Adjusted ratios
hit_rate_adjusted <- (n_hit + 0.5)/(n_hit + n_miss + 1)
fa_rate_adjusted <- (n_fa + 0.5)/(n_fa + n_cr + 1)

This has been fixed in the latest version ☺️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants