==== ====
The sanction which the Society gives to the work, now published under its auspices, extend only to the novelty, ingenuity, or importance of the several memoirs which it contains. Responsibility concernig the truth of facts, the soundness of reason, in the accuracy of calculations is wholly disclaimed: and must rest alone, on the knowledge, judgement, or ability of the authors who have respectfully furnished such communications.
\only<2>{ \vskip .2in \hskip .5in {\it The Memoirs of the Literary Society of Machester, !1785! } }
=! Are we far from this in 21st century? !=
==== Duke controversy ====
- <2-> Very prominent research lab. Huge grants.
- <3-> Famous work on Gene Wide Association Studies
- <4-> Published over 20 papers. Prestigious journals.
- <5-> However, !nobody could reproduce!.
- <6-> Fallout *# <7-> !Retraction of papers! *# <8-> !$1 Million! grant returned *# <9-> !Resignations! *# <10-> !Investigation still ongoing!
==== What is most downloaded paper from PLoS Medicine ? ====
\only<2>{ "Why Most Published Research Findings Are False", Ioannidis, John P. A., 2005 }
=! Do we have this problem at UMBC? !=
==== Do we have this problem at UMBC? ====
Qoutes heard from fellow students:
{\vskip .1in}
\only<1->{ A: You run your calculations on the data in this file. Where do these numbers come from?
B: ''I extracted them from the main dataset using the shell commands.'' \vskip .1in }
\only<2->{ A: So, which commands did you use to extract the data?
B: !I don't remeber!.
}
==== Do we have this problem at UMBC? ====
Qoutes heard from fellow students:
{\vskip .1in}
I thought I was done with my thesis, but my advisor wants me to re-do the same experiments on 4 more datasets. I can't believe he is asking that. That's like several months of work. I'll never be done.
==== Do we have this problem at UMBC? ====
Qoutes heard from fellow students:
{\vskip .2in}
P: So here is the dataset, I copied everything to the public directory on our cluster. Here are the genetic profile data and here is the file with the class labels.
{\vskip .1in}
Q: So, how do you know which labels go with which profiles?
{\vskip .1in}
P: Ahh, sorry forgot the mapping file, !let me email it to you!.
==== Do we have this problem at UMBC? ====
Qoutes heard from fellow students:
{\vskip .2in}
P: Here is a great tool some former students wrote few years ago. Why don't you run it again to get the fresh data for current research.
{\vskip .1in}
A: Sure, how hard could this be.
{\vskip .1in}
A: Ooops, it !doesn't compile!. The server !can't initialize the databse!, SQL scripts are !failing!.
B: I don't think we can get it running any time soon.
==== Modern quote ====
In my own experience, error is ubiquitous in scientific computing, and one needs to work very diligently and energetically to eliminate it. One needs a very clear idea of what has been done in order to know where to look for likely sources of error. I often cannot really be sure what a student or colleague has done from his/her own presentation, and in fact often his/her description does not agree with my own understanding of what has been done, once I look carefully at the scripts. Actually, I find that researchers quite generally forget what they have done and misrepresent their computations.
\vskip .2in \hskip .1in David L. Donoho (2010). ''An invitation to reproducible computational research.'' Biostatistics Volume 11, Issue 3 Pp. 385-388