Significance

[nyuanshin]

Here is one small example (indirectly via Patri), of a multitude I could choose, of why I'm working on what I'm working on right now: in this paper the authors check how damaged the hearts of marathon runners are compared to relatively sedentary people. Surprise! Their hearts tend to actually be in *worse* shape -- i.e. they show more scarring,

Comments

The Earth Is Round (p < .05)

[anonymous]

That's another favorite of mine along the same lines.

www.projectimplicit.net/nosek/teaching/761/cohen.pdf

I need to give you a ring...

brad v.

Re: The Earth Is Round (p < .05)

[nyuanshin]

Yeah, you shared that with me a while ago, but thanks for reminding me -- that's another good one. :)

I've been negligent in answering my e-mails lately, but yours are still starred in my inbox. If you've got time to give me a call I'm free tonight and all day tomorrow (Tues).

Re: The Earth Is Round (p < .05)

[anonymous]

Yeah, I thought I'd shared that with you...

Anyway, I'll give you a ring tomorrow. When's the best time to call?

Re: The Earth Is Round (p < .05)

[nyuanshin]

Any time from 4-9pm EST works.

[tdj]

Gelman & Stern's paper is a good one; I'll add that to my list. The examples attempt a few pretty bizarre comparisons, but in fairness, so do a lot of people in the literature. Don't bother hypotesting until you're sure you're applying the right test.

We don't have access to that journal, so I have to ask - what values were they comparing in that assay?

[nyuanshin]

They're exploiting the fact that blood tends to pool within scar tissue as a way of detecting it -- pump some gadolinium into them and use MRI see how fast it clears back out again. Scar tissue will retain it longer than healthy tissue (whence "late gadolinium enhancement"), so they just checked for how many in each group had patches of abnormal clearance. I'm given to understand it's pretty much a gold standard technique to look for myocardial scarring.

[tdj]

I'd be more likely to buy the statistical insignificance argument if the results of the test sort easily into two groups - approximately normal clearance or abnormal clearance. More "room" for outliers that way. If there's a gradient of clearances, I've another reason to be happy that I don't run marathons.

[nancylebov]

I would like to know whether marathon runners are actually more likely to have heart attacks, and especially if there's a type of heart attack associated with scarring and marathon runners get more of those.

[nyuanshin]

I should have been clearer on this: LGE is itself evidence of an infarction, meaning they've already had (subclinical) heart attacks. Judging by the damage patterns, some were your standard ischemic attacks, but more interesting were the others that showed an atypical pattern more suggestive of microembolisms -- not a full artery blockage, but lots of little blockages in the arterioles of the heart muscle. That latter one seems to suggest something special about the runners, which makes sense -- your clotting factors & such kick into high gear during physical stress. FWIW, four of the runners subsequently had non-fatal heart attacks in the following 21 months of follow-up.

[hypnos7]

Yes, more-interesting/less-probable results are more useful than less-interesting/more-probable results.

Though, you still have to cut bait and run at some point -- that is contextual.

[sthira_sukha]

Hi, I've read your journal for about a year and three-quarters now (though I think it was only last year that I LJ-friended you), and I find it very edifying, to say the least.

I have a relevant question you may be able to readily answer: Sometime last year I encountered an article that reached a conclusion similar to Ioannidis' paper but focused more on the effects of authors referencing other papers. In their analysis, the author(s) apparently (my knowledge of this topic is regrettably limited) employed systems theory to create a network, of sorts, of documents linked by citations and found that it was not uncommon for authors to cite items explicitly described as speculations by the original researchers or even rarely (iirc) hypotheses found unsubstantiated in the very work referenced. It's not the Pfeiffer-Hoffmann article Large-Scale Assessment of the Effect of Popularity on the Reliability of Research, but I do think it's also on PLoS like these other studies. One of the blogs I read covered it last year--I think it was either

[nyuanshin]

You're welcome. :)