tag:blogger.com,1999:blog-1405968300258104460.post6222629471070168166..comments2024-03-27T01:18:24.341-04:00Comments on Andy's Brain Blog: The Noose Tightens: Scientific Standards Being RaisedAndrew Jahnhttp://www.blogger.com/profile/16435706598096921650noreply@blogger.comBlogger4125tag:blogger.com,1999:blog-1405968300258104460.post-77524475365113128672013-04-26T13:48:50.579-04:002013-04-26T13:48:50.579-04:00Hey Sherif,
I see no problem with using smaller s...Hey Sherif,<br /><br />I see no problem with using smaller sample sizes, provided that a power analysis shows that a typical effect for your study will probably reject the null most of the time. This is the motivation behind pilot studies and power analyses, although I think that most people tend to omit them; see one of my earlier posts discussing Jeanette Mumford's fmri_power tool in Matlab.<br /><br />This relates to your other question about exploratory analyses restricted by small sample sizes. I don't think they should be discouraged, necessarily, but I do think that researchers need to have some idea about the effect that they expect to find before simply testing any hypothesis with a small sample. Since most null effects from small samples go unreported, the ones that do get reported are often very big; they have to be, in order to reject the null with relatively few subjects. Again, pilot studies could help this problem by giving some idea about whether the question is worth pursuing in the first place.<br /><br />As for statistical knowledge in general, I think it would help if courses examined the statistics used in actual studies, so that students get a better idea about why they are used and in what contexts, and discuss whether the statistics are appropriate in a given paper...but that's just something I've been thinking about in my free time.<br /><br />-AndyAndrew Jahnhttps://www.blogger.com/profile/16435706598096921650noreply@blogger.comtag:blogger.com,1999:blog-1405968300258104460.post-86020749388730231222013-04-25T19:32:06.223-04:002013-04-25T19:32:06.223-04:00Actually, I'd say it's true of scientists ...Actually, I'd say it's true of scientists in general, not just biologists and neuroscientists.Sherifhttps://www.blogger.com/profile/11511264082943121336noreply@blogger.comtag:blogger.com,1999:blog-1405968300258104460.post-3918366871675527212013-04-25T19:31:10.209-04:002013-04-25T19:31:10.209-04:00As for sample sizes, if one can quantitatively jus...As for sample sizes, if one can quantitatively justify a smaller sample size because there is very little expected variability and therefore you can still get good statistical power with the smaller sample size, then I think that should be fine (but what do I know?).<br /><br />On the other hand, what if you want to study something and there are only few samples available in the first place? Should you avoid the inquiry altogether?Sherifhttps://www.blogger.com/profile/11511264082943121336noreply@blogger.comtag:blogger.com,1999:blog-1405968300258104460.post-59381965040783023572013-04-25T19:27:04.621-04:002013-04-25T19:27:04.621-04:00"Too many biologists [and neuroscientists] st..."Too many biologists [and neuroscientists] still do not receive adequate training in statistics and other quantitative aspects of their subject."<br /><br />Couldn't agree more ... I've done well in statistics courses, and I still feel rather clueless. It's the fault of academic institutions that gloss over it as an afterthought ...Sherifhttps://www.blogger.com/profile/11511264082943121336noreply@blogger.com