This came from a reddit thread about a paper presented in the New England Journal of Medicine in 2012 entitled Chocolate Consumption, Cognitive Function, and Nobel Laureates. The paper needs to be accessed from authorised (aka paid) sources but it was extensively discussed by Stanford professor Sanjay Basu in his public health blog. The NEJM paper claimed that:
chocolate consumption enhances cognitive function
based on the correlation between chocolate consumption and the number of Nobel Laureates in a selected sample of countries (r = 0.791 P < 0.0001). According to the authors, this is due to the beneficial effect of the flavanols contained in cocoa.
Prof Basu rightly called the paper:
the worst example of medical statistical misadventure we’ve seen in years.
Researchers from Belgium wrote in the Journal of Nutrition and sided with Prof Basu in dismissing the paper. They pointed out that this is a classic case of ecological inference fallacy, where conclusions about group data is drawn from individual data with no relationship between group and individuals presented. In other words:
the observed correlation is in fact based on country-averaged chocolate consumption and not on the actual consumption of Nobel Laureates themselves.
The two sets of data points have no commonality whatsoever. Chocolate consumption was over a 2 year period for the entire country, whereas the count of Nobel Laureates was over time. Some of the said laureates weren’t even alive during the 2 year chocolate consumption period. The Belgian researchers found an even higher correlation (r = 0.82 P < 0.0001) between the number of Ikea stores and Nobel Laureates in a country, a correlation they used to illustration the fact that it’s so meaningless that it’s laughable. All correlations do is to give a numerical relationship between data points, it’s up to the researcher to give meaning to the correlation. In some cases, there is no meaning.
The interesting observation is how the original paper even got through peer review into a journal. Was it meant to be ironic or humorous? Who knows. The research design comes into question. I’ve always thought of research as following the process of: do experiments -> make observations -> arrive at conclusion -> propose theory. This is the most traditional research method, especially in the natural sciences. In talking with mm, her professor seems to take the opposite approach: predict desired outcomes for theory -> design experiment/study -> get results that confirm theory. Seems to be a common method in social sciences.
There are different names for these research designs. Bottom up research is called exploratory or inductive research. The opposite, the top down approach, is called confirmatory or deductive research. Which is better, which more effective, that’s a difficult question to answer. It depends on the overall goals and the specific topic, I guess. Doing a lot of experiments lead to more new knowledge. Knowing what results you want may lead to more effective experiments but may not advance the overall knowledgebase. I don’t know the answer. All I know is, eating more chocolate does not, unfortunately, lead to winning Nobel Prizes.