The many ways a scientist can lie

In a paper published in the European Journal of Oncology, Professor John Bailar discusses how he thinks that although it is not often scientists directly lie, it is a common trait to mislead. They can do this without resorting to fraud or other direct lies.

False graphs and charts are not the only way scientists can lie to us

The choice of the topic of the study is a big area. Bailar uses the example of the tobacco industry- their preferred method of lying is through the use of secret research. If the secret research  shows a secondary one will not be harmful to the industry, only then are the results published.

Scientists are clever humans and know how to frame a question that will allow them to reach a pre-determined outcome. A study can also be put into the hands of a person adept at interpreting results in certain ways. Most scientists have peers that are engaged in such methods of interpretation and can invite them to interpret their specific study.

Media coverage and citation bias?

Throughout this project, we’ve been analysing the potential bias within our papers, their citations and authors, and examining the flaws, benefits and necessities of peer review. One thing we haven’t yet looked at is the media coverage our papers received.

Originally we wanted to look at the media coverage for each of our citations, but with over 3500 citations in Web Of Knowledge alone (i.e. not including all the cross-checking required with the PubMed citations) this was unfeasible, especially as it was a side project, rather than the crux of our research.

So instead I sat down this morning with the task of looking at the media coverage of each of our original papers in Nature. I was actually fairly surprised – this is a high impact journal for which most of the big newspaper journalists get pre-embargo press releases sent to them. What I saw, however, was that 60% of the papers received no press coverage at all.

Critical reflection of our methodology

Since we are now finished with generating and analysing the data, it is time for a critical reflection of our methodology. Every time researchers take their instrument of measurement and start with the data generation, they will stumble over shortcomings of their methodology and have to take those into account while performing the analysis. In this post, I will always try not only to point out the weaknesses of our methodology that we discovered, but also how we coped with it in the analysis.

So, here we go.

First of all, how Abi described in her blog post about the methodology, we wanted to randomly pick the papers to make sure our selection was unbiased  The initial thought was, that a random choice of papers would provide us with a solid sample, that we could also use to take as a representative section of the basic population (i.e. all papers published in Nature in 2005).

Author’s prevalence does not imply bias

Today I found out something curious. Let me approach it by challenging you.

Having a quick glimpse at the following picture, how long does it take you to identify the name Ranganathan?

Perhaps not long, but also not as quickly as you read Kelly JW or Bailey-Kellogg C.

If you remember my blog post on the results of the paper I analysed, you might also remember that Rama Ranganathan was the author responsible for most of the bias I found. Although the analysis of the paper had a very low overall bias – about 5 per cent – the bias that was found was, indeed, direclty linked to Ranganathan’s name.

Bias in the world of immunity

Here is yet another post with results. You can find my methodology here, which Abi has kindly written up for us, but if you have any questions or points that you didn’t quite follow, feel free to leave a comment and we’ll try to explain.

The Paper

I investigated a paper from the journal Nature which was published on 22 September 2005. The paper was concerning a protein found in humans, known as complement component C3, and the role it plays in human immunology.

The eight authors of the paper came from four different institutions, two in The Netherlands and two in Sweden.

As you would expect, this paper was cited by a number of articles from immunology and health journals.

How the visual cortex is build. And more fascinating figures.

My paper dealt with “Functional imaging with cellular resolution reveals precise micro-architecture in visual cortex.” In simple words, it analysis how the part of the brain responsible for seeing is build from differenc cells. By comparing rat-brain with cat-brain , the researchers found out that the architecture of the cell structures was different. If you wonder what this is for, let me tell you the last sentence of the abstract: “These results indicate that cortical maps can be built with single-cell precision.” I knew I had a good reason in chosing chemistry and physics over biology in school.

The paper and some basic figures

Anyway, the paper was published in the Nature issue 7026, published on the 10th Feb 2005 in print and on the 19th Jan in the same year online. Five authors had contributed to the paper, all from the huge Harvad Medical School in Boston, Massachusetts.

The research was supported by grants from the National Eye Institute from the United States and fellowships from the Uehara Foundation (Kenichi Ohki), the Goldenson Fund (Sooyoung Chung) and Howard Hughes Medical Institute (Yeang H. Ch’ng)

In total, the paper was cited in 225 papers from its publication to January 2011. About one third of these ciations, in figures 85, where found in PubMed, whereas Web of Knowledge found 216 ciations. Therefore, including PubMed in the analysis only added nine citations that where not covered by Web of Knowledge.