Sunday, November 29, 2009

How And When To Read A Scientific Paper As A Layman

Science is conducted by people, most of whom are bright, a fraction are very bright, and a small fraction are brilliant. These people are, as discussed in the post Peer Review & Skepticism competing for prestige and grant money. The vast majority of papers are rushed at the end to get them ready for presentation at conferences.

The paper publication process starts with publication and presentation at a conference. The paper is distributed to all conference attendees, who hopefully read it before your presentation on it, and there's usually a question and answer phase afterwards.

The paper gets commented on (generally by email) from attendees who read it, went through the presentation, and had questions or insights.

Most paper presenters incorporate those comments into the next draft of the paper (or, sometimes, decide that the line of research needs to be abandoned), and then submit it for peer review and publication in a journal.

Once the paper is in the journal, there will usually be a journal moderated commentary and letters section, where a second round of "let's make sure this says what you think it says" comes into play, usually with responses by the original author put back in.

In a lot of ways, it behaves like USENET discussion groups, but more slowly and with a bit more deliberation in the outcome. (And yes, it has its flame wars and trolls.)

Scientific papers have an abstract, a number of chapters or sections, and a set of end notes. If you're not used to reading them, the abstract is a one or two paragraph summary of what's to be covered in the paper, the notes cover where their sources are, accredit people whose work they've referenced, and sometimes point to "Unpublished Annexes" that give people the ability to dig deeper if they're interested.

This means that - as a layman - you should start reading papers when they have their journal mandated Question and Response come through. This also means, between conference presentations and journal review and question and response that anything that's published within the last 9 months is still probably waiting for that commentary process to run through.

It will also be a difficult read. Most scientists are, to put it mildly, mediocre writers. Science papers also have a particular formalism that makes perfect sense in context, but explains why scientists generally can't write.

The ideal is that the authorial voice comes out to "Experiment X was run in methodologies Y, with results Z, the conclusions derived from the data are ...". This is the near inverse of journalistic writing, where you'd start with the conclusion, work your way through the data gathering, then gloss over the methodologies, and show how the results supported the conclusion.

Just because it's difficult doesn't mean you shouldn't do it, but you should, as a layman, budget time to look over related papers. You cannot read one paper in isolation and get a realistic appreciation of what's going on.

1 comment:

  1. Ken,

    You've got this completely wrong. Papers are not often presented first at conferences. The presentations might be of work in progress or they might be posters, but they are not papers in the usual sense of the term.

    The general sequence, as seen from the outside, is:

    1. Submit to a journal.

    2. Journal accepts for review (or not)

    3. One or more rounds of peer review. This is generally anonymous and secret, but more and more journals, especially those from the EGU, are using an open review process.

    4. At the end of the review process and the accompanying revisions to the paper, it is either accepted for publication or not.

    5. If accepted for publication the paper is published (obviously).

    6. If a reader finds a problem with a paper, they have the option of writing a comment. Comments are generally sent out for review, just as a paper. In addition, the original researcher(s) is given a chance to respond to the comment.

    7. If, and I must heavily emphasize the if, a comment is received and published, this generally closes matters on the paper as far as the journal is concerned. Of course debate may rage in the literature or a paper may just drop out of sight.

    In general I would recommend that laymen, such as myself, limit themselves to papers which have stood the test of time, ie. those which have not been debunked by other papers exploring the same subject or those papers which have been cited many times because they are influential.

    Remember, peer review is necessary but not sufficient condition for a paper to be worth your time.