a typical PhD, students will read probably more than 100 papers. Reading
research papers is a skill that can be acquired and that is very different
from reading a novel. The reading group sessions at MISS are to introduce
students to that skill. In small groups of around 10-15 students per faculty
member, students will discuss papers selected by the school faculty. In
preparation for this, students are expected to study (not just read) the
provided papers in advance, by tracing the ideas in those papers as far
back as possible.
MISS, we will be holding two different types of reading groups:
Reading Group: On Monday at 17:15 after the lectures, we will run
an introductory reading group for all school participants, introducing
the concept of how to critically read a paper, with an example paper
for all to read and work through ahead of school. This will be held
on Monday after the lectures, and will be led by Sir Mike Brady and
read and work through the following paper before arriving at the school:
Heinrich, M Jenkinson, M Bhushan, T Matin, FV Gleeson, Sir M Brady,
JA Schnabel. MIND: Modality Independent Neighbourhood Descriptor for
multi-modal deformable registration. Medical Image Analysis, 16(7):423–435,
2012. DOI: http://dx.doi.org/10.1016/j.media.2012.05.008 (also
available at the following link)
Individual Reading Groups: On Wednesday, 10 parallel reading groups
will be held, led by individual MISS lecturers on a paper of their
suggestions. These will take place in small groups of students informally
over lunch (at everyone’s own expense), and will give you a
great opportunity to personally meet and work closely with one of
the MISS lecturers, and vice versa. We expect this to be a hugely
enjoyable experience, and hope that you will share “your”
lecturer’s enthusiasm on the paper they suggested – especially
if these are their own ones!
are now asked to select, from the list of 10 offered papers [download],
your preferred 3 papers in ranking order by using the following module:
DEADLINE TO SUBMIT YOUR PREFERENCE IS 10 JULY 2014
will strive to allocate one of these three papers to you, but at the same
time we will need to balance out the numbers per reading group and lecturer.
You will be notified of your final paper allocation by 12 July 2014. Please
carefully read and work through your allocated paper before arriving at
is no single right way of how to read a scientific paper, but you may
find the following guidelines helpful:
starting to read a paper, don’t read it front to back straight away.
Instead, you can use the following 3-stage process:
Screening: Read the title and abstract, and then flick through the
pages to look at any illustrations, pictures and plots, and the final
summary and conclusions. Now, set the paper aside and ask yourself
a number of questions: what is the paper about? What problem does
it purport to help solve? Is the problem important or of interest
to me? If I were to tackle this problem, how would I do it? This quick
skim will take you just a few minutes and is often the decisive factor
on whether you actually want to delve deeper into this paper or instead
find a more interesting one to read (of course, in this school, you
will still need to read the paper!). It’s similar to reading
the “blurbs” of a paperback novel, or the first page of
a newspaper. However, if you ask yourself the above questions you
are already actively engaging with the paper in a way that you do
not when you are reading a novel!
the punchline: On your second pass, you should read the paper front
to back, but leave out any equations or complicated descriptions,
so that you don’t slow down your progress through the paper.
Your aim on this pass is to try to understand the key ideas of the
paper, whilst avoiding to get bogged down by any technical/mathematical
details. Again, set the paper aside and ask yourself some more questions:
Does the paper propose a method – in which case, does it work?
Does it work on real data or just on simulated (numerical) data? What
are the limitations of the approach? What are the novelties/strengths?
Is the approach something that you might use in your work? Note that
you can generally answer all of these questions without understanding
a single equation! Again, you are actively engaging with the paper,
this time to establish the key ideas presented, which may whet your
appetite to go one step further.
Understanding the paper: This requires a third pass (and a fourth,
fifth… pass), when you read the paper much more carefully, trying
to work through all the nitty-gritty detail. This will involve looking
at references in this work, and can easily become a recursive problem
as you will start reading more and more papers in order to make sense
of the first one. However, do not get disheartened as you often only
need to find the key idea (steps 1 and 2) of those papers in order
to understand your paper. You can always return later to the more
interesting ones and read them more thoroughly. This is a great way
of building up your own library of papers. Note that Google Scholar,
ResearchGate and other tools allow you to also go “back to the
future”, as for a given paper, you can find out which later
(more recent) papers have cited and built up on it. This is often
used to establish the “impact” of a paper. However, don’t
be completely fooled by any odd “highly-cited papers”
– they could receive a lot of citations for being a particularly
bad example of a technique or application!
any of the steps above, try to switch between roles of reader/reviewer
and author. This will ultimately help you in writing better papers. To
Be critical: Do not believe everything that’s written in a paper,
but use your own judgment and (ever growing) experience. Question
the problem statement, motivation, timeliness and importance of the
work. Consider any assumptions made, and whether much simpler methods
could have been used rather than elaborate new methods. Check whether
a new method was compared to the state-of-the-art and whether it was
properly validated. The number of datasets, any quantitative values
in form of tables, graphs or plots are good indicators. Check for
any “magic parameters” and whether these have been tested.
Make a list of what improvements to this work could be done, like
a scientific reviewer for a journal or conference would do.
Be constructive: Try to find the key idea / main innovation of the
paper. Reviewers are often very critical (see above), but normally
would try to start their review by stating the contributions of the
work, and what they really like about it. When collecting the good
points of a paper, you can also think about how ideas generated by
a paper could be translated to other methods or applications (the
authors usually indicate this in their outlook section). The introductory
reading group paper (see above) is perhaps a useful example of how
an idea generated in Computer Vision could find its way into Medical
courteous: It is very easy to “dress down” a paper –
but putting yourself into the authors’ position may help you
see that some limitations cannot be overcome very easily; comparison
to other methods may not always be possible due to lack of openly
available source code/data or a difference in underlying assumptions.
Sometimes, even if a paper does not contain an entirely new method,
validation, or useful application, there may still be the grain of
an idea for future papers!
Be thorough: highlight passages, take notes, work through mathematical
derivations, and even write a brief summary of a paper that you read.
When presenting a paper in your own lab, you could make up a few slides
with bullet points of the key bits of the paper, using figures from
the paper for better visualisation. If any source code is available,
download it and try it out on provided or your own data – or
even re-implement it yourself.
Set your ego to one side: Often, a review says “This paper is
quite good but omits reference to Foobar et. al. [1,…,2624].
The author will immediately know that the reviewer is Foobar, or one
of his/her colleagues, and any reasonable Editor will ignore this
comment. Science does not progress so that my success is inevitably
your failure. Be generous in your assessment of other peoples’
work, even if they do omit mention to your 2624 defining contributions
on the topic which, in your unbiased judgement close off forever that
line of work and establishing you as the world authority.
your own “verdict”: In rare cases this could be completely
thumbs up or down (after your critical, constructive, courteous, thorough
and generous study process of course!), but more often will turn out
to be much more nuanced, summarising the various pro’s and con’s,
and the potential impact of the paper to stimulate new ideas.
are in fact papers (as well as books, websites and blogs) on how to read
papers – be your own judge on how good those papers are! Here is
just a small selection:
there are papers providing helpful guidelines on how to review a paper,
where our personal favourite stipulates the golden rule of reviewing:
“treat other manuscripts as you would want your own to be treated”!
Here another small selection:
by Julia Schnabel and Sir Mike Brady, 2 July 2014.