Facebook Page Twitter 

29 July - 4 Aug 2018 Favignana, Sicily

Medical Imaging meets Deep Learning

See pictures of previous editions: Set1 - Set2

Reading Group - Rules of Engagement

During a typical PhD, students will read probably more than 100 papers. Reading research papers is a skill that can be acquired and that is very different from reading a novel. The reading group sessions at MISS are to introduce students to that skill. In small groups of around 10-15 students per faculty member, students will discuss papers selected by the school faculty. In preparation for this, students are expected to study (not just read) the provided papers in advance, by tracing the ideas in those papers as far back as possible.

At MISS, we will be holding two different types of reading groups:

  1. Introductory Reading Group: On Monday at 16:45 after the lectures, we will run an introductory reading group for all school participants, introducing the concept of how to critically read a paper, with an example paper for all to read and work through ahead of school. This will be held on Monday after the lectures, and will be led by Julia Schnabel. The introductory reading group is a useful example of how Deep Learning can solve a Medical Imaging task.

    The introductory paper is the following

    Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Konstantinos Kamnitsas, Christian Ledig, Virginia F.J. Newcombe, Joanna P. Simpson, Andrew D. Kane, David K. Menon, Daniel Rueckert, Ben Glocker. Medical Image Analysis 36 (2017) 61-78, doi.org/10.1016/j.media.2016.10.004

    Read it before coming to the school!

  2. Individual Reading Groups: parallel reading groups will be held, led by individual MISS lecturers on a paper of their suggestions. These will take place in small groups of students informally over lunch (at everyone’s own expense), and will give you a great opportunity to personally meet and work closely with one of the MISS lecturers, and vice versa. We expect this to be a hugely enjoyable experience, and hope that you will share “your” lecturer’s enthusiasm on the paper they suggested – especially if these are their own ones! Every student will receive information about his/her group by email. Unfortunately there is no assignment that will satisfy all participants, so in the interest of fairness the groups have been divided randomly. This means that students cannot change their assigned group.

The different groups are reported below in the section “GROUPS”. Please carefully read and work through your allocated paper before arriving at the school.

There is no single right way of how to read a scientific paper, but you may find the following guidelines helpful:

When starting to read a paper, don’t read it front to back straight away. Instead, you can use the following 3-stage process:

  • Screening: Read the title and abstract, and then flick through the pages to look at any illustrations, pictures and plots, and the final summary and conclusions. Now, set the paper aside and ask yourself a number of questions: what is the paper about? What problem does it purport to help solve? Is the problem important or of interest to me? If I were to tackle this problem, how would I do it? This quick skim will take you just a few minutes and is often the decisive factor on whether you actually want to delve deeper into this paper or instead find a more interesting one to read (of course, in this school, you will still need to read the paper!). It’s similar to reading the “blurbs” of a paperback novel, or the first page of a newspaper. However, if you ask yourself the above questions you are already actively engaging with the paper in a way that you do not when you are reading a novel!

  • Getting the punchline: On your second pass, you should read the paper front to back, but leave out any equations or complicated descriptions, so that you don’t slow down your progress through the paper. Your aim on this pass is to try to understand the key ideas of the paper, whilst avoiding to get bogged down by any technical/mathematical details. Again, set the paper aside and ask yourself some more questions: Does the paper propose a method – in which case, does it work? Does it work on real data or just on simulated (numerical) data? What are the limitations of the approach? What are the novelties/strengths? Is the approach something that you might use in your work? Note that you can generally answer all of these questions without understanding a single equation! Again, you are actively engaging with the paper, this time to establish the key ideas presented, which may whet your appetite to go one step further.

  • Understanding the paper: This requires a third pass (and a fourth, fifth… pass), when you read the paper much more carefully, trying to work through all the nitty-gritty detail. This will involve looking at references in this work, and can easily become a recursive problem as you will start reading more and more papers in order to make sense of the first one. However, do not get disheartened as you often only need to find the key idea (steps 1 and 2) of those papers in order to understand your paper. You can always return later to the more interesting ones and read them more thoroughly. This is a great way of building up your own library of papers. Note that Google Scholar, ResearchGate and other tools allow you to also go “back to the future”, as for a given paper, you can find out which later (more recent) papers have cited and built up on it. This is often used to establish the “impact” of a paper. However, don’t be completely fooled by any odd “highly-cited papers” – they could receive a lot of citations for being a particularly bad example of a technique or application!

In any of the steps above, try to switch between roles of reader/reviewer and author. This will ultimately help you in writing better papers. To this end:

  • Be critical: Do not believe everything that’s written in a paper, but use your own judgment and (ever growing) experience. Question the problem statement, motivation, timeliness and importance of the work. Consider any assumptions made, and whether much simpler methods could have been used rather than elaborate new methods. Check whether a new method was compared to the state-of-the-art and whether it was properly validated. The number of datasets, any quantitative values in form of tables, graphs or plots are good indicators. Check for any “magic parameters” and whether these have been tested. Make a list of what improvements to this work could be done, like a scientific reviewer for a journal or conference would do.

  • Be constructive: Try to find the key idea / main innovation of the paper. Reviewers are often very critical (see above), but normally would try to start their review by stating the contributions of the work, and what they really like about it. When collecting the good points of a paper, you can also think about how ideas generated by a paper could be translated to other methods or applications (the authors usually indicate this in their outlook section). The introductory reading group paper (see above) is perhaps a useful example of how an idea generated in Computer Vision could find its way into Medical Imaging.

  • Be courteous: It is very easy to “dress down” a paper – but putting yourself into the authors’ position may help you see that some limitations cannot be overcome very easily; comparison to other methods may not always be possible due to lack of openly available source code/data or a difference in underlying assumptions. Sometimes, even if a paper does not contain an entirely new method, validation, or useful application, there may still be the grain of an idea for future papers!

  • Be thorough: highlight passages, take notes, work through mathematical derivations, and even write a brief summary of a paper that you read. When presenting a paper in your own lab, you could make up a few slides with bullet points of the key bits of the paper, using figures from the paper for better visualisation. If any source code is available, download it and try it out on provided or your own data – or even re-implement it yourself.

  • Set your ego to one side: Often, a review says “This paper is quite good but omits reference to Foobar et. al. [1,…,2624]. The author will immediately know that the reviewer is Foobar, or one of his/her colleagues, and any reasonable Editor will ignore this comment. Science does not progress so that my success is inevitably your failure. Be generous in your assessment of other peoples’ work, even if they do omit mention to your 2624 defining contributions on the topic which, in your unbiased judgement close off forever that line of work and establishing you as the world authority.

  • Give your own “verdict”: In rare cases this could be completely thumbs up or down (after your critical, constructive, courteous, thorough and generous study process of course!), but more often will turn out to be much more nuanced, summarising the various pro’s and con’s, and the potential impact of the paper to stimulate new ideas.

There are in fact papers (as well as books, websites and blogs) on how to read papers – be your own judge on how good those papers are! Here is just a small selection:

Similarly, there are papers providing helpful guidelines on how to review a paper, where our personal favourite stipulates the golden rule of reviewing: “treat other manuscripts as you would want your own to be treated”! Here another small selection:


Every MISS student will receive an email with indication of their group.

GROUP 1: Tuesday 13:30-15:00

Michael Bronstein

MolGAN: An implicit generative model for small molecular graphs. Nicola De Cao, Thomas Kipf. https://arxiv.org/abs/1805.11973

GROUP 2: Tuesday 13:30-15:00

Ender Konukoglu

Anatomically Constrained Neural Networks (ACNNs): Application to Cardiac Image Enhancement and Segmentation. Ozan Oktay, Enzo Ferrante, Konstantinos Kamnitsas, Mattias Heinrich, Wenjia Bai, Jose Caballero, Stuart A. Cook, Antonio de Marvao, Timothy Dawes, Declan P. O'Regan, Bernhard Kainz, Ben Glocker, Daniel Rueckert. https://ieeexplore.ieee.org/abstract/document/8051114

GROUP 3: Tuesday 13:30-15:00

Sebastien Ourselin

Large scale deep learning for computer aided detection of mammographic lesions. Thijs Kooi, Geert Litjens, Bramvan Ginneken, Albert Gubern-Mérida, Clara I.Sánchez, RitseMann. 10.1016/j.media.2016.07.007

GROUP 4: Tuesday 13:30-15:00

René Vidal

Mathematics of Deep Learning. René Vidal, Joan Bruna, Raja Giryes, Stefano Soatto. 2017. https://arxiv.org/abs/1712.04741

GROUP 5: Tuesday 13:30-15:00

Kevin Zhou

Image-to-Image Translation with Conditional Adversarial Networks. Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, Alexei A. Efros. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 5967-5976. 10.1109/CVPR.2017.632

GROUP 6: Tuesday 13:30-15:00

Larry Zitnick

Mask R-CNNK. Kaiming He, Georgia Gkioxari, Piotr Dollár, Ross Girshick. 2017 IEEE International Conference on Computer Vision (ICCV). 10.1109/ICCV.2017.322

GROUP 7: Thursday 13:30-15:00

Thomas Brox

Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs. Maxim Tatarchenko, Alexey Dosovitskiy, Thomas Brox. 2017 IEEE International Conference on Computer Vision (ICCV). https://lmb.informatik.uni-freiburg.de/Publications/2017/TDB17b/

GROUP 8: Thursday 13:30-15:00

Trevor Darrell

Learning to Reason: End-to-End Module Networks for Visual Question Answering. Ronghang Hu, Jacob Andreas, Marcus Rohrbach, Trevor Darrell, Kate Saenko. 2017 IEEE International Conference on Computer Vision (ICCV). 10.1109/ICCV.2017.93

GROUP 9: Thursday 13:30-15:00

Sanja Fidler

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. Chelsea Finn, Pieter Abbeel, Sergey Levine. 2017 International Conference on Machine Learning (ICML). https://arxiv.org/abs/1703.03400

GROUP 10: Tuesday 13:30-15:00

Georg Langs

Anatomical Priors in Convolutional Networks for Unsupervised Biomedical Segmentation. Adrian V. Dalca, John Guttag, Mert R. Sabuncu. 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). http://openaccess.thecvf.com/content_cvpr_2018/papers/Dalca_Anatomical_Priors_in_CVPR_2018_paper.pdf

GROUP 11: Thursday 13:30-15:00

Daniel Rueckert

Dynamic Routing Between Capsules. Sara Sabour, Nicholas Frosst, Geoffrey E Hinton. https://arxiv.org/abs/1710.09829

GROUP 12: Thursday 13:30-15:00

Andrea Vedaldi

The Building Blocks of Interpretability. Olah, Chris and Satyanarayan, Arvind and Johnson, Ian and Carter, Shan and Schubert, Ludwig and Ye, Katherine and Mordvintsev, Alexander. https://distill.pub/2018/building-blocks/

For more information, send an email to: miss@dmi.unict.it