Literature review — sad enough but they’ve got it all wrong!

My journey from Mendeley via Google Drive (with Adobe Reader on-the-side) with a layover at Evernote … till we built our own

Hello researchers, I am super excited to put in my first article in The Academic Rollercoaster. This is a great opportunity to take you back to my PhD days in 2006 and to tell you a story about how, after several misadventures, I finally decided to start building RAx — the ultimate Jarvis for academic researchers, initially as my own small academic project. Back then I was voraciously reading all sorts of paper (yeah, more irrelevant ones than ones that helped me expand my scholarship). And, I had to do a lot of literature reviews (some of us would rather want to call it a critique) and discuss that with my advisor. And to be honest, it was quite a mess.

My first taste of reference managers

I started using some of the early versions of modern reference managers, first Endnote and then later on converting myself to Mendeley. I thought I had become specially empowered in a fundamental way, being one of the early adopters. I was, quite sadly, wrong!

I soon realised that, at the best, I was able to organise the papers that I was reading in one place for future reference. However, I was not able to organize the most important thing — I was losing all the key takeaways while I was going through a paper, all the new insights that I was developing in the process, all the revisions that were happening to my previous understanding, and all the loose ends and gaping holes that needed to be addressed in the future. The result: I left all these reference managers, started using Google Drive (I was quite active in Dropbox for a while till I started loving the design of Drive — the simple, familiar, clean, folder-ish style).

I was able to organize the papers with reference managers but I was not really organising the key takeaways from those papers.

Did I lose anything out?

Actually not. At the end of the day, I realized that it is always on me to give a very meticulous thought to what should go where and also to fill up the metadata properly — all reference managers assume that I will be doing that perfectly, every time, and then based on that they will be giving me all sorts of “cool” features such as auto-formatting of my reference list in standard citation styles, ability to shift my reference list (in case I want to convert myself), and newer additions like suggestions of research paper based on my reference list.

I asked myself that why all such coolness couldn’t stop me from getting into Google Drive. The answer was quite simple — I really did not need formatting and all that from a reference manager since I can do that perfectly when I actually sit down to write my paper draft using a decent Latex editor. My final citation list size for any of my paper is 15 on an average (maybe 50–60 for someone from biosciences or humanities) and it just takes me less than 20 minutes to copy-paste the BibTex format from Google Scholar into the .bib file needed by my Latex editor. Any day I would love to remain in one single yet super simple platform for all my day’s work, and Google Drive just used to do that for me!

It is always on me to give a very meticulous thought to what should go where and also to fill up the metadata properly — all reference managers assume that I will be doing that perfectly…

But then what about organizing my reviews?

The manual way: As mentioned earlier, organizing papers is not the same as organizing reviews. My way of dealing with it (if at all I can call that “organizing” of any sort) was extremely straightforward (and surprisingly commonplace too!). I started jotting down my notes on hard-copy prints of these papers and ended up hoarding them up in my cabinet. I know that’s what most of us still do. But I am pretty sure many of you will also agree to what I found happening to me:

God knows how many times I had to make duplicate hard-copies, lost important pieces of sheets, scribbled down some cryptic side notes which I couldn’t decipher two months later, had a real hard time comparing my notes across so many papers as they got accumulated, and started a frantic search through my pile of papers just because I forgot where I have seen something similar to what I am reviewing currently.

Adobe Reader style: I got to know about the highlight and sticky note feature of Adobe Reader around Jan 2014 (yeah, I was months late to adopt since the feature was out in June 2013) and I started using that together with Drive. I thought I had gotten the holy grail. But I faced an even more obvious problem that I never actually thought of — I was mimicking the same behaviour that I had with hardcopies, except that now I did not need to search through a huge pile of papers if I had to search for something — well, I did have to open each and every paper stored in my Google Drive folder with Reader and then search. And trust me, it can become quite tedious over time.

A tryst with Evernote: Multiple file searchability feature was the reason why I started using Evernote (of course, all modern reference mangers also support this aspect). But then I can’t deny that it was love at first sight and much of the credit should be given to the design. I thought it was a great way to pen down all my ideas, my to-dos, my schedule, and at the same time scan across all my literature for specific search terms (the technology, though, still needs lot of polishing as of date since I do not necessarily use the same terms while expressing similar thoughts in all my sticky notes). So, is this finally the holy grail? Hardly.

Conclusion

I realized that there are three fundamental cognitive processes behind literature review — (i) extracting the key facts and contributions from a paper, (ii) developing deeper understanding and revising my previous understanding of various aspects of the paper, and (iii) documenting my own insight and scholarly opinion regarding the essential ideas in the paper. And to do all these three efficiently I needed a way to ask the right research questions at the right time that would, in turn, trigger me to collate all the facts, figures, and arguments scattered over the paper in the right places for future reference and comparative study. Not just that, it should also help me in systematically addressing those questions that require deeper thinking, understanding, and further investigation. Highlighting, sticky notes, and multi-file term searching simply couldn’t help me much on this. After all, sticky notes get scattered too! :-)