Reading Augmentation
The science of reading shows that Reading While Listening, also called immersive reading, is an underrated and unexplored reading augmentation.
Effective reading is one of the fastest ways to learn and increase your information input, enabling you to understand and solve more and more complex problems.
After giving an overview of the science of reading, I will describe a relatively new type of reading, Reading While Listening, and compare tools that you can use to augment your reading.
The Science of Reading
Models
There are two main scientific models of reading. The simple one is the Simple view of Reading (SVoR), and was proposed by researchers Gough and Tunmer in 1986. It defines reading comprehension as the product of decoding and linguistic comprehension. It is useful to categorize the most frequent reading disabilities:
A more refined model, Scarborough’s Reading Rope was published in 2001 and breaks down the ability to read into Language-comprehension and Word-recognition.
The active view of reading (AVR) model from May 7, 2021, recognizes that some reading abilities like vocabulary comprehension contribute to recognition and comprehension. It also separates active self-regulation, which determines the reading quality based on factors such as working memory, attentional control, and the skill of planning.
Reading While Listening (RWL)
In a meta-study with the title Audio-visual synchronization in reading while listening to texts: Effects on visual behavior and verbal learning. from 2017, published in Computer Speech & Language, the authors reported positive findings:
The first reports of the effect of RWL were very promising, especially for poor readers. In these non-experimental studies, RWL improved comprehension, word recognition, word meaning acquisition, reading fluency, motivation to read, and the “awareness of the end product of learning to read”.
The main reasons the authors in this paper also touch on for this effect seem to be improved comprehension and reading speed:
Improved comprehension
The first factor involved in improved comprehension is the reduced cognitive load. Automaticity theory defines the reading process, listed as part of the models above, simultaneous (i.e., as multi-threaded), which all take up cognitive resources and are limited by the reader’s working memory. This theory suggests that making one or two components automatic may allow readers to pay more attention to other tasks in the reading process.
Similarly, in a different paper, the author conjectures that RWL-readers
benefited from a lighter cognitive load. With additional cognitive resources available for comprehension, these students achieved higher quiz scores. In other words, their improved reading fluency facilitated their reading comprehension.
The second, closely related factor is multimodal information intake. For a closer look at the neuroscience of reading, I recommend this presentation by Stanislas Dehaene, author of Reading in the Brain. Herein, he also shows that information intake of visual reading or listening uses different paths but is both processed in the same brain region responsible for higher-level understanding.
Thus, we can conjecture that the gain in comprehension is achieved through multimodal input of the same information.
Research in fundamental cognitive psychology has shown that such an audio-visual redundancy favors further recall of words (e.g., Bird & Williams, 2002; Lewandowski & Kobus, 1993) compared to unimodal (i.e., either oral or visual) presentations.
Improved reading speed
The increased reading speed is achieved by reducing regressive saccades (eye movement). These unintended reflexive eye movements are driven by our brain trying to achieve 100% prediction accuracy and understanding. In some cases, this is useful to “re-encode” the content or further process it, but often it is redundant since the meaning of a viewed word will often become apparent by reading the following text and context. In his book on Speedreading, Tony Buzan lists other effects such as unconscious back-skipping, visual wanderings, and conscious regressions.
Here Buzan also recommended using a metronome, reading each line at the metronome rate, guiding the eyes using a stick, and gradually increasing the metronome rate.
The RWL software has a similar effect: reducing regressive saccades by guiding your eyes to continue reading. However, the currently available software, which I list below, still has a lot of improvement potential.
After going back to the speed-reading literature, I also noticed potential improvements in the current position/word highlighting of the RWL software. One improvement would be to help the user reduce the number of fixations by chunking words together based on the reader’s ability.
The only downside of using these RWL applications I experienced was that they could reduce your ability to read fast when you don’t use them. We might observe a similar effect when trying to study without nootropics after excessive use of them.
Best Reading while listening Application
Amazon Wispersync
Wispersync is the best RWL option for books. It’s handy if you mainly listen to the audiobook but want to make highlights. You’ll always be in the correct position. My current alternative workflow is that I have the corresponding book in iBooks and search the section to do highlights which is slower. There is not much information about it, but I found this article explaining the market of Wispersync.
Natural Reader
The natural reader is the best option for PDFs and has a fast loading speed. It also auto has some logic to disregard stuff in parentheses, i.e., academic references like (author_x at al. year_y), which improves the listening experience.
The three cons are that there is no mobile app, or bookmarking feature, and you can click on the text to navigate what to read, which interferes with text highlighting.
Speechify
Speechify has a Chrome extension, web, and mobile app. It also has some very high-fidelity voices (I think from Google WaveNet). Speechify supports up to 4.5x speed-up, i.e., 900 WPM.
Matter
Matter is the best reading app that supports RWL I found and has some good social features to follow certain writers. Unfortunately, it only supports up to 2x playback speed. (Edit: as of mid 2023 Matter added support for up to 3x) .
Other reading Tech
Bionic reading: helps focus on the overall shape of the words by bolding text.
Jiffy Reader: open source implementation of the above.
Spritz: presents individual words in fast succession.
Cybersalience: ML attention model that highlights important parts of text.
Scim: Intelligent Skimming Support for Scientific Papers (can auto highlight goals, methods, and results).
Beeline reader: Extension for various applications and browsers that helps your eyes jump to the next line in a text by changing the text color.
Appendix: Improving your Reading process
Now that I explained the science of reading and tech to augment your reading process, let’s look at a few non-technological tricks to augment the entirety of your reading process. This is not a comprehensive list and more so a list of pointers to explore.
Characteristics of a skilled reader: In her book Effective Note Taking, Fiona McPherson describes that
skilled readers read the topic introduction and difficult to relate to the main topic sentences slower and that
they recall more superordinate than subordinate items (i.e., they think about how a given argument or concept fits into the greater whole).
Preparation and selection: This article on how to choose what to read describes that we should ask ourselves, “Is this a book in 260,000?“. Naval Ravikant and others have also talked about having a strong filter and disregarding books fast after one loses interest.
Training micro-skills: One lesson we can draw from the above-discussed models is that reading is a complex skill constituted of many parts that one can train individually. You can, for example, improve the bridging process skill of vocabulary knowledge when you add new words that you come across and learn about to your spaced repetition system.
Reading research papers in an unfamiliar field: Michael Nielsen describes how to improve the exploratory reading and internalization process.
Decide on reading type and depth: This article Reading books vs. engaging with them might be helpful to think about your time investments after you decide on your material. The classical book on reading by Mortimer J. Adler also touches on the four levels of reading (i.e., Elementary Reading, Analytical, and Syntopical). For a shorter overview, I recommend the essay How to Read a Book by Paul Edwards, academic historian and CISAC fellow at Stanford. The German philosopher Arthur Schopenhauer also wrote about the tradeoff between reading and thinking for ourselves. Schopenhauer describes reading as: “letting others think for you”.
Leveraging Curiosity: A friend of mine Adithya Balaji (he also wrote a nice article on audio reading), once told me about his curiosity driving reading strategy rule, which is “read now or disregard”. My current system is that I separate reading from searching for reading material. I like to batch process them because they use different cognitive processes (information search vs. consumption). I also like Adithya’s policy, though, and it seems there is a tradeoff between the use of curiosity and efficiency. If you have a different approach or advice on this, I’d be curious to hear it! (I plan to dive into the topic of curiosity and write about it here in the future).
Conclusion
Choose what you read wisely, use your curiosity as a driver, and augment your reading with technology.
If you’re interested in improving reading and learning technology, join us to build the next version of the RemNote PDF-Annotation tool and much more. If you have been thinking about too or have suggestions, I’d love to hear from you. Also, let’s connect on GoodReads.