Senstory - A Multisensory Book Concept

I've always been captivated by the experience of being surrounded by books in a library. Now, I’m on a mission to transform how we enjoy physical books in the digital age. Imagine immersing yourself in a sensory world with the texture of leather, subtle sounds, warm lighting, and the scent of old paper. I'm excited to introduce my project, 'Natural Reflections,' featuring nine excerpts that explore the connections between nature and human existence. Each excerpt is paired with sensory delights, making 'Natural Reflections' a unique reading adventure that elevates the power of words like never before!

Tools
Adobe Illustrator, Adobe Animate, Adobe Audition, Projection Mapping and Fine Arts.

Concept
Experience Design

Step into a book—not just with your eyes, but with all your senses.

Natural Reflections redefines the reading experience by transforming words into a multi-sensory journey. Feel the texture of leather-bound pages, breathe in the scent of aged paper, and immerse yourself in ambient sounds and warm lighting—all while exploring nine captivating excerpts that bridge the connection between nature and human existence.

This isn’t just reading; it’s an experience. Are you ready to awaken your senses?

The Process

*

The Process *

Problem Statement: To redefine the traditional book reading experience and make it more immersive such that the written words feel more meaningful.

Step 1: Finding Inspiration

Ever since I discovered my love for reading, I’ve wanted to write. I was captivated by the colorful worlds created by authors and imagined ways to enhance the reading experience using technology without overshadowing the written word. This idea inspired me even before I started my graduate studies at NYU’s Interactive Telecommunications Program (ITP). I realized my thesis could be the perfect opportunity to explore this concept and bring my vision to life, even though I started with just a basic idea of my final goal.

Influence 1: Personal Experiences

After two years in hospitality, I became captivated by the operations of Michelin Star establishments and the artistry of wine tasting. Each meticulously curated dining experience felt like a narrative, where flavors contributed to a larger story. Wine tasting, similar to the nuanced world of perfumes, took me on a sensory journey. My interests also led me to explore innovations like 4D cinema and virtual reality, particularly the enchanting 'Tree' experience at ITP, which highlighted the transformative power of immersive media in storytelling. Additionally, I discovered the beauty of tools like Tarot cards, where unique imagery creates malleable narratives. The experience of reading physical books in a library—the scent of leather, soft lighting, and tactile pages—was mesmerizing. While I initially considered using QR codes in books, I opted for a more analog approach, focusing on enriching the written word through multiple sensory stimuli.

Influence 2: Secondary Research

After conducting extensive research by reading various articles and books, as well as gathering personal experiences from individuals, I have developed a methodology for approaching the project and determining the best direction to take. I have gained clarity on my hypothesis that readers would appreciate increased immersion in their reading experience as long as this does not detract from the quality of the writing. Otherwise, they may perceive their experience as a different medium altogether, rather than simply reading a book. Additionally, I have drawn inspiration from children's books and medieval literature, recognizing that children's books have already delved into technologies such as scratch and sniff, as well as employing illustrative imagery similar to the artistic borders and typefaces seen in medieval times.

Influence 3: Exploration

After completing my research, I set out on a quest to immerse myself in a wide range of sensory experiences, exploring various book designs and their presentations firsthand. My journey took me to Olfactory Art Keller, an artist space situated in lower Manhattan. This unique venue provided a platform for artists to showcase scent-based artworks, including books, sculptures, and set designs. I also sought inspiration at the Metropolitan Museum of Art, delving into medieval art and visual displays. Additionally, I made trips to bookstores, libraries, and specialized perfumeries to gain a deeper understanding of real-time sensory stimuli.

The Intended Goal

A book of nature assembled by man.
An ancient book that fits modern times.
A book seeming manufactured, made by hand.
A book of existentialism eliciting comfort.
A 2D book evoking a 4D sensation.

A Book of Juxtapositions

Step 2: Prototyping and User-Testing

With a newfound direction, I embarked on the journey to manifest my goal. I was confident that I would be the author of the book rather than relying on existing or generative content. Though uncertain about the content and writing style, I began with hope, knowing that clarity would come as I made adjustments to my work.

Writing

As I began writing, I discovered that I was spending more time in nature and reading various books to help with creative thinking. I experimented with different types of content, such as romantic tragedy, information on Vedic sciences, the journey of the Fool inspired by tarot, and philosophy. I spent a significant amount of time figuring out what kind of content I wanted to convey to my readers.

Until the day I figured it out…

As I journeyed home from school one day, I gazed up at the bright Moon against the city lights. The sight stirred something within me, inspiring me to capture my feelings in my Notes App. As I pondered, I experienced a sudden realization - the answer had been right in front of me all along, within the beauty of nature.

Nature offers us profound insights into the human condition, revealing simple truths that are often overlooked. By observing nature, we can gain valuable perspective on our lives and uncover meaningful metaphors that provide deeper insights into the essence of being human. This inspired me to write 9 powerful excerpts for my book.

Post writing, I chose to title my book 'Natural Reflections' since the book is meant to depict my 'reflections' on 'nature' which came to me 'naturally' as I observed certain phenomena in the outside world. The metaphor of nature is used to make sense of certain things we feel as humans.

Prototyping and User Testing

During the initial prototyping phase, I used a combination of existing text and my own writing, along with sensory stimuli, to assess user response.

I was torn between visuals that were hand drawn and visuals that were digital and thus I tested both styles. Through testing, I concluded that while users did like visual drawings, they seemed to really be drawn toward imagery that was displayed via projection mapping. I further tried to test what kind of imagery they prefer: abstract, suggestive, or obvious to which I was told to go with the suggestive style.

For scent, I was torn between scratch and sniff paper and diffusing fragrance in the atmosphere. Users seemed to prefer both, stating that they did quite like the child-like imagery of scratch and sniff paper juxtaposed with immaculate text as well as the maturity of more nuanced diffused fragrances.

After conducting the tests, I decided to incorporate a combination of Scratch and Sniff paper, Fragrance Diffusion, Hand-Drawn Visuals, and Projection Mapping to deliver visual and olfactory stimuli. For sound and touch, I chose to use textures and verbal narration.

Step 3: Making

As I was more certain of which direction to take in terms of presentation, I started assigning each excerpt to a sense that I deduced would best bring out the meaning of the excerpt. I conducted the tabular assignment on Miro.

For effective deduction, I evaluated the excerpts based on two categories: Meaning and Emotion Evoked, and then justified which sense would most complement the experience while reading, assigning the final sense accordingly.

Since I wanted to keep this project relatively analogous, gathering the right kind of materials was imperative. Especially for excerpts requiring touch, I gave a lot of thought to the kind of textures I would want to utilize.

To achieve the desired outcome of conveying a sense of the handmade, yet contemporary, and ancient, yet modern, I carefully selected materials that would resonate with audiences today. I chose sensory strips and scratch-and-sniff stickers to evoke familiarity and a calming sensation, complementing the weighty nature of the content. Additionally, I integrated natural elements such as leaves, twigs, water, and sand to harmonize with the overarching theme of nature. Lastly, I embarked on the exciting task of concocting my own unique fragrances to enhance the sensory experience, drawing from my passion for perfumery.

After having collected all the requisite materials, I began the final making of the book which I had envisioned. The making was divided into three parts: 1. Creating animations and sounds for the visual and auditory sensory aspects. 2. Applying touch and texture elements as well as concocting scents. 3. Hand-drawing additional visual elements.

Thought Process Behind Each Sense:

Sense 1: Visuals

After gathering alumni feedback, I decided to adopt a more medieval art style. Initially, my approach featured a modern design with handwritten content and large, realistic projections, but user testing revealed that it was hard to read. Users preferred a more suggestive art style.

Through my exploration of medieval artwork in old books, I noticed that decorations typically framed the text, which kept it legible and allowed me to explore simple 2-D animations along the page borders for visual excerpts.

I initially used Resolume Arena for projection mapping, creating realistic animations with effects reminiscent of live performances. However, I found the harsh contrasts unsatisfactory. I aimed for animations that felt more innocent and comforting.

I then switched to Adobe Animate, producing three easy-on-the-eyes animations to accompany specific excerpts needing visuals. My interactions with the text improved, and I added further visuals through hand-drawn artwork.

Using the Canterbury typeface added a medieval touch, while print formatting kept it modern. The excerpts were cut ruggedly for a distressed look, and I used water pens for the visuals to avoid damaging the fragile paper. After completing the visuals, I moved on to the next sensory element.

Sense 2: Olfactory

To incorporate smell with the writings, there were two potential ways in which I wanted to impart the sense: 1. Diffusing fragrance oil. 2. Scratch and Sniff paper/sticker.

At the time of user testing, I was still torn with the results since there was an equal preference for both of these ways of experiencing the sense. For this reason, I went ahead and incorporated both of these ways of imparting scent. 

Style 1: Stickers: 

While the initial idea was to create a custom scratch-and-sniff scent, the stickers that happened to be readily available did have the nuance required to depict the emotion, to my surprise.

During user testing, the scratch-and-sniff stickers received an unexpectedly positive response. Many people associate them with childhood, and they are commonly found in children’s books. The 2D graphics and familiar scents evoked nostalgia, which users preferred over more dimensional designs. The lightheartedness of the stickers also tapped into something primal, enhancing the self-reflective nature of the pieces.

I had to abandon my initial direction and prioritize user testing results. Although I wanted to create a more nuanced fragrance, I decided to keep it familiar to avoid overstimulation, given the immersive nature of the text. A complex scent would have led to sensory overload.

The added benefit of Scratch and Sniff Stickers – Since these stickers need to be manually scratched and smelled, it adds an additional dimension of touch due to the scratch texture involved in doing so. Due to this reason, I made sure to inculcate this style of emitting smell in pieces that required touch as a sense as well. 

Style 2: Dispersion:

Since diffusing fragrance was also an experience that users enjoyed during the testing phase, I chose to incorporate at least one instance of ambient fragrance in my work.

I considered limiting the use of multiple scents because they can linger and create a confusing mix that may overwhelm the senses. Using it once reduces this risk while still enhancing the overall scent experience.

I chose to create an ambient scent based on seasons, as they are atmospheric and experienced through involuntary sensation. It made sense for the piece to come from the environment rather than having the user seek it out.

I selected four fragrance oils to represent the essence of each season:

- Fallen Leaf (Autumn): A grassy scent with a stale, dried quality, reminiscent of leaves decaying in the soil.

- Surfs Up (Summer): An oceanic and coconut scent, evoking sipping a piña colada by the beach on a warm day.

- Ice Crystals (Winter): A strong menthol scent that cools the nose, like fallen ice.

- Lovely Lilac (Spring): A blend of lilac and orchid blossoms, capturing the freshness of blooming flowers.

These scents effectively convey the changing seasons.

In my experiment, I wanted to explore the multi-dimensionality of scent, building on the linearity and nostalgia of scratch-and-sniff stickers. As a fragrance enthusiast, I appreciate how olfactory notes develop over time. Since the excerpt discusses cycles and the non-linearity of human nature, I aimed to enhance this concept by creating my own scent concoction.

For the final, I aimed to evoke the 5th dimension, where all 4 seasons occur simultaneously. I mixed the scents of each season on cardboard to create a 5th scent, representing the confusion and uncertainty of human cycles compared to nature's predictable rhythms.

I used thick, stiff, A4-sized cardboard to disperse the scent because it absorbs and releases it while users experience the piece. Unlike the mechanical and artificial nature of a fan, cardboard allows for customizable waving motions that mimic natural wind and breeze patterns.

I alternated between no motion, breezy motion, and fast motion to create varying sensations of temperature and scent intensity. This immersive piece combined sound and scent, encouraging involuntary interaction while users had their eyes closed. The wind gusts against the skin and adds a tactile element, making it one of the more experimental works in the book.

Sense 3: Touch/Feel

For Touch, I spent the most time wondering if I should include haptic sensor-based touch elements or keep it purely analog and mess around with materiality instead.

Since I decided upon a more ancient, old feel of the book with a few modern elements being depicted in the visuals, I thought it would be a good idea to keep the touch elements more material-based and natural. It made the most sense since all the writings in my book were inspired by nature and naturally found, and materials were the only way to do complete justice to the final vision that I had planned.

As mentioned above, one of the touch elements was a by-product of scent dispersion, where the breeze was creating sensations felt during changing seasons. While I did intend for the piece to have an elemental sensation, I did not explicitly categorize it as touch when I was allocating a sense to it.

I took inspiration from that development and chose for a piece to have an explicit sensation element coupled with sound. 

This piece, in particular, talks about rain, and it seemed like the obvious choice to include a mist-like sensation while the user was experiencing the piece.

I used a fine-mist spray bottle, which I would spray lightly from up above to create the sensation of rain or drizzle. 

This piece is intended to be coupled with sound, and hence, the user would be keeping their eyes closed and experiencing it involuntarily from the atmosphere.

The sound itself was a narration of the piece supplemented with sounds of thunder and lightning.

The other touch elements that I wanted to incorporate were material-based and tactile. I wanted the user to interact with the piece with their hands while experiencing the words.

Most of the written excerpts are heavy in nature, provoking a lot of emotions of existentialism, reflection, and, to some degree, anxiety. To counteract these feelings or rather juxtapose these with calming elements, which in a way exhibit the comfort that one must find through acceptance of the discomfort that often comes with life, I chose to incorporate materials that are associated with creating a sense of calm, i.e., sensory strips. 

For some of the excerpts, I wanted the material to reflect articles gathered from nature since all of the excerpts are reflections of nature.

Certain excerpts involved trees and sands, for that reason, I chose to inculcate real trees and sand to create texture. 

The other type of texture is 3D paint, which has a 3D glue-like texture on dry down. The swirl patterns are meant to create an unruly feel of sand dunes.

These textures are meant to be felt while reading/hearing excerpts.

The overarching experience: the book itself.

The book itself was chosen to be of an old/grunge style. Some elements of it were already distressed and in addition, I used craft scissors to distress it further for more visuals. The rugged feel of the book elevated the natural textures that I added to it.

Sense 3: Sound

Sound was a sense that was best coupled with other senses since mere listening can sometimes not be immune to distractions and the wandering mind. An additional sense helps ground the user to the piece they are listening to.

Three excerpts, which I thought benefitted from an ambient, environmental experience, were chosen to be sound pieces. The excerpts were either coupled with scent or touch that seemed to be coming from the environment to create the effect of being present in nature/outdoors. This stimulates the imagination of the reader since the eyes are meant to be kept closed. The narration of the excerpts is coupled with environmental sounds as well to further immerse the reader into the ambience. 

Initially, I tried to use a voice synthesizer for the narrations, but that created a robotic effect, so I chose to narrate the pieces manually instead.

I wanted to create an ethereal, voice-from-above effect with the narrations, so I used Adobe Audition to add effects. 

Users were given earphones to listen to the piece and were asked to shut their eyes to experience it.

Instructions on what to do with each excerpt:

On the left side of each page excerpt, there are instructions that let the users know what to do before the start of the excerpt to maximize their experience of it.

During user testing where no instructions were provided, it was difficult and time-consuming for the users to figure out what interaction needed to be done along with the excerpt and hence instructions were mentioned to counteract this issue. 

The handwritten notes are meant to add a human touch to the work such that the reader feels a connection with the writer of the book.

Bonus: Taste

Toward the end of the book, during the live demonstration, a blank page was opened up to the reader to write their own thoughts or reflections as a space for catharsis. It was not a mandatory interaction, but users could take the time to write if they so wished.

Keeping in mind the heavy nature of the book, a candy was given to each user towards the end for comfort and to gently bring them back into reality.

Final Solution

*

Final Solution *

I want to emphasize the time it took to write this project more than anything because I truly wanted to explore literary arts for my thesis. Philosophy has been a subject that has intrigued me for a couple of years now and I wanted to use this opportunity to bring that to the forefront while using technology as a supplement rather than a centerpiece.

This project has been something that I wanted to explore for the last 3 years but I never found the right time or resources to embark on it. I saw an opportunity in my thesis.

Final Showcase

More videos: https://drive.google.com/drive/folders/1O7kHOrcewWaXvQSwAL2NzLwEHBA3QN_3?usp=sharing

Feedback Received + Takeaways

I’m grateful for the positive reception that I received for my Thesis showcase. The feedback that was given to me by my peers shall help me to make improvements for the future scope of this project. Some of them include:

The textures were appropriately used and fun to play with

This project brought out their inner child

The mixture of scents (My experimental concoction) was very interesting and people spent some time deciphering what the scent is, thus grounding them in the piece.

The animations were appropriate but certain portions could have been smoother.

The stickers added to the ‘scrap-book’ feel of olden times and it was well received.

The usage of sound was interesting but could have been used throughout the ambiance.

The scent filled the hallway and kickstarted the experience while walking towards it, thus increasing anticipation and setting the mood. (The scent alone evoked curiosity and pulled a lot of people in.)

The writing was well received.

Overall people felt that the book did a decent job of juxtaposing old with new and manufactured with handmade.

While I feel that I did a satisfactory job in bringing my vision to life, there are certain things that I would consider doing differently when sizing up the scale of this project:

Automating the animations

Making the book extremely medieval or modern without any elements of the other in the art style and making the book either feel manufactured or truly handmade i.e., creating a singular theme instead of experimenting with juxtapositions and contrasts. While I enjoyed incorporating 4 different ways of expression in one book as an experiment, I would want to try and add a very specific theme the next time I recreate this piece.

Deciding to either make this book a handheld experience or an exhibit i.e.; deciding between making it a commercial piece or an installation.