An AR Application designed for people living in memory impairment

concept | research | prototype | 3d modelling

Memory Diary

Memory Diary is a collaboration project with Peng Song and Te Bao. It is a speculative concept which is designed for people living in memory impairment. It is an image diary generated from stories that are made by participants. A tool kit is designed which consists of a set of objects with 21 different shapes, a board (size: 50cm*50cm*6cm) and an AR App on mobile phone. People living in memory impairment are encouraged to tell stories or express their feelings with family members and friends, by placing objects onto the board. By physically touching, and articulating emotions, this project helps stimulate memories. Moreover, as one way to communicate with people living with memory impairment, not to bring them to reality, but try to enter their world.

Prototype of story telling board and 3d printed metaphorical objects.

Example of generated image through the pattern made on the story board.


Illustration of Engagement Process

Illustration of function flow

Objects in different shapes are designed to be metaphors for participants to transform intangible ideas into context. At the end of the story or conversation, participants, for example family members, could take a picture of the board through the AR App, different arrangements of objects will trigger different kinds of shaders that will be added onto the picture. The background color will reflect participants’ emotions, for example, red represents angry and green represents relaxing etc. A sound sensor will be installed beneath the board, which will capture the dialogue and later on Tone Analyzer API will analyze the information and show the result by colors. The final image, which generated by the composition of objects and emotion-representative colors, will be saved as an image diary.