Exploring the use of a pose as a mode of creating and displaying data, building a prototype in which one acts out the experience using pose detection in ML5, visualized in the browser.
Context: CMU M.Des. independent study; advised by Molly Steenson
Time: 16 weeks, part time
Team: individual
Tools: Javascript: p5.js, ml5.js, socket.io, D3.js
“the body is not only the crucial source of all perception and action but also the core of our expressive capability and thus the ground of all language and meaning”
— Merleau-Ponty
In their work Data Feminism, authors and scholars Catherine D’Ignazio and Lauren Klein assert that we obscure bodies in our practices of collecting and visualizing data, we often silence, extract, and monitize those bodies. With this assertion in mind, I wondered how to center body as a source of and mode of expressing data. I began developing a data visualization tool for groups of people to express their feelings about their community through the mode of pose, beginning with this question:
How might I design a data visualization tool for my cohort to remember through observation and encoding of the sensations of body and the associated stories?
Based on my research and discussion with Carnegie Mellon professor and community scholar Hannah Du Plessis, I justified pose as a potent design direction for encoding an experience, manifested as a “stick figure:”
- Unconditional acceptance and noticing of body is fundamental aspect for life-affirming community
- Every experience is embodied
- Abstract form provides not only a sense of anonymity (as opposed to drawn figures), but is more relatable because we can map ourselves onto it
I developed a prototype in which one could pose with their body in response to an emotional prompt, using ml5.js to detect poses:
The tool is designed to be used in group conversations. As people submitted their poses, they can displayed on a projector/large screen:
Delight in expressing the embodied and intangible
What other aspects of the body should be considered? Stephen Neely, Carnegie Mellon professor or Eurythmics, asserts that there remains a “depth, wealth, and wide range of potential interactions available to the experiencing body.” Beyond pose, there are other components of the soma (more than just the figure) that remain to be explored:
- Heavy / light (swinging example from Neely’s TED talk)
- Control / release (Calligraphy vingiette from Neely’s paper on Soma-Deep)
- Gesture (Skateboarding vingiette from Neely’s paper on Soma-Deep)
- Temperature (Feldenkrais Method)