In My Feelings (NYU Thesis)

NYU ITP

I explored how personal data could become self-expression—tracking my emotions by room for three weeks and translating them into a light installation for my NYU ITP thesis.

Timeline

3 months

Tools

Arduino, Myoware muscle sensor, MQTT, Grafana, PostgresSQL Fitbit, Fitbit API, Python, Java

LIGHT INSTALLATION OF EMOTIONS

THE OVERVIEW

During the 2021 pandemic, I began tracking my moods by room in my NYC apartment over three weeks. What started as a personal data experiment evolved into my NYU ITP thesis—a light installation projecting my emotions onto my apartment wall as a creative reflection of my week.

THESIS

An experiment in gathering multiple data points of my emotional state throughout my apartment and whether that data can used for creative expression.

During my thesis, how long did I spend at home?

319:42:44  hours

2  weeks out of 3 weeks

😱

WHY AM I COLLECTING DATA?

THE INSPIRATION

Self-proclaimed tracker

I like tracking things. I mainly use health apps to track things such as steps, food, physical activity.

At the beginning of my thesis, I was considering sending a survey about health habits to others and how to improve health habits. 

Data Artists

However, as I was doing more research I came across data projects more so focused on exploring daily life and observations, not necessarily with the intention to improve one’s life. 

Inspired by Nicholas Feltron's practice of collecting data from his surroundings and the Dear Data project's use of observations for self-insight, I decided to focus on tracking my own life rather than surveying others, allowing for more varied and flexible data collection."

FELTRON Annual Reports

Designer Nicholas Feltron created Annual Reports documenting his life through aggregated personal data, meticulously recorded in his phone's Notes app.

Nicholas Feltron

Dear Data

Lupi and Posavec collected weekly data on topics like how often they said thank you, visualizing it on postcards they mailed to each other throughout the year.

Giorgia Lupi and Stefanie Posavec

THE PROCESS

DATA COLLECTION

There ended up being three stages to my thesis project. It was not a linear process, but when I look back at the process this is how I would characterize the stages.

  1. Experimental data collection

  2. Data analysis

  3. What do I do with all this data?

EXPERIMENTAL DATA COLLECTION

STAGE 1

I started to look around me and gather information about my day to day. Over course of a few days did this and sliced the data different ways, ultimately ended up with these as my main form of analysis : emotion by room.

  • Activity

  • Room

  • Start Time

  • End Time

  • Duration

  • Emotion

  • Emotion Notes

What are the various ways of tracking emotion ?

Manually logging

I manually logged my emotions and activities using Toggl Track, starting timers for each activity and tagging them with emotions and location. I supplemented this with Notes and message timestamps for accuracy.

Emotions

I narrowed my original 37 emotions using Dr. Gloria Willcox's feeling wheel, based on Robert Plutchik's color comparison of emotions, adjusting categories to fit my experience. I applied similar narrowing to my activities.

Angry

Blank

Bad

Disgusted

Fearful

Focused

Happy

Sad

Surprised

Feeling wheel - Dr. Gloria Wilcox. I categorized 37 emotions to 9 emotions (right)

Biometric Data

Humans have physical reactions when they experience emotions (Ex: happy - smile). I wanted to measure body reactions that could indicate a change in emotion.

Heart Rate - there are various studies on emotion and heart rate. Ex: when you’re angry, or anxious your body releases stress hormones which cause your heart to beat more rapidly. 

Muscle Tension - I noticed when I get anxious, I tend to tense my upper body, so I wanted to measure the activities that caused me tension.

Heart Rate

I tracked heart rate via Fitbit API, averaging it for each emotion over one week. The data showed minimal variation, leading me to conclude that three weeks wasn't sufficient—future iterations would require months of data for meaningful analysis.

Average heart rate per emotion

Calculating average heart rate with the Fitbit API and Python

Muscle Tension

I tested muscle tension throughout my day using a Myoware sensor connected to an Arduino, which sent data via MQTT to a PostgreSQL database. For mobility, I used a battery-powered setup in a running belt.

What I looked like with all the sensors attached

My robotic arm - muscle sensor, attached to a battery powered Arduino

Data of muscle tension (labeled as stress) being sent to PostgresSQL

Here is the graph version of the data I collected over time, used Grafana. When I tensed, the graphical interface increases. I’ve labeled the activities I was doing at the time as they correspond to the graph.

The data proved unreliable—extremely noisy despite smoothing filters, and the sensor couldn't distinguish between movement (like typing) and actual shoulder tension.

I concluded this metric required a more controlled environment rather than day-to-day wear."

Attempt 1 at smoothing the data

Attempt 2 at smoothing the data

Diary

At the end of each day, I wrote a diary entry to holistically understand the day.

This entry at the front is about my frustration with this muscle sensor.

ANALYSIS

STAGE 2

I put all my data in a spreadsheet. I used python and python libraries to do many different analyses of my emotional data.

I found my emotion by room and created a heat map for deeper analysis. I also found some other interesting data : emotion by time of day and most frequent activity.

Emotion by room

INSIGHTS

  • Happy was top emotion across rooms

  • Most time spent at desk

  • Experienced most range of emotions at desk 

  • Neutral was top emotion in kitchen

Emotion by time of day

INSIGHTS

  • Felt most bad during the morning : tired

  • Happiest in afternoon

  • Widest spread of emotions in the morning

Most frequent activity associated with each emotion by room

INSIGHTS

  • Thesis gave me most range of emotions -> made me simultaneously feel the most bad and the most happy

  • Routine activities I mainly felt neutral (as on auto-pilot - eating vitamins at my desk, cleaning in the living room

  • Fearful manifested itself in anxiety (anytime I had to prepare talking to someone - thesis advisor, on the phone) 

  • Fearful in the living room - doing physical therapy, anxiety over whether my physical therapy would help

VISUALIZATION

STAGE 3

My thesis goal was to creatively use the data by exploring how I experienced emotions in physical space—each room as an extension of my feelings.

Since I spent most time at home, I created a timed light installation projecting color-coded emotions (inspired by the emotion wheel) onto my apartment walls.

LIVING ROOM - ORIGINAL

LIVING ROOM - LIGHT INSTALLATION

The visualization's duration corresponded to emotion frequency—the more I felt an emotion in a room, the longer the light lasted. I included diary excerpts for context. Though a public thesis project, this visual journal was intended for personal reflection on my week.

ALL EMOTIONS AT A GLANCE

THE WRAPUP

REFLECTIONS

Continue the visualizations for other rooms

Watching my visualization in the living room was a nice encapsulation of my week - I was able to observe and relive my emotions, almost like a highlight reel. If I had more time, I would continue to do emotions for each room (as I only had time to visualize the living room).

Transition between the lights

In between the emotion change or light switch, I’d like to transition between colors (ex: instead of going from yellow to purple at once) for a smoother viewing experience.

Hardware to understand emotions requires specific setup and calibration 

This was my first time experimenting with biometric data - I learned that wearables or hardware need specific setup and calibration.

Muscle tension and heart rate are highly affected by movement, so in the future to measure emotions, I would have a controlled environment (ex: measuring emotion by having a list of specific tasks to do or watch on screen in one sitting position) 

Data introspection

The data collection process was an interesting way to be introspective as I had to write down my emotions in real time. It brought consciousness to what I was feeling in the moment.