Auditory Augmented Reality Data Experience



In a nutshell

Hyperglue originates from a one-week workshop held by Yanni Loukissas from Georgia Tech Data Design Lab. Hyperglue is an Auditory Augmented Reality App, designed during the 5 days of the workshop and developed in the last 48 hours. It matches the GPS signal of the user with the mapped musical bands' stickers on the street to play songs performed by those bands, calibrating volume as the user walks closer to or further from each sticker.

Stickers let bands share the same stage. Get out on the street and listen.


During the workshop, I took care of the developement of a functioning protype that uses live GPS location data and uses it to activate and calibrate acustics within the Web App. The prototype is on github. Please note that for optimal testing this prototype requires a GPS-enabled device, some sort of speaker, a reasonable amount of battery, and to be used at 45°30'13.5"N 9°09'49.1"E on Planet Earth. Below you can see a video simulation that we used as proof of concept before development.

Data Gathering and Concept

We don’t often consider where data come from and how they are shaped by their origins. The road between the railway station and the university campus is one that is traversed many times by hundreds of people every day. As a students’ passage road, many use this street as a place to showcase their identity and projects through stickers placed on walls, poles and other elements of the street. We set out to explore this part of Bovisa’s identity by mapping the stickers placed on the roads around the campus. We tagged them by geo-coordinates, category and surface that they were sticked on.

After this initial data-gathering phase, we saw a big potential in focussing on one peculiar aspects of the stickers: their ability to bridge different places and dimensions, as a hyperlink, eventually erasing distance by spreading a message from where the sticker is placed, with its glue.

Acustic AR for a Humanistic Data Experience

The goal of the workshop was to design a "Data Walk", a 5 minute walk through wich you would experience data about the walk itself. For our data walk, we decided to zoom into the dataset and focus specifically on musical bands' stickers only. We interpretedd each sticker as a speaker that diffuses music from its band. This choice revealed to us the nature of the street that we were studying: multiple tracks from different bands fight for people’s attention just as the stickers themselves are placed one over the other. As you get closer to a band’s sticker, the volume of the track rises, and multiple tracks get on top of the others as multiple stickers are placed close together. This creates a mix of sounds that at first may seem chaotic, but nonetheless generates a soundtrack that is inherently local and dictated by people.

  • hyperglue
  • hyperglue

Features of the App

Starting from our data walk, we designed an App that serves the core experience of the walk. The App features visualizations that show where the band’s stickers lead to. You can also save the stickers that you like on a personal wall. The App could be expanded as a means for independent bands to connect with people in a more local way, where bands place their stickers and augment reality around them with sounds and their songs.

  • hyperglue
  • hyperglue

Excerpt from final delivery

Personal reflection

As this Workshop was my final course before graduation, I intended to make the most out of it by building a team with other colleagues that I respect the most. This played out very well, even better than I expected. We really worked as a team throughout the whole week: surely we study in the same course but each has their own peculiarity and personal skills, which came into action from start to finish.

The concept was designed and discussed many times, together 100%. In a second part of the workshop we divided our forces to what we could take on individually while trusting the others for the development of the other parts. In this phase, I moved 85% to coding the working prototype that the whole class could experience during the Data Walk. I started from scratch based on the knowledge I've gained during last Semester’s Creative Coding class, using P5js. The whole coding part was more of a challenge than a straightforward development of the app. This is because I’ve never used real time location data and neither did I know where to look. But I felt challenged both by my classmate who asked if we were actually going to develop this or fake it, and also by the quote you said in class: “It’s worth to struggle with technology: it means that you are at the cutting edge.” This resonated with me so much I had to do it for real! After several failed attempts and testings I got it even further than what we imagined and what I personally thought I could do. I used the remaining 15% to consult with the team on main decisions across the board. Generally, this workshop has been quite intense, mostly towards the end, but at the same time very much enjoyable and rich of new insights on how we can approach data on a more humanistic level.

Tech Specs

Coding languages and libraries used: HTML5, CSS3, JavaScript, p5.js (+ p5.sound.js + p5.geolocation.js)


Valeria Sonia Aufiero
Andrea Benedetti
Simone Costagliola
Alessandro Zotta

All images on this site are Copyright © 2019 Alessandro Zotta or their respective copyright holders. Do not use without permission.