Data Sonification toolkit coming together!
Today I’m learning about TwoTone and how to resuscitate a dead web audio interface. The wonderful Øystein Fjeldbo comes by to help me navigate this brave data world, and talks me through some of the options I’m exploring to make a proof-of-concept.
First up, based on a tutorial in the Brexification post from MCT Music Commmunications Technology Master’s student blog at NTNU. This process turns out to be not that adaptable as their sonification works by changing the pitch of a generated tone, and for my purposes I need to pitch shift a sound sample.
However, it’s very handy that it comes embedded in Max for Live (Connection Kit) and I get the sense of how easy it could be to use, once I can add my own sound samples to and swap out their GPS coordinates for the weather at specific bridge locations.
I can change the API to another live data stream, but there is no simple way to swap their synthesised sounds for a sample player. It turns out to be a completely different process, applying parameter changes to a tone generated by the patch, rather than making changes to an audio sample. So, we take a look at the patch made by the students, who have adapted it to their own needs – but again, this is too specific and not quite what I want to do. Now I’m a little concerned that I will have to give the process over to a custom build, but when we look into the code for the mysteriously vanishing TwoTone app, it turns out this is something Øystein can help me rebuild.
Or so the twotone web audio app promises – sadly their domain is no longer active. Even though it was launched with great fanfare on the 5th March in 2019. Only two years on, and it’s already obsolete. “TwoTone is imagined and made by Datavized Technologies with support from Google News Initiative. We hope you like it.” Did Data Sonification fall out of fashion so fast? Luckily it’s open source, and thanks to GitHub, the code still exists. But I have no idea what to do with it, or where to start.
Where did it go? Technology graveyard from two years ago. Google labs and all.
The talented Øystein, who teaches Data Sonification manages to get this zombie code going… from the graveyard of TwoTone.io https://github.com/datavized/twotone
It’s such a beautiful and simple concept, a data sonification web-based audio tool. You simply upload your dataset and then can choose instrumentation, key, duration, etc. Once we have it running on my browser, I start working my way through the tutorials.
Nodejs – another thing we had to install to get TwoTone working!
I really have no idea what any of this means, but I trust that Øystein knows!
Despite not being able to swap out my own sounds with the pre-made samples, it’s satisfying working through the options to make interesting sonification compositions.
I’m pretty happy to be able to add my own csv data sets – a couple of examples from my research. Golden Gate Bridge accelerometers recorded on mobile phone by Stephen Wylie, via kaggle https://www.kaggle.com/mrcity/golden-gate-accel-20180512 This is one minute of data from the “Linear Accelerometer” of the Physics Toolbox Suite v1.8.6 for Android. The data was collected from a Pixel 2 phone on the east side of the Golden Gate Bridge at the midpoint between the two towers of the bridge at approximately 3:20 PM local time on May 12, 2018. CCO Public Domain.
the original data sonification composition sounds like this…
the export is doing a glitchy thing where it only gives a minutes of the sound, not all 30 minutes… I manage to change the key and duration of the notes for variation
But it’s only a problem with this one, not my next attempt using pedestrian data from the Brooklyn Bridge.
Now it gets fun to start playing with filters: defining the key, speed and instruments
I’ve figured out a hack to get my own samples in there – just have to name them exactly as the existing sets are named. Going to work on that tomorrow. Here’s a preview:
There’s a range of durations, from 02 seconds to 14 seconds, and I think the actual notes can be sacrificed for a variety of sounds. This is where it starts to get really creative! And sounds like some wild free jazz. Now I need to do some more study in order to really get a sense of the possibilities of sonification. Here’s the lecture series by Thomas Hermann – one of the people who literally wrote the book on sonification.
SONIFICATION AND SOUND DESIGN, MCT 2019
MCT Data Sonification course taught by Thomas Hermann: techniques beyond parameter mapping for applications in data mining and bio feedback
MCT4046 Sonification and Sound Design Seminar Series – Spring 2019
Speaker: Thomas Hermann
Facilitator: Shreejay Shrestha
Video production: MCT students 2018-2020
Video recording: Shreejay Shrestha, Guy Sion
Audio recording: Jørgen Nygård Varpe
A/V editing & processing: Sepehr Haghighi
Design: Karolina Jawad
Music: Sepehr Haghighi
Technical support: Robin Støckert, Anders Tveit, Alexander Refsum Jensenius
Administration: Maj Vester Larsen
Consultant: Sigurd Saue
Seminar Series Curation: Anna Xambó
Recorded in NTNU Trondheim 2019
Norwegian composer Arne Nordheim, recommended by Øystein to listen for generative compositions and experimental music techniques.
Hardanger Bridge monitoring
Next step: contact NTNU engineers and ask for access to their data, pretty please?!
NTNU Case study: The Hardanger Bridge
The Hardanger Bridge is a 1380 meters long suspension bridge, located on the western coast of Norway. It is the longest suspension bridge in Norway, and the 10th longest in the world, which makes it an interesting case study. Our research in the field of suspension bridges requires knowledge from several engineering fields; such as aerodynamics, signal processing, finite element analysis and control theory.
A comprehensive measurement system is operating on the Hardanger Bridge to improve the current understanding of the dynamic behaviour of long-span suspension bridges and their interaction with wind. This includes sensors for measurement of both response and environmental excitation. The system is described in detail under Structural monitoring (Structural monitoring – The Hardanger Bridge).
The Hardanger Bridge monitoring project
The Hardanger Bridge, opened in 2013, is a 1380 m long suspension bridge crossing the Hardanger fjord in western Norway. The main span is 1310 m long, which makes it the longest suspension bridge in Norway and the 10th longest in the world. The two concrete bridge towers are 200 m high, made using the slip forming technique, and supports the steel bridge casing and the cables that are anchored in the mountain side.
The main objective with the monitoring project is to study the dynamic behaviour of the suspension bridge; especially the wind-induced vibrations. All data generated from the extensive measuring system is directly used in research related to the ferry-free costal highway E39 project; both PhD and master theses.
The monitoring system consists of the following sensors (illustrated in Figure 2):
- 20 triaxial accelerometers
- 9 anemometers
System identification and modal analysis
Based on recordings established by the monitoring system, parameters characterizing the system behaviour; typically represented by natural frequencies, damping ratios, and mode shapes; are estimated (system identification). The results are a highly valuable asset for these applications:
- Studying the dynamic behaviour of the bridge
- Updating the numerical model, such that it better describes the real behaviour of the bridge
- Verification and possible improvement of the current state-of-the-art methods used for numerical modelling
Load modelling and identification
Modelling of the wind-induced forces on suspension bridges is crucially important for accurate prediction of the dynamic response.
The modelling of the environmental wind loads hinges on the description of the spatial and temporal characteristics of the wind field. The wind data from the field measurements will be used to characterize the wind field. It is aimed to test the performance of load models and reveal the uncertainty involved in response prediction.
The models for motion-induced loads most commonly used in bridge aerodynamics are linear engineering approximations. It has been shown in several case studies that the models are working well when the response of the bridge is dominated by one vibration mode in each direction. Taking into account that the principle of superposition does not hold in fluid dynamics, it is unknown if the models will be able to predict reliable results for a more complex motion. Thus, there is a need to test the accuracy of the linear assumption introduced in the modelling of the self-excited forces. This challenging task does not only require development of new experimental setup but also identification techniques able to work with an arbitrary motion.
Postdocs and PhD candidates working with suspension bridges: