Data Sonification session one

Data Sonification toolkit coming together! Today I’m learning about twotone and how to resucitate a dead web audio interface. The wonderful Øystein Fjeldbo comes by to help me navigate this brave data world, and talks me through some of the options I’m exploring to make a proof-of-concept. First up, based on a tutorial in the Brexification post from MCT (Music Commmunicationst Technology) Master’s student blog at NTNU turns out to be not that adapatble. It’s very handy that it comes embedded in Max for Live (Connection Kit) and I get the sense of how easy it could be to use.

I can change the API to another live data stream, but there is no simple way to swap their synthesised sounds for a sample player. It turns out to be a completely different process, applying paramater changes to a tone generated by the patch, rather than making changes to an audio sample. So, we take a look at the patch made by the students, who have adapted it to their own needs – but again, this is too specific and not quite what I want to do. Now I’m a little concerned that I will have to give the process over to a custom build, but when we look into the code for the mysteriously vanishing twotone app, it turns out this is something Øystein can help me rebuild.

What is it for?

TwoTone can be used for understanding data through listening. It makes data more accessible.

Data

TwoTone can be used by itself or in tandem with visualization. Just like in the cinema, sounds add another layer to understanding.

Music

TwoTone is a fun and intuitive way to make your own compositions without any prior musical or technical knowledge.

Google NEWS & DATAVIZED TECHNOLOGIES

Or so the twotone web audio app promises – sadly their domain is no longer active. Even though it was launched with great fanfare on the 5th March in 2019. Only two years on, and it’s already obsolete. “TwoTone is imagined and made by Datavized Technologies with support from Google News Initiative. We hope you like it.” Did Data Sonification fall out of fashion so fast? Luckily it’s open source, and thanks to GitHub, the code still exists. But I have no idea what to do with it, or where to start.

https://twotone.io/ introduction

Where did it go? Technology graveyard from two years ago. Google labs and all.

The talented Øystein, who teaches Data Sonification manages to get this zombie code going… from the graveyard of twotone.io https://github.com/datavized/twotone

It’s such a beautiful and simple concept, a data sonification web-based audio tool. You simply upload your dataset and then can choose instrumentation, key, duration, etc. Once we have it running on my browser, I start working my way through the tutorials.

TWOTONE TUTORIALS

Introduction: https://twotone.io/tutorials/introduction-to-twotone/

Advanced: https://twotone.io/tutorials/advanced-features-tutorial/

NODEJS

Nodejs – another thing we had to install to get twotone working!

https://nodejs.org/en/

https://nodejs.org/en/about/

I really have no idea what any of this means, but I trust that Øystein knows!

Despite not being able to swap out my own sounds with the pre-made samples, it’s satisfying working through the options to make interesting sonification compositions.

I’m pretty happy to be able to add my own csv data sets – a couple of examples from my research. Golden Gate Bridge accelerometers recorded on mobile phone by Stephen Wylie, via kaggle https://www.kaggle.com/mrcity/golden-gate-accel-20180512 This is one minute of data from the “Linear Accelerometer” of the Physics Toolbox Suite v1.8.6 for Android. The data was collected from a Pixel 2 phone on the east side of the Golden Gate Bridge at the midpoint between the two towers of the bridge at approximately 3:20 PM local time on May 12, 2018. CCO Public Domain.

and the composition looks like this…

adding sound later – the export is doing a glitchy thing where it only gives a minutes of the sound, not all 30 minutes… But it’s only a problem with this one, not my next attempt using pedestrian data from the Brooklyn Bridge.

NEW YORK CITY OPEN DATASETS

Transporation: Brooklyn Bridge Pedestrian Count

Towards Manhattan / Towards Brooklyn. Weather summary, precipitation:

Now it gets fun to start playing with filters: defining the key, speed and instruments<

I’ve figured out a hack to get my own samples in there – just have to name them exactly as the existing sets are named. Going to work on that tomorrow.

There’s a range of durations, from 02 seconds to 14 seconds, and I think the actual notes can be sacrificed for a variety of sounds. This is where it starts to get really creative! And sounds like some wild free jazz. Now I need to do some more study in order to really get a sense of the possibilities of sonification. Here’s the lecture series by Thomas Hermann – one of the people who literally wrote the book on sonification.

SONIFICATION AND SOUND DESIGN, MCT 2019

MCT Data Sonification course taught by Thomas Hermann: techniques beyond parameter mapping for applications in data mining and bio feedback

MCT4046 Sonification and Sound Design Seminar Series – Spring 2019
Speaker: Thomas Hermann

Facilitator: Shreejay Shrestha
Video production: MCT students 2018-2020
Video recording: Shreejay Shrestha, Guy Sion
Audio recording: Jørgen Nygård Varpe
A/V editing & processing: Sepehr Haghighi
Design: Karolina Jawad
Music: Sepehr Haghighi
Technical support: Robin Støckert, Anders Tveit, Alexander Refsum Jensenius
Administration: Maj Vester Larsen
Consultant: Sigurd Saue
Seminar Series Curation: Anna Xambó
Recorded in NTNU Trondheim 2019

ARNE NORDHEIM

Norwegian composer Arne Nordheim, recommended by Øystein to listen for generative compositions and experimental music techniques.

Hardanger Bridge monitoring

Next step: contact NTNU engineers and ask for access to their data, pretty please?!

NTNU Case study: The Hardanger Bridge

The Hardanger Bridge is a 1380 meters long suspension bridge, located on the western coast of Norway. It is the longest suspension bridge in Norway, and the 10th longest in the world, which makes it an interesting case study.  Our research in the field of suspension bridges requires knowledge from several engineering fields; such as aerodynamics, signal processing, finite element analysis and control theory. 

Experimental surveys

A comprehensive measurement system is operating on the Hardanger Bridge to improve the current understanding of the dynamic behaviour of long-span suspension bridges and their interaction with wind. This includes sensors for measurement of both response and environmental excitation. The system is described in detail under Structural monitoring (Structural monitoring – The Hardanger Bridge).

Source: https://www.ntnu.edu/kt/research/dynamics/monitoring/hardanger

Figure 2: Illustration of the front view of the Hardanger Bridge, showing accelerometer and anemometer positions. Illustration by NTNU/Heidi Kvåle.

The Hardanger Bridge monitoring project

The Hardanger Bridge, opened in 2013, is a 1380 m long suspension bridge crossing the Hardanger fjord in western Norway. The main span is 1310 m long, which makes it the longest suspension bridge in Norway and the 10th longest in the world. The two concrete bridge towers are 200 m high, made using the slip forming technique, and supports the steel bridge casing and the cables that are anchored in the mountain side.

The main objective with the monitoring project is to study the dynamic behaviour of the suspension bridge; especially the wind-induced vibrations. All data generated from the extensive measuring system is directly used in research related to the ferry-free costal highway E39 project; both PhD and master theses. 

The monitoring system consists of the following sensors (illustrated in Figure 2):

  • 20 triaxial accelerometers
  • 9 anemometers

System identification and modal analysis

Based on recordings established by the monitoring system, parameters characterizing the system behaviour; typically represented by natural frequencies, damping ratios, and mode shapes; are estimated (system identification). The results are a highly valuable asset for these applications:

  • Studying the dynamic behaviour of the bridge
  • Updating the numerical model, such that it better describes the real behaviour of the bridge
  • Verification and possible improvement of the current state-of-the-art methods used for numerical modelling
Modal shape of the Hardanger Bridge. Model and animation by NTNU/Øyvind Wiig Petersen.

Load modelling and identification

Modelling of the wind-induced forces on suspension bridges is crucially important for accurate prediction of the dynamic response.

The modelling of the environmental wind loads hinges on the description of the spatial and temporal characteristics of the wind field. The wind data from the field measurements will be used to characterize the wind field. It is aimed to test the performance of load models and reveal the uncertainty involved in response prediction.

The models for motion-induced loads most commonly used in bridge aerodynamics are linear engineering approximations. It has been shown in several case studies that the models are working well when the response of the bridge is dominated by one vibration mode in each direction. Taking into account that the principle of superposition does not hold in fluid dynamics, it is unknown if the models will be able to predict reliable results for a more complex motion. Thus, there is a need to test the accuracy of the linear assumption introduced in the modelling of the self-excited forces. This challenging task does not only require development of new experimental setup but also identification techniques able to work with an arbitrary motion.

People:

Postdocs and PhD candidates working with suspension bridges:

SOURCE: https://www.ntnu.edu/kt/research/dynamics/research/long-span/suspension-bridges

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s