Graphical Sound

On April 18, 2013 by jodirose

Andrey Smirnov, Moscow, 2011

Graphical (drawn) Sound is a technology for synthesizing sound from light that was invented in Soviet Russia in 1929 as a consequence of the newly invented sound-on-film technology which made possible access to the sound as a trace in a form that could be studied and manipulatedIt also opened up the way for a systematic analysis of these traces such that they could be used to produce any sound at will…

The first practical sound-on-film systems were created almost simultaneously in the USSR, USA and Germany. In Soviet Russia Pavel Tager initiated developments in 1926 in Moscow. Just a few months later in 1927, Alexander Shorin started his research in Leningrad.

avra30

Tager’s system, the Tagephon, was based on intensive variable density optical recording on film while in Shorin’s Kinap system the method of transversal variable area optical recording on film was realized.

By 1936 there were several main, relatively comparable trends of Graphical Sound in Russia:
– Hand-drawn Ornamental Sound, achieved by means of shooting still images of drawn sound waves on an animation stand, with final soundtracks produced in a transversal form (Arseny Avraamov, early Boris Yankovsky);
– Hand-made Paper Sound with final transversal soundtracks (Nikolai Voinov);
– The Variophone or Automated Paper Sound with soundtracks in both transversal and intensive form (Evgeny Sholpo, Georgy Rimsky-Korsakov);
– The Syntones method, based on the idea of spectral analysis, decomposition and resynthesis, developed in 1932-1935 by a pupil of Arseny Avraamov, the young painter and acoustician Boris Yankovsky.

At exactly the same time very similar efforts were being undertaken in Germany by Rudolf Pfenninger in Munich and, somewhat later, by the animator and filmmaker Oskar Fischinger in Berlin. Among the researchers working with Graphical Sound after World War II were the famous filmmaker Norman McLaren (Canada) and the composer and inventor Daphne Oram (UK).

Ornamental Sound

The ornamental sound technique, developed in 1929-1930 by Arseny Avraamov, was similar to German animator and filmmaker Oscar Fischinger’s sounding ornaments first presented in 1932. In 1930, however, Avraamov was the first to demonstrate experimental sound pieces – based on geometric profiles and ornaments – produced purely through drawing methods. This was achieved by means of shooting still images of drawn sound waves on an animation stand.

ornamental_300

In December 1930 Mikhail Tsekhanovsky wrote in his article About the Drawn Sound Film: “with the invention of new drawn sound techniques (developed by Arseny Avraamov in Moscow, Sholpo and [Georgy] Rimsky-Korsakov in Leningrad) we are achieving a real possibility of gaining a new level of perfection: both sound and the visual canvas will be developing completely in parallel from the first to the last frame […] Thus the drawn sound film is a new artistic trend in which for the first time in our history music and art meet each other.” [2]

In autumn 1930 Avraamov founded the Multzvuk group at Mosfilm Productions Company in Moscow. To produce his first drawn ornamental sound tracks he had on staff a special draughtsman, cameraman Nikolai Zhelynsky, animator Nikolai Voinov and acoustician Boris Yankovsky who was responsible for the translation of musical scores into Avraamov’s microtonal Welttonsystem as well as Samoilov’s Ober-Unter-Tone Harmony system. The final scores were coded in Yankovsky’s 72- step ultrachromatic scale with the dynamics and speed variations indicated by the number of  frames. Yankovsky was also involved in the production of

acoustic experimental studies, developing methods for the synthesis of sounds with glissando, timbre crossfades, timbre variations and polyphony by means of multiple shooting on the same optical soundtrack (alternative to multi-track recording which was not available yet).

From 1930-34 more than 2000 meters of sound track were produced by Avraamov’s Multzvuk group, including the experimental films Ornamental Animation, Marusia Otravilas, Chinese Tune, Organ Cords, Untertonikum, Prelude, Piruet, Staccato Studies, Dancing Etude and Flute Study. In autumn 1931 the Multzvuk group moved to NIKFI (Scientific Research Institute for Cinema and Photography) and was renamed Syntonfilm Laboratory. In December 1932 NIKFI stopped supporting Syntonfilm and the group moved to Mezhrabpomfilm where in 1934 it was closed as it was unable to justify itself economically. The whole archive was kept for many years at Avraamov’s apartment, where in 1936-37, during Avraamov’s trip to the Caucasus, it was burned by his own sons, making rockets and smoke screens with the old nitro-film tapes, which were highly flammable.

Because of the cross-disciplinary nature of the new technique, people involved in it had to be skilled not only in music, but in acoustics, mathematics, sound-on-film technology and engineering. As a result even skilled journalists often could not understand the physical meaning of the phenomena under consideration or specific technological ideas. Having no developed terminology, many mistakes and unexpected “puzzles” appeared in their writings. Moreover, there were several known research groups – competitors in Russia and Germany working in parallel. It led to a very specific problem – encryption of the information. For instance, in the well-known photograph Oscar Fischinger holds ‘fake’ rolls made by his Studio for publicity purposes as he did not want his competitors to learn his actual techniques. He never used rolls as large as this – they were fakes. [3] Yankovsky had a very special way of making notes on his ideas. It is impossible to understand the construction of his tools from reading one description without referring to several other manuscripts that offer important keys for understanding it.

Syntones and Audio Computing

In 1931-32 Boris Yankovsky (1904-1973) was on the staff of the Multzvuk group. In 1932, however, disappointed with its Ornamental Sound approach, he left the group. Unlike most of his colleagues he understood that the waveform does not represent the tone colour uniformly and that only the spectrum of sound developed in time with all the nuances of its temporal transitions can give a complete picture. Of all the early graphical sound pioneers, Yankovsky alone pursued the approach of spectral analysis, decomposition and re-synthesis. His concept was based on the belief that it is possible to develop a universal library of sounds similar to Mendeleev’s table of chemical elements. His curves were spectral templates, semiotic entities that could be combined to produce sound hybrids. As an option he developed several sound processing techniques including pitch shifting and time stretching based on the separation of spectral content and formants, resembling recent computer music techniques of cross synthesis and the phase vocoder. To realize these ideas he invented a special instrument, the Vibroexponator – the most paradigm-shifting proposition of the mid-1930s.

In 1935 in one of his manuscripts Yankovsky wrote: ‘It is important now to conquer and increase the smoothness of tone colours, flowing rainbows of spectral colours in sound, instead of monotonous colouring of stationary sounding fixed geometric figures [wave shapes], although the nature of these phenomena is not yet clear. The premises leading to the expansion of these phenomena – life inside the sound spectrum – give us the nature of the musical instruments themselves, but “nature is the best mentor” (Leonardo da Vinci) […] The new technology is moving towards the trends of musical renovation, helping us to define new ways for the Art of Music. This new technology is able to help liberate us from the cacophony of the well-tempered scale and related noises. Its name is Electro-Acoustics and it is the basis for Electro-Music and Graphical Sound’.

vibroexp

Read the full text by Andrey Smirnov here – and more in his book with Jeremy Deller, Sound in Z Experiments in sound and electronic music in early 20th-century Russia, published May 2013:

“Russia, 1917 – inspired by revolutionary ideas, artists and enthusiasts developed innumerable musical inventions, instruments and ideas often long ahead of their time – a culture that was to be cut off in its prime as it collided with the totalitarian state of the 1930s.

Andrey Smirnov’s account of the period offers an engaging introduction to some of the key figures and their work, including Arseny Avraamov’s open-air performance of 1922 featuring the Caspian flotilla, artillery guns, hydroplanes and all the town’s factory sirens, and Alexei Gastev, the polymath who coined the term ‘bio-mechanics’.

Shedding new light on better-known figures such as Leon Theremin (inventor of the world’s first electronic musical instrument, the Theremin), the publication also investigates the work of a number of pioneers of electronic sound tracks using ‘graphical sound’ techniques.

Sound in Z documents an extraordinary and largely forgotten chapter in the history of music and audio technology.” 

Leave a Reply

Your email address will not be published. Required fields are marked *